Effectiveness of e-learning in higher education

Effectiveness of e-learning in higher education José Fernández-Cerero1*, Marta Montenegro-Rueda2 1Department of Teaching and Educational Organization, University of Sevilla, [email protected] 2Department of Teaching and Educational Organization, University of Sevilla, [email protected] * Corresponding Author: José Fernández Cerero (email address: [email protected]). University of Seville. Department of Teaching and Educational Organization. C/ Pirotecnia s/n, Seville, 41013, Spain Abstract Nowadays, the knowledge society demands to be prepared in the use of technology, educational institutions cannot remain on the side-lines of this trend and are adapting their curricula to cope with these demands. The present study explored the results of recent research on the effectiveness of the use of e-learning in higher education, to provide a clear synthesis on the state of research, gaps, and trends in the use of e-Learning in the teaching-learning process at the university stage. For this purpose, a systematic literature review (SLR) method is used. Out of 238 initial articles, 19 were selected for study (published between 2012 and 2022). Among the results obtained, we highlight that doeslearning does not necessarily lead to better learning results but requires the involvement of the entire educational community to obtain a positive effect. Thus, the success of this educational model depends on how it is implemented by teachers. Keywords: Electronic Learning, Higher Education, Information and Communication Technologies, Systematic Review Introduction The world of higher education is constantly evolving and currently undergoing a digital transformation that is changing the way we think about teaching and learning. Information and Communication Technologies (hereinafter ICT) are being integrated into all aspects of the educational process, from management to delivery (Guerrero, 2018). This process implies a methodological, conceptual, and procedural change in methodologies and teaching-learning processes and electronic learning (hereinafter elearning) is taking centre stage in universities (Parlakkiliç, 2017). E-learning is a distance learning modality that is facilitated by the intensive use of the internet. This type of learning makes it possible to offer asynchronous training and access to a wide range of resources, without being bound by the constraints of time and place. It is a modality that promotes lifelong learning and helps individuals to stay updated with the latest knowledge in their field (Cardona Román & Sánchez Torres, 2011). As soon as the first experiments with e-learning began to be carried out in universities, it was realised that its adoption as a mode of education would bring about a global restructuring in the way of conceiving teaching, but also, essentially, in the way of understanding learning (Luque, 2004). Thus, in general terms, e-Learning refers to a set of techniques and processes of interaction between the subjects of the educational process to acquire new knowledge through Internet technologies in synchronous and asynchronous mode (Kornílov et al., 2020). Currently, e-learning is not only gaining popularity in the online mode of study, but also in face-to-face and blended learning modes. Blended learning is a promising method of teaching that combines traditional lectures with e-Learning elements. It provides the possibility to use a variety of media and tools for didactic problem solving. In terms of blended technology, teaching can be applied directly in the classroom, if a teacher interacts with students on an online platform (Allen & Seaman, 2013; Parlakkiliç, 2017). Massive Open Online Courses (MOOCs) are another e-learning initiatives that has gained popularity in many universities. MOOCs are open access online courses that allow unlimited (mass) participation with no special restrictions on participation. Many MOOCs provide interactive elements to encourage interaction among students and between students and teachers. However, the latter is not a defining requirement (Kaplan & Haenlein, 2016). However, we must bear in mind that e-learning to support teaching and learning processes, specifically those of a university nature, have limitations. One of the main limitations is access to technology. Access to technology, particularly access to the internet, is not equal. In poor and developing countries, access to technology is less than in developed and rich countries. Even within developed countries, access to technology is not the same for all inhabitants (Gutiérrez Campos, 2012). People with some kind of disability also face a disadvantaged situation when trying to approach the use of technology. Another limitation is the lack of access to computer equipment in rural areas, especially in the most depopulated and depressed areas. And finally, of course, teacher training in digital skills is also lacking in many cases (Fernández Batanero et al. 2022). Despite these limitations, there is a need to further explore the possibilities and challenges of e-learning in higher education, and to demonstrate whether it is a promising and effective tool for university students, understood as the ability of technology to enhance the learning process and the outcomes achieved by students, and can be measured through a variety of indicators, including knowledge retention, transfer of learning to practical situations, student satisfaction and academic performance (Kirkwood & Price, 2014).In this sense, this study aims to explore the effectiveness of e-learning in higher education and to discover the challenges and opportunities for implementing successful e-learning programmes. Previous reviews As technology has advanced, education has followed suit and has evolved to include new forms of online learning. E-learning has gained popularity in recent years, especially during the coronavirus outbreak (COVID-19) that forced the closure of educational institutions in most countries and has become an increasingly attractive option for university students (Katsaris & Vidakis, 2021). It can be defined as the use of ICT to facilitate learning and teaching in a virtual environment (Ruiz et al., 2006). E-learning has been widely used in higher education, offering benefits such as flexibility or accessibility of learning (Ally, 2004). In turn, e-learning can be divided into different types, such as MOOCs (massive open online courses), and blended learning, among others. MOOCs are open online courses that allow access to many students from all over the world. MOOCs are usually free and are often designed to encourage active participation and collaboration among learners (Liyanagunawardena et al., 2013, Hollands & Tirthali, 2014). Many see MOOCs as an alternative to the traditional education system due to the flexibility of learning time and the absence of economic and geographical constraints (Papadakis, 2023). However, there are certain challenges that need to be addressed. These revolve around student motivation and engagement, since being free and having no formal qualification, some students may feel less motivated or committed to the course, thus increasing dropout rates (Weller, 2014). Blended learning, also known as combined learning, is a learning modality that combines the use of information and communication technologies (ICT) with face-to-face teaching. This learning modality seeks to leverage the advantages of face-to-face teaching and elearning, providing a richer and more complete learning experience (Garrison & Kanuka, 2004). Blended learning has been widely used in higher education and has been shown to be effective in improving academic performance and student satisfaction (Graham et al., 2013). This has led to the emergence of several research studies that have explored the impact of e-learning in higher education, focusing both on quality assessment (Ortiz López et al., 2022) and on the possibilities presented by different models, such as blended learning, mobile learning, or MOOCs, on the academic performance of university students (Wang et al., 2011). These reviews conclude that e-learning will be effective if teacher training is provided, educational resources are designed to accommodate different learning styles, and assessments are designed to ensure a wide range of participation options (CrisolMoya et al., 2020). These systematic reviews provide an overview of existing research on e-Learning in higher education and can help educators make informed decisions about the design and implementation of effective e-learning programs. Since the primary purposes of systematic reviews are to define relevant concepts, synthesize evidence, identify previously used methodologies, and distinguish research gaps of interest (Baker, 2016). It is important to keep in mind that e-learning may not be the right option for all students, and not all higher education institutions may be prepared to offer high-quality online education. In this regard, we consider research on the effectiveness of e-learning in higher education to be necessary. Research can help to identify the challenges and opportunities of e-learning in higher education, and to develop best practices for implementing and managing successful e-learning programs. Thus, the purpose of this study is to synthesize recent research findings on the effectiveness of using e-learning in higher education and to provide a deeper understanding of the existing research and its implications for educational practice. To achieve this, the following research questions (RQ) were posed: RQ1. What are the research trends in e-learning in higher education? RQ2. What is the impact of the use of e-learning in higher education? RQ3. What are the possible future directions for the advancement of e-learning in higher education? Method To address the objective and answer the research questions guiding this study, a systematic literature review (SLR) method is used. Systematic reviews are an explicit process for identifying and synthesising evidence and answering research questions (Moher et al., 2014). Accordingly, this article aims to provide an overview of research related to e-learning in higher education, identifying trends and other relevant indicators through content analysis of recent publications, and identifying gaps and opportunities for research in this field, using VOSViewer software (version 1.6.15). The use of this visualisation software offers the possibility to present the data graphically through category maps. Search strategy In this study, the literature search was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines (Moher et al., 2009). To ensure the level of research quality, only articles from peer-reviewed journals, published in the last decade (2012 - 2022), were included. The search period was set at 10 years, i.e., publications between January 2012 and December 2022, considering that, in this last decade, virtual educational environments have had a remarkable growth in terms of techno-pedagogical contributions. A comprehensive electronic and manual search was conducted. Hand searching the reference list of selected articles will identify relevant studies that may have been overlooked in the electronic search. Even the most exhaustive searches may miss some important studies due to errors in indexing, language restrictions or lack of appropriate search terms. The educational databases recommended by our university librarian were Web of Science, Scopus, and Google Scholar. In addition, we justified this choice as previous researchers have indicated that major academic databases are more likely to report high quality studies (Bai et al, 2020). Our search strings and eligibility criteria were designed collaboratively by the authors. A Boolean AND/OR search was used focusing on the two main themes of the systematic review: (1) e-Learning and (2) higher education. The Boolean search used AND to connect the two parts of the search string and OR for synonyms in each of the themes. This ensured that the most relevant articles on the topic were collected during the search process. The search was conducted by abstract, title, and keywords. Table 1 presents the search terms that were selected to represent each of the themes: Table 1. Selected search terms. Topic Search terms 1 “E-learning” OR “Electronic Learning” OR “E-learning Platform” OR “E-learning Technology” 2 “Higher Education” OR “University” OR “Tertiary education” The hand search was conducted on the reference lists of identified articles that were relevant to the topic of study. The electronic and manual search resulted in a total of 238 studies. Inclusion and exclusion criteria As the main objective of the study was to investigate the effectiveness of e-learning, only studies that met the following inclusion criteria were included (1) scientific articles published in peer-reviewed journals, (2) studies published between 2012 and 2022, (3) studies in the context of higher education; and (4) studies investigating the effectiveness of e-learning in higher education. The following studies were excluded (1) according to study format: review articles, books, abstracts, conferences and conference proceedings, (2) studies published outside the selected period, (3) studies not aimed at higher education, (4) duplicate studies. Search results After the first search, 232 studies were identified in the different databases. In addition, 6 manually identified records were added to the references of these studies. After eliminating duplicates, 133 were evaluated for their eligibility according to the inclusion criteria, excluding 115 records. The 21 full-text studies were then reviewed for quality, excluding two studies. A total of 19 articles have been included. The flow chart of the search strategy is shown in Figure 1. Figure 1. Flow chart of the selection process. Quality assessment and data mining To ensure the review process, the PRISMA recommendations were rigorously followed. The methodological quality of the 21 identified articles was assessed through the Johanna Briggs checklist (JBI), which was examined by an independent critical review (elevenpoint checklist) (Aromataris & Munn, 2020). In order to assess the internal quality of the proposed study, the checklist was assessed by two independent researchers (Appendix I) to the research in a masked form in order to avoid assessment bias by the authors themselves. The checklist followed 7 assessment criteria: 1. Is the purpose of the research clearly specified? 2. Does it specify the type of technology applied in the study? 3. Is the impact of the use of e-learning in higher education specified? 4. Are the data extraction instruments adequate? 5. Are the results obtained useful for the scientific community? 6. Are the authors' conclusions supported by the data analysed? 7. Are recommendations made for future research? These quality assessment criteria were applied to all eligible studies that were reviewed at full text (n=21). To be included in the study they had to meet five or more assessment criteria. Two studies were excluded (Gupta & Gupta, 2020; Kadirbayeva et al., 2021) that did not meet at least five of the above assessment criteria. Therefore, a total of 19 studies were included in this review. The researchers individually examined the retrieved records, and subsequently validated that they met the specific criteria. The researchers reached 100% agreement on the coding, after resolving disagreements by consensus. Data analysis and mining The following data were extracted from the included publications for further use in the cross-analysis: (1) authors, (2) year of publication, (3) location and geographical focus of publication, (4) methodology, (5) instrument, (6) sample, and (7) e-learning platform. We manually classified, synthesised, and analysed the studies. The main characteristics of the analysed studies are shown in Table 2. Table 2. Overview of the studies included in the systematic review. Author(s) (Year) Location and geographical focus Method Instrument Sample E-learning platform Costa et al. (2012) Portugal (Europe) - Developed country Mixed Interview, Questionnaire and Documentary Analysis 278 participants (students) Moodle Khan & Badii (2012) Pakistan (Asia) - Developing country Quantitative Questionnaire 205 participants (students) Moodle Abou El-Seoud et al. (2014) Egypt (Africa) - Developing country Quantitative Questionnaire 124 participants (students) Moodle Ngampornchai & Adams (2016) EE.UU. (America) - Developed country Quantitative Questionnaire 84 participants (students) Not specified Quadri et al. (2017) Saudi Arabia (Asia) - Developing country Quantitative Questionnaire 257 participants (students and teachers) Not specified Hernández Godoy et al. (2018) Guatemala (America) - Developing country Quantitative Questionnaire 47 participants (students) Not specified Shahmoradi et al. (2018) Iran (Asia) - Developing country Quantitative Questionnaire 300 participants (students) Not specified Hennig & Nazarkulova (2019) Austria (Europe) - Developed country Quantitative Questionnaire 86 participants (students and teachers) MOOC, Moodle Kim et al. (2019) South Korea (Asia) - Developed country Quantitative Questionnaire 614 participants (students) MOOC Ayu (2020) Indonesia (Asia) - Developing country Quantitative Questionnaire 35 participants (students) Moodle Lange & Costley (2020) South Korea (Asia) - Developed country Quantitative Questionnaire 654 participants (students) MOOC Rivera, & Tamayo (2021) Ecuador (America) - Developing country Quantitative Questionnaire 81 participants (students) MOOC Milicevic et al. (2021) Serbia (Europe) - Quantitative Questionnaire 367 participants (students) Not specified Developing country Ullah et al. (2021) Pakistan (Asia) - Developing country Quantitative Questionnaire 200 participants (students) Zoom, Learning Management System (LMS), Moodle, Mobile Learning Younis et al. (2021) Iraq (Asia) - Developing country Quantitative Questionnaire 650 participants (students) Not specified Amer et al. (2022) Egypt (Africa) - Developing country Quantitative Questionnaire 1858 participants (students) Moodle Bouyzem et al. (2022) Morocco (Africa) - - Developing country Quantitative Questionnaire 331 participants (students) Not specified Maatuk et al. (2022) Libya (Africa) - Developing country Quantitative Questionnaire 160 participants (students and teachers) Not specified Qashou et al. (2022) Palestine (Asia) - Developing country Quantitative Questionnaire 95 participants (students) Not specified In addition to the quantitative analysis of the collected literature, we conducted a qualitative synthesis using a thematic approach (Xu & Zammit, 2020) to obtain information on the main findings of the studies. Results The scope of the present review is to provide an overview of the studies that have so far focused on the effectiveness of e-learning in higher education. In this section, we provide the results of the research. Year of publication The time frame selected for this review is from 2012 to 2022. Thus, it can be seen in relation to the evolution of the studies by year of publication (Figure 2), how the period from 2012 to 2022 was a decade in which e-learning experienced significant growth in higher education, with the number of publications increasing progressively over the years, but without yet reaching the barrier of five publications per year in peer-reviewed journals. Figure 2. Articles by year of publication. Methodology, data collection and sample size The size of the participant population ranged from 35 (Ayu, 2020) to 1858 participants (Amer et al., 2022). There were five contributions with between 1-99 participants, two between 100-199 participants, four between 200-299, three between 300-399 and five with more than 400 participants. The participants were mostly undergraduate students. Among the methodologies used in the studies, it reveals that the majority used quantitative methods (94.73%), with the questionnaire as the instrument used in the research. Only one study used mixed methods, applying interviews, questionnaire, and documentary analysis (5.26%). No study used a qualitative approach. Location and geographical focus of the studies The studies have been conducted in different countries around the world, mainly in Asia (47%), followed by Africa (21%), America (15%) and Europe (15%). Furthermore, the studies have been conducted mainly in developing countries (66.67%), i.e., those whose standard of living, income, economic and industrial development are still below average, including Pakistan, Egypt, Saudi Arabia, Guatemala, Iran, Iraq, Indonesia, Serbia. The rest of the studies are conducted in developed countries (33.33%), such as the USA, Portugal, Austria, or South Korea. Impact of e-learning in higher education The results of the existing research revealed the impact after the implementation of eLearning in higher education (Table 3) mainly highlighting the improvement of students' academic performance (30.43%) and the flexibility of the e-learning system (21.74%). Table 3. Impact of the e-learning system in Higher Education. Impact Studies Percentage Accessibility (Hernández Godoy et al., 2018; Lange & Costley, 2020; Rivera, & Tamayo, 2021) 13.04% Quality (Henning et al., 2019) 4.35% Flexibility (Abou El-Seoud et al., 2014; Quadri et al., 2017; Hernández Godoy et al., 2018; Henning & Nazarkulova, 2019; Milicevic et al., 2021; Qashou, 2022) 21.74% - Research on how personalisation of learning in e-learning can improve the effectiveness of learning in higher education, including how artificial intelligence can be used to tailor learning to the individual needs and preferences of students. - Research on how social interaction in e-learning can enhance learning effectiveness in higher education, including how online learning communities can promote collaboration, communication, and peer-to-peer learning. - Research on how accessibility and inclusion in e-learning can improve the effectiveness of learning in higher education, including how accessibility can be improved for students with disabilities and how inclusion can be promoted for students from different cultural and linguistic backgrounds. Funding This research is part of the R&D&I project, PID2019-108230RB-I00, funded by MCIN/AEI/10.13039/501100011033. References Abou El-Seoud, M.S., Taj-Eddin, I.A.T.F., Seddiek, N., El-Khouly, M. & Nosseir, A. (2014), E-Learning and Students’ Motivation: A Research Study on the Effect of E-Learning on Higher Education. International Journal of Emerging Technologies in Learning (iJET), 9(4), 20–26. https://doi.org/10.3991/ijet.v9i4.3465 Allen, E. & Seaman, J. (2013). Cambio de rumbo: diez años de seguimiento de la educación en línea en los Estados Unidos. Babson Survey Research Group y Quahog Research Group, LLC. Ally, M. (2004). Foundations of educational theory for online learning. In Theory and Practice of Online Learning (pp. 15-44). Athabasca University Press. Ally, M., & Prieto-Blázquez, J. (2014). What is the future of MOOCs? The International Review of Research in Open and Distributed Learning, 15(5), 1-14. https://doi.org/10.19173/irrodl.v15i5.1954 Amer, T.S., Al Musawi, A.S. & Muhammad, A.A.N. (2022). Higher education students’ perceptions of e-learning quality. Journal of Education and e-learning research, 9(2), 71-78. Area, M. (2008), Modelos formativos de eLearning, Universidad de La Laguna, Tenerife. https://manarea.webs.ull.es/modelos-formativos-de-elearning/ Ayu, M. (2020). Online learning: leading e-learning at higher education. The journal of English literacy education, 7(1), 47-54. Bai, S., Hew, K. F., & Huang, B. (2020). Does gamification improve student learning outcome? Evidence from a meta-analysis and synthesis of qualitative data in educational contexts. Educational Research Review, 30, 100322. https://doi.org/10.1016/j.edurev.2020.100322 Baker, J. D. (2016). The purpose, process, and methods of writing a literature review. AORN journal, 103(3),265-269. https://doi.org/10.1016/j.aorn.2016.01.016 Bates, T. (2005), Technology, e-learning and distance education. Second Edition. Ed. Routledge Taylor & Francis Group. Bouyzem, M., Ghilane, H., Moustakim, O. & Tsouli, V. (2022). Higher Education in Morocco in the Covid-19 era: What perception of the usefulness and ease of use of e-learning? International Journal of Business and Technology Studies and Research, 3(4), 1-7. Chen, Y. (2016). E-learning in higher education: A review of the research on implementation models and success factors. Higher Education Studies, 6(2), 1-10. https://doi.org/10.5539/hes.v6n2p1 Costa, C., Alvelos, H. & Teixeira, L. (2012). The use of Moodle e-learning Platform: A study in a Portuguese University. Procedia Technology, (5), 334-343. https://doi.org/10.1016/j.protcy.2012.09.037 Crompton, H., Burke, D., Jordan, K. & Wilson, S.W.G. (2021). Learning with technology during emergencies: A systematic review of K-12 education. British Journal of Educational Technology, 52(4). https://doi.org/10.1111/bjet.13114 Dybå, T. & Dingsøyr, T. (2008). Empirical studies of agile software development: A systematic review. Information and Software Technology, 50(9), 833–859. Fernández Batanero, J.M., Cabero Almenara, J., Román Graván, P. & Palacios Rodriguez A. (2022). Knowledge of university teachers on the use of digital resources to assist people with disabilities. The case of Spain. Education and Information Technologies. https://doi.org/10.1007/s10639-022-10965-1 Follorunso, O., Ogunseye, S. & Sharma, S.K. (2006). An exploratory study of the critical factors affecting the acceptability of e-learning in Nigerian universities. Information Management & Computer Security, (14), 496-505. https://doi.org/10.1108/09685220610717781 Garrison, D. R., & Kanuka, H. (2004). Blended learning: Uncovering its transformative potential in higher education. Internet and Higher Education, 7(2), 95-105. Graham, C. R., Woodfield, W., & Harrison, J. B. (2013). A framework for institutional adoption and implementation of blended learning in higher education. Internet and Higher Education, 18, 4-14. Gordillo, A., López Perna, S. & Barra; E. (2019). Effectiveness of MOOCs for teachers in safe ICT use training. Comunicar, 61, 103-112. https://doi.org/10.3916/C612019-09 Guerrero, C. (2018). Innovación y Transformación Digital en la Educación Superior. En C. Cáceres, N. Esteban, M. d. C. Gálvez y B. Rivas (Eds.). En Innovación y transformación digital: Estrategias y metodologías docentes en Educación Superior (pp. 19-26). Dykinson. Gupta, S. B., & Gupta, M. (2020). Technology and E-learning in higher education. Technology, 29(4), 1320-1325. Gutiérrez Campos, L. (2012). Conectivismo como teoría de aprendizaje: conceptos, ideas, y posibles limitaciones. Revista Educación y Tecnología, (1), 111-122. Hennig, S. & Nazarkulova, A. (2019). Benefits and challenges of elearning in central Asia. International Journal of Geoinformatics, 15(4), 43-41. Hernández Godoy, V., Fernández Morales, K. & Pulido, J.E. (2018). Attitude towards elearning in university students. RIERevista de Investigación Educativa, 36(2), 349-364. https://doi.org/10.6018/rie.36.2.277451 Hollands, F.M. & Tirthali, D. (2014). MOOCs: expectations and reality. Full report. Center for Benefit-Cost Studies of Education, Teachers College, Columbia University. Ijaz Baig, M., Shuib L. & Yadegaridehkordi, E. (2021). E-Learning adoption in higher education: A review. Information Development. https://doi.org/10.1177/02666669211008224 Kadirbayeva, D., Sabanbayev, N., Abdurayeva, G. & Zhanseitov, A. (2021). Advancement of elearning in higher education in the case of Kazakhstan. Journal of Educators Online, 18(2), 1-8. Katsaris, I., & Vidakis, N. (2021). Adaptive e.learning systems through learning styles: A review of the literature. Advances in Mobile Learning Educational Research, 1(2), 124.145. https://doi.org/10.25082/AMLER.2021.02.007 Khan, K.U. & Badii, A. (2012). Impact of E-Learning on Higher Education: Development of an E-Learning Framework. Life Science Journal, 9(4), 4073-4082. Kaplan, A.M. & Haenlein, M. (2016). Higher Education and the Digital Revolution: About MOOC, SPOCs, Social Media, and the Cookie Monster. Business Horizons, 59, 441-450. https://doi.org/10.1016/j.bushor.2016.03.008 Kim, H.J., Hong A.J. & Song, H.D. (2019). The roles of academic engagement and digital readiness in students’ achievements in university e-learning environments. International Journal of Educational Technology in Higher Education, 16(21), 118. https://doi.org/10.1186/s41239-019-0152-3 Kitchenham, B. (2004). Procedures for performing systematic reviews. (NICTA Technical Report). Kornilov, Iu.P., Danilov, DA, Kornilova, AG, Golikov, AI & Gosudarev, IB (2020). Diferentes enfoques para el desarrollo del aprendizaje en línea en la educación superior. Propósitos y Representaciones, 8(SPE3), e706. http://dx.doi.org/10.20511/pyr2020.v8nSPE3.706 Lange, C. & Costley, J. (2020). Improving online video lectures: learning challenges created by media. International Journal of Educational Technology in Higher Education, 17(16), 1-18. https://doi.org/10.1186/s41239-020-00190-6 Liyanagunawardena, T. R., Adams, A. A., & Williams, S. A. (2013). MOOCs: A systematic study of the published literature 2008-2012. The International Review of Research in Open and Distributed Learning, 14(3), 202-227. Luque, M. (2004). Visión americana de la perspectiva social del e-learning en la educación superior. RUSC, 1(1), 1-10. Maatuk, A., Elberkawi, E.K., Aljawarneh, S., Rashaideh, H. & Alharbi, H. (2022). The COVID-19 pandemic and E-learning: challenges and opportunities from the perspective of students and instructors. Journal of Computing in Higher Education, 34, 21-38. Mayorga-Albán, A., Aveiga, C.E., Fierro, W. & Cepeda, L. (2020). E-learning models in the development of collaborative learning in higher education. Dominio de las Ciencias, Vol 6(2), 847-865. http://dx.doi.org/10.23857/dc.v6i2.1198 Milicevic, V., Denic, N., Milicevic, Z., Arsic, L., Spasic-Stojkovic, M., Petkovic, D., Stojanovic, J., Krkic, M., Milovancevic, N.S. & Jovanovic, A. (2021). E-Learning perspectives in higher education institutions. Technological forecasting and social change, 166. https://doi.org/10.1016/j.techfore.2021.120618 Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P. & Stewart, L. (2015). Preferred reporting items for systematic review and metaanalysis protocols (PRISMA-P) 2015 statement. Systematic Review, 4(1), 1-9. https://doi.org/10.1186/2046-4053-4-1 Molina-García, P., Molina-García, A. & Jentry-Jones, J. (2020). E-learning and the evolution in the teaching and learning of higher education. Dominio de las Ciencias, 6(4), 491-500. http://dx.doi.org/10.23857/dc.v6i5.1666 Morrison, D. (2003). E-learning Strategies. How to get implementation and delivery right first time. Wiley & Sons. Ngampornchai, A. & Adams, J. (2016). Students’ acceptance and readiness for E-learning in Northeastern Thailand. International Journal of Educational Technology in Higher Education, 13(1), 1-13. Ortiz López, A., Olmos Miguelañez, S., & Sánchez Prieto, J.C. (2022). E-Learning quality assessment in Higher Education: A systematic literature review. Education in the Knowledge Society, 23-37. https://doi.org/10.14201/eks.26986/e2698 Ouzzani, M., Hammady, H., Fedorowicz, Z., & Elmagarmid, A. (2016). Rayyan—a web and mobile app for systematic reviews. Systematic Review, 5(1), 210-221. Papadakis, S. (2023). MOOCs 2012.2022: An overview. Advances in Mobile Learning Educational Research, 3(1), 682.693. https://doi.org/10.25082/AMLER.2023.01.017 Parlakkiliç, A. (2017). Change Management in Transition To E-learning System. Qualitative and Quantitative Methods in Libraries, 3(3), 637-651. Qashou, A. (2022). Obstacles to effective use of e-learning in higher education from the viewpoint of faculty members. Turkish Online Journal of Distance Education, 23(1), 144-177. Quadri, N.N., Muhammed, A., Sanober, S., Qureshi, M.R.N., & Shah, A. (2017). Barriers effecting successful implementation of e-learning in Saudi Arabian Universities. International Journal of Emerging Technologies in Learning (iJET), 12(6), 94107. https://doi.org/10.3991/ijet.v12i06.7003 Rivera, L.D.F., & Tamayo, C.M. (2021). Análisis comparativo del b-learning y e-learning en competencias TIC para la docencia en educación superior. Revista Innova Educación, 3(4), 173-190. Rosenberg, M.J. (2001). E-learning strategies for delivering knowledge in the digital age. McGraw-Hill. Ruiz, J. G., Mintzer, M. J., & Leipzig, R. M. (2006). The impact of E-learning in medical education. Academic Medicine, 81(3), 207-212 Ruipérez, G. (2003). E-learning - Educación Virtual. Fundación Auna. Shahmoradi, L., Changizi, V., Mehraeen, E., Bashiri, A., Jannat, B. & Hosseini, M. (2018). The challenges of E-Learning system: Higher educational institutions perspective. Journal of Education and Health Promotion, 7 (116). 1-15. Ullah, M.S., Razaq, N., Iqbal, N., Ul Ain, Q., Hassan, U., Ahmed, A.. & Noor-Ul-Ain. (2021). Effect of E-Learning on Higher Education. Pakistan Journal of Medical & Health Sciences, 15(9), 2903-2905. http://doi.org/10.53350/pjmhs211592903 Xu, W., & Zammit, K. (2020). Applying thematic analysis to education: A hybrid approach to interpreting data in practitioner research. International Journal of Qualitative Methods, 19, 1609406920918810. https://doi.org/10.1177/1609406920918810 Younis, N.M., Ahmed, M.M. & Hussein, A.A. (2021). University Students’ Attitude Towards E-Learning. Bahrain Medical Bulletin, 43(2), 460-462. Wang, Q., Chen, L., & Liang, Y. (2011). The effects of e-learning on students' performance in tertiary education: A meta-analysis. Journal of Educational Technology & Society, 14(2), 124-136. Weller, M. (2014). The battle for open: How openness won and why it doesn't feel like victory. Ubiquity Press. Appendix A Check list. References Journals Q1 Q2 Q3 Q4 Q5 Q6 Q7 Costa et al. (2012) Procedia Technology Khan and Badii (2012) Life Science Journal Abou El-Seoud et al. (2014) International Journal of Emerging Technologies in Learning Ngampornchai & Adams (2016) International Journal of Educational Technology in Higher Education Quadri et al. (2017) International Journal of Emerging Technologies in Learning Hernández Godoy et al. (2018) Revista de Investigación Educativa Shahmoradi et al. (2018) Journal of Education and Health Promotion Hennig & Nazarkulova (2019) International Journal of Geoinformatics Kim et al. (2019) International Journal of Educational Technology in Higher Education Ayu (2020) The journal of English literacy education Gupta & Gupta (2020) Technology Lange & Costley (2020) International Journal of Educational Technology in Higher Education Kadirbayeva et al. (2021) Journal of Educators Online Rivera & Tamayo (2021) Revista Innova Educación Milicevic et al. (2021) Technological forecasting and social change Ullah et al. (2021) Pakistan Journal of Medical & Health Sciences Younis et al. (2021) Bahrain Medical Bulletin Amer et al. (2022) Journal of Education and elearning research Bouyzem et al. (2022) International Journal of Business and Technology Studies and Research Maatuk et al. (2022) Journal of Computing in Higher Education Qashou et al. (2022) Turkish Online Journal of Distance Education Note: Green colour means that it meets the requirements. Red means that it does not meet the requirements.