Analysing the Impact of Artificial Intelligence and Computational Sciences on Student Performance: Systematic Review and Meta-analysis

ORIGINAL JOURNAL OF NEW APPROACHES IN EDUCATIONAL RESEARCH 2023, VOL. 12, NO. 1, 171-197, e-ISSN: 2254-7339 https://doi.org/10.7821/naer.2023.1.1240 Received2022-09-24 Revised2022-10-14 Accepted2022-12-05 Published2023-01-15 CorrespondingAuthor InmaculadaGarcía-Martínez, [email protected] DepartamentodeDidácticay OrganizaciónEscolar. Facultad deCienciasdelaEducación. CampusdeCartujas/n18071 Granada,Spain. DOIhttps://doi.org/10.7821/ naer.2023.1.1240 Pages: 171-197 Funding:SpanishState ResearchAgency,Spain (Award:PID2019-108230RB-I00) Distributedunder CCBY-NC4.0 Copyright: ©TheAuthor(s) Analysing the Impact of Artificial Intelligence and Computational Sciences on Student Performance: Systematic Review and Meta-analysis InmaculadaGarcía-Martínez1 ,JoséMaríaFernández-Batanero2 ,JoséFernández-Cerero2 andSamuelP.León3 1Department of Didactics and School Organization, University of Granada, Spain 2Department of Didactics and School Organization, University of Seville, Spain 3Department of Pedagogy, University of Jaén, Spain ABSTRACT Artificial intelligence (AI) and computational sciences have aroused a growing interest in education. Despite its relatively recent history, AI is increasingly being introduced into the classroom through different modalities, with the aim of improving student achievement. Thus, the purpose of the research is to analyse, quantitatively and qualitatively, the impact of AI components and computational sciences on student performance. For this purpose, a systematic review and meta-analysis have been carried out in WOS and Scopus databases. After applying the inclusion and exclusion criteria, the sample was set at 25 articles. The results support the positive impact that AI and computational sciences have on student performance, finding a rise in their attitude towards learning and their motivation, especially in the STEM (Science, Technology, Engineering, and Mathematics) areas. Despite the multiple benefits provided, the implementation of these technologies in instructional processes involves a great educational and ethical challenge for teachers in relation to their design and implementation, which requires further analysis from the educational research. These findings are consistent at all educational stages. Keywords ARTIFICIAL INTELLIGENCE, LEARNING ACHIEVEMENT, EDUCATION, TEACHING METHODS, EDUCATIONAL IMPROVEMENT 1INTRODUCTION Nowadays, society is becoming increasingly oriented towards a massive process of technologicalisation in all spheres (political, economic, educational, social, etc.). This trend of adaptating to new technological interaction communities has created a variety of technologies that allow communication with the user, called “virtual assistants”, which use computer algorithms to emulate human intelligence so that users have the feeling that they are interacting with another person. This concept is known as “artificial intelligence” (AI) (Ocaña, How to cite this article (APA): García-Martínez, I., Fernández-Batanero, J. M., Fernández-Cerero, J., & León, S. P. (2023). Analysing the Impact of Artificial Intelligence and Computational Sciences on Student Performance: Systematic Review and Meta-analysis.JournalofNewApproaches in Educational Research, 12(1),171-197. doi: 10.7821/naer.2023.1.1240 171 García-Martínez,Inmaculada;etal. Analysingthe ImpactofArtificialIntelligenceand ComputationalSciences Valenzuela, & Garro, 2019;Yang, Zhuang, & Pan, 2021). In educational environments, AI has taken special interest, given the high possibilities of communication that are established between teachers and students when using virtual information assistants, since from its execution there is a simulation of responses that approach a human conversation and, as the tool is used, the interaction with the user is learned and recognised intuitively. Nevertheless, in the current global context of the technological revolution, there are human qualities that cannot yet be reproduced by AI, such as creativity or the ability to produce new ideas or to improvise and constantly evolve. In this regard, current trends in AI and computational science are moving towards human-centred AI, that considers people’s characteristics and contexts to reduce the biases that may be associated with the management and processing of the algorithms that support it (Yang et al., 2021). According to UNESCO (2019), the link between AI and education consists of three areas: learning with AI (using AI tools in the classroom), learning about AI (its technologies and techniques) and preparing for AI (enabling all citizens to understand the potential impact of AI on human life). It also believes that AI has the potential to address some of the biggest challenges facing education today, namely, to develop innovative teaching and learning practices guided by the fundamental principles of inclusion and equity while helping to accelerate progress towards SDG 4. Thus, AI has a strong potential to accelerate the process of realisation and development of the global goals around education by reducing barriers to access to learning, by automating management processes, and by optimising methods to improve student performance and, as a result, learning outcomes (Moreno, 2019). In short, AI as a technological tool applied to education may contribute innovative methods and ways with the use of ICT that improve the teaching and learning process from the perspective of the student and the teacher. 1.1 Artificial Intelligence in the Educational Context The educational field is constantly changing and adapting to new generations and their educational needs (Halili, 2019). If we consider that all these developments go in parallel with technological advances, we can say that the speed at which education is being updated may be the fastest ever (Harrison, 1986). Although these advances are studied from their many forms, they all have a common factor: AI. To address the concept of AI, as discussed above, it must be understood to include any resource or machine that carries out human work. Humans create these machines to mechanise the tasks they perform every day, with the purpose of accomplishing more in less time. Popenici and Kerr (2017) define it as computing systems that can engage in similar processes to the human ones, such as learning, adapting, synthesising, self-correcting and using data for complex processing tasks. More specifically in education, educational artificial intelligence (EAI) refers to the use of AI to support personalised and automated feedback and guidance in the educational field Song and Wang (2020). The growing demand in education in recent years has given rise to a thriving new field of research which integrates AI and education, which has resulted in an expansion JournalofNewApproachesin EducationalResearch,12(1) |2023|https://doi.org/10.7821/naer.2023.1.1240 172 García-Martínez,Inmaculada;etal. Analysingthe ImpactofArtificialIntelligenceand ComputationalSciences of the existing literature on EAI. Furthermore, and updating this terminology, EAI is related to different fields, such as robotics (Jawaid et al., 2020), applications for smart devices (Petko, Schmid, Müller, & Hielscher, 2019), electronic devices (Pyörälä et al., 2019), e-learning (Reister & Blanchard, 2020;Singer-Brodowski, Brock, Etzkorn, & Otte, 2019), or virtual (VR) and augmented reality (AR) (Bower, Dewitt, & Lai, 2020;Kavanagh, Luxton-Reilly, Wuensche, & Plimmer, 2017), intelligent conversational software agents (chatbot) (Schachner, Keller, & Wangenheim, 2020), virtual assistants (Jee, 2019), and online platforms for self-learning (Moreno, 2019). All these areas have a common goal: to learn, to teach and to solve problems (Baker, Smith, & Anissa, 2019). Several studies have highlighted the different current trends in EAI (Roll & Wylie, 2016); thus, we can highlight the role of AI in special education (Guilherme, 2017), such as the transformative collaboration between teachers and students (Drigas & Ioannidou, 2013), the global advance and trends of various intelligent tutoring systems (Han, Zhao, Jiang, Oubibi, & Hu, 2019), the potential of EAI in higher education (Crompton, Bernacki, & Greene, 2020), etc. Hence, the importance to contribute to the improvement of academic performance by optimising instructional processes and reducing students’ learning difficulties. In our study, we understand academic performance as a measure of the student’s abilities, which expresses what the student has learned throughout the learning process. It also implies the student’s ability to respond to educational stimuli (Martínez, Karanik, Giovannini, & Pinto, 2015). Academic performance is affected by a multiplicity of heterogeneous factors (internal and external) that condition student performance, including aptitude and motivation (Castrillón, Sarache, & Herrera, 2020). Research in the field of EAI and academic performance is a key area in education, since through its study the effectiveness of all these intelligent resources in the teaching and learning processes can be evaluated. Considering all the above, this study aims to conduct an empirical analysis of the evidence found within the EAI literature. Despite previous systematic reviews on the inclusion of AI in education (Hooshyar, Yousefi, & Lim, 2019;Roll & Wylie, 2016;Song & Wang, 2020; Zawacki-Richter, Marín, Bond, & Gouverneur, 2019), the effect size on performance has not yet been calculated, quantified or meta-analysed. Thus, the main objective of the research is to analyse the impact of AI components and computational sciences on student performance. Focusing on EAI, this study attempts to address the following research questions: • Does EAI improve student performance? • What effects does EAI have on students? • What type of AI and computational science is the most common in the educational field? • Is EAI effective at all educational stages? JournalofNewApproachesin EducationalResearch,12(1) |2023|https://doi.org/10.7821/naer.2023.1.1240 173 García-Martínez,Inmaculada;etal. Analysingthe ImpactofArtificialIntelligenceand ComputationalSciences 2METHODS 2.1 Search Procedures This study follows the guidelines of the PRISMA Statement (Page et al., 2021). The search was conducted on Web of Science (WOS) and Scopus databases, due to their prestige and scope in education research. For this purpose, the following search equation was introduced: SUBJECT: (AI-based education) OR SUBJECT: (artificial intelligence in education) AND SUBJECT: (e-learning) OR SUBJECT: (automated tutor) OR SUBJECT: (intelligence agent) OR SUBJECT: (simulation) OR SUBJECT: (artificial agent) AND SUBJECT: (student) AND SUBJECT: (experimental design) OR SUBJECT: (quasi-experimental design). The following inclusion criteria were adopted: (a) only articles, excluding the so-called grey literature; (b) whose language was English, as it is the internationally recognised language in the scientific field; (c) published between 2010 and 2020. In the search process, the volatility of educational technology was assumed, so it was decided to focus on empirical research published in the last 10 years; (d) belonged to the area of “Education Educational research or Education Scientific disciplines” in the case of WOS, and the research areas “social sciences, art and humanities, psychology and neuroscience” in Scopus. From the application of the initial inclusion criteria, the search was set at 480 manuscripts: 455 WOS and 25 in Scopus. There were 87 duplicates, so the initial sample was set at 393. A first screening of the papers, resulting from the initial search, was conducted by examining the titles and abstracts. This phase was carried out by two of the authors. Unpublished dissertations, reviews and meta-analyses were excluded at this stage. This first screening resulted in a total of 41 articles that were fully read independently by the authors to verify whether the inclusion criteria were met. Of the 41 articles, only 18 fully achieved the inclusion criteria. A top-down search of articles citing or cited by those two articles was then performed to identify possible additional studies. This allowed the inclusion of five new articles. Finally, we looked for previous systematic reviews on AI, or some of its components, and analysed all the papers cited in those reviews for possible inclusion, considering two more studies in this last search. Thus, the final sample of articles reviewed for inclusion in this study was 25 articles. Figure 1 summarises the search process in a PRISMA flowchart. 2.2 Selection Criteria The studies analysed must meet the following criteria to be included in this review: (a) the objective of the study was to measure quantitatively the impact of EAI, or one of its components, on the academic development of students; (b) the studies had to follow a prepost design with control groups; (c) the dependent variable had to be related to academic performance; (d) the sample had to be made up of students. 2.3 Data Extraction and Coding The review process of the manuscripts to reach the sample was carried out independently by two of the authors who signed the article. For this purpose, the data from the articles JournalofNewApproachesin EducationalResearch,12(1) |2023|https://doi.org/10.7821/naer.2023.1.1240 174 García-Martínez,Inmaculada;etal. Analysingthe ImpactofArtificialIntelligenceand ComputationalSciences Figure 1 Sample selection flowchart were analysed in an Excel file, according to the parameters provided by the inclusion and exclusion criteria, where a cell was added in order to determine whether or not they were included in the review and the reason for them. After completing each phase of the review (reading the title and abstract and the full text), the results obtained were contrasted. The methodological quality of the selected articles was also assessed using Joanna Briggs’ checklist (JBI), where it was examined by means of critical and independent review (elevenpoint checklist) (Aromataris & Munn, 2020). For the internal assessment of the quality of the proposed study, the checklist was evaluated blindly by two researchers external to the research, with the aim of avoiding assessment bias by the authors themselves. For the first complete reading made by the authors in the first search, the degree of agreement of inclusion of the papers was 96%. Disagreements were resolved by discussion and consensus between the two researchers until there was a 100% agreement. In the second search, the degree of agreement was 100%. The studies included were finally analysed in depth by the authors. All the relevant information of each article has been coded for the analysis and discussion procedure using a database, whose information has been interpolated into the figures and tables. Table 1 shows a summary of the detailed analysis of the 25 articles included in the systematic review. JournalofNewApproachesin EducationalResearch,12(1) |2023|https://doi.org/10.7821/naer.2023.1.1240 175 García-Martínez,Inmaculada;etal. Analysingthe ImpactofArtificialIntelligenceand ComputationalSciences Table 1 Articles included Autors and Year Journal Country N Stage Type of interventor Intervention duration DV Tests used to measure DV Results Type of IA J. L. Anderson and Barnett (2013) Journal of Science Education and Technology USA 91 CG = 32 EG = 59 Supercharged! 7 class periods SA CT EG has better SA. Ap Barbalios, Ioannidou, Tzionas, and Paraskeuopoulos (2013) Computer & Education Greece 24 3D virtual reality modelling language - SA CT EG has better SA. VR Ibáñez, Serio, Villarán, and Kloos (2014) Computer & Education Spain 64 AR-based methodology vs. Web-based methodology 5 sessions SA; P CT The AR-based application was more effective than the web-based application. AR Bortnik, Stozhko, Pervukhina, Tchernysheva, and Belysheva (1968) Research in Learning Technology Russia 50 CG = 25 EG = 25 U Virtual chemistry lab combined with classroom vs. Traditional teaching - SA R of EG and CG; CT; SP EG has better SA. Sim Chin et al. (2010) Educational Technology Research and Development USA Include 2 studies: 1) EG = 28 CG = 30 2) n = 104 Teachable agents application 1) 3 weeks 2) 15 weeks SA CT EG has better SA. Ap Civelek, Ucar, Ustunel, and Aydın (2014) Eurasia Journal of Mathematics Science and Technology Education Turkey EG = 106 CG =109 Immersive virtual reality environment (VRE) 2 weeks SA; A CT EG has better SA. VR Yelamarthi and Drake (2014) IEEE Transactions on Education USA EG = 17 CG = 24 U Flipped course Twice a week for 75 min over the 16 weeks SA CT Significant improvements in EG students’ performance and their perceptions of their learning experience. Sim Continued on next page JournalofNewApproachesin EducationalResearch,12(1) |2023|https://doi.org/10.7821/naer.2023.1.1240 176 García-Martínez,Inmaculada;etal. Analysingthe ImpactofArtificialIntelligenceand ComputationalSciences Table 1 continued Autors and Year Journal Country N Stage Type of interventor Intervention duration DV Tests used to measure DV Results Type of IA Olde, De Jong, and Gijlers (2013) Educational Technology & Society NetherlandEG = 21 CG = 28 TVS Lookexperiment- design (LED) by computer simulation 3 two-hour sessions SA CT EG has better SA. Sim Dickerson and Clark (2018) Computer Applications in Engineering Education USA CG = 31 EG = 22 U SPICE simulation 12 weeks SA I, CT EG has better and deeper SA. Sim Fang and Guo (2016) Journal of Computer Assisted Learning 142 G = 65 EG = 77 U Computer simulation and animation (CSA) 12 months SA CT Better results with CSA, although they point out that the teacher’s role cannot be replaced. Ap Fidan and Tuncel (2019) Computer & Education Turkey EG1 = 30 EG2 = 31 CG = 30 AR based PBL with FenAR software 11 weeks SA; A CT; AS The experimental results indicated that the integration of AR into the activities of PBL increased SA and AS. AR Jiménez, Bravo, and Bacca (2010) Computer Applications in Engineering Education Mexico EG= 31 CG= 31 U Web-based gamified software 20 days SA CT; MQ EG scored better on the content test and displayed higher motivation than CG. BL Continued on next page JournalofNewApproachesin EducationalResearch,12(1) |2023|https://doi.org/10.7821/naer.2023.1.1240 177 García-Martínez,Inmaculada;etal. Analysingthe ImpactofArtificialIntelligenceand ComputationalSciences Table 1 continued Autors and Year Journal Country N Stage Type of interventor Intervention duration DV Tests used to measure DV Results Type of IA Madathil et al. (2017) Computers in Education Journal USA 165 U Instructional model with VR integrated 1 hour SA, Sa, U; PE CT Although no significant learning differences were found between the conditions, students’ perceptions of their learning showed significant improvements in the VR and control groups over the photo-based group. VR improves the learning experience. VR Neri, Noguez, Robledo-Rella, Escobar-Castillejos, and Gonzalez- Nucamendi (2018) Journal of Educational Technology & Society Mexico EG = 47 CG = 38 U Visuo-haptic simulators 5 months SA PQ, R, CT Incorporating haptic technology in learning simulators to teach physics enhance the understanding of physics concepts motivation and SA. Ap Pellas and Vosinakis (2018) Education and Information Technologies Greece 50 Scratch and OpenSim with Scratch4SL 4-weeks period with 6 sessions SA CT EG performed better on problem-solving measures and algorithmic thinking. Sim Stieff (2011) Journal of Research in Science Teaching USA 460 Connected Chemistry: Discovering Matter! 180 minutes with different distributions in each case: 4x45; 2x90... RC; SA CT Connected Chemistry class yield only small to modest improvements in SA but higher RC. Sim Continued on next page JournalofNewApproachesin EducationalResearch,12(1) |2023|https://doi.org/10.7821/naer.2023.1.1240 178 García-Martínez,Inmaculada;etal. Analysingthe ImpactofArtificialIntelligenceand ComputationalSciences Table 1 continued Autors and Year Journal Country N Stage Type of interventor Intervention duration DV Tests used to measure DV Results Type of IA Zacharia and Olympiou (2011) Learning and Instruction Cyprus 234 U Traditional vs. Physical manipulative experimentation vs. Virtual manipulative experimentation (VME) 2 semesters Duration: 30.5 hours SA CT Better results in the experimental groups. VR J. Anderson and Barnett (2011) Journal of Science Education and Technology USA CG = 65 EG = 71 U Labs with and without Supercharged! 2 labs of 2 h each SA CT; LN Video games can lead to positive learning outcomes and support student scientific understanding. Ap Bozkurt and Ilik (2010) Procedia Social and Behavioral Sciences Turkey 152 U Computer simulations (E) vs. Traditional (C) 1 semester SA; B CT Groups who study with computer simulations are more successful than those who study with traditional methods. Sim Pareto (2014) International Artificial Intelligence in Education Society Sweden 443 P Teachable agents application 3 months SA CT Teachable agents in educational games can help achieve deeper levels of learning and motivational power. Ap Continued on next page JournalofNewApproachesin EducationalResearch,12(1) |2023|https://doi.org/10.7821/naer.2023.1.1240 179 García-Martínez,Inmaculada;etal. Analysingthe ImpactofArtificialIntelligenceand ComputationalSciences Figure 2 Forest plot for g∆(panel A) and gp(panel B) JournalofNewApproachesin EducationalResearch,12(1) |2023|https://doi.org/10.7821/naer.2023.1.1240 186 García-Martínez,Inmaculada;etal. Analysingthe ImpactofArtificialIntelligenceand ComputationalSciences Figure 3 Funnel plots for g∆(panel A) and gp(panel B) 4DISCUSSION AND CONCLUSIONS The systematic review carried out aimed at analysing the impact of different AI components and computational sciences on student performance. After applying the inclusion and exclusion criteria mentioned above, the search resulted in 25 quasi-experimental studies with experimental and control groups, and/or hybrids, incorporating, in addition, qualitative strategies, which examined the positive effects of applying AI and computational sciences with respect to traditional methods. JournalofNewApproachesin EducationalResearch,12(1) |2023|https://doi.org/10.7821/naer.2023.1.1240 187 García-Martínez,Inmaculada;etal. Analysingthe ImpactofArtificialIntelligenceand ComputationalSciences Specifically, it was intended to provide answers to four research questions, which will be answered below: 4.1 Does EAI Improve Student Performance? All the studies found have shown the usefulness of employing EAI-based methods over more traditional methods. Among the pedagogical intentions and considerations that justify the use of EAI-based methods, the importance of placing students into the heart of their learning, providing them with opportunities to take an active role in that construction, stands out. The pedagogical possibilities offered by EAI are oriented to achieve significant learning in students, encouraging the visual component that has its different modalities, such as simulations, VR, AR, or applications such as games. One of the great benefits of EAI is the flexibility to adapt educational programs to the rhythms and circumstances of each student (Deng, Benckendorff, & Gannaway, 2019; Vilkova & Shcheglova, 2020). Considering the advances in society and technology, teaching and guidance by teachers may often be done in a non-face-to-face modality. These modalities are the basis of distance education. This type of education arises due to the incompatibility between individuals’ studies and their work, or to situations of major importance, such as the global pandemic we are experiencing. Moreover, one of the main advantages of using technology in distance education is video tutorials (Wilkie & Liefeith, 2020). Students obtain personalised feedback on their teaching-learning process through them. On the other hand, teachers can record their classes or brief extracts of any content and students can watch it as often as necessary for their proper learning (Elliot, Gehret, Valadez, Carpenter, & Bryant, 2020;López-Rodríguez & Barac, 2019). 4.2 What Effects Does EAI Have on Students? Based on our research findings, we have identified that the different EAI modalities not only affect the quantity of what students learn, but also lead to higher levels of motivation, which is demonstrated by a greater willingness to be involved in their learning. At the same time, it has been shown that most studies on EAI have been contextualised in STEM knowledge areas, which require higher levels of abstraction and greater complexity to achieve a proper understanding of knowledge. Depending on the level of education, different resources can be used so that the students are encouraged to manipulate them. In this regard, EAI not only helps to keep students focused while they are building something, but also encourages their creative ability to shape their thoughts (Barak & Zadok, 2009). Likewise, several studies use and have demonstrated the effectiveness of the AI tools in education (Fabregas et al., 2016;Jiménez et al., 2010). In addition, many educational institutions around the world are employing the STEM teaching methodology. This methodology is characterised using a series of new and up-to-date tools for the teaching of different school subjects. Moreover, it allows the design and development of a computational model based on learning and teaching conditions controlled on any subject with a high visual and multimedia content, which facilitates the acquisition and understanding of the contents through the ongoing interaction with the computer (Vlachopoulos & Makri, 2017). JournalofNewApproachesin EducationalResearch,12(1) |2023|https://doi.org/10.7821/naer.2023.1.1240 188 García-Martínez,Inmaculada;etal. Analysingthe ImpactofArtificialIntelligenceand ComputationalSciences In some of the studies included, besides checking the effectiveness of EAI on the quantity and quality of student learning, it has been shown how these modalities have led to changes in students’ attitudes towards these knowledge areas. In this regard, it may be determined that incorporating technologies, resources and EAI strategies into teaching methodologies is a significant advance on the road to achieving a meaningful and integrated student learning. 4.3 What Type of AI and Computational Science is the Most Common in the Educational Field? Regarding the type of AI modality used, it was found that the different studies included address the potential of EAI on student performance through applications (n = 8), simulations (n = 8), VR (n = 4) and AR (n = 2). In relation to the applications, the literature suggests that their use in education is increasingly widespread at all levels (Wirjawan et al., 2020). Given their high impact on students, subject-based learning is contextualised and updated to the digital society. Applications are contextualised to the subject matter and educational level of the student. There are applications that allow the user to interact with their environment, use question and answers, find some element, orientate, watch instructional videos, create a portfolio, learn mathematics, play video games, and even intelligent tutoring. Likewise, research can be found that focuses on the use of some application carried out at any educational stage (Dunleavy et al., 2019;Hoplock, Lobchuk, & Lemoine, 2020;Petko et al., 2019). It is also important to note that apps tend to be used more frequently by teachers, due to their greater accessibility compared to other AI modalities, such as Big Data, which requires a higher level of literacy and involve other variables such as ethical issues (Gao, Li, & Liu, 2021). We can also find how there are studies in which different applications are used with people with autism (Law, Dutt, & Neihart, 2019) or in people with attention deficit hyperactivity disorder (Butt et al., 2020;Păsărelu, Andersson, & Dobrean, 2020). This implies that this type of AI is easier to implement in the design, adjustment and development of instructional processes for learners with special educational needs (Zhai et al., 2010). Regarding the simulations, they provide a variety of scenarios in which students can learn by discovering applying what they have learned to progress at different levels, playing games or solving day-to-day problems (Masson & Rennie, 2006). Their easy access, potential for individualisation and low cost are some of the benefits that encourage its widespread use in education (Cabero-Almenara & Costas, 2016). Among the findings obtained in these studies, there are common results and objectives: many of them sought to understand complex concepts in subjects classified as “difficult” and to strengthen and improve the students’ attitude towards the subject and, in some cases, to establish collaborative strategies among students. Likewise, the research that used the simulations as an AI, AR and VR modality was oriented to offer more real learning situations for the students, demonstrating an effectiveness similar to that of real laboratories. These experiences allow students to interact with an outside environment within the classroom (Lau & Lee, 2015). Furthermore, the use of VR in education enables students to interpret signs, JournalofNewApproachesin EducationalResearch,12(1) |2023|https://doi.org/10.7821/naer.2023.1.1240 189 García-Martínez,Inmaculada;etal. Analysingthe ImpactofArtificialIntelligenceand ComputationalSciences whether visual, auditory, or haptic, and to build their knowledge through their movement and their interaction with their own environment (Beck, 2019;Vesisenaho et al., 2019). In order to implement this technology into practice at school, teachers need to experiment with it, learn how to use it and then contextualise the content to the students’ own environment Tondeur, Roblin, Van Braak, Voogt, and Prestridge (2017). In higher education, university studies in medicine or nursing include the use of VR to perform operations or treatments (Baxter & Hainey, 2019;Bernardo, 2017). 4.4 Is EAI Effective at All Educational Stages? The systematic review and meta-analysis have included studies from different educational stages. Thus, there are three studies in primary education, eleven contextualised studies in secondary education, one in vocational studies and ten studies in university. In all of them, the effectiveness of AI has been shown. However, the superiority of research in secondary education and university seems to point to a tendency to develop more studies in advanced educational stages, due, among other reasons, to the potential intellectual growth of students at these stages. Similarly, small differences in terms of the type of AI and computational science have been observed in the different educational stages. According to the results obtained, there is a more restricted trend towards the use of AI and computer science, with applications and simulations becoming the most widely used in primary education. On the other hand, in secondary education and university, there is a tendency to use more types of AI and computational science, also due to the wide variety of subjects that can be studied and the tendency to encourage more computational thinking in students than in previous educational stages. However, this differs from Sun, Guo, and Hu (2021), who state that the use of gamification strategies to enhance computational thinking tends to be more effective at lower educational stages, making gamification a factor to be considered in this analysis. 4.5 Strengths and Limitations of the Study In this regard, there is a change of trend in relation to the development of studies that use AI and promote computational thinking in students due to the pressure from international organisations to include computational thinking in the curricula at all educational stages. Thus, an increasing number of research studies is being found in the literature that uses AI and promotes computational thinking at different educational stages (McCormick & Hall, 2022;Merino-Armero, González-Calero, & Cozar-Gutierrez, 2022). This paper has a set of limitations that need to be emphasized. Firstly, the limitation related to the lack of studies that empirically demonstrate the relationship between AI and student achievement was considered. Another issue to take into account is determined by the databases themselves. Problems are often perceived in the registration of manuscripts, depending on the type of document or language, which may imply any possible bias in the elaboration of the systematic review. Related to this, the choice to use exclusively the WOS and Scopus databases and to ignore other publication sources, such as ERIC, PSYINFO, or the grey literature, may lead to bias. However, it was considered appropriate to focus on them because of its prestige and JournalofNewApproachesin EducationalResearch,12(1) |2023|https://doi.org/10.7821/naer.2023.1.1240 190 García-Martínez,Inmaculada;etal. Analysingthe ImpactofArtificialIntelligenceand ComputationalSciences since they are the databases that include the most manuscripts in the research area. In addition, this paper also has several strengths that should be noted. For example, this review has quantitatively assessed the effect of different EAI-based studies on student performance. In addition, studies from different countries have been collected, which provides an overview of this topic. 4.6 Contribution of This Paper Experiences of AI in education can be found in the literature. Most of the works highlight the potential of AI to facilitate the understanding of complex and abstract knowledge. However, the negative points emphasize that AI requires specific software and the design of materials and technological resources. This particular paper provides scientific evidence of the real benefits of incorporating AI into educational processes, based on the meta-analysis conducted. Specifically, the size of the effect of the different AI modalities and computational sciences on the academic performance of students has been measured, and the benefits have been found to be real. Despite the high economic cost, this work gathers experiences of AI in different educational stages, which opens the way towards the implementation of teaching methodologies based on AI in both early childhood and higher education, with a view to decreasing learning difficulties in students and increasing their motivation towards learning. Similarly, it is important to note the impact on teachers themselves. While it is true that the use of AI and computer science in the field of education can provide significant support for teachers, who have seen their roles increase over the last few years. However, it also brings with it several challenges that need to be clarified. Firstly, there is digital literacy, where they must know how these modalities work, their viability and usefulness in their subject, as well as how to extrapolate it to the reality of their classroom in order to build meaningful learning situations for their students. Secondly, it must consider the sustainable objectives proposed by UNESCO (2021) both in the design of learning situations and in their implementation, in order to ensure that ethical assumptions are met and that human potential is enhanced rather than hindered (Flores-Vivar & García-Peñalvo, 2023;UNESCO, 2019). Accordingly, challenges that teachers must take on Table 3 Challenges for teachers for the inclusion of EAI. Adaptation of Flores-Vivar & García-Peñalvo (2023; pp. 4-5). Dimensions and proposals Challenges Educational stage Accessibility Design and use of new educational resources. All educational stages Intelligent tutoring Monitoring of intelligent system-generated responses to ensure that the ethical guidelines of SDG 4 are met. University Virtual facilitators Design of new pedagogical models that include AI and computer science. All educational stages Intelligent content Ethical and legal issues related to intellectual property. Secondary Education and University Teacher and AI collaboration Digital literacy plans. All educational stages Content management and analytics Ethical issues associated with data processing. University Out-of-class tutoring Human-machine interaction paradigm. Secondary Education and University Learning management automation Resource optimization. All educational stages JournalofNewApproachesin EducationalResearch,12(1) |2023|https://doi.org/10.7821/naer.2023.1.1240 191 García-Martínez,Inmaculada;etal. Analysingthe ImpactofArtificialIntelligenceand ComputationalSciences in the inclusion of AI in the field of education have been identified (see Table 3). 4.7 Recommendations for Further Research Based on the findings and limitations identified, the study should be extended to other databases, such as ERIC or PsycINFO, or the grey literature, which provide a worldwide view of current research on AI and students performance. During the search for studies carried out, it has been observed that the problem has grown exponentially over time. 5AUTHORS' CONTRIBUTION Conceptualization: Inmaculada García-Martínez and José María Fernández-Batanero; Data curation: Samuel P. León; Formal analysis: Samuel P. León; Funding acquisition: José María Fernández-Batanero; Investigation: Inmaculada García-Martínez and José Fernández-Cerero; Methodology: Inmaculada García-Martínez and Samuel P. León; Software: Samuel P. León; Writing – original draft: Inmaculada García-Martínez, José Fernández-Cerero and Samuel P. León; Writing – review & editing: Inmaculada García- Martínez and José María Fernández-Batanero; Supervision: All authors have reviewed and approved this manuscript. ACKNOWLEDGEMENTS Funded by: Spanish State Research Agency, Spain Funder Identifier: http://dx.doi.org/10.13039/501100011033 Award: PID2019-108230RB-I00 REFERENCES Anderson, J., & Barnett, M. (2011). Using video games to support pre-service elementary teachers learning of basic physics principles. Journal of Science Education and Technology,20(4), 347– 362. https://doi.org/10.1007/s10956-010-9257-0 Anderson, J. L., & Barnett, M. (2013). Learning physics with digital game simulations in middle school science. Journal of Science Education and Technology,22(6), 914–926. https://doi.org/ 10.1007/s10956-013-9438-8 Aromataris, E., & Munn, Z. (2020). Chapter 1: JBI Systematic Reviews. JBI manual for evidence synthesis. Joanna Briggs Institute: Joanna Briggs Institute. Retrieved from https://doi.org/ 10.46658/JBIMES-20-02 https://doi.org/10.46658/JBIMES-20-02 Baker, T., Smith, L., & Anissa, N. (2019). Educ-AI-tion rebooted? Exploring the future of artificial intelligence in schools and colleges. Retrieved from https://media.nesta.org.uk/documents/ Future_of_AI_and_education_v5_WEB.pdf Barak, M., & Zadok, Y. (2009). Robotics projects and learning concepts in science, technology and problem solving. International Journal of Technology and Design Education,19(3), 289–307. https://doi.org/10.1007/s10798-007-9043-3 Barbalios, N., Ioannidou, I., Tzionas, P., & Paraskeuopoulos, S. (2013). A model supported interactive virtual environment for natural resource sharing in environmental education. Computers JournalofNewApproachesin EducationalResearch,12(1) |2023|https://doi.org/10.7821/naer.2023.1.1240 192 García-Martínez,Inmaculada;etal. Analysingthe ImpactofArtificialIntelligenceand ComputationalSciences & Education,62, 231–248. https://doi.org/10.1016/j.compedu.2012.10.029 Baxter, G., & Hainey, T. (2019). Student perceptions of virtual reality use in higher education. Journal of Applied Research in Higher Education,12(3), 413–424. https://doi.org/10.1108/jarhe -06-2018-0106 Beck, D. (2019). Augmented and Virtual Reality in Education: Immersive Learning Research. Journal of Educational Computing Research,57(7), 1619–1625. https://doi.org/10.1177/ 0735633119854035 Bernardo, A. (2017). Virtual reality and simulation in neurosurgical training. World Neurosurgery, 106, 1015–1029. https://doi.org/10.1016/j.wneu.2017.06.140 Borenstein, M., Hedges, L. V., Higgins, J., & Rothstein, H. R. (2009). When does it make sense to perform a meta-analysis. Introduction to Meta-Analysis, 357–364. https://doi.org/10.1002/ 9780470743386.ch2 Bortnik, B., Stozhko, N., Pervukhina, I., Tchernysheva, A., & Belysheva, G. (1968). Effect of virtual analytical chemistry laboratory on enhancing student research skills and practices. Research in Learning Technology,25, 1968. http://doi.org/10.25304/rlt.v25.1968 Bower, M., Dewitt, D., & Lai, J. W. (2020). Reasons associated with preservice teachers’ intention to use immersive virtual reality in education. British Journal of Educational Technology, 1–19. https://doi.org/10.1111/bjet.13009 Bozkurt, E., & Ilik, A. (2010). The effect of computer simulations over students’ beliefs on physics and physics success. Procedia-Social and Behavioral Sciences,2(2), 4587–4591. https://doi.org/ 10.1016/j.sbspro.2010.03.735 Butt, S., Hannan, F. E., Rafiq, M., Hussain, I., Faisal, C. N., & Younas, W. (2020). Say-It & Learn: Interactive Application for Children with ADHD. International Conference on Human- Computer Interaction (pp. 213–223). Springer. https://doi.org/10.1007/978-3-030-49913-6 _18 Cabero-Almenara, J., & Costas, J. (2016). Simulators use for students training. Prisma Social,7, 343–372. Castrillón, O., Sarache, & Herrera, R. (2020). Predicción del rendimiento académico por medio de técnicas de inteligencia artificial. Formación Universitaria,13(1), 93–102. http://doi.org/ 10.4067/S0718-50062020000100093 Chin, D. B., Dohmen, I. M., Cheng, B. H., Oppezzo, M. A., Chase, C. C., & Schwartz, D. L. (2010). Preparing students for future learning with teachable agents. Educational Technology Research and Development,58(6), 649–669. https://doi.org/10.1007/s11423-010-9154-5 Civelek, T., Ucar, E., Ustunel, H., & Aydın, M. K. (2014). Effects of a haptic augmented simulation on K-12 students’ achievement and their attitudes towards physics. Science and Technology Education,10(6), 565–574. https://doi.org/10.12973/eurasia.2014.1122a Crompton, H., Bernacki, M., & Greene, J. A. (2020). Psychological foundations of emerging technologies for teaching and learning in higher education. Current Opinion in Psychology,36, 101–105. https://doi.org/10.1016/j.copsyc.2020.04.011 Deng, R., Benckendorff, P., & Gannaway, D. (2019). Progress and new directions for teaching and learning in MOOCs. Computers & Education,129, 48–60. https://doi.org/10.1016/j.compedu .2018.10.019 Dickerson, S. J., & Clark, R. M. (2018). A classroom-based simulation-centric approach to microelectronics education. Computer Applications in Engineering Education,26(4), 768–781. https://doi.org/10.1002/cae.21918 Drigas, A. S., & Ioannidou, R. E. (2013). A review on artificial intelligence in special education. Communications in Computer and Information Science, 385–391. https://doi.org/10.1007/978 -3-642-35879-1_46 JournalofNewApproachesin EducationalResearch,12(1) |2023|https://doi.org/10.7821/naer.2023.1.1240 193 García-Martínez,Inmaculada;etal. Analysingthe ImpactofArtificialIntelligenceand ComputationalSciences Dunleavy, G., Nikolaou, C. K., Nifakos, S., Atun, R., Law, G. C. Y., & Car, L. T. (2019). Mobile digital education for health professions: systematic review and meta-analysis by the digital health education collaboration. Journal of Medical Internet Research,21(2). https://doi.org/10.2196/ 12937 Elliot, L., Gehret, A., Valadez, M. S., Carpenter, R., & Bryant, L. (2020). Supporting Autonomous Learning Skills in Developmental Mathematics Courses with Asynchronous Online Resources. American Behavioral Scientist,64(7), 1012–1030. https://doi.org/10.1177/ 0002764220919149 Fabregas, E., Farias, G., Dormido-Canto, S., Guinaldo, M., Sánchez, J., & Bencomo, S. D. (2016). Platform for teaching mobile robotics. Journal of Intelligent & Robotic Systems,81(1), 131– 143. https://doi.org/10.1007/s10846-015-0229-8 Fang, N., & Guo, Y. (2016). Interactive computer simulation and animation for improving student learning of particle kinetics. Journal of Computer Assisted Learning,32(5), 443–455. https:// doi.org/10.1111/jcal.12145 Fidan, M., & Tuncel, M. (2019). Integrating augmented reality into problem based learning: The effects on learning achievement and attitude in physics education. Computers & Education, 142, 103635. https://doi.org/10.1016/j.compedu.2019.103635 Flores-Vivar, J. M., & García-Peñalvo, F. J. (2023). Reflexiones sobre la ética, potencialidades y retos de la Inteligencia Artificial en el marco de la Educación de Calidad (ODS4). Comunicar, 31(74). https://doi.org/10.3916/C74-2023-03 Gao, P., Li, J., & Liu, S. (2021). An Introduction to Key Technology in Artificial Intelligence and big Data Driven e-Learning and e-Education. Mobile Networks and Applications,26(5), 2123– 2126. https://doi.org/10.1007/s11036-021-01777-7 Guilherme, A. (2017). AI and education: the importance of teacher and student relations. AI & Society,34(1), 47–54. https://doi.org/10.1007/s00146-017-0693-8 Halili, S. H. (2019). Technological advancements in education 4.0. The Online Journal of Distance Education and E-Learning,7, 63–69. Han, J., Zhao, W., Jiang, Q., Oubibi, M., & Hu, X. (2019). Intelligent Tutoring System Trends 2006- 2018: A Literature Review. 2019 Eighth International Conference on Educational Innovation through Technology (EITT) (pp. 153–159). IEEE. https://doi.org/10.1109/eitt.2019.00037 Harrison, N. (1986). Patterns of participation in higher education for care-experienced students in England: why has there not been more progress? Studies in Higher Education,45(9), 1986– 2000. https://doi.org/10.1080/03075079.2019.1582014 Hooshyar, D., Yousefi, M., & Lim, H. (2019). A systematic review of data-driven approaches in player modeling of educational games. Artificial Intelligence Review,52(3), 1997–2017. https:// doi.org/10.1007/s10462-017-9609-8 Hoplock, L. B., Lobchuk, M. M., & Lemoine, J. (2020). Perceptions of an evidence-based empathy mobile app in post-secondary education. Education and Information Technologies,26, 1273– 1292. https://doi.org/10.1007/s10639-020-10311-3 Ibáñez, M. B., Serio, A. D., Villarán, D., & Kloos, C. D. (2014). Experimenting with electromagnetism using augmented reality: Impact on flow student experience and educational effectiveness. Computers & Education,71, 1–13. https://doi.org/10.1016/j.compedu.2013.09.004 Jawaid, I., Javed, M. Y., Jaffery, M. H., Akram, A., Safder, U., & Hassan, S. (2020). Robotic system education for young children by collaborative-project-based learning. Computer Applications in Engineering Education,28(1), 178–192. https://doi.org/10.1002/cae.22184 Jee, C. (2019). Best chatbot building platforms. Techworld. Retrieved from https://bit.ly/2Ate94F Jiménez, E., Bravo, E., & Bacca, E. (2010). Tool for experimenting with concepts of mobile robotics as applied to children´s education. IEEE Transactions on Education,53(1), 88–95. https:// JournalofNewApproachesin EducationalResearch,12(1) |2023|https://doi.org/10.7821/naer.2023.1.1240 194 García-Martínez,Inmaculada;etal. Analysingthe ImpactofArtificialIntelligenceand ComputationalSciences doi.org/10.1109/TE.2009.2024689 Jiménez-Hernández, E. M., Oktaba, H., Díaz-Barriga, F., & Piattini, M. (2020). Using webbased gamified software to learn Boolean algebra simplification in a blended learning setting. Computer Applications in Engineering Education,28(6), 1591–1611. https://doi.org/10.1002/ cae.22335 Kavanagh, S., Luxton-Reilly, A., Wuensche, B., & Plimmer, B. (2017). A systematic review of Virtual Reality in education. Themes in Science and Technology Education,10(2), 85–119. Lau, K. W., & Lee, P. Y. (2015). The use of virtual reality for creating unusual environmental stimulation to motivate students to explore creative ideas. Interactive Learning Environments,23(1), 3–18. https://doi.org/10.1080/10494820.2012.745426 Law, G. C., Dutt, A., & Neihart, M. (2019). Increasing intervention fidelity among special education teachers for autism intervention: A pilot study of utilizing a mobile-app-enabled training program. Research in Autism Spectrum Disorders,67, 101411. https://doi.org/10.1016/ j.rasd.2019.101411 López-Rodríguez, M. I., & Barac, M. (2019). Valoración del alumnado sobre el uso de Clickers y vídeo tutoriales en educación superior. Research in Education and Learning Innovation Archives,22, 29–44. Retrieved from https://doi.org/10.7203/realia.22.14582 https://doi.org/ 10.7203/realia.22.14582 Madathil, K. C., Frady, K., Hartley, R., Bertrand, J., Alfred, M., & Gramopadhye, A. (2017). An empirical study investigating the effectiveness of integrating virtual reality-based case studies into an online asynchronous learning environment. Computers in Education Journal,8(3), 1–10. Martínez, D. L., Karanik, M., Giovannini, M., & Pinto, N. (2015). Perfiles de Rendimiento Académico: Un Modelo basado en Minería de datos. Campus Virtuales,4(1), 12–30. Masson, R., & Rennie, F. (2006). ELearning. The key concepts. Routledge. McCormick, K. I., & Hall, J. A. (2022). Computational thinking learning experiences, outcomes, and research in preschool settings: a scoping review of literature. Education and Information Technologies,27, 1–36. https://doi.org/10.1007/s10639-021-10765-z Merino-Armero, J. M., González-Calero, J. A., & Cozar-Gutierrez, R. (2022). Computational thinking in K-12 education. An insight through meta-analysis. Journal of Research on Technology in Education,54(3), 410–437. https://doi.org/10.1080/15391523.2020.1870250 Moreno, R. D. (2019). The arrival of artificial intelligence to education. RITI Journal,7(14), 260–270. https://doi.org/10.36825/RITI.07.14.022 Morris, S. B. (2008). Estimating effect sizes from pretest-posttest-control group designs. Organizational Research Methods,11, 364–386. https://doi.org/10.1177/1094428106291059 Neri, L., Noguez, J., Robledo-Rella, V., Escobar-Castillejos, D., & Gonzalez-Nucamendi, A. (2018). Teaching of Classical Mechanics Concepts using Visuo-haptic Simulators. Educational Technology & Society,21(2), 85–97. Ocaña, Y., Valenzuela, L., & Garro, L. (2019). Artificial Intelligence and its Implications in Higher Education. Propósito y Representaciones,7(2), 536–568. http://doi.org/10.20511/ pyr2019.v7n2.274 Olde, G. C. V.-D., De Jong, T., & Gijlers, H. (2013). Learning by Designing Instruction in the Context of Simulation-based Inquiry Learning. Educational Technology & Society,16(4), 47–58. Page, M. J., Mckenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., & Moher, D. (2021). Declaración PRISMA 2020: una guía actualizada para la publicación de revisiones sistemáticas. Revista Española de Cardiología,74(9), 790–799. https://doi.org/10.1016/j.recesp .2021.06.016 Pareto, L. (2014). A teachable agent game engaging primary school children to learn arithmetic JournalofNewApproachesin EducationalResearch,12(1) |2023|https://doi.org/10.7821/naer.2023.1.1240 195