Online (versus face-to-face) augmented reality experience on nursing students’ leg ulcer competency: Two quasi-experimental studies

Nurse Education in Practice 71 (2023) 103715 Available online 10 July 2023 1471-5953/© 2023 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/bync-nd/4.0/). Online (versus face-to-face) augmented reality experience on nursing students’ leg ulcer competency: Two quasi-experimental studies Carlos Rodríguez-Abad a , 1 , Alba-Elena Martínez-Santos a , b , c , 1 , Josefa-del-Carmen Fern´ andez-de-la-Iglesia d , Raquel Rodríguez-Gonz´ alez a , * a University of Santiago de Compostela, Faculty of Nursing, Department of Psychiatry, Radiology, Public Health, Nursing and Medicine, Santiago de Compostela, Spain b DIPO Research Group, Galicia Sur Health Research Institute (IIS Galicia Sur). SERGAS-UVIGO, Spain c Dermatology Department, Complexo Hospitalario Universitario de Pontevedra, Pontevedra, Spain d University of Santiago de Compostela, Faculty of Education Sciences, Department of Pedagogy and Didactics, Santiago de Compostela, Spain ARTICLE INFO Keywords: Augmented reality Nursing education Research-Based Leg ulcer Online education ABSTRACT Aim: The aim of this study was two-fold. Firstly, to describe the influence of an augmented reality (AR)-based online educational experience on the academic performance and learning determinants of nursing students related to the study of leg ulcer care. We also set out to compare these online results with those obtained in the same experience developed a year before but in a face-to-face design. Background: Undergraduate education in nursing aims to train students towards an acceptable level of competence for professional practice. Sometimes, some essential clinical areas of knowledge are particularly challenging for both students and nurse educators. One of these is the care of chronic wounds, particularly in leg ulcers where nurses play a key role. Currently face-to-face teaching methods are evolving to more active contexts and alternative means such as online learning and AR are becoming widely used, garnering encouraging results. Design: A quasi-experimental design was used. Two experiments were carried out to meet the aims of the study, the first one was a pre-test post-test design in one group and the second one a post-test two experimental groups design. Methods: Participants were undergraduate nursing students from the School of Nursing of the University of Santiago de Compostela (Spain). The two experimental groups received the same training but in different environments: online during 2019/2020 (n =111) and onsite during 2018/2019 (n =72). The instruments selected were a knowledge and skills test and two validated questionnaires. Results: Regarding academic performance, the study demonstrated significant differences towards learning after the AR-based online experience (Z = − 9.074; p ≤0.001). The participants also showed good results in relation to the learning determinants studied. Compared with students receiving AR-based face-to-face teaching, the students with the online learning approach scored better in the dimensions Autonomous Learning (U=3104.5; p = 0.020) and 3D Comprehension (U=3167.0; p =0.035). However, the participants in the onsite experience scored higher in the Attention dimension (U=3163.0; p =0.035). Conclusions: AR positively influences academic performance and diverse educational variables when this experience is carried out online. Since both the online and face-to-face experience show statistically significant benefits in different dimensions of the learning determinants, the need to improve the AR learning experience through blended-learning environments should be promoted. 1. Introduction Currently, Information and Communication Technologies (ICT) are reliable and essential tools to foster higher education worldwide (Adarkwah, 2021). These technologies nurture students both personally and academically, while improving performance and making communication easier (Guven Ozdemir and Sonmez, 2021). Including ICT in training programs is widespread and benefits educators, educationalists * Correspondence to: University of Santiago de Compostela, Faculty of Nursing, Avenida de Xo´ an XXIII s/n, Campus Norte, 15782 Santiago de Compostela, Spain. E-mail address: [email protected] (R. Rodríguez-Gonz´ alez). 1 These authors contributed equally to this work and are co-first authors Contents lists available at ScienceDirect Nurse Education in Practice journal homepage: www.elsevier.com/locate/issn/14715953 https://doi.org/10.1016/j.nepr.2023.103715 Received 2 February 2023; Received in revised form 21 June 2023; Accepted 1 July 2023 Nurse Education in Practice 71 (2023) 103715 2 and students by improving the teaching-learning and research processes (Aljaraideh and Bataineh, 2019). Thus, implementing them in nursing curricula helps future nurses provide high-quality and evidence-based nursing care (Raman, 2015). On March 11, 2020, the WHO decreed a state of public health emergency caused by SARS-COV-2. Consequently, there was a gradual closure of educational centers around the world, disrupting student learning (UNESCO, 2020; Sahu, 2020; Uymaz and Uymaz, 2022). The suspension of face-to-face teaching forced many universities to shift to online teaching, which was a great challenge for the entire university community (Gewin, 2020; Lira et al., 2020; Masha’al et al., 2020). In this context, online learning was postulated as a solution for uninterrupted teaching (Adarkwah, 2021; Sahu, 2020). 1.1. Background Undergraduate education in nursing aims to train students towards an acceptable level of nursing competence for professional practice to meet the needs of society (Immonen et al., 2019). However, there is room for improvement in some curricula, causing nurses to lack sufficient knowledge and skills in some essential clinical areas (Atkin et al., 2019; Missen et al., 2016). One of these clinical areas is the care of chronic wounds and, specifically, leg ulcers, where nurses play a key professional role (Bobbink et al., 2022; Kielo et al., 2019). Leg ulcer prevalence is estimated to increase in the coming years, significantly reducing the quality of life of the patient and becoming a considerable economic burden for health systems (Atkin et al., 2019; Martinengo et al., 2019; Welsh, 2018). Therefore, nursing curricula must train students to provide care based on the robust existing evidence (Kielo et al., 2019), since it has been proven that there are deficiencies in the knowledge and skills of nurses in relation to leg ulcer care (Atkin et al., 2019; Martínez-Santos et al., 2019). In the current context of globalization and uncertainty due to migratory flows, travels, new diseases, wars or natural disasters due to climate change, face-to-face teaching methods are under threat. Thus, alternative means such as online learning are becoming widespread (Huun, 2018; Song, Park, 2021). In this vein, online learning must differ from other variants of distance education. Since it relies on the Internet to access learning experiences it is considered the most modern version of distance learning (Moore et al., 2011) where the educator promotes and supports the learning process. Despite the numerous advantages of online education, the learning outcomes of nursing students are still being studied. In fact, a metaanalysis performed by Liu et al. (2016) concluded that its effectiveness is not sufficiently proven. Recent studies have stated that blended learning (combination of online and face-to-face learning) can improve the performance, skills and satisfaction of students, compared with traditional learning methods (Grønlien et al., 2021; Li et al., 2019). On the other hand, according to Kim and Park (2021) some studies have developed educational strategies aimed at nursing students in different learning environments. They found no differences regarding student performance between online and traditional learning. In online learning, it is a fact that an online format in nursing teaching is on the rise and is supported by the development of technologies such as virtual reality or augmented reality (AR). AR is a variation of virtual reality that enriches the physical environment with digital content and virtual images (combining real and virtual world) in real time through different technological media such as smartphones, tablets or smartglasses (Mendez et al., 2020). Among other e-learning tools, AR has positive implications in health sciences education, by aiding the teaching-learning process and improving current educational methods in clinical topics (Rodríguez-Abad et al., 2021). Thus, AR has not only been shown to have positive effects on motivation, which leads to multiple benefits in the teaching-learning process, but also positively influences academic performance or three-dimensional vision (Bork et al., 2021; Rodríguez-Abad et al., 2022). In addition, students perceive that this tool increases their knowledge and their skills while providing multiple benefits to active educational and self-learning processes (Uymaz and Uymaz, 2022). According to M´ endez et al. (2020) and Wüller et al. (2019), the effect of AR in nursing education has not been studied enough and its implementation in nursing programs is scarce, so it is necessary to provide new evidence to the field. In fact, little is known about the development of AR through online environments and its potential for nursing education. To this end, the objectives of this study were: (1) to describe the influence of an AR-based online education experience on the academic performance and learning determinants such as attention, satisfaction or motivation of nursing students towards leg ulcer care; and (2) to contrast differences between the results of learning determinants in the online experience with those of the same experience onsite. 2. Methods 2.1. Design and participants A quasi-experimental design was used. This study was conducted during the academic year 2019/2020 in the School of Nursing at the University of Santiago de Compostela (North-west Spain). To ensure adequate data reporting, the TREND Statement was used (supplementary file). Second-year nursing students enrolled in the course "Clinical Nursing I" were recruited as participants, so no sampling procedure was used. Participation in the study was voluntary and all students were taught by the same nurse educator. Two experiments were carried out to meet the objectives of the study, which is a new step of a project on the use of AR in nursing education (Rodríguez-Abad et al., 2022) (Fig. 1): For the first experiment (related with the first aim), we used a pretest post-test design in one group to find out if the students in the online AR-based experience showed significant differences in the knowledge and skills test depending on when the assessment took place (pre/ post experience). Once the experience was over, we also obtained descriptive data to find out if AR influenced different learning determinants through two validated questionnaires. For the second experiment (related with the second aim), we chose a post-test two experimental groups design. Experimental group 1 (2019/ 2020 academic year students) received an online AR-based educational intervention while experimental group 2 (2018/2019 academic year students) received the intervention only face-to-face. Subsequently, we verified that the groups did not obtain significant differences based on gender distribution and average grade of the student record. In this way, we tested if the distance method (online/onsite) modifies any of the learning determinants studied. 2.2. The online learning project based on AR To meet our institution’s demands for remote work as well our students’ needs to continue acquiring knowledge and skills of specific nursing contents during the pandemic lockdown, we created an AR- based remote learning experience. The content of the educational module was related to leg ulcer care. The learning materials were published using the learning content management platform Moodle® and were structured in four virtual stations following the same structure and contents as the previous onsite experience (Fig. 2). Before starting the following AR-based stations, students had to comprehensively read the instructions of the experience and perform a pre-test of knowledge and skills: •Virtual learning station 1 “Introduction to leg ulcers”. The teaching materials for this station consisted of five multimedia presentations with content on leg ulcers (such as physiopathology, diagnosis, therapeutic management). Different QR codes were used as triggers C. Rodríguez-Abad et al. Nurse Education in Practice 71 (2023) 103715 3 to the presentations stored in the cloud (hosted by Dropbox®) and accessed by the app or browser. •Virtual learning station 2 “Ankle-Brachial Index”. QR codes were used as hyperlinks to several multimedia elements such as videos, infographics and images of the devices and clinical data needed for this diagnostic test, the procedure and interpretation of the results. •Virtual learning station 3 “Wound bed preparation and dressing”. The teaching materials used in this station consisted of 1) a multimedia presentation created by the nurse educator on the main dressings used in the care of chronic wounds, 2) a catalog of products used in moist wound healing financed by our public health service and 3) images of the main dressings (used in leg ulcer care through moist wound healing) to which videos, images, text and audio were linked using the AR software Blippar®. •Virtual learning station 4 “Lower limb compression therapy”. The fundamental aspects of this treatment are shown through 1) a multimedia presentation prepared by the nurse educator is accessed using a QR code and 2) an image of a multilayer bandage system with associated digital content using AR technology. The students could log into Moodle® whenever they wished for two weeks. In addition, three synchronous sessions took place with all the participants and the nurse educator on the Microsoft Teams® communication platform. Fig. 1. Design of the two experiments of the study. In the first experiment, a pre-test post-test design in one group was used to find out if the online AR-based experience influenced academic performance and learning determinants. For the second experiment, a post-test two experimental groups design was used to analyze differences regarding learning determinants between an educational AR-based intervention carried out online (group 1) and face-to-face (group 2). AR: Augmented Reality. Fig. 2. Contents and structure of the AR-based leg ulcer care learning project. Contents were organized in four virtual learning stations in which QR codes or images of care-related products (for example, dressings or bandages) were used as triggers for different teaching materials presented as multimedia elements (presentations, videos, images) to be viewed by the students. AR: Augmented Reality. C. Rodríguez-Abad et al. Nurse Education in Practice 71 (2023) 103715 4 Once the 4 virtual stations were finished, students were given the opportunity to complete a post-test on knowledge and skills, as well as questionnaires about the methodology used during the entire experience. 2.3. Instruments The knowledge and skills pre-test and post-test were designed by 5 nursing experts in chronic wounds and 2 experts in pedagogy. The test was then piloted in third- and fourth-year nursing students, who confirmed that it was understood and well expressed. Each test had 20 multiple-choice questions, with 1 point awarded for correct answers and 0 points for incorrect answers, so the maximum score was 20 points. The selected questions and answers were the same in both tests but were presented in random order. In addition, two validated questionnaires were used to measure different learning determinants after the experience: •The first questionnaire validated by Ferrer-Torregrosa et al. (2016) included 16 items belonging to 3 dimensions: Autonomous Work (initiative and stimulation of work by oneself), Attention and Motivation (interest in the topic and low distraction) and 3D Comprehension (perception of elements and three-dimensional understanding). This instrument was scored using a 4-point likert scale ranging from 1 (never) to 4 (always). •The Instructional Material Motivational Survey (IMMS) initiated by Keller (2010) and adapted to our language by Barroso et al. (2016) comprised 36 items and the following 4 dimensions: Confidence (perceptions of personal control and hope for success), Attention (learners’ interest, orientation reflex, curiosity and sensation seeking), Satisfaction (sense of fulfillment achieved during the learning process) and Relevance (how adequate an instrument is to fulfil needs and personal satisfaction). This instrument was scored using a 7-point likert scale ranging from 1 (maximum disagreement) to 7 (maximum agreement). In both questionnaires, the scores of the seven dimensions were obtained using the average of the items that compose them. The Cronbach’s alpha values of the questionnaires were 0.930 and 0.946, respectively, indicating a high reliability. 2.4. Data analysis Descriptive statistics (means (M) and standard deviations (SD)) were obtained for all the outcome variables in the online learning sample. For this, the SPSS® for Mac (IBM Inc.) software was used. They include the three dimensions of the questionnaire by Ferrer-Torregrosa et al. (2016), the four dimensions of the IMMS, the results of the knowledge and skills tests (pre and post-test). Descriptive statistics were also collected for the post-test results of knowledge and skills performed in the face-to-face sample. Wilcoxon test for related samples was performed comparing the results of the knowledge and skills tests with two levels: pre-test and posttest. This was done to contrast the effectiveness of the online learning project and since the data did not fit a normal distribution (Kolmogorov- Smirnov test). Non-parametric (Mann-Whitney U) tests were performed for the results in the two questionnaires used in this research to compare the results of the learning project methodology (online vs face-to-face) on the learning determinants of the use of AR. 2.5. Ethical considerations Full ethical approval was granted by the Bioethics Committee of the university where the experiment was carried out. This approval guaranteed ethical access to students as research participants, as well as making it necessary to clearly explain to students that their participation in the study would not affect their progress or grades. To guarantee all this, an informed consent was obtained, where the students gave access to their academic data and knew all their rights as research participants. To ensure the freest possible participation, the students were informed that another educator would be in charge of collecting the research data. 3. Results 3.1. Sample characteristics The sample consisted of 111 undergraduate nursing students, who carried out the experience during the SARS-COV-2 pandemic lockdown in our country. Of the total, 89.2% were females (n =99) and 10.8% were males (n =12). The average age was 20.88 years (SD=3.94). The average grade in the individual academic record was 7.37 out of 10 points (SD=0.51), with a minimum of 6.07 and a maximum of 8.52 points. 3.2. Effect of an AR-based teaching methodology on knowledge and skills test A positive effect of an AR-based teaching methodology was obtained on the level of knowledge. Thus, while the average grade of the pre-test was 9.87 out of 20 points (SD=2.60), in the post-test an average of 17.64 (SD=2.35) was acquired. This was confirmed by the Wilcoxon test, revealing that student knowledge showed significant differences depending on the moment of their evaluation (pre/post experience). The knowledge obtained by the students after online learning project based on AR was superior to that obtained previously (Z = − 9.074; p≤0.001). 3.3. Influence of AR on learning determinants in the experimental group 3.3.1. Ferrer-Torregrosa et al. (2016) questionnaire In the Ferrer-Torregrosa et al. questionnaire the participants obtained the same result in both the Attention and Motivation dimension (M=3.33 out of 4; SD=0.51) and in the Autonomous Work dimension (M=3.33; SD=0.66). However, the 3D Comprehension dimension scored slightly lower (M=3.17; SD=0.56) (see Table 1). Nursing students perceived that the online AR teaching methodology helped them revise at home (M=3.44; SD=0.75), strengthening their autonomous learning (M=3.39; SD=0.70). Likewise, the online AR project made it easier to understand leg ulcer care (M=3.39; SD=0.69) and to fix their attention (M=3.36; SD=0.63) through specifically designed content (see Table 1 for more details). 3.3.2. IMMS questionnaire In the IMMS questionnaire, the nursing undergraduates showed a mean of 5.75 (SD=0.80) out of 7 in the Relevance dimension followed by the Attention dimension (M=5.54; SD=1.06), then the Satisfaction dimension (M=5.50; SD=1.16) and lastly the Confidence dimension (M=5.33; SD=0.80) (see Table 2). The participants highlighted the relevance of the contents in their needs as future nurses (M=6.49; SD=1.09), clearly understanding the AR-based lesson material (M=6.37; SD=1.16). The students also believed that the contents would be useful (M=6.34; SD=1.06) and that the audiovisual materials were fun (M=6.26; SD=1.11) (see Table 2 for more details). 3.4. Influence of learning methodology (online vs face-to-face) on learning determinants after AR experience The Mann-Whitney U tests revealed a significant effect of the learning methodology on two of the three dimensions of the Ferrer-Torregrosa et al. (2016) questionnaire: Autonomous Learning C. Rodríguez-Abad et al. Nurse Education in Practice 71 (2023) 103715 5 (U=3104,5; p =0.020) and 3D Comprehension (U=3167,0; p =0.035). Thus, for the same AR experience the online group scored higher in the dimensions Autonomous Learning (M=3.34; SD=0.66) and 3D Comprehension (M=3.17; SD=0.56) than their face-to-face counterparts (M=3.12; SD=0.62 and M=2.97; SD=0.54, respectively). On the other hand, the Mann-Whitney U tests revealed a significant effect of the learning methodology on the Attention dimension of the IMMS (U=3163,0; p =0.035). In this case, the face-to-face group scored higher (M=5.93; SD=0.66) than their online counterparts (M=5.54; SD=1.06). 4. Discussion Our study enriches the understanding of the influence of an AR-based online learning experience on students’ academic performance in clinical contents related to leg ulcer care. It also compares the results related to different learning determinants obtained in this online experience with those garnered in a face-to-face experience a year before. To the best of our knowledge, this project is the first to delve into the influence of AR-based online learning on academic performance, motivation and satisfaction towards learning in nursing students. In relation to academic performance, Abbasi et al. (2020) stated that most health sciences students believe that e-learning is not effective in acquiring clinical skills. However, our findings have shown significant differences associated to having carried out the online AR-based experience, with knowledge and skills after learning being higher in accordance with other studies (Bork et al., 2019; Mendez et al., 2020). Thus, we agree with several authors that AR facilitates the teaching-learning process, improves learning methods in online education environments and offers nursing educators interesting possibilities for teaching innovation (Mendez et al., 2020; Uymaz and Uymaz, 2022). Regarding motivation, satisfaction, attention, confidence, comprehension and autonomous work, participants have shown high scores in the different dimensions measured through two validated questionnaires. These results are in line with those found in different health sciences degrees such as medicine (Bork et al., 2021; Khan et al., 2019; Kugelman et al., 2018) or podiatry (Ferrer-Torregrosa et al., 2016). However, little is known about the influence of AR on nursing students in particular, especially when working on content related to our own professional competences. Regarding the IMMS questionnaire, the relevance dimension obtained the highest score. Recent studies have shown that knowledge acquisition and self-directed learning improve when AR is used (Garrett et al., 2015) and that nursing students accept this technology to a high degree (Uymaz and Uymaz, 2022). Although the environment is not a typical face-to-face laboratory, it seems logical to think this technology makes students perceive content as relevant. When analyzing the differences between the online and onsite groups, we observed statistically significant differences in relation to the attention dimension in favor of the face-to-face group. This could be because the physical presence of the educator and peers as well as the absence of external stimuli allow students to focus more on contents and facilitate their interaction and support. In relation to the Ferrer-Torregrosa et al. (2016) questionnaire, students showed a high degree of motivation towards learning in both online and face-to-face experiences. Similar results were found in realistic e-learning cases during the lockdown (Rahm et al., 2021). We found significant differences in the autonomous learning and 3D comprehension dimensions where scores were higher in the online experience. The differences between the two experiences may be due to students attending in-person and were only able to access the content during class. Meanwhile, the online experience makes learning more flexible, allowing the student to access the content at any time and place (Adarkwah, 2021; Goh, 2016). According to The Council of Independent Colleges (2016), students feel more engaged when traditional learning is combined with online Table 1 Means & standard deviations of each of the dimensions and items of the Ferrer-Torregrosa et al. (2016) questionnaire measuring attention and motivation, autonomous work and comprehension. M SD Item Likert scale (%) M SD 1 Never 2 3 4 Always FERRER-TORREGROSA ET AL (2016) QUESTIONNAIRE DIMENSIONS ATTENTION AND MOTIVATION 3.33 0.51 It helps me to fix my attention 0.0 8.1 47.7 44.1 3.36 0.63 It helps me to retain the contents 0.0 12.6 51.4 36.0 3.23 0.66 It motivates me to learn 0.9 12.6 41.4 45.0 3.31 0.72 It makes it possible to study in different ways thus avoiding feeling frustrated 1.8 11.7 36.9 49.5 3.34 0.76 It helps me to see/to imagine very clearly what I am being explained 1.8 8.1 45.0 45.0 3.33 0.70 It helps me to understand nursing care for leg ulcers 0.9 9.0 40.5 49.5 3.39 0.69 It helps me to understand the course without excessive explanations from the professor 1.8 10.8 45.9 41.4 3.27 0.73 It helps me to revise at home 1.8 9.9 30.6 57.7 3.44 0.75 AUTONOMOUS WORK 3.34 0.66 Active learning stimulates me 1.8 15.3 35.1 47.7 3.29 0.79 It strengthens my autonomous learning 0.0 12.6 36.0 51.4 3.39 0.70 COMPREHENSION 3.17 0.56 It is able to make me recognize the etiology of leg ulcers 1.8 18.0 49.5 30.6 3.09 0.75 It has allowed me to learn to guide the treatment of choice 0.9 14.4 55.9 28.8 3.13 0.68 I can recognize the signs and symptoms of infection and take action 0.9 22.5 51.4 25.2 3.01 0.72 I have improved my ability to select the most appropriate dressing based on the characteristics of the ulcer 0.0 11.7 53.2 35.1 3.23 0.65 I have improved my ability to visualize the placement of the different dressings 1.8 12.6 42.3 43.2 3.27 0.75 It has helped me to know how to apply basic care for leg ulcers (prevention, diagnosis and care) 0.0 7.2 54.1 38.7 3.32 0.60 Means (M), standard deviation (SD). All items were valued on a Likert-type scale rated from 1-never to 4-always. C. Rodríguez-Abad et al. Nurse Education in Practice 71 (2023) 103715 6 Table 2 Means & standard deviations of each of the dimensions and items that compose the IMMS questionnaire measuring the ARCS motivational model. M SD Item Likert scale (%) M SD 1 2 3 4 5 6 7 absolutely disagree absolutely agree IMMS QUESTIONNAIRE DIMENSIONS ATTENTION 5.54 1.06 There was something interesting at the beginning of this lesson that got my attention 3.6 3.6 0.9 9.9 20.7 30.6 22.5 5.23 1.59 This AR-technology is eyecatching 6.3 8.1 0.9 0.9 15.3 28.8 31.5 5.32 1.83 The quality of the AR material helped to hold my attention 2.7 6.3 3.6 9.9 13.5 33.3 30.6 5.48 1.61 This material is so abstract that it was hard to keep my attention on it § 1.8 1.8 2.7 4.5 10.8 32.4 45.9 6.01 1.32 The images, videos and notes of this lesson look dry and unappealing § 0.0 1.8 6.3 4.5 17.1 29.7 40.5 5.88 1.27 The way the information is arranged using this technology helped keep my attention 0.0 5.4 5.4 10.8 13.5 34.2 30.6 5.58 1.44 The information discovered through experience stimulated my curiosity 0.0 7.2 3.6 0.9 19.8 35.1 25.2 5.48 1.43 The amount of repetition in this activity caused me to get bored sometimes § 2.7 5.4 8.1 12.6 17.1 32.4 21.6 5.20 1.59 I learned some things about AR that were surprising or unexpected 8.1 9.9 6.3 16.2 18.0 22.5 18.9 4.69 1.88 The variety of audiovisual material helped keep my attention on the lesson. 1.8 4.5 1.8 7.2 20.7 34.2 29.7 5.62 1.41 The audiovisual material is boring § 0.0 0.9 3.6 4.5 7.2 27.0 56.8 6.26 1.11 There is so much content that it is irritating § 1.8 2.7 6.3 5.4 11.7 32.4 39.6 5.78 1.47 CONFIDENCE 5.33 0.80 When I first looked at this lesson, I had the impression that it would be easy for me 4.5 11.7 18.0 31.5 17.1 13.5 3.6 4.00 1.45 This material was more difficult to understand than I would like § 1.8 2.7 9.9 9.0 13.5 36.9 26.1 5.45 1.50 After reading the introductory information, I felt confident that I knew what I was supposed to learn from this lesson 0.9 2.7 5.4 14.4 22.5 32.4 21.6 5.39 1.34 There was so much information that it was difficult for me to remember the important points § 7.2 5.4 16.2 16.2 19.8 24.3 10.8 4.52 1.71 As I worked on this lesson, I was confident that I could learn the content 0.9 3.6 5.4 9.0 18.0 37.8 25.2 5.54 1.37 It was difficult to discover the digital information associated with the real image § 1.8 2.7 0.9 9.9 10.8 31.5 42.3 5.89 1.38 After working on this lesson for a while, I was confident that I would be able to pass a test on it 1.8 2.7 6.3 15.3 33.3 26.1 14.4 5.12 1.33 I could not really understand quite a bit of the material in this lesson § 1.8 1.8 0.9 0.9 1.8 33.3 59.5 6.37 1.16 The content was so well organized that it helped me be confident that I would learn this material 0.0 3.6 0.9 10.8 23.4 32.4 28.8 5.67 1.23 RELEVANCE 5.75 0.80 It is clear to me how the content of this material is 2.7 7.2 11.7 18.9 18.9 33.3 7.2 4.73 1.52 (continued on next page) C. Rodríguez-Abad et al. Nurse Education in Practice 71 (2023) 103715 7 learning. In this way, based on our findings and according to Mendez et al. (2020), once online content continues to grow and AR becomes popular, nurse educators will be able to generate advanced knowledge. All this makes us consider the possibility of developing a blended learning experience combining the benefits of each of the teaching modalities. Counting on AR is fundamental to promote the teaching-learning process in the field of Nursing in a cost-effective way. 4.1. Strengths and limitations To the best of our knowledge, this study represents the first research in describing an online AR experience in nursing as well as comparing the influence of AR in an online and face-to-face experience. Therefore, this article contributes to mitigate the lack of research on the effects of AR specifically in nursing education. Creating and incorporating the materials into the virtual campus is a heavy workload for the educators team but could provide a valuable alternative to face-to-face education in the metaverse era. In this way, the continuous increase in students in nursing schools, changes in the syllabus or time limitations for teaching could be compensated. However, this study has the typical biases of nonrandomized quasi-experimental designs. It is necessary to carry out larger and multicenter studies to generalize the results. 5. Conclusions Our study has shown that AR positively influences academic performance and diverse educational variables related to motivation when this experience is carried out online. It is necessary to underline that, compared with the face-to-face experience, the online one scored higher in autonomous learning and 3D comprehension. This may happen happens because these aspects adjust to the pace of each student. However, attention was better in the face-to-face experience. Therefore, there is room for improvement in future AR ranged experiences. These findings show that blended-learning experiences, widely promoted since the pandemic, have benefits which should be taken advantage of, such as improving the educational experience as well as enhancing learning determinants and academic performance. Table 2 (continued) M SD Item Likert scale (%) M SD 1 2 3 4 5 6 7 absolutely disagree absolutely agree related to things I already know There were no images, videos and texts that showed me how this material is related to things that I already know § 1.8 4.5 3.6 9.0 13.5 25.2 42.3 5.73 1.54 Completing this lesson successfully was important to me 0.0 1.8 8.1 9.0 22.5 31.5 27.0 5.55 1.30 The content of this material is relevant to my interests 4.5 3.6 0.9 5.4 9.0 30.6 45.9 5.86 1.60 There are explanations or examples of how people use the knowledge in this lesson 0.9 1.8 12.6 17.1 16.2 36.0 15.3 5.15 1.39 The content and audiovisual material in this lesson convey the impression that its content is worth knowing 0.0 3.6 1.8 8.1 15.3 34.2 36.9 5.86 1.26 This lesson was not relevant to my needs because I already knew most of it § 1.8 0.0 1.8 1.8 3.6 19.8 71.2 6.50 1.09 I could relate the content of this lesson to things I have seen, done, or thought about in my own life 0.9 2.7 6.3 9.0 27.0 32.4 21.6 5.42 1.32 The content of this lesson will be useful to me 0.0 1.8 0.9 4.5 7.2 25.2 60.4 6.34 1.06 SATISFACTION 5.50 1.16 Completing the exercises in this lesson gave me a satisfying feeling of accomplishment 0.0 1.8 8.1 9.0 22.5 31.5 27.0 5.54 1.30 I enjoyed this lesson so much that I would like to know more about this topic 0.9 7.2 6.3 14.4 27.0 25.2 18.9 5.11 1.49 I really enjoyed studying this lesson 0.9 1.8 7.2 9.9 17.1 31.5 31.5 5.61 1.38 The wording of feedback after the exercises, or of other comments in this lesson, helped me feel rewarded for my effort 2.7 2.7 4.5 9.9 26.1 28.8 25.2 5.41 1.45 It felt good to successfully complete this lesson 0.9 4.5 2.7 17.1 19.8 29.7 25.2 5.40 1.41 It was a pleasure to work on such a well-designed lesson 0.9 1.8 4.5 8.1 14.4 24.3 45.9 5.90 1.36 Means (M), standard deviation (SD). All items were valued on a Likert-type scale rated from 1-maximum disagreement to 7-maximum agreement. §Negative statements were reverse coded, so a higher score indicates greater disagreement. C. Rodríguez-Abad et al. Nurse Education in Practice 71 (2023) 103715 8 Ethical approval The study was approved by the Bioethics Committee of the University of Santiago de Compostela to ensure participant and student rights. Funding source This study has been partially financed by the competitive call for teaching innovation projects of the National Conference of Deans of Nursing in Spain -CNDE-(PID_02CNDE2020). Declaration of Competing Interest We wish to confirm that there are no known conflicts of interest associated with this publication and there has been no financial support for this work that could have influenced its outcome. We confirm that the manuscript has been read and approved by all named authors and that there are no other persons who satisfied the criteria for authorship but are not listed. We further confirm that the order of authors listed in the manuscript has been approved by all of us. We confirm that we have given due consideration to the protection of intellectual property associated with this work and that there are no impediments to publication, including the timing of publication, with respect to intellectual property. In so doing we confirm that we have followed the regulations of our institutions concerning intellectual property. We further confirm that any aspect of the work covered in this manuscript that has involved human patients has been conducted with the ethical approval of all relevant bodies and that such approvals are acknowledged within the manuscript. We understand that the Corresponding Author is the sole contact for the Editorial process (including Editorial Manager and direct communications with the office). She is responsible for communicating with the other authors about progress, submissions of revisions and final approval of proofs. We confirm that we have provided a current, correct email address which is accessible by the Corresponding Author and which has been configured to accept email from your journal and publishing company. Acknowledgements We are grateful to Diego Lopez-Cao for his services in editing the manuscript. Author statement Carlos Rodríguez-Abad: design of the work, acquisition and analysis of the data and drafting the work. Alba-Elena Martínez Santos: design of the work, acquisition of the data and drafting the work. Josefa-del-Carmen Fern´ andez-de-la-Iglesia: design of the work, analysis and interpretation of the data and drafting the work. Raquel Rodríguez-Gonz´ alez: project administration, design of the work, acquisition of the data and drafting the work. All of them gave final approval of the version to be published. References Abbasi, M.S., Nasser, A., Batool, S., Abdullah, A., Sumera, S., Shaur, S., Rana, A., Fahim, V., Tariq, A., 2020. E-Learning perception and satisfaction among health sciences students amid the COVID-19 pandemic. Work 67 (3), 549–556. https://doi. org/10.3233/WOR-203308. Adarkwah, M.A., 2021. “I’m not against online teaching, but what about us?”: ICT in Ghana post Covid-19. Educ. Inf. Technol. 26 (2), 1665–1685. https://doi.org/ 10.1007/s10639-020-10331-z. Aljaraideh, Y., Bataineh, K.A., 2019. Jordanian students’ barriers of utilizing online learning: a survey study. Int. Educ. Stud. 12 (5), 99–108. Atkin, L., Bu´ cko, Z., Conde Montero, E., Cutting, K., Moffatt, C., Probst, A., Romanelli, M., Schultz, G.S., Tettelbach, W., 2019. Implementing TIMERS: the race against hard-to-heal wounds. J. Wound care S1–S50. https://doi.org/10.12968/ jowc.2019.28.Sup3a.S1. Barroso, J.M., Cabero Almenara, J., Moreno-Fern´ andez, A.M., 2016. La utilizaci´ on de objetos de aprendizaje en realidad aumentada en la ense˜ nanza de la medicina. Int. J. Technol. Educ. Innov. 2 (2), 77–83. https://doi.org/10.20548/innoeduca.2016. v2i2.1955. Bobbink, P., et al., 2022. E-Learning and blended-learning program in wound care for undergraduate nursing students. J. Nurs. Educ. 61 (1), 53–57. https://doi.org/ 10.3928/01484834-20211203-03. Bork, F., et al., 2019. The benefits of an augmented reality magic mirror system for integrated radiology teaching in gross anatomy. Anat. Sci. Educ. 12, 585–598. https://doi.org/10.1002/ase.1864. Bork, F., et al., 2021. The effectiveness of collaborative augmented reality in gross anatomy teaching: A quantitative and qualitative pilot study. Anat. Sci. Educ. 14 (5), 590–604. https://doi.org/10.1002/ase.2016. Ferrer-Torregrosa, J., et al., 2016. Distance learning ects and flipped classroom in the anatomy learning: Comparative study of the use of augmented reality, video and notes. BMC Med. Educ. 16, 230. https://doi.org/10.1186/s12909-016-0757-3. Garrett, B.M., et al., 2015. Augmented reality m-learning to enhance nursing skills acquisition in the clinical skills laboratory. Interact. Technol. Smart Educ. 12 (4), 298–314. https://doi.org/10.1108/ITSE-05-2015-0013. Gewin, V., 2020. Five tips for moving teaching online as COVID-19 takes hold. Nature 580 (7802), 295–296. https://doi.org/10.1038/d41586-020-00896-7. Goh, P.S., 2016. eLearning for technology enhanced learning in medical education—Hope, not hype. Med. Teach. 38, 957–958. https://doi.org/10.3109/ 0142159X.2016.1147538. Grønlien, H.K., et al., 2021. A blended learning teaching strategy strengthens the nursing students’ performance and self-reported learning outcome achievement in an anatomy, physiology and biochemistry course-A quasi-experimental study. Nurse Educ. Pract. 52, 103046 https://doi.org/10.1016/j.nepr.2021.103046. Guven Ozdemir, N., Sonmez, M., 2021. The relationship between nursing students’ technology addiction levels and attitudes toward e-lear ning during the COVID-19 pandemic: a cross-sectional study. Perspect. Psychiatr. Care 57 (3), 1442–1448. https://doi.org/10.1111/ppc.12710. Huun, K., 2018. Virtual simulations in online nursing education: align with quality matters. Clin. Simul. Nurs. 22, 26–31. https://doi.org/10.1016/j.ecns.2018.07.002. Immonen, K., Oikarainen, A., Tomietto, M., K¨ a¨ ari¨ ainen, M., Tuomikoski, A., Kauˇ ciˇ c, B. M., Mikkonen, K., 2019. Assessment of nursing students’ competence in clinical practice: a systematic review of reviews. Int. J. Nurs. Stud. 100, 103414 https://doi. org/10.1016/j.ijnurstu.2019.103414. Keller, J.M., 2010. Motivational Design for Learning and Performance: The ARCS Model Approach. Springer, New York, NY. Khan, T., et al., 2019. The impact of an augmented reality application on learning motivation of students. Adv. Hum. -Comput. Interact. 2019, 7208494. https://doi. org/10.1155/2019/7208494. Kielo, E., et al., 2019. Competence areas for registered nurses and podiatrists in chronic wound care, and their role in wound care practice. J. Clin. Nurs. 28 (21–22), 4021–4034. https://doi.org/10.1111/jocn.14991. Kim, S.H., Park, S., 2021. Influence of learning flow and distance e-learning satisfaction on learning outcomes and the moderated mediation effect of social-evaluative anxiety in nursing college students during the COVID-19 pandemic: A cross-sectional study. Nurse Educ. Pract. 56, 103197 https://doi.org/10.1016/j.nepr.2021.103197. Li, C., et al., 2019. The effects of blended learning on knowledge, skills and satisfaction in nursing students: A meta-analysis. Nurse Educ. Today 82, 51–57 https://doi. org/ 10.1016/j.nedt.2019.08.004. Lira, A.L.B.C., et al., 2020. Nursing education: challenges and perspectives in times of the COVID-19 pandemic. Rev. Bras. De. Enferm. 73 (suppl 2), e20200683 https://doi. org/10.1590/0034-7167-2020-0683. Liu, Q., Peng, W., Zhang, F., Hu, R., Li, Y., Yan, W., 2016. The effectiveness of blended learning in health professions: systematic review and meta-analysis. J. Med. Internet Res. 18 (1), e4807 https://doi.org/10.2196/jmir.4807. Martinengo, L., et al., 2019. Prevalence of chronic wounds in the general population: systematic review and meta-analysis of observational studies. Ann. Epidemiol. 29, 8–15. https://doi.org/10.1016/j.annepidem.2018.10.005. Martínez-Santos, A.-E., Tiz´ on Bouza, E., Pesado Cartelle, J.A., 2019. Conocimiento enfermero sobre prevenci´ on, diagn´ ostico y cuidados de las úlceras de extremidad inferior en un ´ area sanitaria. Gerokomos 30 (1), 34–41 (Retrieved from). 〈http://scie lo.isciii.es/scielo.php?script=sci_arttext&pid=S1134-928×2019000100034&lng=es &tlng=pt〉. Masha’al, D., et al., 2020. Distance learning-related stress among undergraduate nursing students during the COVID-19 pandemic. J. Nurs. Educ. 59 (12), 666–674. https:// doi.org/10.3928/01484834-20201118-03. Mendez, K.J., et al., 2020. Virtual and augmented reality: implications for the future of nursing education. Nurse Educ. Today 93, 104531. https://doi.org/10.1016/j. nedt.2020.104531. Missen, K., et al., 2016. Qualified nurses’ perceptions of nursing graduates’ abilities vary according to specific demographic and clinical characteristics. A descriptive quantitative study. Nurse Educ. Today 45, 108–113. https://doi.org/10.1016/j. nedt.2016.07.001. Moore, J.L., et al., 2011. e-Learning, online learning and distance learning environments: are they the same? Internet High. Educ. 14 (2), 129–135. https://doi.org/10.1016/j. iheduc.2010.10.001. Raman, J., 2015. Mobile technology in nursing education: where do we go from here? A review of the literature. Nurse Educ. Today 35 (5), 663–672. https://doi.org/ 10.1016/j.nedt.2015.01.018. Rodríguez-Abad, C., Fern´ andez-de-la-Iglesia, J.D., Martínez-Santos, A.E., Rodríguez- Gonz´ alez, R., 2021. A systematic review of augmented reality in health sciences: a C. Rodríguez-Abad et al. Nurse Education in Practice 71 (2023) 103715 9 guide to decision-making in higher education. Int. J. Environ. Res. Public Health 18 (8), 4262. https://doi.org/10.3390/ijerph18084262. Rodríguez-Abad, C., Rodríguez-Gonz´ alez, R., Martínez-Santos, A.E., Fern´ andez-de-la- Iglesia, J.D., 2022. Effectiveness of augmented reality in learning about leg ulcer care: a quasi-experimental study in nursing students. Nurse Educ. Today 119, 105565. https://doi.org/10.1016/j.nedt.2022.105565. Sahu, P., 2020. Closure of universities due to coronavirus disease 2019 (COVID-19): impact on education and mental health of students and academic staff. Cureus 12 (4), e7541. https://doi.org/10.7759/cureus.7541. Song, C.E., Park, H., 2021. Active learning in E-learning programs for evidence-based nursing in academic settings: a scoping review. J. Contin. Educ. Nurs. 52 (9), 407–412. https://doi.org/10.3928/00220124-20210804-05. UNESCO. (2020). COVID-19 Educational Disruption and Response. Retrieved from https://en.unesco.org/covid19/educationresponse. Uymaz, P., Uymaz, A.O., 2022. Assessing acceptance of augmented reality in nursing education. PloS One 17 (2), e0263937. https://doi.org/10.1371/journal. pone.0263937. Welsh, L., 2018. Wound care evidence, knowledge and education amongst nurses: a semi-systematic literature review. Int. Wound J. 15 (1), 53–61. https://doi.org/ 10.1111/iwj.12822. Wüller, H., et al., 2019. A scoping review of augmented reality in nursing. BMC Nurs. 18, 19. https://doi.org/10.1186/s12912-019-0342-2. C. Rodríguez-Abad et al.