The motivation of technological scenarios in augmented reality (AR): results of different experiments

applied sciences Article The Motivation of Technological Scenarios in Augmented Reality (AR): Results of Different Experiments Julio Cabero-Almenara 1,†and Rosabel Roig-Vila 2,†,* 1Facultad de Ciencias de la Educación, Universidad de Sevilla, C/Pirotecnia, S/N, 41013 Sevilla, Spain 2Facultad de Educación, Universidad de Alicante, Campus Sant Vicent del Raspeig, 03080 Alicante, Spain *Correspondence: rosabel.r[email protected]; Tel.: +34-965-903-721 †These authors contributed equally to this work. Received: 19 June 2019; Accepted: 17 July 2019; Published: 19 July 2019   Abstract: Augmented Reality (AR) is an emergent technology that is acquiring more and more relevance in teaching every day. Together with mobile technology, this combination arises as one of the most effective binomials to support significant and ubiquitous learning. Nevertheless, this binomial can only prove valid if the student is motivated to use it during the learning process. An attempt was made through the implementation of Keller’s Instructional Material Motivational Survey model o determine the degree of motivation of Pedagogy, Medicine and Art students from the University of Seville for using AR-enriched notes available by means of mobile devices in the classroom. Three applications designed for the subjects of Educational Technology, Anatomy and Art served to assess it positively in terms of the motivation raised by the participation in the experiment, as well as regarding academic performance improvement. It can additionally be stated that our main finding was a link between students’ motivation to use the enriched notes and the performance obtained in the subject in which they use them. Evidence was also found that the utilization of Augmented Reality benefits the learning process. Keywords: mobile devices; augmented reality; education; higher education; students’ perceptions 1. Introduction Augmented Reality (AR) is becoming an emergent technology with great potential in various and largely different fields of application, such as geography, with regard to watershed control [ 1 ], medicine, cultural heritage [ 2 ] or industry [ 3 ]. It is likewise being used in the educational context [ 4 – 9 ], amongst other reasons because AR permits easy access to information—most often through mobile devices. It must be remembered that these devices are highly present in the life of university students all over Latin America [10,11]. According to Azuma [ 12 ], AR can be defined as a system with the following characteristics: it combines real and virtual; it is interactive in real time, and it is registered in 3D. AR technology consequently allows us to combine digital and physical information in real time by means of different technological media such as tablets, smartphones [ 13 ] or Augmented Reality glasses [ 3 ], to quote but three, thus providing the chance to create a new reality [ 14 ]. “Augmented reality visibly introduces the knowledge that students need to learn inside its real environment” [15]. Such a mixed integration of physical and digital reality may take place at various levels, ranging from the utilization of QR codes, the use of images, the exploitation of 3D objects and the Global Positioning System (GPS)-mediated mobilization of coordinates to thermal footprints [ 6 ]. Furthermore, one can classify AR systems depending on their location or according to images. “Location-based AR Appl. Sci. 2019,9, 2907; doi:10.3390/app9142907 www.mdpi.com/journal/applsci Appl. Sci. 2019,9, 2907 2 of 16 systems take advantage of the data about the position of mobile devices, determined by the Global Positioning System (GPS) or other Wi-Fi-based positioning systems . . . Instead, image-based AR focuses on image recognition techniques that serve to identify the position of physical objects in a real environment, so that the virtual contents related to such objects can be properly situated” [16]. The relevance of AR for learning has to do: (a) with its specific characteristics: being a mixed reality, it offers the possibility for a real-time integration both of different information layers and of the various formats where that information may appear (texts, URLs, videos . . . ). Furthermore, as an interesting, user-friendly technology, it allows us to enrich or alter the information about reality, adding further contents to it [ 6 ]; and (b) with the fact that the devices most frequently used for its observation, e.g., smartphones, are technologies within easy reach of university learners. Numerous authors [ 17 – 28 ] have highlighted that AR utilization opens a wide range of possibilities for us, namely: (1) removing information that might hinder the assimilation of significant information by students; (2) increasing or enriching the information about reality so that it can be more easily understood by students; (3) being able to observe an object from different points of view, thus enabling the student to select the time and place of observation; (4) fostering ubiquitous learning; e) creating safe “artificial” scenarios, such as laboratories or simulators; (5) enriching the materials printed for students with additional information in different formats; (6) turning students into “proconsumers” of learning objects in AR format; and (7) achieving an improved learning through its use. Furthermore, AR can be used in a variety of disciplines and at different educational levels [ 29 , 30 ], even though university learning admittedly stands out as the context where it appears the most often. As for its impact on education, the number of research studies carried out so far is still limited—even through that number is gradually increasing [ 31 ]. This possibly constitutes one of the most serious problems for its incorporation into teaching. Some aspects have been stressed in those works, however: that students show favorable attitudes towards AR and that its utilization enhances motivation towards learning [ 32 , 33 ]; that it helps create a constructivist learning context [ 16 , 34 ]; that it favors an active teaching environment [ 35 ]; that it reduces the cognitive load to which students are subjected in learning tasks [ 36 , 37 ]; that students have a high degree of acceptance as well as positive attitudes in relation to AR technology [ 38 – 43 ]; that it increases satisfaction levels amongst both students and teachers [ 24 , 44 – 46 ]; that it improves students’ spatial skills and orientation [ 47 , 48 ]; and finally, that its use enhances academic performance [ 26 , 30 , 49 ]. Additionally, it deserves to be highlighted that experiments are already being carried out for its incorporation into e-learning-based learning initiatives [ 31 , 50 ], as well as into others where students act as learning object producers [51,52]. In any case, we must bear in mind that the incorporation of AR into teaching faces a number of obstacles: this is a new technology; no development-focused educational experiments have been carried out yet; no learning resources and objects produced in AR format exist; teachers have a poor training; teachers need to develop positive attitudes towards AR incorporation into educational practice; no educational experiments have so far been performed in the development of AR-based learning objects; screens do not offer a suitable observation size; there are no conceptual frameworks which can help us find innovative practices in AR implementation; support centers for teachers need to be created so that they will find it easier to produce AR-based objects, as well as their maintenance in servers; a base/basic technology must be available for its observation by students; and an information overload clearly exists [28,52]. Concerning the last obstacle mentioned above, some precautions should be adopted when designing AR objects. Modules must have short and direct contents and be flexible as well as straightforward. They have to reflect an accessible and tolerant approach with a multimedia design, oriented to the student’s action and involvement. Tablets and smartphones do not constitute the goal of the educational process, and therefore, the methodology must focus on practicality and interactivity and adapt to the diversity of devices where it can be observed [53]. Appl. Sci. 2019,9, 2907 3 of 16 This paper shows an experiment carried out within a university context, additionally providing a detailed description of the AR products generated. This serves as the starting point to explain the educational research performed, where the Instructional Material Motivational Survey (IMMS) developed by Keller was used as the assessment tool [ 4 ]. The paper finishes with a summary of the results obtained as well as the corresponding conclusions drawn from them. 2. Materials and Methods 2.1. Research Aims The different research works described below pursued the following objectives: 1. Ascertaining the degree of motivation measured by means of Keller’s IMMS model and the dimensions shaping it (attention/relevance/satisfaction) that the utilization of AR objects generated amongst students. 2. Analyzing if the degree of motivation (attention/relevance/satisfaction) measured using Keller’s IMMS model influenced students’ knowledge acquisition. Below can be found the hypotheses that we pose in this study: 1. H0 (null hypothesis): There are no significant differences, with a 0.05 alpha risk of us being wrong with regard to the scores achieved by students in the pre-test and the post-test. 2. H1 (alternative hypothesis): There are significant differences, with a 0.05 alpha risk of us being wrong with regard to the scores achieved by students in the pre-test and the post-test. 2.2. Research Design It must be stressed from the very beginning that the research was replicated with AR objects specifically made for investigation purposes that belonged to three different areas of knowledge: Educational Sciences; Fine Arts; and Medicine. Two types of single-case pre-experimental design were used: one intended to analyze the relevance that the interaction with subjects had for the degree of motivation raised, which used a post-test measure; and the other referred to performance, which revolved around the analysis of academic performance and included pre-test and post-test measures [ 54 , 55 ]. More precisely, a treatment (the AR object created) was administered in both designs, complemented with a measurement (the student’s motivation to develop the experiment and to improve academic performance). The experiment took place following the same sequences in all cases: 1. Explaining to students what AR was all about; 2. Showing the different objects produced and their possibilities; 3. Specifying the website where they could download the different guides and apps for the objects produced; 4. Encouraging students to work on their own using their mobile devices with the different objects; and 5. Completing questionnaires 2.3. The Sample All the students involved in the experiment were enrolled in degrees in Medicine (Anatomy) and Pedagogy (Educational Technology), and in the master’s degree in “Art: ideas and production” —all of them taught at the University of Seville—Table 1. The samples had a non-probabilistic nature; in other words, they were “convenience” or “causal” samples [ 56 , 57 ] determined by the researcher’s ease of access to the subjects making up the sample in question. Appl. Sci. 2019,9, 2907 4 of 16 Table 1. Number of participants in each one of the experiments. Educational Technology Medicine Art Total Men Women Total Men Women Total Men Women 302 77 225 50 21 29 20 5 15 25.50% 74.50% 42% 58% 25% 75% It is worth highlighting that the experiment with the Art group took place after a three-hour-long interaction of students with the learning objects, as opposed to the one-week-long interaction period in the other two groups. 2.4. The Objects Produced The experiment involved three AR objects specifically developed for this study and referred to contents associated with (a) the discipline of Educational Technology: “ways to use video in teaching-learning processes”; (b) Medicine (more precisely, Anatomy); and (c) the field of Fine Arts. A program guide prepared for students told them where they had to download the apps which allowed for the operation of the objects as well as the actions that they could perform with those objects. This was all complemented with a reference bibliography for students to delve deeper into each thematic area. The three aforementioned objects had different typologies; those related to contents framed within “Educational Technology” (educational roles played by video use) and “Anatomy” can be categorized as belonging to Type II [ 6 ] in the format of “notes enriched” with AR objects, transferring the former with a video resource, and the latter with a 3D object that the student could manipulate ( Figure 1shows images of the object created for the “Educational Technology” subject). Figure 1. Image of the Augmented Reality (AR) object and its utilization in “Educational Technology”. Four AR objects were created for the “Anatomy” area, namely: Coxofemoral2; Shoulde2r; Ankle2; and Cervical2. All of them shared the following features: 3D animations; stop and monitoring of the object; enlargement of the size of the component presented so that details become visible; and transfer to an information extending video located on YouTube (Figure 2). Appl. Sci. 2019,9, 2907 5 of 16 Figure 2. Pictures of one of the AR objects created for the Anatomy area. The final object produced can be classified as belonging to the “Mixed Reality” category due to the combination of elements from both “Augmented Reality” and “Virtual Reality.” It shows the Church of the Annunciation in Seville, which belongs to the University of Seville, but initially was the “Professed House of the Company/Society of Jesus”. This church dates back to 1565, and its façade stands out as one of the masterpieces of Sevillian Mannerism. The application developed allows the user to view—on a marker—3D models of the sculptures that Mart í nez Mont á ñez made of San Ignacio de Loyola and San Francisco de Borja. These figures, sculpted between 1610 and 1624, are located at the Main Altarpiece of this church. Users can likewise have access to a gigapixel photograph of the Altarpiece, where an extremely high zoom allows them not only to look at every single detail but also to experience a 360 º viewing of the central nave, the entrance, and the crossing. They can also watch a video filmed with the help of a drone that shows the interior of the central nave (Figure 3). Figure 3. Images of the object produced at the Church of the Annunciation. Most importantly, the material generated in this case included a series of questions designed to find out if students had paid attention during their interaction (Figure 4). A final result corresponding to the number of questions posed appears at the end of the questionnaire. Appl. Sci. 2019,9, 2907 6 of 16 Figure 4. An example of the questions which appear in the program. We can classify the material produced on this occasion as: Level-2 AR; Immersive virtual reality; 360◦vision; 3D modelling; incorporation of audio and gigapixel which helps enlarge the image (Figure 5). Figure 5. Gigapixel image of the Church of the Annunciation. The production of these objects was possible thanks to the use of the different software programs listed in Appendix A, which also specifies the type of use given to each one of them. As can be seen, we used a variety of programs, including Metaio Creator, Eclipse and Ffmpeg, since the production of the respective objects required some specific characteristics. It deserves to be highlighted that all the programs were prepared in such a way that they could work on Android as well as on iOS devices. Appendix Bcontains the URLs which feature the videos showing the operation of objects and where the apps can be downloaded. 2.5. Information Collection Instruments The basic tool to diagnose the degree of motivation raised by the participation in the experiment was the Instructional Material Motivational Survey (IMMS), developed by Keller (2010). According to Keller [ 4 ], motivation “refers to the magnitude and direction of behavior. In other words, it has to do with the choice made by an individual in relation to what they experience or the goals that they will try to achieve or avoid, and the degree of effort that they will make for that purpose. Motivation depends on internal looks and external characteristics. Individuals respond to their context on the basis of internal reflections, impulses, perceptions and goals, and according to the perceived base and the real opportunities, together with the reinforcement of the external context”. Appl. Sci. 2019,9, 2907 7 of 16 (Figure 6). In turn, Cheng [ 41 ] stresses that this is an internal state or condition which leads us to action, which guides us and persists in our behavior, involving us in certain activities. It also refers—in the classroom context—“to subjective experiences, particularly students’ predisposition to take part in class activities and their reasons to do so” [41]. Figure 6. Keller’s ARCS (Attention, Relevance, Confidence, Satisfaction) model (Keller, 1983). Keller used a number of different works [ 1 , 58 , 59 ] to formulate his ARCS model for the motivation applied to instructional design. He sees motivation in instructional contexts as being determined by the interaction of four dimensions: attention (A); relevance (R); confidence (C); and satisfaction (S) (http://www.arcsmodel.com/). For Keller [59], Attention includes human characteristics such as orientation reflection, curiosity and the search for sensations. Relevance refers to those things which a person perceives as a tool to meet needs and to achieve personal Satisfaction, including the fulfilment of personal objectives. Confidence comprises several motivational constructs. Finally, Satisfaction constitutes the final step that will allow us to maintain the motivation to learn. The model illustrates how “a person’s curiosity (“attention”) and the motives or values (“relevance”), combined with their hope of success (“confidence”) will determine which objectives acquire the highest prominence, consequently resulting in an intentional effort to reach the goal. Environmental influences such as teacher enthusiasm and social values, instruction quality, clear expectations and the availability of resources will also have some bearing on an objective effort and orientation towards some degree of fulfilment. “Individuals are thus influenced by their own knowledge, capacity and skills too” [ 1 ]. Therefore, this model assumes that, if the instruction strategy is considered useful to achieve the goals set, then that strategy acquires importance for students [ 60 ], and they will be likely to learn the contents and to acquire the skills. Keller’s IMMS consists of 36 items which collect information about four dimensions: attention (12 items); confidence (9 items); relevance (9 items); and satisfaction (6 items). The instrument has a Likert-type construction with seven response levels (from 1 =“Extremely unlikely/disagree with” to 7=“Extremely likely/I agree”). As stressed by Loorbach, Peters, Karreman and Steehouder [ 61 ], the model formulated by Keller “has been used on countless occasions to implement motivation strategies for instruction materials—and to test their effects. Despite being originally designed to influence students’ motivation within a traditional learning environment based on a face-to-face teacher-student interaction, this model has also been applied and thoroughly tested in contexts that require other adjustments, e.g., computer-assisted teaching and virtual or distance education”. More specifically, Keller’s model proved helpful to check the degree of motivation raised by different technological resources amongst students. Examples are: videogames [ 62 ]; the inclusion of video in e-learning learning sessions [ 63 ]; audio podcasts [64]; Massive Open Online Courses [65,66]; or AR [67–69]. Appl. Sci. 2019,9, 2907 8 of 16 We obtained the reliability index for this instrument using Cronbach’s alpha—one of the most suitable tools to obtain it with Likert-type scales [ 70 , 71 ]. Table 2provides the values obtained both for the instrument as a whole and for the different dimensions shaping it, in all three applications. Table 2. ReliabilityindexcorrespondingtotheInstructionalMaterialMotivationalSurvey(IMMS)instrument. Video Use Anatomy Art Total for the instrument 0.940 0.898 0.951 Attention 0.876 0.704 0.893 Confidence 0.780 0.582 0.900 Relevance 0.764 0.746 0.789 Satisfaction 0.821 0.775 0.843 The values reached allow us to state that the instrument has a high reliability level, both globally and in the different dimensions shaping it [ 70 , 72 ]. Only one dimension (Confidence) and one of the experiments (Anatomy) reveal values which can be considered moderate. The scores suggest high reliability indices in the other cases. As for the tool created to evaluate academic performance, it actually adopted the format of two multiple-choice tests administered under the pre-test +post-test modality. These tests included 15 items which collected information about the three following categories extracted from Bloom’s taxonomy: Apply (4 items); Remember (6 items); and Understand (5 items). Both modalities contained the same items in the pre-test and the post-test, albeit in a different presentation order. 3. Results We present the results of the average scores reached in each of the items of the instrument used. We present the total results and the results of the three experiments in Appendix C. It becomes visible in most items that the scores obtained exceed the central value of the scale (3.5), which suggests to us that the experiments were positively valued; furthermore, the items receiving the lowest scores from students are those linked to negative attitudes towards the experiment. As for the means and standard deviations obtained for the instrument as a whole, in its different dimensions, in its different applications, and in the application as a whole, see Table 3. Table 3. Means and standard deviations for the IMMS instrument and for its dimensions. Video Use Anatomy Art Total M SD M SD M SD M SD Total for the instrument 4.39 0.58 4.31 0.71 4.55 0.61 4,44 0.63 Attention 4.49 0.62 4.49 0.71 4.66 0.67 4.28 0.67 Confidence 4.16 0.62 4.16 0.77 4.24 0.63 4.33 0.67 Relevance 4.63 1.01 4.60 0.85 4.85 0.98 4.64 0.95 Satisfaction 4.58 0.69 4.60 1.06 4.83 0.69 4.53 0.81 The average scores obtained tell us that students very positively assessed their participation in this experiment, regardless of the type of AR objects produced and of the knowledge area we were dealing with. They expressed a high level of satisfaction after taking part in the interaction experiment with AR objects. Most importantly, the items which obtained slightly higher scores were the ones which combined AR and VR objects. The following two hypotheses were posed for the purpose of checking whether the pre-test and the post-test differed significantly in terms of scores obtained: Appl. Sci. 2019,9, 2907 9 of 16 • H0 (null hypothesis): There are no significant differences, with a 0.05 alpha risk of us being wrong with regard to the scores achieved by students in the pre-test and the post-test. • H1 (alternative hypothesis): There are significant differences, with a 0.05 alpha risk of us being wrong with regard to the scores achieved by students in the pre-test and the post-test. Seeking to verify H0, a Student’s statistical t-test for related samples was applied to the pre-test and post-test scores. This statistical analysis separately focused on the group of students who interacted with the AR object “Video use” and those who did so with “Anatomy”. Table 4shows the Student’s t-values reached and their significance level. After applying Student’s t-test, we obtained the values of t =16.107 for “Video use” and t =17.323 for “Anatomy”. These values allow us to reject H0 (null hypothesis) and to accept H1 (alternative hypothesis) at a significance level of p ≤ 0.001 in both cases, which ultimately leads us to conclude that students’ participation in the learning experience with AR objects proved useful to acquire the contents offered and, therefore, for them to enhance their learning. Table 4. Student’s t-values (note: ** =significant at p ≤0.01). Student’s t-Value Significance Video use 16.107 0.000 (**) Anatomy 17.323 0.000 (**) Art 14.468 0.000 (**) Pearson’s correlation coefficient served to verify whether the motivation shown by students and the levels of confidence, attention, satisfaction and relevance assessed by means of IMMS influenced the performance reached after their participation in the AR experiment. The results achieved in both experiments can be seen in Table 5. Table 5. Correlations between the performance and motivation shown by students and the levels of confidence (C), attention (A), satisfaction (S) and relevance (R), by identified t-values (note: significant at * =0.05 and **=0.01). Pearson’s Correlation Coefficient Global Attention Confidence Relevance Satisfaction Video use 0.321(**) 0.348(**) 0.289(*) 0.256(**) 0.201(*) Anatomy 0.52(**) 0.491(**) 0.323*(*) 0.321(**) 0.383(**) Art 0.432 (**) 0.398 (**) 0.315 (**) 0.2989 (**) 0.356 (**) It follows from the values obtained that: (a) the relationships between the different variables compared (performance and motivation; attention-confidence-relevance-satisfaction) are direct, i.e., if one of them increases, so does the other; (b) relationships are moderate; and (c) all of them are significant at a 0.05 or lower level. 4. Discussion and Conclusions A number of conclusions can be drawn from our study: 1. The diagnostic instrument—Keller’s IMMS—appears to be a good predictor to explain the motivation, attention, confidence, relevance and satisfaction raised by the interaction with AR objects. This is true regardless of the specific AR object that they interacted with and of the subject in which the students were enrolled. In this respect, our findings match those obtained in other works carried out by different authors [24,73–79]. 2. The instrument utilized has a high reliability index, once again showing a coincidence with the results achieved by other authors. Appl. Sci. 2019,9, 2907 16 of 16 72. Garc í a-L ó pez, L.; Guti é rrez, D.; Pastor, J.; Romo, V. Validity and Reliability of a Questionnaire on Primary and Secondary School Teachers’ Perception of Teaching a Competence-based Curriculum Model. J. N. Approaches Educ. Res. 2018,7, 46–51. [CrossRef] 73. Mateo, J. La investigaci ó n ex-post-facto. In Metodolog í a de la Investigaci ó n Educativa; Bisquerra, B., Ed.; La Muralla: Madrid, Spain, 2004; pp. 195–230. 74. Lin, T.; Been-Lirn, H.; Li, N.; Wang, H.; Tsai, C.C. An investigation of learners’ collaborative knowledge construction performances and behavior patterns in an augmented reality simulation system. Comp. Educ. 2013,68, 314–321. [CrossRef] 75. Bongiovani, P. Realidad Aumentada en la Escuela: Tecnolog í a, Experiencias e Ideas. 2013. Available online: http://www.educacontic.es/blog/realidad (accessed on 3 May 2019). 76. Chang, H.; Wu, K.; Hsu, Y. Integrating a mobile augmented reality activity to contextualize student learning of a socioscientific issue. Br. J. Educ. Technol. 2013,44, E95–E99. [CrossRef] 77. Buitrago-Pulido, R. Incidencia de la realidad aumentada sobre el estilo cognitivo: Caso para el estudio de las matemáticas. Educ. Educ. 2015,18, 27–41. [CrossRef] 78. Gazc ó n, N. Libros Aumentados: Extensi ó n del Concepto, Exploraci ó n e Interacciones. PhD Thesis, Universidad Nacional del Sur, Bahía Blanca, Argentina, 2015. 79. Mehmet, H. The classification of augmented reality books: A literature review. In Proceedings of the INTED 2016 Conference, Valencia, Spain, 7–9 March 2016; pp. 4110–4118. 80. Mar í n, V.; Cabero, J.; Gallego, O. Motivaci ó n y realidad aumentada: Alumnos como consumidores y productores de objetos de aprendizaje. Aula Abierta 2018,47, 337–346. [CrossRef] © 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).