Virtual environments and augmented reality applied to heritage education. An evaluative study

applied sciences Article Virtual Environments and Augmented Reality Applied to Heritage Education. An Evaluative Study Alex Ibañez-Etxeberria 1, Cosme J. Gómez-Carrasco 2, Olaia Fontal 3 and Silvia García-Ceballos 4,* 1Department of Didactics of Social Sciences, University of the Basque Country UPV-EHU, 20018 San Sebastián-Donostia, Spain; [email protected] 2Department of Didactics of Social Sciences, University of Murcia, 30100 Murcia, Spain; [email protected] 3Department of Didactics of Plastic Expression, University of Valladolid, 47011 Valladolid, Spain; [email protected] 4Department of Specific Didactics University of Zaragoza, 50009 Zaragoza, Spain *Correspondence: [email protected] Received: 28 February 2020; Accepted: 26 March 2020; Published: 30 March 2020   Abstract: Technological advancements have provided heritage with new learning environments via the use of virtual and augmented reality, which can foster the accessibility and understanding of culture and propose new ways of interacting with heritage. Therefore, in this study, a systematic evaluation is carried out of n=197 heritage education programs listed in the database of the Observatorio de Educaci ó n patrimonial en España (OEPE) (the Spanish Heritage Education Observatory–SHEO) which, in their descriptions, integrate the use of virtual environments and/or augmented reality to promote learning on the part of the user. The objectives of this study are: (1) to analyse the state of the art, (2) to evaluate the quality of their educational designs via the “analysis and assessment sequential method for heritage education programs” (SAEPEP-OEPE) and (3) to identify variables which can be improved or which have a significant influence on the quality of the programs. Highlights of the results include: (a) the increasing implementation of these technologies in heritage education programs, with the greater presence of virtual resources than of learning environments, (b) the low level of the scope of educational quality in their designs, particularly their assessment, and (c) the inclusion of advanced technologies slightly decreases the specificity of the educational design. Keywords: heritage education; virtual environment; augmented reality; heritage programs; evaluation 1. Introduction 1.1. Virtual Reality and Augmented Reality Applied to Heritage Education The emergence of digital and portable technology, which enables us to experience heritage in situ, provides great opportunities for heritage education, and the changing role that this type of technology provides to the transmission of knowledge must be recognised. Nowadays, the digital realm is the perfect context for heritage education [ 1 ], due to the fact that processes of knowledge, comprehension, enhancement, awareness, and enjoyment of cultural heritage take place on the internet, making it a means of identity in a universal context [ 2 , 3 ]. Therefore, several studies [ 4 , 5 ] have analysed the extent to which the digital environment democratises, educates, and socialises heritage, which has gone from being a physical place to possessing a virtual facet. Learning in these digital environments is a process which is free from restrictions; it proposes new rules, tools for communication, and contexts for interaction. In this regard, virtualisation has supposed one of the greatest impacts on the mediation of heritage, the most common use of which, up to the present moment, has been the creation of educational itineraries in historical sites [6]. Appl. Sci. 2020,10, 2352; doi:10.3390/app10072352 www.mdpi.com/journal/applsci Appl. Sci. 2020,10, 2352 2 of 20 The first uses of virtual reality (VR) applications focusing on cultural heritage in the classroom employed gamification as a learning technique [ 7 ]. Nowadays, there is an abundance of research on educational proposals for working with heritage based on games created ex profeso [ 8 ] or making use of pre-existing videogames and their historical settings, such as Assassins Creed or its educational versions, Classcraft and Minecraft Education edition [ 9 ]. In the field of heritage education, there is currently a significant increase in the presence of applications based on geolocation, augmented reality (AR), and virtual reality (VR). Some authors consider AR and VR as new tools for accessing information and that they facilitate learning processes [ 10 ], and this premise has already been addressed by some studies that raise research on how the use of these tools contributes significantly to the teaching–learning processes from different areas of knowledge [ 11 , 12 ]. In this way, if VR supplants reality and makes it possible to create inexistent settings (be they real or imaginary), AR complements it, enabling the learner to interpret, for example, archaeological remains or modified urban areas. Both techniques make it possible to develop immersive contexts which offer a greater degree of sensorial perception [ 13 ] via the recreation of environments [ 14 ] and inexistent places and constructions [ 15 ], to visualise the whole or a part of an object without damaging the original [ 16 ], to walk a route upon which virtual information is superimposed on places of interest [ 17 ], to be accompanied by a virtual guide during a route [ 18 ], and to play interactive games in which virtual and real aspects are combined [ 19 , 20 ]. All of these possibilities can be offered via specific digital platforms, virtual museums [ 21 ], and even in situ, making use of GPS [ 18 ] and QR codes [ 22 ]. In addition, the use of other devices, such as Google Glass [ 23 ] or other types of visors, have been used for this objective, albeit with the limitation that it is an added resource which the user must have at his/her disposal. At the present time, work with 3D and 360 ◦ images, the use of global positioning systems, the introduction of GoPro cameras, drones, and photogrammetry are in the vanguard of education. This new technological generation is modifying the way we relate to heritage, leading to a broadening of knowledge, an improvement in the understanding of cultural assets, a greater degree of global accessibility, and an increase in approaches and methodologies with which we can interact with heritage. These new trends can be seen in projects such as Inception, in which 3D representations are created and added to a European cultural heritage database which allows access to any user with the aim of facilitating the understanding of heritage as a relationship of unique identities to be transmitted to future generations by way of devices and innovative methodologies [ 24 , 25 ]. The Formapps project is also worthy of note, which promotes educational innovation in its digital competence via virtual spaces, networks, and devices [ 26 ]; the Ba é tica project, which creates and develops learning communities by way of multidisciplinary research projects on archaeological sites which culminates in a virtual hypothesis of the site and the 3D digitalization of the monuments [ 27 ]; the SmartMarca project, an Italian proposal for the transmission of content in AR and VR on cultural heritage, which seeks to provide students with learning methods and content construction through these applications. Moreover, the study evaluates its potential as an attractive, interactive, realistic, and close-to-the-student medium that improves the reworking and acquisition of new knowledge [ 28 , 29 ]. As far as applications are concerned, there are apps such as VirTimePlace, which allows the user to visit historical sites with the aid of a 3D visor; the online assistant Virgo, which includes a catalogue of items in 360 degrees and makes it possible to create exhibitions in three-dimensional virtual spaces; and the Smart Coolture resources, which provides the user with virtual experiences to disseminate and preserve heritage and to make it accessible. 1.2. This Study In order to identify what work is being carried out on these new relationships between heritage sites, digital applications (particularly AR and VR and virtual environments) and education processes, we evaluated a selection of cultural heritage education programs which specify the use of virtual environments or virtual or augmented reality in their educational design. The main aim of this study Appl. Sci. 2020,10, 2352 3 of 20 was to analyse the educational components of heritage education programs which make use of these emerging technologies. The relationship between heritage education and technology is gaining presence in research in Spain [ 30 ]. Led by the Observatorio de Educaci ó n Patrimonial en España (the Spanish Heritage Education Observatory—SHEO), there has been a large number of educational research projects carried out in recent years in heritage contexts with ICT resources (Information and Communication Technology). Particularly worthy of note are the analyses of the use of digital technologies in the areas of archaeology and heritage education [ 31 ], the use of ICT and contemporary heritagization processes [ 32 ], and the most recent projects regarding the study of apps [ 33 – 35 ] which follow in the footsteps of the initial projects carried out in Great Britain [ 23 ]. Over the past few years, studies on virtual heritage [ 36 ], Big Data [ 37 ] and heritage, ICT, and inclusion [ 38 ] have been carried out in other parts of the world. Although there are previous studies on the evaluation of heritage education programs, there are none which evaluate whether the fact of introducing these types of devices into the educational design supposes an alteration of their planning, or whether these types of devices are already totally integrated or if the process of implementation is still in progress. The initial hypothesis of this research suggests that we are facing an emerging process in which there is an increase in heritage educational programs that integrate virtual environments, AR and VR, which do not have a quality didactic or evaluative structure. Therefore, the specific objectives of this study are: SO1: To analyse the typology of heritage education programs which use VR, AR, or virtual environments. SO2: To evaluate the quality of the educational design of heritage education programs which specify the use of virtual environments, AR and VR in their planning. SO3: To analyse the correlations between the quality standards of these heritage education programs. SO4: To differentiate the quality of the programs according to whether they are publicly or privately owned, the specificity of the program and the category of heritage in which these digital resources are implemented. 2. Materials and Methods The approach of this study is based on the Analysis and Assessment Sequential Method for Heritage Education Programs (SAEPEP–OEPE) [ 39 ] in its most advanced phase of implementation, corresponding to the evaluation of programs based on basic standards via the application of the Q-Edutage [ 40 ] scale. This scale evaluates the level of educational quality of heritage education programs with proof of validity and reliability for the implementation of educational practices based on the design of effective actions. 2.1. Sample The selection of the study sample was extracted from the database of the Spanish Heritage Education Observatory (SHEO). This institution currently has n=2156 educational programs relating to heritage. A search of these programs was carried out based on the descriptors “virtual environment”, “VR”, and “AR”, which resulted in a valid sample of n=197 programs for this study. This sample was, in turn, analysed in two successive phases: one encompassing the whole of the sample and another comparative analysis of two subsamples selected following a discrimination of the programs in relation to the consistency of the descriptors of interest, those which appear in a partial (they are only named) or specific way (their use, objective or manner of implementation is described). These subsamples correspond to n=113 and n=84 respectively. Appl. Sci. 2020,10, 2352 4 of 20 2.2. Data Collection Tool The Q-Edutage scale is an assessment tool which forms part of the previously mentioned SAEPEP-OEPE method, designed by way of a review of literature based on quality indicators of heritage education programs and on the methodological criteria derived from the Plan Nacional de Educaci ó n y Patrimonio (National Education and Heritage Plan) [ 41 ], following the assessment model proposed by Stake [ 42 ]. The scale has previously been validated via expert judgement [ 43 ] consisting of 17 members from 9 knowledge areas, who have analysed its empirical and content validity [ 44 ] in accordance with its coherence, relevance and congruence with regard to the object of evaluation on a 4-point scale, calibrated via the item response theory (IRT) on an assessment of 330 programs [40]. The scale consists of 14 items resulting from this procedure which respond to various facets and aspects of the quality of the programs. This tool possesses a high degree of reliability and discrimination between the 4 levels of the variables [ 45 ], which makes it possible to adequately identify the quality of the programs included here. It is a brief and precise tool which does not present bias independently of the assessor applying it. The 14 items which make up the scale [ 45 ], make it possible to accurately discriminate four levels of quality via a Likert scale which contributes towards detecting outstanding programs and to optimising the rigour of the assessment and the planning of programs (“is not achieved”, “is achieved with conditions”, “is achieved”, and “is achieved with quality”). For its correct use, a rubric for assessment is used which helps to determine the scoring of the items (this can be requested from the first author). The indicators (see Table 1) respond to aspects of identification of the program, such as the contact and the descriptors which define it, the holistic conception of the heritage, if the heritage is presented in a complete and multiple way in all of its categories, the specification of the typology of the project carried out, the description of its bases or principles or the audience to which it is directed and if it has annexes; and to the key elements of an educational plan: justification, objectives, contents, methodology, resources, assessment and repercussion. Table 1. Coding of variables according to quality standards. Item Standard Coding i01 Contact information with the management and/or design team, planning and implementation. Contact i02 Descriptors that define the program. Descriptors i03 Holistic conception of heritage in its nature (material and immaterial) and its qualities (archaeological, historical, documentary, artistic, etc.). Heritage i04 Specification of the type/typology of the project developed (educational program, educational project, educational design, educational action, isolated activity, etc.). Typology i05 Description of the bases, principles, and criteria on which the program is established. Criteria i06 Specification of the target audience. Audience i07 Incorporation of documentary annexes (memory, images, videos, teaching materials, etc.). Annexes i08 Project justification. Justification i09 Description of the objectives to be achieved in the development of the program. Objectives i10 Presentation of the contents of the program. Contents i11 Methodological approach and teaching and learning strategies. Methodology i12 Definition of resources, formats, media, and technology used. Resources i13 Determination of assessment systems or tools. Evaluation i14 Evaluation of the impact and repercussion of the proposal. Impact and Repercussion The Cronbach’s alpha (Table 2) and Guttman (Table 3) statistics show that the questionnaire is suitably reliable for analysis. The criterion agreed upon by different authors is that a Cronbach’s alpha value of between 0.70 and 0.90 indicates a good internal consistency for a one-dimensional scale [ 46 , 47 ]. Appl. Sci. 2020,10, 2352 5 of 20 Table 2. Cronbach’s alpha reliability statistics. Cronbach’s alpha Standardized Cronbach’s alpha 0.91 0.91 Table 3. Guttman’s reliability statistics. Guttman L1 Guttman L2 Guttman L3 Guttman L4 Guttman L5 Guttman L6 0.84 0.91 0.91 0.95 0.89 0.92 2.3. Data Analysis As far as the procedure for data analysis is concerned, the sample and resulting subsamples were assigned to three expert assessors familiarised with the assessment rubric of the items and, in a previous stage, have gone through a registration sequence which implies searching for and locating programs, a discrimination phase for their inclusion in the database, and the later inventory via a tool with 42 fields to be completed. The data were analysed with the XLSTAT v.2019.3.2 program by way of descriptive statistic processing of the items which make up the scale. After this, Pearson correlation tests were performed, along with non-parametric tests (Mann–Whitney U test and Kruskal–Wallis) on the items and the variables “type of entity”, “type of project”, and according to the specificity of the project. This test has enabled us to check if there are any statistically significant differences among the result variables and the variables of identification of the project. 3. Results 3.1. Typology of Programs This is a heterogeneous sample originating from public (n =144; 73.09%) and private entities ( n=53 ; 26.90%), implemented internationally, which tackles different categories of heritage (see Table 4) and which encompasses 17 typologies of educational design (see Table 5). All of the programs which make up the sample deal, in part or totally, with different categories of heritage education and also reflect the use of virtual environments, VR and AR. Table 4. Frequencies obtained according to heritage category. Code Heritage category Frequency 1 Cultural 111 2 Immaterial 29 4 Natural and cultural heritage 14 6 Monuments: Architectonic works 10 7 Monuments: Archaeology 9 5 Archaeological sites 7 3 Places: created by man 6 9 Digital heritage 5 8 Monuments: Pictorial works 2 10 Monuments: Cave art 1 11 Ensembles: Isolated constructions 1 12 Monuments: Group of elements of importance 1 13 Cultural and natural 1 In relation to the typology of the educational designs which make up the sample, a greater frequency is detected for those which are defined as educational projects (21.31%), programs (18.27%), and educational designs (12.18%). This piece of data is a common denominator in all of the studies extracted from the observatory’s database, independently of the descriptors which define the search, as these suppose a high percentage of the total sample. However, the appearance of educational Appl. Sci. 2020,10, 2352 6 of 20 resources (12.18%) among the three most frequent typologies is worthy of note. This detail is a differential indicator, due to the fact that this typology does not appear in all cases. Rather, it depends on the descriptors of the search which define the sample. With the aim of contrasting these results with previous studies [ 48 , 49 ], a statistical graph was created (see Figure 1) which allows us to obtain the trends for each typology of program. In this case, a slight increase in frequency can be observed in comparison with other studies in the cases of educational resource and educational tool, both of which are teaching aids to facilitate the teaching and learning processes in educational activities and, furthermore, should ensure interaction on the part of the learner [ 10 ]. Therefore, it seems clear that using VR and AR in an attempt to bring heritage environments closer to the classroom in order to achieve existential experiences supposes an increase in these typologies. Also worthy of note is the increase in research projects which elaborate on issues of new forms of information technology (ICT) and new technology for learning and knowledge (TLK) in response to the inclusion of VR and AR in learning processes. Finally, the absence of improvement projects must be highlighted, given that in the same way as research projects, VR and AR should be one of the main objectives for achieving the advancement and improvement of the quality of education, as well as going more deeply into programs which respond to the descriptors of inclusion and attention to diversity. Table 5. Frequencies obtained according to typology of educational design. Code Typology Frequency 2 Educational project 42 1 Educational program 36 3 Educational design 24 4 Educational resource 24 9 Research project 10 5 Educational tool 9 6 Educational action 9 16 Course 9 8 Workshop 7 15 Scientific event 6 11 Competition 5 12 Educational itinerary/route/visit 5 13 Plan 4 14 Net 4 7 Isolated activity 3 10 Improvement project 0 17 Social networks 0 Appl. Sci. 2019, 9, x FOR PEER REVIEW 7 of 21 Figure 1. Frequency of actions according to typology. 3.2. Assessment of the Quality of the Programs In general, the assessment carried out by the items via the Q-Edutage scale shows an extremely low opinion of heritage education programs which use virtual environments, VR, and AR (see Table 6). Only two items were scored above an average of three points, with only one program achieving a score of between 2.5 and three points. Eleven of the fourteen items received scores below 2.5 and a median of two or one (in other words, they did not achieve a sufficient level of quality, or only did so with conditions). Table 6. Statistics of the items which make up the Q-Edutage scale. Statistic i01 i02 i03 i04 i05 i06 i07 i08 i09 i10 i11 i12 i13 i14 Number 197 197 197 197 197 197 197 197 197 197 197 197 197 197 Minimum 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 2.00 Maximum 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Median 3.00 3.00 2.00 3.00 2.00 2.00 1.00 2.00 2.00 2.00 2.00 2.00 1.00 2.00 Average 3.24 2.63 2.01 3.13 2.36 2.19 1.64 2.42 2.30 2.35 2.30 2.08 1.47 2.27 Variance (n) 0.60 0.99 0.84 0.88 1.01 0.65 1.04 0.95 0.67 1.15 1.19 1.08 0.69 0.31 Typical deviation (n) 0.77 1.00 0.92 0.94 1.01 0.81 1.02 0.98 0.82 1.07 1.09 1.04 0.83 0.56 Asymmetry (Pearson) - 0.57 - 0.10 0.49 - 0.77 0.20 0.56 1.25 0.09 0.45 0.28 0.27 0.46 1.63 1.92 The three Items with the highest scores are 1, 2, and 4 (see Figure 2). These items are mainly related to the identification of the program and the contact information. As can be seen in figures a (i01 contact), b (i02 descriptors) and c (i04 typology), the interquartile range (Q1–Q3) is situated between three and four points in Items 1 and 4 (a–c), and between two and three points in Item 2 (b). Therefore, the majority of the programs reached the level of quality required for these items of identification. Figure 1. Frequency of actions according to typology. Appl. Sci. 2020,10, 2352 7 of 20 3.2. Assessment of the Quality of the Programs In general, the assessment carried out by the items via the Q-Edutage scale shows an extremely low opinion of heritage education programs which use virtual environments, VR, and AR (see Table 6). Only two items were scored above an average of three points, with only one program achieving a score of between 2.5 and three points. Eleven of the fourteen items received scores below 2.5 and a median of two or one (in other words, they did not achieve a sufficient level of quality, or only did so with conditions). Table 6. Statistics of the items which make up the Q-Edutage scale. Statistic i01 i02 i03 i04 i05 i06 i07 i08 i09 i10 i11 i12 i13 i14 Number 197 197 197 197 197 197 197 197 197 197 197 197 197 197 Minimum 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 2.00 Maximum 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 4.00 Median 3.00 3.00 2.00 3.00 2.00 2.00 1.00 2.00 2.00 2.00 2.00 2.00 1.00 2.00 Average 3.24 2.63 2.01 3.13 2.36 2.19 1.64 2.42 2.30 2.35 2.30 2.08 1.47 2.27 Variance (n) 0.60 0.99 0.84 0.88 1.01 0.65 1.04 0.95 0.67 1.15 1.19 1.08 0.69 0.31 Typical deviation (n) 0.77 1.00 0.92 0.94 1.01 0.81 1.02 0.98 0.82 1.07 1.09 1.04 0.83 0.56 Asymmetry (Pearson) − 0.57 − 0.10 0.49 − 0.77 0.20 0.56 1.25 0.09 0.45 0.28 0.27 0.46 1.63 1.92 The three Items with the highest scores are 1, 2, and 4 (see Figure 2). These items are mainly related to the identification of the program and the contact information. As can be seen in figures a (i01 contact), b (i02 descriptors) and c (i04 typology), the interquartile range (Q1–Q3) is situated between three and four points in Items 1 and 4 (a–c), and between two and three points in Item 2 (b). Therefore, the majority of the programs reached the level of quality required for these items of identification. Appl. Sci. 2019, 9, x FOR PEER REVIEW 8 of 21 (a) (b) (c) Figure 2. The most notable box plots: contact, descriptors, and typology. The three items with the lowest scores were 3, 7, and 13: holistic conception of heritage, annexed documents, and determination of assessment systems (see Figure 3). As can be seen in figures a (i03 heritage), b (i07 annexes), and c (i13 assessment), the interquartile ranges of the scores for these programs are situated between one and two in the case of the items on annexed documentation and assessment systems, and between one and three points for the case of the holistic conception of heritage. In other words, a large proportion of these programs did not fulfil the quality criteria, or only did so with conditions. It is also important to point out that all of the items on the educational design of the program (objectives, contents, methodology, materials and resources, and evaluation) have an average score of between two and 2.5 and a median of two or one, in other words, all these items are reached with conditions that imply a very superficial design of the programs. In the assessment carried out of the programs which use virtual environments, VR, and AR, very few fulfilled the criteria for quality required in the educational design. (a) (b) (c) Figure 3. The most insufficient box plots: heritage, annexes, and assessment. 3.3. Correlations in the Assessment of the Quality of the Programs The table of Pearson correlations (see Table 7; Table 8) shows moderate to high correlation coefficients among all of the items, with the exception of Item 1, which has low correlations (between 0.17 and 0.25) with the other items. The rest maintain statistically significant coefficients as can be seen in the table of p-values. Thus, 87 of the 89 correlations have values p<0.05. Figure 2. The most notable box plots: contact, descriptors, and typology. ( a ) (i01 contact); ( b ) (i02 descriptors); (c) (i04 typology). The three items with the lowest scores were 3, 7, and 13: holistic conception of heritage, annexed documents, and determination of assessment systems (see Figure 3). As can be seen in figures a (i03 heritage), b (i07 annexes), and c (i13 assessment), the interquartile ranges of the scores for these programs are situated between one and two in the case of the items on annexed documentation and assessment systems, and between one and three points for the case of the holistic conception of heritage. In other words, a large proportion of these programs did not fulfil the quality criteria, or only did so with conditions. It is also important to point out that all of the items on the educational design of the program (objectives, contents, methodology, materials and resources, and evaluation) have an average score of between two and 2.5 and a median of two or one, in other words, all these items are reached with conditions that imply a very superficial design of the programs. In the assessment carried out of the programs which use virtual environments, VR, and AR, very few fulfilled the criteria for quality required in the educational design. Appl. Sci. 2020,10, 2352 8 of 20 Appl. Sci. 2019, 9, x FOR PEER REVIEW 8 of 21 (a) (b) (c) Figure 2. The most notable box plots: contact, descriptors, and typology. The three items with the lowest scores were 3, 7, and 13: holistic conception of heritage, annexed documents, and determination of assessment systems (see Figure 3). As can be seen in figures a (i03 heritage), b (i07 annexes), and c (i13 assessment), the interquartile ranges of the scores for these programs are situated between one and two in the case of the items on annexed documentation and assessment systems, and between one and three points for the case of the holistic conception of heritage. In other words, a large proportion of these programs did not fulfil the quality criteria, or only did so with conditions. It is also important to point out that all of the items on the educational design of the program (objectives, contents, methodology, materials and resources, and evaluation) have an average score of between two and 2.5 and a median of two or one, in other words, all these items are reached with conditions that imply a very superficial design of the programs. In the assessment carried out of the programs which use virtual environments, VR, and AR, very few fulfilled the criteria for quality required in the educational design. (a) (b) (c) Figure 3. The most insufficient box plots: heritage, annexes, and assessment. 3.3. Correlations in the Assessment of the Quality of the Programs The table of Pearson correlations (see Table 7; Table 8) shows moderate to high correlation coefficients among all of the items, with the exception of Item 1, which has low correlations (between 0.17 and 0.25) with the other items. The rest maintain statistically significant coefficients as can be seen in the table of p-values. Thus, 87 of the 89 correlations have values p<0.05. Figure 3. The most insufficient box plots: heritage, annexes, and assessment. ( a ) (i03 heritage); ( b ) (i07 annexes); (c) (i13 assessment). 3.3. Correlations in the Assessment of the Quality of the Programs The table of Pearson correlations (see Table 7; Table 8) shows moderate to high correlation coefficients among all of the items, with the exception of Item 1, which has low correlations (between 0.17 and 0.25) with the other items. The rest maintain statistically significant coefficients as can be seen in the table of p-values. Thus, 87 of the 89 correlations have values p<0.05. Table 7. Pearson coefficient correlation. Variables i01 i02 i03 i04 i05 i06 i07 i08 i09 i10 i11 i12 i13 i14 i01 1.00 i02 0.25 1.00 i03 0.25 0.41 1.00 i04 0.29 0.29 0.40 1.00 i05 0.19 0.53 0.50 0.34 1.00 i06 0.22 0.32 0.33 0.26 0.35 1.00 i07 0.19 0.39 0.40 0.25 0.53 0.36 1.00 i08 0.17 0.50 0.48 0.31 0.88 0.34 0.59 1.00 i09 0.21 0.41 0.33 0.29 0.60 0.37 0.51 0.56 1.00 i10 0.25 0.33 0.37 0.27 0.59 0.39 0.53 0.55 0.67 1.00 i11 0.27 0.36 0.40 0.33 0.59 0.46 0.50 0.59 0.56 0.53 1.00 i12 0.20 0.40 0.49 0.33 0.54 0.37 0.52 0.54 0.56 0.56 0.62 1.00 i13 0.21 0.39 0.47 0.26 0.49 0.36 0.41 0.45 0.49 0.39 0.50 0.52 1.00 i14 0.24 0.33 0.39 0.24 0.41 0.33 0.52 0.44 0.38 0.30 0.49 0.38 0.49 1.00 Table 8. P-values of the Pearson correlation table. Variables i01 i02 i03 i04 i05 i06 i07 i08 i09 i10 i11 i12 i13 i14 i01 0.00 i02 0.00 0.00 i03 0.00 <0.01 0.00 i04 <0.01 <0.01 <0.01 0.00 i05 0.07 <0.01 <0.01 <0.01 0.00 i06 0.02 <0.01 <0.01 0.00 <0.01 0.00 i07 0.07 <0.01 <0.01 0.00 <0.01 <0.01 0.00 i08 0.02 <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 0.00 i09 0.03 <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 0.00 i10 0.00 <0.01 <0.01 0.00 <0.01 <0.01 <0.01 <0.01 <0.01 0.00 i11 0.00 <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 0.00 i12 0.04 <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 0.00 i13 0.04 <0.01 <0.01 0.00 <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 0.00 i14 0.01 <0.01 <0.01 0.01 <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 <0.01 0.00 Appl. Sci. 2020,10, 2352 9 of 20 The highest correlations are among items related to the implementation of the educational design of heritage education programs (Items 9–13): the implementation of educational objectives, contents, methodology, resources, and evaluation and impact of the proposal. The correlations among these items are located between 0.50 and 0.67 and all of them have p-values <0.01. As can be seen in the image of the correlation matrix (see Figure 4), the greatest correlations arise among these items (light green). It can also be observed how Items 1 and 4 are those which present a lower correlation coefficient with the rest (these are items of identification). From these data, it is observed how the items of the educational structure (Items 9–13) find a very positive correlation, which means that regardless of the degree of scope in which they are made explicit, they are related and the proper unit of design between its parts. Appl. Sci. 2019, 9, x FOR PEER REVIEW 10 of 21 Figure 4. Image of the correlation matrix. 3.4. Hypothesis Testing The following detail the hypothesis tests that determine the differentiation between the quality of the programs based on the typology of the institution and between the specificity of the program are detailed, as they are integrated into its conception in a general or specific way the virtual environments, VR, and AR. 3.4.1. Public/Private For hypothesis testing, items on the educational design of the program were selected as, on the whole, they present a similar average (under 2.5) and high correlation coefficients. This demonstrates a high consistency between the elements which define the educational structure of the program as units of information which are designed and systematically recorded and organised [50]. In order to determine the existence of differentiated values between the publicly (73%) and privately (27%) funded programs, the items relating to the educational design were compared. As can be seen in Table 9, no big differences were found. The items with the biggest differences concern the description of the objectives and the systems or tools for evaluation, the quality of which are slightly lower in the privately funded programs. One reason for this may be the need for transparency in projects which belong to public institutions. As a consequence, the publicly funded programs include a larger quantity of information. However, as can be observed, the average differences are not high. The Mann–Whitney U test (see Figure 5) shows that this difference is not statistically significant in items which define the educational design of the program (p>0.05 in all of the items). Table 9. Descriptive statistics of the items regarding the educational design of the program according to whether it is public or private. Variable Observations Minimum Maximum Mean Typical dev. i09 | Public 144.00 1.00 4.00 2.34 0.86 i09 | Private 53.00 1.00 4.00 2.21 0.69 i10 | Public 144.00 1.00 4.00 2.35 1.09 i10 | Private 53.00 1.00 4.00 2.36 1.04 i11 | Public 144.00 1.00 4.00 2.33 1.11 i11 | Private 53.00 1.00 4.00 2.23 1.05 i12 | Public 144.00 1.00 4.00 2.03 1.04 i12 | Private 53.00 1.00 4.00 2.21 1.04 i13 | Public 144.00 1.00 4.00 1.51 0.85 i13 | Private 53.00 1.00 4.00 1.36 0.76 Figure 4. Image of the correlation matrix. 3.4. Hypothesis Testing The following detail the hypothesis tests that determine the differentiation between the quality of the programs based on the typology of the institution and between the specificity of the program are detailed, as they are integrated into its conception in a general or specific way the virtual environments, VR, and AR. 3.4.1. Public/Private For hypothesis testing, items on the educational design of the program were selected as, on the whole, they present a similar average (under 2.5) and high correlation coefficients. This demonstrates a high consistency between the elements which define the educational structure of the program as units of information which are designed and systematically recorded and organised [50]. In order to determine the existence of differentiated values between the publicly (73%) and privately (27%) funded programs, the items relating to the educational design were compared. As can be seen in Table 9, no big differences were found. The items with the biggest differences concern the description of the objectives and the systems or tools for evaluation, the quality of which are slightly lower in the privately funded programs. One reason for this may be the need for transparency in projects which belong to public institutions. As a consequence, the publicly funded programs include a larger quantity of information. However, as can be observed, the average differences are not high. The Mann–Whitney U test (see Figure 5) shows that this difference is not statistically significant in items which define the educational design of the program (p>0.05 in all of the items). Appl. Sci. 2020,10, 2352 16 of 20 these concepts are not clear for those who design the proposals, given that “augmented reality” (AR) enables the users to visualise elements or spaces via a digital device in the real world [ 17 ] and “virtual reality” (VR) transports the user to a projected context which may be immersive [ 14 ] via physical accessories such as glasses or headphones. In turn, we consider that this analogy in terms can be extended in society as it is an emerging technology [59,60]. After analysing the possibilities that the projects show with the implementation of virtual technologies, certain aspects stand out: the elimination of barriers of space and time, facilitation in the exchange of experiences among users, cooperation and collaboration among agents, the opening of new paths for transmission and communication, an increase in motivation, the development of attitudes of awareness and respect towards the role of heritage, the appearance of new ways of approaching heritage and educational experiences, and new forms of interaction. Last of all, one aspect which is repeated in many cases must be highlighted: the inclusion provided by new technologies in promoting the understanding of cultural assets and the global accessibility to heritage environments, which, in contrast, presents a low manifestation in the “accessibility” variable, which is only explicit in 7.6% of cases. The presence of adapted material and human resources in the programs is almost non-existent, which is one aspect for imminent improvement, both in their development and in their presence in the educational design, an indicator which has already been detected and which does not show any progress. From this analysis, certain unavoidable necessities can also be extracted which authorities and institutions must bear in mind if they are to play an active role in the management, support, and financing of these applications to promote their implementation for the benefit of teaching: that (a) develop policies for incorporation, application, and development of virtual environments to heritage cultural; (b) promote research initiatives and projects on the effects of the use of virtual environments, VR and AR in the dissemination of cultural heritage; and (c) boost research on models, interactive designs and virtual prototypes to allow knowledge of heritage to be possible for a wide spectrum of the population. Finally, it should be pointed out that some of the conclusions included here were drawn from variables which were not dealt with in a detailed manner during this study, but which were observed during the prior analysis of the programs and were deemed relevant. Among them, we can highlight the audiences which the majority of the programs are aimed at, given that the implementation of these technologies is closely related to digital natives. The biggest range of target users consists of the stages of compulsory education, particularly young people of between 14 and 20 years of age: compulsory secondary education and baccalaureate (25.9%), primary education (19.8%), and “all audiences” including those mentioned previously (25.9%). This high percentage is a clear indicator of the educational context in which these virtual learning environments and technological resources are developed. Programs which do not provide data on the community, population, or audience which the project is aimed at (16.7%) were excluded from the sample and the remaining percentage refers to specific audiences of different kinds (11.7%). The final conclusion to be drawn alludes to the category of heritage which the programs refer to (see Table 4). In this regard, according to their frequencies, 56.3% of the programs are aimed at cultural heritage. Although it is true that the majority of the programs deal with historical, cultural, and artistic heritage, this aspect is a common denominator in the different assessment studies, the main cause of which can be considered to be that a high number of institutions use the term cultural heritage generically as a container for different categories and kinds of heritage. To conclude, following the analysis and the review of literature, it can be stated that these cutting-edge technologies are extremely costly to perfect and implement; this is currently an embryonic field whose funds are mainly assigned to the documentation, reconstruction, restoration, and dissemination of heritage [ 61 ]. In the field of education, there are many more steps to take, but this research has enabled us to define the state of the art, to identify variables, to identify the quality of the educational proposals Appl. Sci. 2020,10, 2352 17 of 20 which currently exist, to extract the main advantages of the use of virtual environments (VR and AR), and to unravel key necessities with the perspective of educational improvement. Author Contributions: Conceptualization, C.J.G.-C.; Funding acquisition, A.I.-E. and C.J.G.-C.; Investigation, O.F.; Methodology, S.G.-C.; Project administration, O.F.; Resources, O.F.; Supervision, A.I.-E.; Visualization, S.G.-C.; Writing—original draft, C.J.G.-C. and S.G.-C.; Writing—review & editing, A.I.-E. All authors have read and agreed to the published version of the manuscript. Funding: This research was carried out with the aid of funding from the Spanish Ministry of Science, Innovation and Universities, and was co-funded by ERDF funds from the European Union for the research projects EDU2015-65716-C2-1-R, EDU2015-65716-C2-2-R, and PGC2018-094491-B-C33. It has also been made possible by funding from the Fundaci ó n S é neca of the Region of Murcia for the project 20638/JLI/18 and the Education Department of the Regional Government of the Basque Country (IT1193-19). Conflicts of Interest: The authors declare no conflict of interest. The funders had no role in the design of the study, in the collection, analyses, or interpretation of data, in the writing of the manuscript, or in the decision to publish the results. References 1. Pineiro, V.; Jose, J.; Rodr í guez, I. Identity-related implications of the dissemination of cultural heritage through the Internet: A study based on Framing Theory. Commun. Soc. 2018,31, 1–21. 2. Feldman, J. Memorylands: Heritage and Identity in Europe Today. Am. Ethnol. 2017 ,44, 145–146. [CrossRef] 3. Chng, K.S.; Narayanan, S. Culture and social identity in preserving cultural heritage: An experimental study. Int. J. Soc. Econ. 2017,44, 1078–1091. [CrossRef] 4. Enhuber, M. Art, space and technology: How the digitisation and digitalisation of art space affect the consumption of art—A critical approach. Digit. Creat. 2015,26, 121–137. [CrossRef] 5. Osuna, S.; López, J. Educommunicative evaluation model in virtual education. Opción2015,31, 832–853. 6. Ib á ñez-Etxeberria, A.; Kortabitarte, A. Apps, Redes Sociales Y Dispositivos M ó viles En Educaci ó n Patrimonial. 2006. Available online: http://www.culturaydeporte.gob.es/planes-nacionales/dam/jcr:4cc8d5a2-3084-409e- a134-b46b62f7126e/apps-y-socialmedia-en-educacionpatrimonial.pdf (accessed on 13 March 2019). 7. Chrast, O.; Beseda, J. New technologies and new ICT and gamification-based approaches for the cultural heritage education and comparison of their use in some Czech Republic’s cultural heritage sites, cultural monuments and virtual museums. In Proceedings of the DISCO, 13 International Conference on Overcoming the Challenges and Barriers in Open Education, Prague, Czech Republic, 25–26 June 2018; pp. 401–411. 8. Grammatikopoulou, A.; Laraba, S.; Sahbenderoglu, O.; Dimitropoulos, K.; Douka, S.; Grammalidis, N. An adaptive framework for the creation of exergames for intangible cultural heritage (ICH) education. J. Comput. Educ. 2019,6, 417–450. [CrossRef] 9. Garc í a-Fern á ndez, J.; Mateus, L. Solution supporting the communication of the built heritage: Semi-automatic production path to transfer semantic LIDAR data to Minecraft environment. Digit. Appl. Archaeol. Cult. Herit. 2019,14, e00112. [CrossRef] 10. Barrado-Tim ó n, D.A.; Hidalgo-Giralt, C. The Historical City, Its Transmission and Perception via Augmented Reality and Virtual Reality and the Use of the Past as a Resource for the Present: A New Era for Urban Cultural Heritage and Tourism? Sustainability 2019,11, 2835. [CrossRef] 11. McGovern, E.; Moreira, G.; Luna-Nevarez, C. An application of virtual reality in education: Can this technology enhance the quality of students’ learning experience? J. Educ. Bus. 2019, 1–7. [CrossRef] 12. Rivas, Y.C.; Valdivieso, P.A.V.; Rodr í guez, M.A.Y. Virtual reality and 21st century education. Int. Res. J. Manag. IT Soc. Sci. 2020,7, 37–44. 13. Rivero, P.; Feliu, M. Aplicaciones de la arqueolog í a virtual para la Educaci ó n Patrimonial: An á lisis de tendencias e investigaciones. Estudios Pedagógicos 2017,43, 319–330. [CrossRef] 14. Zhang, Y.; Han, M.; Chen, W. The strategy of digital scenic area planning from the perspective of intangible cultural heritage protection. EURASIP J. Image Video Process. 2018,1, 130. [CrossRef] 15. Petrucco, C.; Agostini, D. Teaching our cultural heritage using mobile augmented reality. J. E Learn. Knowl. Soc. 2016,12, 115–128. Appl. Sci. 2020,10, 2352 18 of 20 16. Chang, K.; Chang, C.; Hou, H.; Sung, Y.; Chao, H.; Lee, C. Development and behavioral pattern analysis of a mobile guide system with augmented reality for painting appreciation instruction in an art museum. Comput. Educ. 2014,71, 185–197. [CrossRef] 17. Furata, H.; Takahashi, K.; Nakatsu, K.; Ishibashi, K.; Aira, M. A mobile application system for sightseeing guidance using augmented reality. In Proceedings of the 6th International Conference on Soft Computing and Intelligent Systems and the 13th International Symposium on Advanced Intelligence Systems, Kobe, Japan, 20–24 November 2012. 18. Chatzidimitris, T.; Kavakli, E.; Economou, M.; Gavalas, D. Mobile Augmented Reality edutainment applications for cultural institutions. In Proceedings of the 4th International Conference on Information, Intelligence, Systems and Applications—IISA 2013, Mikrolimano, Greece, 10–12 July 2013. 19. Angelopoulou, A.; Economou, D.; Bouki, V.; Psarrou, A.; Jin, L.; Pritchard, C.; Kolyda, F. Mobile Augmented Reality for Cultural Heritage. In Mobile Wireless Middleware, Operating Systems, and Applications, Proceedings of the 4th International ICST Conference, Mobilware, London, UK, 22–24 June 2011, 1st ed.; Venkatasubramanian, N., Getov, V., Steglich, S., Eds.; Springer: New York, NY, USA, 2011; pp. 15–22. 20. Perra, C.; Grigoriou, E.; Liotta, A.; Song, W.; Usai, C.; Giusto, D. Augmented reality for cultural heritage education. In Proceedings of the 2019 IEEE 9th International Conference on Consumer Electronics, ICCE, Berlin, Germany, 8–11 September 2019; pp. 333–336. 21. Cozzani, G.; Pozzi, F.; Dagnino, F.M.; Katos, A.V.; Katsouli, E.F. Innovative technologies for intangible cultural heritage education and preservation: The case of i-Treasures. Pers. Ubiquitous Comput. 2017 ,21, 253–265. [CrossRef] 22. Arias-Espinoza, P.; Medina-Carri ó n, A.; Robles-Bykbaev, V.; Robles-Bykbaev, Y.; Pes á ntez-Avil é s, F.; Ortega, J.; Matute, D.; Rold á n-Monsalve, V. e-Pumapunku: An Interactive App to Teach Children the Cañari and Inca Indigenous Cultures During Guided Museum Visits. In Proceedings of the 2018 Congreso Internacional De Innovación Y Tendencias En Ingeniería (CONIITI), Bogota, Colombia, 3–5 October 2018; pp. 1–5. 23. Economou, M.; Meintani, E. Promising beginnings? Evaluating museum mobile phone apps. In Proceedings of the Conference Given in the Congress ‘Re-Thinking Technology in Museums, Emerging Experiences’, University of Limerick, Limerick, Ireland, 26–27 May 2011; Available online: http://eprints.gla.ac.uk/104173/ (accessed on 13 March 2019). 24. Maietti, F.; Di Giulio, R.; Medici, M.; Ferrari, F.; Ziri, A.E.; Turillazzi, B.; Bonsma, P. Documentation, Processing, and Representation of Architectural Heritage through 3D Semantic Modelling: The INCEPTION Project. In Impact of Industry 4.0 on Architecture and Cultural Heritage; IGI Global: Hershey, PA, USA, 2020; pp. 202–238. 25. Maietti, F.; Piaia, E.; Mincolelli, G.; Di Giulio, R.; Imbesi, S.; Marchi, M.; Giacobone, G.A.; Brunoro, S. Accessing and Understanding Cultural Heritage through Users Experience within the INCEPTION Project. In Digital Heritage. Progress in Cultural Heritage: Documentation, Preservation, and Protection: EuroMed. Lecture Notes in Computer Science, Nicosia, Cyprus, 29 October–3 November 2018; Ioannides, M., Fink, E., Brumana, R., Patias, P., Doulamis, A., Martins, J., Wallace, M., Eds.; Springer: Cham, Switzerland, 2018; Volume 11196. 26. BOCyL. ORDEN EDU/832/2015, De 2 De Octubre, Por la que se regula el proyecto denominado “FORMapps” para la implantaci ó n de nuevas metodolog í as de formaci ó n permanente del profesorado en su competencia digital, en centros educativos sostenidoscon fondos p ú blicos de la comunidad de castilla y le ó n 2015. Available online: http://bocyl.jcyl.es/boletines/2015/10/14/pdf/BOCYL-D-14102015-1.pdf (accessed on 20 September 2019). 27. Grande, A.; de Andaluc í a, J. Proyecto Baetica, Programa Educativo Andaluz 2006. Available online: http://www.culturaclasica.com/files/proyecto_baetica.pdf (accessed on 20 September 2019). 28. Pierdicca, R.; Frontoni, E.; Puggioni, M.P.; Malinverni, E.S.; Paolanti, M. Evaluating Augmented and Virtual Reality in Education through a User-Centered Comparative Study: SmartMarca Project. In Virtual and Augmented Reality in Education, Art, and Museums; IGI Global: Hershey, PA, USA, 2020; pp. 229–261. [CrossRef] 29. Frontoni, E.; Paolanti, M.; Puggioni, M.; Pierdicca, R.; Sasso, M. Measuring and Assessing Augmented Reality Potential for Educational Purposes: SmartMarca Project. In International Conference on Augmented Reality, Virtual Reality and Computer Graphics; Springer: Cham, Switzerland, 2019; pp. 319–334. 30. Ib á ñez-Etxeberria, A.; Fontal, O.; Rivero, P. Educaci ó n Patrimonial y TIC en España: Marco normativo, variables estructurantes y programas referentes. Arbor 2018,194, 448. [CrossRef] 31. Vicent, N.; Rivero, P.; Feliu, M. Arqueolog í a y tecnolog í as digitales en Educaci ó n Patrimonial. Educatio Siglo XXI 2015,33, 83–102. [CrossRef] Appl. Sci. 2020,10, 2352 19 of 20 32. G ó mez-Redondo, C.; Fontal, O.; Ib á ñez-Etxeberria, A. Procesos de patrimonializaci ó n e identizaci ó n patrimonial con uso de TIC en torno al arte contemporáneo. Artnodes 2016,17, 52–63. 33. Ib á ñez-Etxeberria, A.; Kortabitarte, A.; De Castro, P.; Gillate, I. Digital competence using heritage theme apps in the DigComp framework. Revista Electr ó nica Interuniversitaria De Formaci ó n Del Profesorado 2019 ,22, 13–27. [CrossRef] 34. Luna, Ú .; Ib á ñez Etxeberr í a, A.; Rivero, P. El patrimonio aumentado. 8 apps de Realidad Aumentada para la enseñanza-aprendizaje del patrimonio (The augmented heritage. 8 apps of Augmented Reality for the heritage teaching-learning). Revista Interuniversitaria De Formacion Del Profesorado 2019,94, 43–62. 35. Luna, U.; Rivero, P.; Vicent, N. Augmented reality in heritage apps: Current trends in Europe. Appl. Sci. 2019,9, 2756. [CrossRef] 36. Ibrahim, N.; Ali, N.M.; Yatim, N.F.M. Factors facilitating cultural learning in virtual architectural heritage environments: End user perspective. J. Comput. Cult. Herit. 2015,8, 1–20. [CrossRef] 37. Romanelli, M. Museums creating value and developing intellectual capital by technology: From virtual environments to Big Data. Meditari Account. Res. 2018,26, 483–498. [CrossRef] 38. Kosmas, P.; Galanakis, G.; Constantinou, V.; Drossis, G.; Christofi, M.; Klironomos, I.; Zaphiris, P.; Antona, M.; Stephanidis, C. Enhancing accessibility in cultural heritage environments: Considerations for social computing. Univers. Access Inf. Soc. 2019, 1–12. [CrossRef] 39. Fontal, O. The Spanish heritage education observatory/El observatorio de educaci ó n patrimonial en España. Cult. Educ. 2016,28, 254–266. [CrossRef] 40. Fontal, O.; Garc í a-Ceballos, S.; Arias Mart í nez, B.; Arias Gonz á lez, V. Assessing the Quality of Heritage Education Programs: Construction and Calibration of the Q-Edutage Scale. Revista De Psicodid á ctica 2019 ,24, 31–38. [CrossRef] 41. Ministerio de Educaci ó n, Cultura y Deporte Plan Nacional De Educaci ó n Y Patrimonio. Available online: https://sede.educacion.gob.es/publiventa/d/20704C/19/0(accessed on 27 January 2020). 42. Stake, R. Evaluación Comprensiva. Evaluación Basada En Estándares; Ed. Grao: Barcelona, Spain, 2006. 43. Escobar-P é rez, J.; Cuervo-Mart í nez, A. Validez de contenido y juicio de expertos: Una aproximaci ó n a su utilización. Avances En Medición2008,6, 27–36. 44. Hern á ndez-Sampieri, R.H.; Fern á ndez-Collado, C.; Baptista-Lucio, P. Metodolog í a De La Investigaci ó n; McGraw-Hill: Mexico City, Mexico, 2003. 45. Fontal, O.; Garc í a-Ceballos, S. Evaluaci ó n de programas de Educaci ó n Patrimonial: Est á ndares de calidad. ENSAYOS, Revista De La Facultad De Educaci ó n De Albacete 2019 ,34, 1–15. Available online: http://www. revista.uclm.es/index.php/ensayos (accessed on 16 April 2019). 46. Gonz á lez, J.; Pazmiño, M. C á lculo e interpretaci ó n del Alfa de Cronbach para el caso de validaci ó n de la consistencia interna de un cuestionario, con dos posibles escalas tipo Likert. Revista Publicando 2015 ,2, 62–77. 47. Oviedo, H.C.; Campo-Arias, A. Aproximaci ó n al uso del coeficiente alfa de Cronbach; An Approach to the Use of Cronbach’s Alfa. Revista Colombiana De Psiquiatría2005,34, 572–580. 48. Mar í n-Cepeda, S.; Garc í a-Ceballos, S.; Vicent, N.; Gillate, I.; G ó mez-Redondo, C. Educaci ó n Patrimonial Inclusiva en OEPE: Un estudio prospectivo. Educación2017,375, 110–135. 49. Fontal, O.; Mart í nez-Rodr í guez, M. Evaluaci ó n de programas educativos sobre Patrimonio Cultural Inmaterial. Estudios Pedagógicos 2018,43, 69–89. [CrossRef] 50. P é rez Juste, R. La evaluaci ó n de programas educativos Conceptos b á sicos, planteamientos generales y problemática. Revista De Investigación Educativa 2000,18, 261–287. 51. Alfonso, I. Elementos conceptuales b á sicos del proceso de enseñanza-aprendizaje. ACIMED 2003 ,11, 16. Available online: http://bvs.sld.cu/revistas/aci/vol11_6_03/aci17603.htm (accessed on 23 April 2019). 52. Rivero, P.; Fontal, O.; Garc í a-Ceballos, S.; Mart í nez-Rodr í guez, M. A Model for Heritage Education through Archaeological Sites: The Case of the Roman City of Bilbilis. Curator 2018,61, 315–326. [CrossRef] 53. Cuenca, J.M. El Patrimonio En La Did á ctica De Las Ciencias Sociales: An á lisis De Concepciones, Dificultades Y Obst á culos Para Su Integraci ó n En La Enseñanza Obligatoria. Ph.D. Thesis, University of Huelva, Huelva, Spain, 2002. 54. Desantes, M. Hacia una visión holística del Patrimonio Cultural. RIIPAC 2013,3, 1–22. 55. Heyd, T. Nature, culture, and natural heritage: Toward a culture of nature. Environ. Ethics 2005 ,27, 339–354. [CrossRef] 56. Fontal, O.; Ib á ñez-Etxeberria, A. Strategies and Tools for Heritage Education in Spain. Educatio Siglo XXI 2015,33, 15–32. Appl. Sci. 2020,10, 2352 20 of 20 57. Fontal, O. Educaci ó n patrimonial: Retrospectiva y prospectivas para la pr ó xima d é cada. Estudios Pedag ó gicos 2016,42, 415–436. [CrossRef] 58. G ó mez-Redondo, C.; Calaf, R.; Fontal, O. Design of an instrument of analysis for heritage educational resources. Cadmo 2017,1, 63–80. [CrossRef] 59. Carceller, I. La realidad aumentada como herramienta de enriquecimiento del proceso de aprendizaje. Edetania. Estudios Y Propuestas Socioeducativas 2020,56, 169–184. 60. Jung, T.; tom Dieck, M.C.; Lee, H.; Chung, N. Effects of Virtual Reality and Augmented Reality on Visitor Experiences in Museum. In Information and Communication Technologies in Tourism; Inversini, A., Schegg, R., Eds.; Springer: Cham, Switzerland, 2016. 61. Castro Calviño, L.; L ó pez-Facal, R. Heritage education: Felt needs of the teachers of preschool, primary and secondary education. RIFOP 2019,94, 97–114. © 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).