Development of the Teacher Digital Competence Validation of DigCompEdu Check-In Questionnaire in the University Context of Andalusia (Spain)

sustainability Article Development of the Teacher Digital Competence Validation of DigCompEdu Check-In Questionnaire in the University Context of Andalusia (Spain) Julio Cabero-Almenara , Juan-Jesús Gutiérrez-Castillo * , Antonio Palacios-Rodríguez and Julio Barroso-Osuna Department of Teaching and Educational Organization, University of Seville, 41013 Seville, Spain; [email protected] (J.C.-A.); apr[email protected] (A.P.-R.); [email protected] (J.B.-O.) *Correspondence: [email protected] Received: 19 June 2020; Accepted: 22 July 2020; Published: 29 July 2020   Abstract: We are currently witnessing a moment in history in which sustainable education practices are being principally modified by the proliferation of technologies and their wider use in every level of society, which makes necessary their integration into education contexts. This is found in the crosshairs of different institutions, which propose a series of competency frameworks, such as DigCompEdu. This framework provides suggestions for the competences that educators should be trained on under pedagogic-didactic criteria. The present work intends to measure the reliability and validity of the questionnaire DigCompEdu Check-In with the participation of 2262 professors from different public Andalusian universities. The known-groups method was utilized to elucidate if the tool is able or not to discriminate different variables of interest between known groups. The study reveals that the instrument has high indices of reliability, globally, and in the different dimensions that comprise it. Furthermore, it verifies that the instrument is sufficiently robust to discriminate the subjects who are clearly differentiated by variables related with technology. Along this line, the recommendation is given to continue working on the creation of new instruments focused on the mastery of this competency. Keywords: digital competence; questionnaires; validation; pre-service professors; DigCompEdu 1. Introduction Information and communication technologies (ICT) have transformed the practices of literacy, earning a great importance in the functioning of the contexts of the so-called Society of Knowledge. However, being immersed in a digital society does not ensure the same opportunities related to access and use for all citizens [ 1 ]. Therefore, it is necessary for the population to master its languages and to possess a multiple literacy and communication competency. The founding Treaty of the European Union ensures that “the Community will contribute with the development of high-quality education, fomenting the cooperation between Member States” [ 2 ] (p. 66). In one of the recommendations given to the member states, the European Council of Lisbon proposed the creation of a new European framework of reference to “define the new basic qualifications that permanent learning should provide” [ 3 ] (p. 11). This measure is considered essential for providing a response to the globalization phenomenon, transforming the socio-economic panorama with policies based on the Society of Knowledge. Along this line, the European Council of Stockholm recommends, as the main objective, an “improvement of the basic skills, especially in digital subjects and information technology” [ 4 ] (p. 4). This priority also highlights the need to improve education policies based on the permanent competence building. Sustainability 2020,12, 6094; doi:10.3390/su12156094 www.mdpi.com/journal/sustainability Sustainability 2020,12, 6094 2 of 14 European policies are aimed at providing new basic skills, underlining the fundamental role of digital technology for the permanent training of the population [ 5 ] and the importance of integrating it into the world of work [ 6 ]. For this, the so-called points of reference are adopted, which are the basis for the development of key skills for life-long learning [7]. The eight key competences for life-long learning are defined in the EU as “a combination of knowledge, skills and attitudes that are adequate to the context” [ 8 ] (p. 7) and provide a common reference framework destined to the decision makers, educators, social partners and the students themselves. Ultimately, these are the set of knowledge, skills and strategies needed by a person to solve any obstacle found in the Society of Knowledge itself. One of these key competences is the digital competence, understood as that which “involves the confident, critical and responsible use of the technologies from the society of information for work, entertainment and education” [ 8 ] (p. 9). The ability to use technology to live, work and learn throughout life is considered a fundamental and transversal subject in light of the development of any community education program. From this point of view, the importance of taking advantage and integrating the technology in an efficient manner in the training/teaching centers should be highlighted. Likewise, the proposal is made for the development of international education plans that aim to develop the professional skills of the general population and the agents in charge of their training [ 9 ]. Altogether, the European Strategy 2020 highlights the key enabling role that the ICT should play within the Society of Information [10]. The above suggests that the existing relationship between the ICT and competency and education policies is becoming ever more important for a society found within the Society of Knowledge. For this reason, and beginning with the previous references, the need is justified to count with useful instruments that are current, valid and reliable for the study of digital competence. Furthermore, these tools must be adapted to the contexts where they are utilized [ 11 ]. Thus, the objective of the present work is to validate the self-diagnosis questionnaire “DigCompEdu Check-In” in the higher education context. 2. Teachers Digital Competence Education institutions are currently facing the challenge of seeking to innovate ways of conducting their education work, especially in light of the recent technological, economic and social changes that are currently taking place [ 12 ], with the aim of training and educating new generations for an uncertain and disconcerting future. Never in history has humanity had so many ICTs available as in the present. Technologies that are duplicated at great speeds thanks to digitalization, resulting in transmedia, i.e., the viewing of media content in different devices. This fact multiplies the manners in which people can interact with information [ 13 , 14 ]. This problem is in constant evolution, just as the development of technology [ 15 ]. Thus, the development of competences should be perceived within the idea of lifelong learning. Furthermore, the forecasts indicatethat inthe“FourthIndustrialRevolution”, withinwhichwefind ourselves in, most of the future jobs will require digital skills [ 16 ]. For this reason, and without forgetting that this same technological penetration entails the so-called technological unemployment, the only way to overcome it is through the development of competences related with digital technology [ 17 , 18 ]. As a result, the development of digital skills, inherent to our society, are directly related with success, employability, creativity and the prosperity of each individual. Within the professional competences of the teacher, different authors allude to those skills or abilities related with the use of technological tools for conducting their professional activity in the classroom, which are different in many aspects to the use that could be given to them in the household environment [ 19 ]. The modern education institutions require the addition of technologies from the society in which they operate, which at the moment are digital in nature. This demands that the teachers possess a significant Teachers Digital Competence (TDC) for the mastery of the Information and Communication Technologies (ICT) and their integration into the teaching and learning processes [ 20 , 21 ]. This competence is understood as a set of capacities, abilities, knowledge or skills that Sustainability 2020,12, 6094 3 of 14 teachers possess to solve educational problems by integrating ICT [ 20 ]. At the same time, the mastery of the TDC empowers the teacher for the use of the ICT not only as support for their existing practices, but also to transform them [22]. Some factors are presented as mobilizing variables of the TDC, among which we will discard some, especially having in mind their significance when we use them in the experimental part of the work: 1. Training of the teacher: the teacher’s work experience, initial training and the degree of knowledge of the ICT tools are a fundamental factor for the development of TDC [23–25]. 2. Resources: quality of the infrastructure and availability of the digital devices and technologies necessary [26]. There are teachers who assert their pre-disposition for integrating ICT resources into the teaching-learning practices if they had the necessary means [27]. 3. Usage time: dedication to the use of the ICT in and out of the classroom as an element that favors the teacher’s digital competence. Likewise, the lack of time available to prepare the sessions through the use of technologies is an opposing element [28]. 4. Attitude towards technology: the attitudes and beliefs the teacher has with respect to the possibilities provided by the ICT for teaching are critical variables that will determine the addition of the ICT to the teaching practice of the teacher, and not only their addition but also the manner in which they are introduced and the functions assigned to them [ 29 – 33 ]. This attitude is perceived in the teacher’s use of certain technologies such as social networks [30,34]. At the same time, the use of technologies in the classroom has been promoted by education policies and didactic and pedagogic trends [35]. Ultimately, the TDC is related with all of the skills, attitudes and knowledge required by the teachers in a digital world [ 36 ]. Altogether, it is related with the use of the ICT from a didactic-pedagogic perspective in a professional education context which has an effect on the teaching strategies that are directly or indirectly related with technology [ 37 ]. Furthermore, it has become an important line of research within the field of Education Technology [38]. For the acquisition of the teacher’s digital competence, a series of competence frameworks have been suggested at the institutional level from different institutions, on which teachers should be trained. All of these seek to discover in which manner the Technologies should be integrated and used in teaching, to identify the training needs and to propose personalize training itineraries [ 39 ]. Furthermore, in agreement with comments by different authors [ 40 – 45 ], the following can be considered the most consolidated: the European Digital Competence Framework for Educators, DigCompEdu; ISTE standards for Educators; the UNESCO ICT Competency Framework for Teachers; the Spanish Common Framework of Teacher’s Digital Competence; the British Digital Teaching Professional Framework; the Colombian ICT Competencies for the professional development of teachers; the Chilean ICT Competencies and Standards for the teaching profession. Altogether, an evaluation study through the use of expert competence coefficient highlighted the DigCompEdu model as the most adequate for evaluating the Teachers Digital Competence [42]. 3. The DigCompEdu Framework At the end of 2017, the Joint Research Centre (JRC) published the European Digital Competence Framework for Educators: DigCompEdu [46]. The DigCompEdu framework is the result of different professional meetings, workshops, debates and deliberations with experts and professionals. It is based on an initial bibliographic review and it entails the synthesis of existing instruments at the local, national, European and international levels [ 31 , 46 ]. The result implies a consensus on the main areas and elements of digital competence of teachers, and follows a logical progression in each competency area. Sustainability 2020,12, 6094 4 of 14 It is model of digital competence for teachers. Its competence areas are: 1. Professional engagement: it is focused on the work environment of the teachers. The digital competence of the teachers is expressed in their ability to use the digital technologies to improve teaching, butalsotoprofessionallyinteractwith peers, students, familyanddifferentagentfromthe education community. This area is also focused in digital continuous professional development. 2. Digital resources: related with the sources, creation and distribution of digital resources. One of the key competences that any teacher should develop is the identification of education resources. Also, the teachers should be able to modify, create and share them so that they fit with their objectives, students and style of teaching. At the same time, the teachers should be able to responsibly use and administer the digital content, respecting the guidelines of copyrights and protecting personal data. 3. Teaching and Learning: know how to design, plan and implement the use of digital technologies in the different stages of the process of teaching and learning. Also, a change of approaches and methodologies that are centered on the students is advocated. 4. Assessment: linked to the use of digital tools and strategies in the evaluation and improvement of the processes of teaching-learning. The digital technologies can improve the existing strategies of assessment and give way to new and improved evaluation methods. 5. Empowering learners: one of the key strengths of the digital technologies in education is their potential to promote the active participation of the students in the process of learning and their autonomy within it. Also, the digital technologies can be utilized to offer learning activities adapted to the level of competency for each student, their learning interests and needs. 6. Facilitating learners’ digital competence: to develop and facilitate digital competence of the students. This proposal is found within the “DigComp Initiatives”, and contributes to the “Competences for Europe Agenda” of the European Commission [ 8 , 47 ] and the Initiative of the European Strategy 2020 on New Skills for New Jobs [ 15 , 48 ]. The DigCompEdu framework is also used as a reference in some programs from the Autonomous regions of Spain for the digitalization of Spanish education centers, such as the Andalusian project #PRODIG [ 49 ]. At the same time, DigCompEdu is compatible with the Common Digital Competence Framework for Teachers, published by the INTEF, in its different dimensions and levels [ 50 , 51 ]. Finally, DigCompEdu shares characteristics with other frameworks used internationally. Table 1compares with other frameworks. Table 1. Comparison between international Digital Competence Frameworks for Educators. Framework Technological Competences Pedagogical Competences European Digital Competence Framework for Educators DigCompEdu Professional Engagement Teaching and Learning Digital resources Assessment Empowering learners Facilitating learners’ digital competence ISTE Standards for Educators Designers Apprentices Analysts Leaders Citizens Collaborators Facilitators UNESCO Framework of ICT Competence for Teachers Organization and administration Understanding ICT in educational policy Professional learning Curriculum and evaluation Pedagogy Digital skills application Sustainability 2020,12, 6094 5 of 14 Table 1. Cont. Framework Technological Competences Pedagogical Competences Spanish Common Framework of Digital Teacher Competence Information and information literacy Communication and collaboration Digital content creation Problem solving Security British Framework for Digital Education Specific Teaching Pedagogical Planning Self-development Pedagogical Approach Student Employability Evaluation Accessibility and Inclusion ICT competences for the professional development of Colombian teachers Technological Pedagogical Communicative Management Investigation ICT Competencies and Standards for Chilean teachers Technical or Instrumental Pedagogical Management Social, Ethical and Legal Development and Responsibility Professional It can be seen how almost all the frameworks have levels of competence, those that contemplate the progressive development of the competences. In this case, all of these classify these levels into three intervals (basic, intermediate and advanced). Regarding the dimensions and competences, all the frameworks contemplate technological and pedagogical competences. Technological: those professional competences that every teacher must develop within an educational institution committed to the Knowledge Society. Pedagogical: those competences directly linked to the teaching, learning and development processes of student citizenship. Along the line of similar studies, where questionnaire-type tools were developed, based on the self-perception of the teacher and utilizing a Likert-type scale [ 48 , 52 ], DigCompEdu is utilized as the basis for the development of an online self-perception tool named DigCompEdu Check-In, which is considered to be the main object of study in this article. 4. Methodology This cross-sectional research study intends to calculate the reliability and validity of the questionnaire DigCompEdu Check-In in the University Context of Andalusia (Spain). For this, the known-groups method is employed, replicating the procedure utilized by Ghomi and Redecker, where it was validated if the tool is able or not to discriminate the variable “digital competence” and its areas of between groups that are known [31]. 4.1. Sample The sample was composed of 2262 professors from different public Andalusian universities, of which 1026 (45.4%) were men and 1236 (54.6%) women, with most being 50–54 years old (37.3%) and 666 between the ages of 40 and 49 (29.4%). Table 2shows the data collected with respect to the University of origin, field of study and years of teaching experience. Lastly, it should be indicated, as a descriptive characteristic of the sample, that most of the teachers polled had been using information and communication technologies as educational tools for quite some time: 594 (26.3%) for more than 20 years, 466 (20.6%) 15–19 years, 448 (19.8%) 20 or more years, 312 (13.8%, from 6–9 years, 202 (8.9%) from 1 to 3 years and 38 (1.7%) less than a year. Not many teachers pointed to not using the technologies, more specifically, 34 (1.5%). This is shown in Table 3. Sustainability 2020,12, 6094 6 of 14 Table 2. University, field of knowledge and years of teaching experience of the professors. University f % Seville 808 35.7 Pablo de Olavide 214 9.5 Granada 178 7.9 Huelva 182 8.0 Cadiz 276 12.2 Cordoba 150 6.6 Malaga 162 7.2 Jaen 150 6.6 Almeria 142 6.3 Field of Study Arts and Humanities 360 15.9 Science 346 15.3 Health Sciences 300 13.3 Engineering and Architecture 434 19.2 Social and Judicial Sciences 822 36.3 Years of teaching experience 1–3 years 194 8.6 4–5 years 124 5.5 6–9 years 194 8.6 10–14 years 308 13.6 15–19 years 268 11.8 More than 20 1174 51.9 Table 3. Percent of time dedicated to the use of the technologies in class. Percent of Time Dedicated to the Use of Technology in Class f % 0–10% 172 7.6 11–25% 572 25.3 26–50% 674 29.8 51–75% 542 24.0 76–100% 302 13.4 0–10% 172 7.6 11–25% 572 25.3 26–50% 674 29.8 It should be indicated that a great majority of the professors indicated that programs dedicated to the ICT existed in their center, and that they participated in it (f =1156, 51.1%). Only 22.2% (f =502) did not participate in their center’s existing programs, and 604 (26.7%) pointed that their center did not participate in these activities. As for their competence in the use of different technologies, the answers provided are shown in Table 4. Table 4. Degree of agreement in technological domain. In My Day-to-Day, I Know How to Use . . . Media C.d. I.d N.d/N.a I.a. C.a. Computer 84 (3.7%) 2 (0.1%) 22 (1.0%) 314 (13.9%) 1840 (81.3%) Tablet 110 (4.9%) 16 (0.7%) 100 (4.4%) 434 (19.2%) 1602 (70.8%) Smartphones 100 (4.4%) 16 (0.7%) 102 (4.5%) 412 (18.2%) 1632 (72.1%) Internet 84 (3.7%) 14 (0.6%) 90 (4.0%) 572 (25.3%) 1502 (66.4%) Note: C.d. =Complete disagreement; I.d. =In disagreement; N.d/N.a =Not disagree/Not agree; I.a =In agreement; C.a. =Complete agreement. Sustainability 2020,12, 6094 7 of 14 It should be indicated that most of the professors who completed the questionnaire were curious about new digital applications, programs and resources; more specifically, 28.7% (f =650) were in agreement with the statement, and 1248 (f =55.2%) were in complete agreement, together comprising more than 80% of the distribution, as opposed to 82 (3.6%) who were in complete disagreement, and 44 (1.9%) who disagreed. Only 238 (10.5%) of the professors were neither in agreement or disagreement. As for the social networks utilized by the professors, Table 5shows the results found. Table 5. Number of social networks utilized by the professor. Number f % None 118 5.2 1 350 15.5 2 486 21.5 3 612 27.1 4 312 13.8 5 216 9.5 More than 6 168 7.4 In the questionnaire, the professors were asked two times to score their digital competence as an educator, by placing themselves in the following groups: newcomer, explorer, integrator, expert, leader and pioneer. The scoring was done before completing the questionnaire, and after it was finished. Table 6shows the results of the assessment performed by the professors before and after finishing. Table 6. Assessment by the professors. Initial f % Final f % Newcomer 66 2.9 Newcomer 74 3.3 Explorer 390 17.2 Explorer 390 17.2 Integrator 944 41.7 Integrator 976 43.1 Expert 616 27.2 Expert 590 26.1 Leader 170 7.5 Leader 174 7.7 Pioneer 76 3.4 Pioneer 58 2.6 4.2. Data Collection Instrument The data collection instrument used was the DigCompEdu Check-In questionnaire [ 41 ], the analysis instrument of the European Digital Competence Framework for Educators DigCompEdu. This competence framework was selected through expert judgement in a previous study [42]. The instrument is composed by 22, which refer to the 6 competences analyzed in “DigCompEdu”. They are related to the areas of competence: (A) Professional Engagement (four items), (B) Digital Resources (three items), (C) Teaching and Learning (four items), (D) Assessment (three items), (E) Empowering Learners (three items) and (F) Facilitating Learners’ Digital Competence (five items). Each of the items measures the different competences that shape the competence framework: A1. Organizational communication; A2 Professional collaboration; A3 Reflective practice; A4 Digital Continuous Professional Development; B1 Selecting digital resources; B2 Creating and modifying digital resources; B3 Managing, protecting and sharing digital resources; C1 Teaching; C2 Guidance; C3Collaborativelearning; C4Self-regulatedlearning; D1 Assessment strategies; D2 Analyzing evidence; D3 Feedback and planning; E1 Accessibility and inclusion; E2 Differentiation and personalization; E3 Actively engaging learners; F1 Information and media literacy; F2 Digital communication and collabo-ration; F3 Digital content creation; F4 Responsible use; and F5 Digital problem solving. In the questionnaire, as previously mentioned, the professors were asked to assess their digital competence before starting the questionnaire, into one of the following categories: newcomer explorer, integrator, leader or pioneer. This was to be done after completing the questionnaire as well. The meanings of each level were the following: Sustainability 2020,12, 6094 8 of 14 1. Newcomer (A1): little experience and contact with education technology. Needs continuous guidance to improve his or her digital competence level. 2. Explorer (A2): little contact with education technology. Has not developed specific strategies for including ICT in the classroom. Needs external guidance to improve his or her digital competence level for teachers. 3. Integrator (B1): experiments with education technology and reflects on its adequacy for different educational contexts. 4. Expert (B2): utilizes a wide range of education technologies critically and with confidence and creativity. Seeks the continued improvement of teaching practices. 5. Leader (C1): able to adapt his or her needs to different resources, strategies and knowledge within his or her reach. A source of inspiration for others. 6. Pioneer (C2): questions the contemporary digital and pedagogic practices, of which they are experts themselves. Lead innovation with ICT and are a role model to follow for other educators. Another series of demographic questions were added to the questionnaire: gender, years of service, time dedicated to the use of technologies in class, digital tools used in class, civil digital competence of the teacher, participation in social networks, etc. 4.3. Procedure of Data Collection and Analysis The administration of the instrument was conducted through the internet (EuSurvey). It was administered in the first months of year 2020 in the different Andalusian public universities that provide regulated in-person teaching. The anonymity of the data was ensured. In addition, all teachers accepted confidentiality clauses. To verify the reliability and validity of the questionnaire, the following two procedures were followed: acquisition of the reliability index and validation of the questionnaire. For this, the procedure from Ghomi and Redecker, the authors who created “DigCompEdu”, was used when checking for reliability and validation [31]. To verify the reliability of the questionnaire, the Omega coefficient from McDonald was used, as it possesses a number of advantages compared to Cronbach’s Alpha; its resistance with samples of heterogeneous populations does not need the requirement of compliance with the tau-equivalence and the lack of correlated errors [44,53–57]. In addition, to validate it, the known-groups method was used, which was applied for analyzing the validity of an instrument and to discriminate the existence of significant differences in a variable of interest among known groups [ 38 , 57 , 58 ]; it is a method utilized in different projects as the validation test in different areas of knowledge [31,59–61]. For the statistical analysis, the following non-parametric tests were utilized: Kruskal–Wallis, Mann–Whitney U and Spearman’s rho [ 62 , 63 ]. These tests were used since it has been verified that the data is not normally distributed through the asymmetry and kurtosis study. The “Kolmogorov–Smirnov goodness-of-fit test” confirmed this check, with significance p=0.000 for all items (non-normal distribution). 4.4. Hypothesis To establish the known groups, the following hypotheses were formulated from the sociodemographic items: Hypothesis 1. The more time spent utilizing technology as an education tool, the greater is the digital competence of the professor, and therefore, the general result of the test as well. Hypothesis 2. The more time dedicated for using the technology in class, the greatest the digital competency, and therefore the general result of the test. Sustainability 2020,12, 6094 9 of 14 Hypothesis 3. The professors who know how to use different technology resources (computer, tablet, smartphones and internet), possess a greater digital competence, and obtain a better result on the test. Hypothesis 4. The professors who are curious about new applications, possess a greater digital competence, and obtain a higher score on the test. Hypothesis 5. The professors who are committed with some ICT or digitalization program at their university, promoted by the administration, obtain better results on the test. Hypothesis 6. The greater the self-perception of the digital competence of the professor be-fore completing the questionnaire, the greater the digital competence, and therefore, the gen-eral result of the test. Hypothesis 7. The greater the self-perception of the teacher’s digital competence after finishing the questionnaire, the greater the digital teaching competence, and therefore, the general result of the test. Hypothesis 8. The greater the number of social networks utilized by the teacher, of which he or she is a user, the greater the digital teaching competence, and therefore, the better the results from the test. The hypothesis that will be formulated will follow the following structure:  Null hypothesis (H0): there are no significant differences between the different groups com-pared, with an alpha risk of rejecting the null hypothesis of 0.05, and the total scores and the general results obtained by the teacher in the instrument “DigCompEdu”.  Alternative hypothesis (H1): there are significant differences between the different groups compared, with an Alpha risk of 0.05, and the total scores and general results obtained by the educator in the instrument “DigCompEdu”. 5. Analysis and Results The analysis starts with the results of reliability (Table 7). Table 7. McDonald’s coefficient omega. Dimension Ω General 0.967 Professional engagement 0.842 Digital resources 0.807 Teaching and learning 0.821 Assessment 0.790 Empowering learners 0.784 Facilitating learners’ digital competences 0.898 All these values, in agreement with O’Dwyer and Bernauer, have high levels of reliability, for the overall instrument, as well as for the different dimensions that comprise it [64]. It should be pointed out that for the items that had different response options, the Kruskal–Wallis test was utilized, and for the dichotomous answers, the Mann–Whitney U test was used. Table 8shows the Kruskal–Wallis values and level of significance for rejecting the H0. As for the values obtained in the Mann–Whitney U test, these are shown on Table 9. The results obtained with both tests allow us to reject the H0 in every case at significance level p≤0.001 . As a result, it can be concluded that the instrument is valid for discriminating the level of TDC between the different response groups provided in the different items.