Study of the impact federated erubrics on assessing competences in the practicum

16. ICT in Education and Training Alternative EERA Network: 09. Assessment, Evaluation, Testing and Measurement Topics: NW 16: Assessment and evaluation of the impact of ICT in education and training Keywords: Practicum, assessment, ICT, ePortfolios, Rubric. Study Of The Impact Of Federated Erubrics On Assessing Competences In The Practicum Manuel Cebrian -de-la-Serna1, María Jesús Gallego-Arrufat2, José Serrano-Angulo1, Violeta Cebrián-Robles3 , Fernando Tellado -González3, Vanesa María Gámiz-Sánchez2 1Malaga University, Spain; 2Granada University, Spain; 3Vigo University, Spain Presentin g Author: Cebrian-de-la-Serna, Manuel; Cebrián-Robles, Violeta Presentation Research of the Practicum and externships has a long history and involves important aspects for analysis. For example, the recent changes taking place in university grades allot more credits to the Practicum course in all grades, and the Company - University collaboration has exposed the need to study in new learning environments. The rise of ICT practices like ePortfolios, which require technological solutions and methods supported by experimentation, study and research, require particular examination due to the dynamic momentum of technological innovation. Tutoring the Practicum and externships requires remote monitoring and communication using ePortfolios, and competence - based assessment and students’ requirement to provide evidence of learning require the best tutoring methods available with ePortfolios. Among the elements of ePortfolios, eRubrics emerge as a tool for design, communication and competence -assessment. This project aims to consolidate a research lin e on eRubrics, already undertaken by another project -I+D+i [EDU2010-15432]- in order to expand the network of researchers and Centres of Excellence in Spain and other countries: Harvard University in USA, University of Cologne in Germany, University of Colima in Mexico, Federal University of Parana, University of Santa Catarina in Brasil, and Stockholm University in Sweden(1). This new project [EDU2013 -41974P](2) examines the impact of eRubrics on tutoring and on assessing the Practicum course and externships. Through technology, distance tutoring grants an extra dimension to human communication. New forms of teaching with technological mediation are on the rise and are highly valuable, not only for formal education but especially in both public and privat e sectors of nonformal education, such as occupational training, unemployed education and public servant training. Objectives Obj. 1. To analyse models of technology used in assessing learning in the Practicum of all grades at Spanish Faculties of Educati on. Obj. 2. To study models of learning assessment measured by eRubrics in the Practicum. Obj. 3. To analyse communication through eRubrics between students and their tutors at university and practice centres, focusing on students’ understanding of compete nces and evidences to be assessed in the Practicum. Obj. 4. To design assessment services and products, in order to federate companies and practice centres with training institutions. Among many other features, it has the following functions CoRubric(3) 1. The possibility to assess people, products or services by using rubrics. 2. Ipsative assessment. 3. Designing fully flexible rubrics. 4. Drafting reports and exporting results from eRubrics in a project. 5. Students and teachers talk about the evaluation and application of the criteria Methodology, Methods, Research Instruments or Sources Used The project will use techniques to collect and analyse data from two methodological approaches: 1. In order to meet the first objective, we suggest an initial exploratory descriptive study (Buendía Eisman, Colás Bravo & Hernández Pina, 1998), which involves conducting interviews with Practicum coordinators from all educational grades across Spain, as well as analysing the contents of the teaching guides used in all educational grades across Spain. 55 academic managers were interviewed from about 10 faculties of education in public universities in Spain (20%), and course guides 376 universities from 36 public institutions in Spain (72%) are analyzed. 2. In order to satisfy the second objective, 7 universities have been selected to implement the project two instruments aimed at tutors practice centers and tutors of the faculty. All instruments for collecting data were validated by experts using the Delphi method. The selection of experts had three aspects: years of professional experience, number and quality of publications in the field (Practicum, Educational Technology and Teacher Training), and self-rating of their knowledge. The resulting data was calculated using the Coefficient of Competence (Kcomp) (Martínez, Zúñiga, Sala & Meléndez, 2012). Results in all cases showed an average experience of more than 0.09 points. The two instruments of the first objective were validated during the first half of 2014-15 year, data collected during the second half. And the second objective during the first half of 2015-16 year and data collection for the second half. The set of four instruments (two for each objective 1 and 2) have the same dimensions as each of the sources (Coordinators, course guides, tutors of practice centers and faculty) as they were: a. Institution-Organization, b. Nature of internships, c. Relationship between agents, d. Management Practicum, e. Assessment. F. Technological support, g. Training and h. Assessment Ethics. Conclusions, Expected Outcomes or Findings The first results respond to Objective 1, where we find different conclusions depending on each of the six dimensions. In the case of internal regulations governing the organization and structure of the practicum, we note that most traditional degrees (Elementary and Primary grades) share common internal rules, in particular development methodology and criteria against other grades (Pedagogy and Social Education ). It is also true that the centers of practices in last cases are very different from each other and can be a public institution, a school, a company, a museum, etc. The memory with a 56.34% and 43.67% daily activities are more demands on students in all degrees, Lesson plans 28.18% 19.72% Portfolio 26.7% Didactic units and Others 32,4%. The technical support has been mainly used the platform of the University 47.89% and 57.75% Email, followed by other services and tools 9.86% and rubric platforms 1.41%. The assessment criteria are divided between formal aspects of 12.38%, Written expresión 12.38%, treatment of the subject 14.45%, methodological rigor of work 10.32%, and Level of argument Clarity and relevance of conclusions 10.32%. In general terms, we could say that there is a trend and debate between formative assessment against a accreditation. It has not yet had sufficient time to further study and confront other dimensions and sources of information. We hope to provide more analysis and conclusions in the conference date. Notes (1) Spanish Plan of R+D+i Excellence (2014-2016) No. EDU2013-41974P. http://erubrica.org (2) Federated PLE-Portfolios of Multimedia Evidence https://gteavirtual.org/ (3) Access to CoRubric tool http://corubric.com Reference s Buendía Eisman, L., Colás Bravo, P. & Hernández Pina, F. (1998). Métodos de investigación en psicopedagogía [Research methods in psychology]. Madrid: McGraw-Hill. Cebrián -de-la-Serna, M., Bartolomé-Pina, A., Cebrián-Robles, D., & Ruiz-Torres, M. (2015). Study of portfolio in the Practicum: an Analysis of PLE-Portfolio. RELIEVE, 21(2). http://dx.doi.org/10.7203/relieve.21.2.7479 Cebrián de la Serna, M., Serrano Angulo, J., & Ruiz Torres, M. (2014). eRubrics in Cooperative Assessment of Learning at University. Comunicar, 22(43), 153–161. http://doi.o rg/10.3916/C43-2014-15 Cebrián Robles, D., Serrano Angulo, J., & Cebrián de la Serna, M. (2014). Federated eRubric Service to Facilitate Self-Regulated Learning in the European University Model. European Educational Research Journal, 13(5), 575–583. http://doi.org/10.2304/eerj.2014.13.5.575 Gallego, M. J., Gámiz, V., Pérez, M. P., & Romero, M. A. (2009). Desarrollo de competencias en el Prácticum con materiales y actividades online [Skills development in materials and Practicum Online]. Pixel-Bit: Revista de Medios Y Educación, 34, 135–150. Martínez, V. G., Zúñiga, S. P. A., Sala, A. G. & Meléndez, A. M. (2012). El uso del método Delphi como estrategia para la valoración de indicadores de calidad en programas educativos a distancia [Using the Delphi method as a strategy for assessing quality indicators in distance education programs]. Calidad en la Educación Superior, 3(1), 200–222. Martínez -Figueira, E., Tellado-González, F., & Raposo-Rivas, M. (2013). La rúbrica como instrumento para la autoevaluación: un estudio piloto. REDU. Revista de Docencia Universitaria, 11(2), 373–390. McConnell, K. D. (2013). Rubrics as catalysts for collaboration: a modest proposal. European Journal of Higher Education, 3(1), 74–88. http://doi.org/10.1080/21568235.2013.778043 Tejad a Fernández, J. (2005). El trabajo por competencias en el prácticum: cómo organizarlo y cómo evaluarlo [The work in the practicum competences: how to organize and how to evaluate it.]. Revista Electrónica de Investigación Educativa, 7(2). Retrieved from http://redie.uabc.mx/index.php/redie/article/view/192 Tejada Fernández, J. (2009). La organización y la evaluación del aprendizaje en el prácticum: líneas programáticas de actuación, 133–160. Zabalza Beraza, M. A. (2011). A vueltas con el prácticum [A round with the practicum]. REDU. Revista de Docencia Universitaria, 9(2), 9. Zabalza Berraza, M. A. (2013). El practicum y las prácticas de empresas: la formación universitaria [The practicum and practices in enterprises: university education]. Madrid: Narcea Ed iciones.