Technology – Based Assessment of Soft Skills in Higher Education

Čubrić, Goran; Salopek Čubrić, Ivana Conference Paper Technology – Based Assessment of Soft Skills in Higher Education Provided in Cooperation with: IRENET - Society for Advancing Innovation and Research in Economy, Zagreb Suggested Citation: Čubrić, Goran; Salopek Čubrić, Ivana (2016) : Technology – Based Assessment of Soft Skills in Higher Education, In: Proceedings of the ENTRENOVA - ENTerprise REsearch InNOVAtion Conference, Rovinj, Croatia, 8-9 September 2016, IRENET - Society for Advancing Innovation and Research in Economy, Zagreb, pp. 416-421 This Version is available at: https://hdl.handle.net/10419/183745 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. https://creativecommons.org/licenses/by-nc/4.0/ 416 ENTRENOVA 8-9, September 2016 Rovinj, Croatia Technology – Based Assessment of Soft Skills in Higher Education Goran Čubrić University of Zagreb Faculty of Textile Technology, Croatia Ivana Salopek Čubrić University of Zagreb Faculty of Textile Technology, Croatia Abstract The importance of soft skills, both for the learning process and employment, is well outlined in a number of documents and strategies. At the moment, digital badges seem to be an important technology that can be used to assess soft skills. Digital badges, or more specifically, Open Badges, provide a whole new way of issuing credentials to individuals that demonstrated certain skill. Furthermore, badges allow individuals to complete with themselves or others, and are good instrument that can motivate continued engagement. Taking all this into concern, it should be pointed out that digital badges have great potential to validate evidences of educational achievements in formal education. This paper describes the 4-phase model SAGRADA, developed within the EU LLP project GRASS, in order to support awarding badges. The whole process is additionally explained trough application case developed at one partner institution. The paper also gives overview of student’s reactions towards the whole badging principle and discusses the outcomes. Keywords: ICT, learning, skills, open badges, research JEL classification: I23 Acknowledgments: This paper has been funded with support from the European Commission within Lifelong Learning Programme and presents results of the project Grading soft skills: GRASS, Project No. 543029-LLP-1-2013-1-RS-KA3-KA3MP. This publication reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein. Introduction There is a number of ICT tools that teachers use in class to help their students to develop soft skills. Teachers, educational institutions and other relevant organizations use different tools and services for teaching and assessing learners’ soft skills. Most of these tools are general-purpose teaching tools and services, or even ICT tools developed with wider objectives in mind. An important and relatively new technology that facilitates recognition and credentialing of different skills and learning achievements, and can further increase learning motivation, is that of digital badges, and more specifically Open Badges (OBs), (Jovanovic et al., 2015). The concept of awarding badges is inspired by award of ribbons and medals that has historically been used for the achievements in sport, professional services, etc. 417 ENTRENOVA 8-9, September 2016 Rovinj, Croatia Digital badges are an online record of achievements, tracking the recipient’s communities of interaction that issued the badge, and the work completed to get it (Open badge, 2016).In other words, the badge is animage file, it contains information about who earned it, what they had to do, when the badge was issued and who issued it. This badge can then easily be added to the personal portfolio, and can be added in clusters that are shared with various groups. In addition, the badge can include a link to the evidence of meeting the criteria for obtaining a badge (Glover, 2013). According to Anderson et al. (2015), the use of the digital badge is more powerful than traditional grades or scores, because:  it represents the effectiveness of skills (as opposed to the oral tests),  badges are based on the standards or criteria (not to normative assessment),  badges encourage students to take control of their learning process as well as publicly available process,  they also enable learning as a social process where assessments of professors and colleagues may be part of the final award of badges and finally,  badges allow connections with other metadata (e.g. the national/international standards to deal with this and how). There is a whole range of organizations that support badges like Badge Alliance (global organization standards for badges), the Mozilla Foundation, Collective shift, the MacArthur Foundation, World the Badge and so on. Currently, a number of institutions and organizations award badges, such as IBM, Microsoft, NASA, Design of America, Michigan State University, NYC Department of Education, Smithsonian American Art Museum, University of Illinois and others (Mozilla, 2016). This paper presents the results accomplished within the EU LLP research project “Grading soft skills - GRASS”, at one partner institution – University of Zagreb, Faculty of Textile technology. The overall objective of this project is to create mechanisms that enable to continuously support, monitor, assess, and acknowledge development of learners' soft skills with the help of state-of-the-art ICT tools in resolving the above stated challenges, thus leading the OBs concept to its full potential (Project GRASS, 2016). In this paper is given an overview of the badging concept, starting from the definition of skills that will be in the focus of badging, continuing with the development of pedagogical approach and badging system, and finally concluding with evaluation of the whole process Methodology The methodological approach presented in this paper refers to the description of developed SAGRADA model and setting up of an application case in higher education, at one institution that is a member of GRASS consortium. Development of SAGRADA mode A 4-phase model, called SAGRADA was developed to support the process of awarding badges. The core of the SAGRADA model is a set of metrics, carefully developed for specific practical application cases following pedagogical approaches (Devedžić et al., 2015). The term SAGRADA stands for SAmpling, GRAding, Displaying and Acknowledging students' soft skills. The first phase, i.e. Sampling, assumes that teaching practice includes the use of any type of ICT tools and services. The teacher set of soft skills metrics in order to periodically observe and measure students' soft skills. All the observations need to be captured and stored as the evidence of the soft skills developed. The second phase within this model is related to grading soft skills by awarding Open Badges. Based on the observations made in the first phase, teacher can 418 ENTRENOVA 8-9, September 2016 Rovinj, Croatia award badges directly, or can involve students into the grading process. The badges can be awarded for different skills and different levels of each skill. The award of badges can be conducted using different platforms, such as BadgeOS, Moodle, Credly, etc. Once students have earned open badges for a certain skill, they can place them in their backpacks or on a number of different websites, social networks, personal profiles and etc. Therefore, this phase of SAGRADA model is called Displaying. Finally, the last phase is called Acknowledging. Within this phase, the third person, for example an employer who wants to hire an employee with pronounced specific soft skills, can see the applicant's collection of badges for soft skills and make the final decision easier. Setting up an Application case In the first phase, it was important to define soft skills that are most important for the profile of the students that study at the University of Zagreb, Faculty of Textile Technology and are participants in this study. As main instruments for the definition are used questionnaire and focus group. On the basis of conducted survey and discussion within the focus group at the Faculty, it is decided to focus at badging of the following soft skills: creativity, communication and collaboration (Salopek Cubric et al., 2014; Stracenski Kalauz et al., 2015). After the definition of important soft skills, an application case for badging is developed. For each of the defined soft skills (i.e. creativity, collaboration and communication) is clearly specified an importance statement, pedagogical approach and specific metric. For grading soft skills with Open Badges, the following badges are designed: Creative individual, Communicator, Collaborator (all three to be given after one successfully completed task), as well as Exceptionally Creative individual, Great Communicator and Perfect collaborator (all three to be given to students with 3 badges earned for the same skill), figure 1. Figure 1 Designed OBs Source: Author’s illustration In the grading process participated BSc students attending the course Machine and automata in clothing technology. Students are requested to work on different tasks either in small groups or individually and to present the outcomes in the classroom. The task is to be done in 3-4 classes (i.e. 135-180 minutes), or within 4 weeks (if the 419 ENTRENOVA 8-9, September 2016 Rovinj, Croatia task is to be done at home). Before the start of the task, teacher showed students similar tasks completed by previously enrolled students, with no further guidance. Students showed completed task in the front of all students. The skill level is discussed and assessed both by professor and all students using a with 5-point scale. The guidelines to be followed during next task are given by teacher, who also gives open badges to the most successful students. Students' reactions towards OBs In order to receive students' reactions towards the use of Open Badges and towards the importance of grading soft skills, second questionnaire is developed. The questionnaire includes 10 questions (7 to be answered using a 5-point Likert scale, and 3 open-ended). Students filled out questionnaires after the end of the badging process. The results are given in the table 1. Table 1 The results of students’ reaction towards OBs and towards the importance of soft skills Question Mean Median Std.D ev. Coef.V ar. Standard Err. Grade the importance of creativity for your future employment 4,73 5,00 0,47 9,88 0,14 Grade the importance of communication for your future employment 4,55 5,00 0,52 11,49 0,16 Grade the importance of collaboration for your future employment 4,73 5,00 0,47 9,88 0,14 I understand the concept of open badges and how badges can be used. 3,64 4,00 0,67 18,54 0,20 I like the use of badging system within this course. 4,55 5,00 0,52 11,49 0,16 The possibility of earning a badge motivates me to perform better. 4,46 5,00 0,69 15,44 0,21 I think that the concept of badging will be important for my future career. 4,00 4,00 0,78 19,37 0,23 Source: Author The results of second questionnaire indicated that students consider all three observed soft skills (creativity, communication and collaboration) very important for future employment in clothing industry. The average grades on Likert scale are: 4,73 for creativity; 4,55 for communication and 4,73 for collaboration. Besides those skills, they also proposed openness and accuracy, as very important skills for the future employment. The overview of results showed that students like the idea of badging (the average grade is 4,55) and consider it very motivating (the average is 4,46). The results also indicated that some students do not completely understand the whole 420 ENTRENOVA 8-9, September 2016 Rovinj, Croatia concept of badging (the average grade is 3,64), but think that open badges will be important for their future career (average grade for the importance is4,00). Among the three observed soft skills, creativity is considered as the most important. Furthermore, half of the students considered that it is not necessary to include additional skills to be graded. Conclusion The paper presents the badging concept that is developed within the EU LLP project GRASS and is applied at University of Zagreb. The concept starts with the definition of skills that will be in the focus of badging, continues with the development of pedagogical approach and badging system, and concludes with students' reactions towards the use of Open Badges. The survey among students indicated that students have very positive attitude towards badging concept and consider all three observed skills very important for the future employment. Furthermore, they also reported that the whole badging process motivated them to be even more successful in completing the tasks within the course. References 1. Anderson, D.M., Staub, S. (2015), „Postgraduate digital badges in higher education: Transforming advanced programs using autheentic online instruction and assessment to meet the demands of a global marketplace”, Procedia – Social and Behavioral Sciences, Vol. 195, pp. 18-23. 2. Devedžić, V., Jovanović, J., Tomić, B., Ševarac, Z., Milikić, N., Domitrijević, S., Đurić, D. (2015), „Grading soft skills with open badges”, In: D. Hickey, J. Jovanovic, S. Lonn, J.E. Willis, III (eds.), Proceedings of the Open Badges in Education (OBIE 2015), Poughkeepsie, New York, USA, 16-Mar-2015 3. Glover, I. (2013), „Open Badges: A visual method of recognising achievement and increasing learner motivation”, Student Engagement and Experience Journal, Vol. 2 No. 1. 4. Jovanović, J., Devedžić, V. (2015), „Open badges: Novel means to motivate, scaffold and recognize Learning”, Technology, Knowledge and Learning, Vol. 20 No.1, pp. 115-122. 5. Mozilla, available at: http://openbadges.org/participating-issuers/ (19/1/ 2016) 6. Open Badges, Badges/onboarding-issuer, available at: https://wiki.mozilla.org/Badges/Onboardingissuer#A.Mozilla_Open_Badge_Infrastructu re_.28OBI.29 (19/1/2016) 7. Project GRASS, available at: https://sites.google.com/site/llpgrassproject/(15/2/2016) 8. Salopek Čubrić, I., Stracenski Kalauz, M., Hudec, G. (2014), „Soft skills' evaluation in textile engineering and design”, Proceedings of ICED 2014 - Educational Development in Changing World, pp. 1-9. 9. Stracenski Kalauz, M., Salopek Čubrić, I., Hudec, G. (2015), „Perception of soft skills in textile designers' education”,Global Journal on Humanities & Social Sciences, Vol. 1No. 1, pp. 33-37. About the authors Goran Čubrić is assistant professor at the University of Zagreb, Faculty of Textile Technology, Department of Clothing Technology. His main area of research is the use of automation and robots in the textile and clothing industry. He is also a member of 2 EU projects. Author can be contacted at goran.[email protected]. Ivana Salopek Čubrić is assistant professor at the University of Zagreb, Faculty of Textile Technology. She completed B. sc. studies at Faculty of Textile Technology and 421 ENTRENOVA 8-9, September 2016 Rovinj, Croatia Faculty of Political sciences, M. Sc. study at Faculty of Textile Technology and Ph.D. at the same institution. She was a member of a number of educational projects and at the moment works as a lead researcher and a member of 3 EU projects related to education. Also, she is a head of part-time study. Author can be contacted at ivana.salo[email protected].