Engaging and Assessing Students through their Electronic Devices and Real Time Quizzes

Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2016.6375 Social and Technological Sciences EISSN: 2341-2593 Ferrándiz et al. (2016) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 3 Nº 2 (2016): 173-184 | 173 engaging and assessing students through their electronic devices and real time quizzes Ferrándiz, E.*, Puentes, C., Moreno, P.J., Flores, E. Department of Economics, Universidad de Cádiz *esther[email protected]; Received: 2016-01-23; Accepted: 2016-03-13 Abstract This paper describes a teaching experience using Socrative, a third party electronic tool, for real-time questioning in lectures of Econometrics. Econometrics is a theoretical-practical subject, but traditionally a large proportion of our students tend to focus on the practical and discard the theory, often skipping classes on theory and avoiding studying its content, probably motivated by its complexity. As a consequence, students’ marks obtained in the theoretical part of the exam are usually low. In this context, we put forward a change in our teaching methodology to include the use of Socrative, a freely available app, that allows students to answer teachers’ short, true/false, or multiple choice questions posed during each class using their smartphones (or other electronic devices with Internet connection). The objectives of this project are twofold: 1) to engage students and increase attendance at lectures; 2) to improve feedback on the learning process. The results of a survey of a sample of 186 students reveal that Socrative has been an effective tool for achieving these objectives. Keywords Students response system, clicker, quizzes, active learning, student engagement, Socrative, gamification Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2016.6375 Social and Technological Sciences EISSN: 2341-2593 Ferrándiz et al. (2016) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 3 Nº 2 (2016): 173-184 | 174 1. Introduction Information technology (IT) offers a powerful opportunity for increasing student engagement (Roblyer & Wiencke 2003). Prominent examples of technology include student response systems (SRSs), 1 which allow an instructor to pose questions to the class, allow students to enter their answers in some kind of device (e.g., clicker or smartphone) and instantly summarize and graphically represent results for the instructor (Beatty 2004), who can decide whether the responses are anonymous or not. Terrion and Aceti (2012) have highlighted the importance of student engagement because unengaged students do not listen, process, or attend to the learning process. Several papers have analysed the role of SRSs as a means of engaging students and making them active in the classroom, among other benefits (for a review see Aljaloud et al. 2015). Although much of the research in SRS has traditionally been related to the use of clickers, the development of cloud-based software that enables any device to become an SRS overcomes some of the criticism of clickers such as cost, and allows educators to exploit the potential of personal electronic devices, mostly smartphones, for educational purposes. Although education is not among the main uses of these devices yet, this use is expected to grow in the coming years as new applications are developed. Companies such as Apple have already foreseen this possibility, developing iTunes U, an application store especially focused on education. The highest rates of smartphone ownership are among the richer economies and the top countries in the rank include South Korea (88%), 77% of Australia (77%), Israel (74%), USA (71%) and Spain (71%), with a greater presence among young people (Pew Research Center, 2016). Gaming (65%), social networking (54.3%), and instant messaging (48.3%) are among the most common uses minors and younger people have for their smartphones’ advanced features (INTECO, 2011). These statistics are in line with Yusof et al. (2012), who argue that smartphones are mainly used for entertainment and social networking. Therefore, the potential of smartphones for education (m-learning) has yet to be unlocked. 1 SRSs are also refered to as classroom performance systems, audience response systems, personal response systems, classroom communication systems, electronic response systems, electronic voting systems, polling systems, or clicker systems (Aljaloud et al. 2015). Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2016.6375 Social and Technological Sciences EISSN: 2341-2593 Ferrándiz et al. (2016) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 3 Nº 2 (2016): 173-184 | 175 Additionally, the subject of Econometrics taught in the Business and Administration Degree program relies on theoretical reasoning and computer practices that students often find difficult to understand. This, jointly with the limited time available in lectures and the complexity of the subject, hinder the learning, comprehension, and motivation of students when they deal with Econometrics. Motivated by all the above-mentioned reasons, instructors of Econometrics at the University of Cadiz have proposed an innovation project based on Socrative, a cloud-based SRS. The main objectives of the project were two: 1) increase attendance and engagement of students in theoretical classes and 2) improve the feedback process to students regarding their learning processes. This paper is structured as follows. Section 2 summarizes the advantages and disadvantages of SRSs. Section 3 describes the methodology and the implementation of the study. Section 4 describes the results; and finally, in section 5 we draw some conclusions and comment on future work. 2. Benefits and drawbacks of using student response systems Kay and LeSage (2009a) and, more recently, Aljaloud et al. (2015) review the literature on the benefits of using SRSs, which are related not only to engagement, but also to interactivity and academic performance. Focusing on the former, previous research has shown that SRSs enhance student engagement in learning by creating a fun learning environment. Further, the researchers found increased attendance and positive student attitudes and an increase in the students’ desire to improve their performance by identifying areas of improvement. Because of increased engagement, an SRS has the potential to improve academic performance. Finally, SRSs improve the process of instructor feedback since the instructor has more information about the learning processes of their students. However, there are also some drawbacks, such as time required for teachers to formulate good questions and deal with technical problems. Other often-cited inconveniences of clickers, such as cost or time to deliver the clickers to students in each class, are overcome thorough the use of cloud-based software such as Socrative, which enables any electronic device (such as students’ smartphone or tablets) to be used as a clicker. Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2016.6375 Social and Technological Sciences EISSN: 2341-2593 Ferrándiz et al. (2016) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 3 Nº 2 (2016): 173-184 | 176 3. Methodology This project focused on 385 students enrolled in Econometrics classes in the Business and Administration Degree program at the University of Cadiz. Econometrics is taught in the second semester to third-year students on three different campuses, involving seven teachers. The average number of participants was 160, spread among five classes (i.e., 32 students per class). The project mainly consisted of the regular use of Socrative quizzes during lectures. The procedure to develop the activity followed several steps: 1. Design and training of teachers. This phase includes the following tasks: a) Elaboration of questions in Socrative for each of the 10 lessons of the syllabus. Each quiz had 3-4 questions and was focused on relevant contents explained in class. We included multiple choice and true/false questions, bearing in mind that the most effective questions to be used in SRS are those (Kay and LeSage 2009b) that allow students to apply knowledge recently acquired; are higher level; focus on process and reasoning as opposed to factual content; identify and help resolve misconceptions; and support a comprehensive review of a specific set of concepts. b) Statement of the rules of the game. - The participation with Socrative is individual unless the teacher says the opposite, and is limited to students who have attended all the classes. - The use of electronic devices is only allowed for responding to teachers’ questions. - Students are responsible of bringing a working electronic device with Internet connection. The University provides free working Wi-Fi for students in their facilities. c) Selection of the reward. Students who regularly attend class and provide correct answers to at least 75% of the questions would get 0.5 extra points added to their final mark. d) Training of teachers. The coordinator of the project developed a training session for teachers and elaborated a visual guide on how to use Socrative, covering some of the following aspects: - How to create a personal account in Socrative.com - How to import a quiz previously elaborated by the coordinator and how to start it. Regarding the latter, we used the following options:  Students pace the quiz, which allows them to navigate through it at their own pace. Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2016.6375 Social and Technological Sciences EISSN: 2341-2593 Ferrándiz et al. (2016) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 3 Nº 2 (2016): 173-184 | 177  Disable immediate feedback. We chose this option because we wanted to keep the correct answer concealed until the entire class had submitted their answers to make cheating less attractive.  Random questions and answers. We used these options to make cheating more difficult. - How to review the class’ answers, save reports and get bar graphs to give feedback to the students. - 2. Information session for students and system testing. In the first class of the course, each teacher introduced the subject to his/her class, including the methodology and the evaluation system. Regarding the latter, the instructor introduced Socrative and presented the rules of the game. Subsequently, we made a trial test in class to check that the Internet connection worked well in the classroom and to get students familiar with the system. In order to facilitate the link between student names and their marks in Socrative, avoiding misspellings, students were asked to log in with their identity card number instead of their names. 3. Classes, quizzes and exit tickets. Once students and teachers knew how to use Socrative, we began the course. In each session, after the teacher explained the theoretical content, he/she gave the students a quiz. By the end of the class, they had to obtain an “exit ticket” that included the following questions: How well did you understand today’s material? What did you learn in today’s class? and, optionally, a teacher’s open question. The latter often polled the student’s desire to explain a point or points again or to develop more deeply some specific materials or concepts that were not clear enough for them. After completing each quiz, the instructor should review and project the aggregated performance of the class for each question, comment on the correct answer, and save the report automatically generated by Socrative. The total estimated in-class time devoted to quizzes and exit tickets should be 10-12 minutes. Before the next class, lecturers had to review the exit tickets to check students’ needs and their requests in the open question. When all the quizzes had been run, teachers had to merge the marks of each quiz with the students’ names. 4. Students’ feedback on the project. By the end of the course, we gathered students’ opinion through a survey to evaluate the contribution of Socrative to their learning process. Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2016.6375 Social and Technological Sciences EISSN: 2341-2593 Ferrándiz et al. (2016) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 3 Nº 2 (2016): 173-184 | 178 4. Results In order to get information about students’ satisfaction with the experience, they were asked to answer questions related to the contribution and limitations of Socrative. This survey was answered through Google Forms. We use a Likert scale from 1 to 5, where 1 is strongly disagree, 2 somewhat agree, 3 neutral, 4 somewhat agree, and 5 strongly agree. There were 186 respondents. For several questions, a relatively high percentage of answers was concentrated in the central value, 3, but this is not unusual because central tendency is a common bias when using a Likert scale. Figure 1 shows that 47.31% of students strongly/somewhat agree that Socrative enhances comprehension of content and acquisition of competence related to Econometrics (20.43% strongly/somewhat disagree). Figure 2 shows that 56.99% of students agree that Socrative has motivated them to attend to class versus 23.12% who disagree. Figure 1. Socrative improves comprehension of contents and acquisition of competence related to the subject. Figure 2. Socrative has motivated me to attend class. It is notable that 73.12% of the students agree that Socrative is a motivation for listening to the teacher (Figure 3). Figure 4 shows that 55.91% of the students agree that Socrative has contributed to helping them memorize some basic concepts, versus 20.97% of the students who disagree. According to Figure 5, 65.05% of the students agree that Socrative 3,23% 17,20% 32,26% 38,71% 8,60% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 1 2 3 4 5 9,14% 13,98% 19,89% 32,26% 24,73% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 1 2 3 4 5 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2016.6375 Social and Technological Sciences EISSN: 2341-2593 Ferrándiz et al. (2016) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 3 Nº 2 (2016): 173-184 | 179 is helpful in testing their level of understanding of the contents explained by the teacher (13.44% disagree). Figure 6 shows that 50% of the students agree that Socrative has increased their selfconfidence for answering teachers’ questions versus 17.74% who disagree. Figure 7 demonstrates that 44.62% of the students agree that Socrative increased their selfconfidence in their learning process in contrast with 23.66% of the students, who disagree. Further, as represented in Figure 8, 56.99% of the students prefer Socrative to the traditional system of raising hands (23.66% do not). Figure 3. Socrative has motivated me to listen to the teacher. Figure 4. Socrative has helped me to memorize some basic concepts. 4,84% 4,84% 17,20% 45,70% 27,42% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 1 2 3 4 5 6,45% 14,52% 23,12% 34,95% 20,97% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 1 2 3 4 5 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2016.6375 Social and Technological Sciences EISSN: 2341-2593 Ferrándiz et al. (2016) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 3 Nº 2 (2016): 173-184 | 180 Figure 5. Socrative has contributed to testing my level of understanding. Figure 6. Socrative has increased my self-confidence in answering teachers’ questions. Figure 7. Socrative has increased my self-confidence in my learning process. Figure 8. Socrative is better than traditional questioning systems (e.g., raising hands, direct questions from the instructor). 3,23% 10,22% 21,51% 43,01% 22,04% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 12345 8,06% 9,68% 32,26% 34,95% 15,05% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 1 2 3 4 5 7,53% 16,13% 31,72% 34,41% 10,22% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 12345 10,75% 12,90% 19,35% 25,27% 31,72% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 1 2 3 4 5 Multidisciplinary Journal for Education, http://dx.doi.org/10.4995/muse.2016.6375 Social and Technological Sciences EISSN: 2341-2593 Ferrándiz et al. (2016) http://polipapers.upv.es/index.php/MUSE/ Mult. J. Edu. Soc & Tec. Sci. Vol. 3 Nº 2 (2016): 173-184 | 181 Figure 9 makes clear the perceived contribution to feedback in comparison to other theoretical subjects, with 67.2% of the students agreeing with this item (only 9.14% disagree). Figure 10 shows that 61.29% of the students agree that Socrative facilitates interaction while only 15.05% of students disagree with this statement. Although we do not present a detailed graph here, 68.81% of the students strongly/somewhat agree that Socrative contributes to making classes more pleasant and fun (only 10.75% of the students disagree). Finally, Table 1 presents the results of questions about the importance of some of the inconveniences students could face with the use of Socrative. Our results show that the most relevant inconvenience was related to Internet connections and problems with the device. Our results suggest that using Socrative does not hinder class dynamics and does not prevent students from going back to the lesson and listening to the instructor. From the instructors’ viewpoint, the experience is also positively evaluated, and we are motivated by the results obtained in the students’ survey. Although it represented an extra effort that required teacher commitment and slight changes in the teaching methodology, the system is easy, fast, and convenient and can be used on a regular base. Figure 9. It gives more feedback to the teacher than in other theoretical subjects. Figure 10. It facilitates interaction. 4,30% 4,84% 23,66% 35,48% 31,72% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 12345 6,99% 8,06% 23,66% 37,10% 24,19% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 1 2 3 4 5