Synergy: An Online Platform for Dialogic Peer Feedback at Scale

Synergy: An Online Platform for Dialogic Peer Feedback at Scale Erkan Er, Universidad de Valladolid, erka[email protected]a.es, Yannis Dimitriadis, Universidad de Valladolid, [email protected] Dragan Gasevic, Monash University, [email protected] Abstract: In this paper, we introduce an online platform called Synergy that is developed to support dialogic peer feedback at scale. The design of Synergy is founded on a theoretical model of dialogic feedback. In this model, dialogic feedback is conceptualized as a collaborative learning activity with three interconnected phases, involving different levels of regulated learning. Grounded in this model, Synergy comprises tools to support learning activities during dialogic feedback. These tools incorporate scripting and learning analytics support to guide learners. By using Synergy as an example, we discuss the importance of informing the design of CSCL tools with theories. Introduction The goal of this paper is to introduce a CSCL platform called Synergy that utilizes scripting and learning analytics to support dialogic peer feedback at scale and to describe its constituent model. The model outlines regulatory learning processes necessary to coordinate, maintain, and make use of dialogic feedback. Synergy assists learners in regulating their learning and collaborative activities as defined in the model. Scripting support is integrated to facilitate learners’ individual and collective actions while learning analytics support is integrated to allow learners to monitor their activities and to make changes to improve their engagement. Synergy aims to overcome existing practices which mainly focus on dialogue with instructors and lack capacity to scale dialogic feedback. In this paper, we favour theory-driven approaches to the design of CSCL tools. We support our stance by providing a detailed description of the alignment between the design of Synergy and the underlying theoretical model. We intend to envision how the design of the Synergy tool would be without a theoretical support, and we refer to several existing feedback tools that are theory-free to highlight the importance of having solid theoretical foundations to inform the design. Thus, rather than providing a detailed description of the tool, in this paper we focus more on answering the following question: why should the design of CSCL tools be informed by theories? Conceptualizing dialogic peer feedback Dialogic approaches have been proposed to boost the power of feedback for learning (Zhu & Carless, 2018). As a dialogic activity, feedback is translated into a collaborative learning activity that involves social interactions between the students to help them construct meaning from feedback and regulate their learning (Ajjawi & Boud, 2017). When learning occurs at scale, instructor dialogue with each student is unaffordable. However, in such contexts, large learning cohorts can be exploited to conduct dialogic feedback with peer support. Yet, there is a lack of theoretical models to capitalize on this potential to design solid feedback practices. The literature is limited to the definition of dialogic feedback as a process where students engage in a dialogue to understand the feedback. We present a model of dialogic peer feedback in Figure 1. To the best of our knowledge, this model is the first to provide a comprehensive conceptualization of dialogic peer feedback targeting large scale online or blended learning environments. This model postulates that dialogic feedback is composed of three interconnected phases: (1) negotiation and coordination of the feedback activities, (2) dialogic interactions for the uptake of the feedback, and (3) translation of the feedback into task progress. Within each phase, several iterations might be necessary to complete the targeted activities (e.g., several iterations of discussion between peers to build a consensus on the focus of the feedback). Additionally, these phases are not linear in nature, and they may run parallel to each other (e.g., continuing to coordinate the feedback provision activities while engaging in dialogue). That is, the model embraces flexibility to design dialogic feedback practices for different tasks and contexts. In the first phase, peers providing feedback work together to coordinate feedback provision. The goal of this phase is to ensure that later during the dialogue with the target student, peers generate coherent feedback based on a shared task understanding and participate according to a common plan and goal. Inconsistent peer feedback may disorient students and damage their learning (Hounsell, Mccune, Hounsell, & Litjens, 2008). The product of this phase is the plan for the feedback (e.g., the focus of the feedback, changes to be suggested, daily contributions to the dialogue). Peers can update their plan collectively based on ongoing dialogue with the student. In the second phase, peers provide the planned feedback and engage in dialogue with the student to support the uptake of the feedback. This phase is literally the dialogue component, which has been the main focus of the literature (Zhu & Carless, 2018). The outcome of this phase is the planning of the actions that students agree to perform to enhance their learning and to progress on the task. In the last phase, students enact the planned CSCL 2019 Proceedings 1005 © ISLS activities, aiming to translate the feedback into strategic task engagement and progress toward the learning goals. During this phase, when facing a difficulty, students can refer to the dialogue and ask for further peer support. In this model, we hypothesize that each phase is driven by different levels of regulated learning. The first phase involves the peers’ socially shared regulation of learning (SSRL) (Hadwin, Järvelä, & Miller, 2011) to negotiate and coordinate the feedback activities. In the second phase, during the dialogue learners engage in coregulation of learning (CoRL) as peers guide students’ regulation of learning (Hadwin, Oshige, Gress, & Winne, 2010). Through CoRL during, students’ transition toward self-regulation (i.e., the last phase of dialogic feedback) is enhanced (Hadwin et al., 2011). The last phase is students’ self-regulation of their learning (SRL) framed by the dialogic they were (or are being) engaged in (Winne & Hadwin, 1998). Table 1 outlines the regulatory events occurring in each phase. These events are derived based on Winne and Hadwin's (1998) model, which is chosen because it can be applied to identify the regulatory processes at both individual and social learning. Figure 1. A model of dialogic peer feedback. Table 1: Macroand micro-regulatory events in each phase of dialogic peer feedback Negotiation & coordination of feedback activities (SSRL between the peers providing feedback) Dialogic interactions for the uptake of the feedback (CoRL between the peers and the student receiving feedback) Translation of the feedback into task progress (SRL by the student) 1. UNDERSTAND THE FEEDBACK TASK 1.1. Get to know each other 1.2. Reach agreement regarding the goals of the feedback task [SCRIPT] 2. AGREE ON THE FEEDBACK 2.1. Assess the student work with a rubric 2.2. Align the perspectives toward the student work [SCRIPT] 2.3. Identify the focus of the feedback [SCRIPT] 3. PLAN THE PARTICIPATION 3.1. Identify the responsibilities and decide on the activities [SCRIPT] 3.2. Set standards for engagement in feedback provision [SCRIPT] MONITOR AND UPDATE 3.3. Monitor and evaluate the collective activities [LA] 3.4. Decide on the changes to improve the feedback activities [SCRIPT] 1. PROVIDE THE FEEDBACK 2. ENGAGE IN THE FEEDBACK DIALOGUE 2.1. Support the task understanding [SCRIPT] 2.2. Discuss the feedback with the student to enhance the understanding of the feedback 2.3. Guide the student when building the plan for the changes [SCRIPT] MONITOR AND UPDATE 2.4. Monitor and support the student’s task engagement [LA] 2.5. Support the student to monitor and evaluate the task engagement [LA & SCRIPT] 2.6. Help the student decide on the changes to improve the task engagement [SCRIPT] 1. APPLY THE PLANNED CHANGES 2. MONITOR AND UPDATE 2.1. Self-monitor and selfevaluate one's own engagement [LA] 2.2. Refer to the feedback dialogue to inquire further support [LA] 2.3. Decide on changes to improve the task engagement As indicated in Table 1, scripting and learning analytics support are incorporated to support learners’ various regulatory activities in different phases of dialogic feedback. In particular, scripts guide learners’ activities during SSRL in the first phase and during CoRL in the second phase. Given the complexity of activities, scripting support aims to shape the interactions between learners. Learning analytics support aims to enable learners to monitor and evaluate their (collective or individual) engagement and progress based on certain standards, and accordingly to make adaptations in their task perceptions, goals, and strategies. These supports are critical given the limited facilitation of instructors in crowded classrooms. CSCL 2019 Proceedings 1006 © ISLS Synergy: An online platform for dialogic peer feedback Synergy is an online platform developed to design and facilitate dialogic peer feedback in online or blended learning contexts. Synergy is designed based on the theoretical model described above. Corresponding to the phases of dialogic feedback, the Synergy platform is composed of three tools: The Coordinator (to support peers’ negotiation and coordination of feedback activities), The Dialoguer (to support peers’ feedback activities and to maintain their dialogue with the student to enhance the uptake of the feedback), and The Task Booster (to support students’ engagement on the task and help them progress). Scripting and learning analytics support are incorporated into these tools as guided by the model (see Table 1). Figure 2 below outlines the sub-components included in these three tools and illustrate their alignment with the theoretical model. As seen in the figure, every component of Synergy is designed with the purpose of supporting a certain action conceptualized in the model. Being informed by the theoretical model, the platform inherently holds an internal organization of its components that sequences and connects various activities of learners to support dialogic feedback. It is noteworthy that the complexity of the model is reflected in the design of the tool, which comprises several components for learners’ use to complete different tasks with different roles. Scripting and learning analytics support is incorporated to guide learners when they are working on these tasks during dialogic feedback. As an example, Figure 3 illustrates the design of scripting support in the Let’s Start component (to guide peers’ negotiation of the task goals) and learning analytics support in the Let’s Monitor component (to help peers monitor and evaluate their collective activities) of the Coordinator Tool. Figure 2. Design of Synergy (tools and their sub-components) and the alignment with the theoretical model. (a) (b) Figure 3. (a) Scripting support in Let’s Start and (b) learning analytics support in Let’s Monitor components. Why theory matters? We favour the position that the design of CSCL tools should be grounded in theories. To support this stance, we follow two approaches. First, we discuss the design of Synergy platform if it were a theory-free tool developed based on practical needs to facilitate dialogic peer feedback. Second, we compare Synergy with other pragmatic feedback tools from the literature to highlight the advantages of being theory driven. An alternative scenario of the design process could be rather simple if it were informed by the current practice of dialogic feedback noted in the literature. It would be driven by the technological affordances that can THE COORDINATOR Let’s Assess and Discuss Let’s Plan Let’s Monitor Let’s Start 2. Agree on the feedback 3. Plan the participation 4. Monitor and update 1. Understand the task Negotiation & coordination of the feedback activities (SSRL between the peers) THE DIALOGUER Send the Feedback Dialogue Let’s Monitor 1. Provide the feedback 2. Engage in the feedback dialogue 3. Monitor and update Dialogic interactions for the uptake of the feedback (CoRL between the student & peers) THE TASK BOOSTER Revising the Work Monitoring 2. Monitor and update 1. Apply the planned changes Translation of the feedback into task progress (SRL by the student) THE DESGIN THE MODEL CSCL 2019 Proceedings 1007 © ISLS facilitate the classical processes involved in peer review. In particular, the platform would include basic tools to allow students to upload their work to be reviewed by peers and to enable the peers to send their feedback, as well as a discussion or a chat tool to facilitate the feedback dialogue (synchronously or asynchronously). Although facilitating these tasks is useful, the Synergy platform would not be able to support learners’ critical regulatory actions during the preparation for providing feedback, during the dialogue to discuss the feedback received, and during task engagement informed by the feedback. In that case, the impact of feedback on learning would optimistically rely on presumed coherent peer feedback that satisfies the task requirements (without peers being aware of each other’s understanding of the feedback task and perspectives toward the student work) and presumed active peer participation in dialogue (without a collective goal and plan). That is, when designed with a pragmatic approach with no theoretical groundings, the Synergy platform would still help implement the peer feedback activity in practice; however, unsurprisingly it would not support critical regulatory processes of dialogic feedback and guarantee productive feedback interactions since by design these processes would not be taken into account. There exist many online feedback tools in the literature that were designed without a solid theoretical foundation. These tools were generally built on the same premise that feedback provided online offers several advantages that favour learning such as studying the feedback without time limitations and the ability to refer to it whenever needed (Hepplestone, Holden, Irwin, Parkin, & Thorpe, 2011). As a result, these tools developed independently carry very similar features to facilitate a very similar feedback task flow (e.g., uploading the work and sending the feedback). One exception is the peer review system proposed by Yang (2011), designed based on the six processes suggested by cognitive apprenticeship theory. The system included distinct features to support students during these processes. According to the results of the study, the tool supported these processes and resulted in greater learning gains (Yang, 2011). Similarly, Synergy is built based on a theoretical model and it contains particular tools to support learners’ various regulation activities during different phases of dialogic feedback. Although grounded in a certain theoretical stance, Synergy allows instructors’ (or instructional designers’) customization (e.g., changing the script content) for creating different feedback designs depending on the characteristics of the learning environment and the task. That is, we argue that having a theoretical stance should not necessarily limit the capacity of a CSCL tool for adapting to various learning settings. References Ajjawi, R., & Boud, D. (2017). Researching feedback dialogue: An interactional analysis approach. Assessment and Evaluation in Higher Education, 42(2), 252–265. https://doi.org/10.1080/02602938.2015.1102863 Hadwin, A. F., Järvelä, S., & Miller, M. (2011). Self-Regulated, Co-Regulated, and Socially Shared Regulation of Learning. In B. J. Zimmerman & D. H. Schunk (Eds.), Handbook of Self-Regulation of Learning and Performance (pp. 65–84). https://doi.org/10.4324/9780203839010.ch5 Hadwin, A. F., Oshige, M., Gress, C. L. Z., & Winne, P. H. (2010). Innovative ways for using gStudy to orchestrate and research social aspects of self-regulated learning. Computers in Human Behavior, 26(5), 794–805. https://doi.org/10.1016/j.chb.2007.06.007 Hepplestone, S., Holden, G., Irwin, B., Parkin, H. J., & Thorpe, L. (2011). Using technology to encourage student engagement with feedback: a literature review. Research in Learning Technology, 19(2), 117–127. Hounsell, D., Mccune, V., Hounsell, J., & Litjens, J. (2008). The quality of guidance and feedback to students. Higher Education Research & Development, 27(1), 55–67. https://doi.org/10.1080/07294360701658765 Winne, P. H., & Hadwin, A. F. (1998). Studying as self-regulated learning. In D. J. Hacker, J. Dunlosky, & A. C. Graesse (Eds.), Metacognition in educational theory and practice (pp. 277–304). Hillsdale, NJ: Erlbaum. Yang, Y.-F. (2011). A reciprocal peer review system to support college students’ writing. British Journal of Educational Technology, 42(4), 687–700. https://doi.org/10.1111/j.1467-8535.2010.01059.x Zhu, Q., & Carless, D. (2018). Dialogue within peer feedback processes: Clarification and negotiation of meaning. Higher Education Research & Development, 37(4), 883–897. Acknowledgements This research has been fully funded by the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement 793317, and partially funded by the European Regional Development Fund and the National Research Agency of the Spanish Ministry of Science, Innovations and Universities under project grants TIN2017-85179-C3-2-R and TIN2014-53199-C3-2-R, by the European Regional Development Fund and the Regional Ministry of Education of Castile and Leon under project grant VA257P18, by the European Commission under project grant 588438-EPP-1-2017-1-ELEPPKA2-KA, and. Access to the data used in this paper was granted by Canvas Network. CSCL 2019 Proceedings 1008 © ISLS