A Survey on Approaches to Modeling Collaborative Practices in E-Learning Platforms

A Survey on Approaches to Modeling Collaborative Practices in E-Learning Platforms Sara Ghaoui 1, Sofiane Mounine Hemam2, and Tarek Djouad3 1ICOSI Laboratory, Abbes Laghrour University, Khenchela, Algeria, [email protected] 2Abbes Laghrour University, Khenchela, Algeria, National High School of Cyber-Security, Algiers, Algeria. , [email protected], [email protected] 3ICOSI Laboratory, Abbes Laghrour University, Khenchela, Algeria, [email protected] Abstract Evaluating collaborative practices for peers/groups of learners in collaborative e-learning is a crucial issue in distance learning platforms. It is a complex task that requires the development of advanced methods and tools to ensure continuous and real-time evaluation of collaboration. The aim of our work is to propose and implement an algorithm, method, and tool for evaluating collaborative learning. We seek to identify and extract collaborative fragments by applying operators to modeled traces in order to pinpoint sequential episodes of collaboration. Additionally, we aim to design and compute collaboration indicators. The objective of this work is to simplify the process of evaluating a group of learners on a collaborative distance-learning platform, enabling non-computer scientists to design their own collaboration indicators and automate their calculation. Keywords:Collaborative e-learning, distance learning platforms, Sequential episodes of collaboration, collaboration indicator. 1 Introduction Distance learning platforms are environments that support, accompany, and validate learning, where learners collaborate to achieve a common goal[13]. Collaborative e-learning, as a pedagogical approach, relies on the sharing and construction of knowledge among learners using technology[11]. Collaboration is defined as ”the mutual engagement of participants in a coordinated effort to solve a problem together”[20]. Cooperative work, on the other hand, ”is defined as a form of work organization where each operator is responsible for his or her part. Collaborative work, in contrast, is a form of work organization in which everyone is responsible for the whole”[8]. In e-learning, the term ’collaboration’ is generally preferred over ’cooperation,’ despite both terms meaning ’working together.’ The aim of learning is not simply to complete a task collectively and produce a final product, but to ensure that all learners achieve the same concepts and reach the desired objectives. The main goal of collaborative learning is to enable a group of learners to work together through a computer system to achieve a collaborative task. This task may involve completing a project, solving an exercise, or understanding a concept. Collaboration can take the form of sharing, exchanging, or discussing information, ideas, and concepts, enabling learners to develop the cognitive skills and knowledge necessary to enhance their competencies. According to [2], collaborative working increases employee productivity and results in higher-quality outcomes. Moreover, regardless of their status, employees report higher levels of satisfaction and responsiveness. The lack of information about the level of collaboration within a group or between groups presents challenges for teachers who wish to evaluate learners’ collaborative behavior. They must answer questions such as: Who participates? Who doesn’t? Who helped whom? Who did what? These questions are often difficult to answer when analyzing the dynamics of a collaborative group. To understand the behavior of a learner or group of learners involved in e-learning, and to provide relevant and adequate information to the teacher or trainer monitoring progress, whether globally or individually, it is necessary to track traces. These traces can be defined as a set of temporally situated elements. 1.1 Research Problem The problem addressed in much of the research in related fields is how to evaluate the collaborative work of one or more groups of learners on a collaborative e-learning platform. 123 In the context of our work, several research questions were posed: 1. How can collaborative activities be detected within an e-learning platform? 2. How can these collaborative practices be evaluated? The central problem concerns how to analyze the traces obtained during a collaborative learning session in order to answer the above questions. Our main contribution is to propose an algorithm, a method, models, and a tool for extracting and evaluating collaborative practices through modeled traces. 1.2 Research Objectives To achieve these results, we have set multiple objectives: 1. Proposing an algorithm for extracting sequential episodes of collaboration in order to identify collaboration fragments. 2. Proposing an MDA-based method for calculating collaboration indicators. 3. Proposing a tool for the design and automatic calculation of collaboration indicators. 1.3 Paper Organization To achieve the above objectives, the rest of the paper is organized as follows:  The first part provides the theoretical framework for collaborative work and its evaluation.  The second part presents our contribution and the proposed approach to achieving our goals.  Finally, we conclude the paper with a summary of our work. 2 State of the Art 2.1 Computer-Assisted Collaborative Learning Collaborative learning is a learner-centered approach in which students actively construct their knowledge, with the instructor playing the role of facilitator. This model contrasts with the traditional teacher-centered approach. The integration of Information and Communication Technologies (ICT) in distance learning platforms has transformed pedagogy by fostering the emergence of collective learning through tools such as forums, wikis, and blogs. These platforms overcome obstacles like physical distance and learner diversity, enhancing collaboration and mutual support. The role of different actors (teachers, tutors, and learners) is crucial for ensuring a conducive learning environment. This section explores the advantages, limitations, and challenges associated with collaborative e-learning, with a particular focus on the evaluation of collaboration to prevent isolation and improve learner engagement. It also discusses the definition of collaborative learning, the approaches to learning supported by distance learning platforms, the roles of various actors in these learning environments, and the importance of evaluating collaboration in an e-learning context. 2.1.1 Definition of Collaborative Learning There are several approaches to learning, including traditional (teacher-centered) and collaborative (learner-centered) approaches. According to Henri and Lundgren-Cayrol[11], ”Collaborative learning is an active approach in which the learner works to construct his or her own knowledge. The trainer plays the role of learning facilitator, while the group participates as a source of information, a motivator, a means of mutual help and support, and a privileged space for the collective construction of knowledge.” In this type of learning, the learner takes responsibility for their own personal development and engages in collaboration with group members to achieve a common goal—learning. Throughout this process, collaboration within the group allows members to share, negotiate, and validate their newly constructed knowledge. 124 2.1.2 Towards Collaborative Learning Supported by Distance Learning Platforms The deployment of Information and Communication Technologies (ICT) in distance learning platforms has brought about significant changes in pedagogy. The variety of collaborative tools available on these platforms, such as forums, wikis, blogs, and others, has fostered the emergence of collective learning. A collaborative learning environment supported by a distance learning platform promotes the desire to exchange, communicate, and share, as well as to participate and collaborate. 2.1.3 Actors in a Collaborative Learning Situation In a collaborative learning environment supported by a distance learning platform, the following roles are typically considered the main ones: teacher, IT designer, tutor, learner, and administrator[20]. 2.1.4 The Role of the Tutor The online tutor assumes various roles, such as coach, facilitator, instructor, and evaluator. They adopt and implement strategies aligned with the learning/teaching paths chosen for the collaborative learning situation. Additionally, they hold a supervisory role, supporting learners, stimulating learning, and communicating rules within the learning environment[18]. 2.1.5 Advantages of Collaborative E-Learning In addition to the flexibility of time and place that collaborative e-learning offers to learners, it also fosters cognitive and personal growth. Learners develop by working together toward a common goal[8]. In this collaborative learning process, the learner adapts to the benefits and demands of collaboration and learns to use discussion and negotiation in their interactions with group members to build knowledge. 2.1.6 Limitations of Collaborative Learning Despite the advantages of collaborative e-learning, there are several limitations to consider, whether in terms of balance, heterogeneity, group size, or assessment procedures[8]. This work focuses more specifically on the limitations related to assessment procedures. 2.1.7 Why We Should Evaluate Collaboration in an E-Learning Situation One of the main problems with most e-learning platforms is student drop-out. A primary factor contributing to this issue is the absence of support and social relationships, which can lead to feelings of isolation. 2.2 Evaluating Collaborative Processes Evaluating the collaborative e-learning process is a delicate task that has prompted researchers to engage in various theoretical and methodological investigations to address its challenges. Before the discovery of the concept of M-traces, the description of elements stored during an elearning session was limited to textual documentation, such as log files or RSS feeds. This made human exploitation of the data very challenging and almost impossible when dealing with large volumes of interactions. In 2006, Yannick Pri´e and his colleagues [21] introduced a new computer object called the ”M-trace” (modeled trace), which associates each collection of observed elements with a model to formally describe the structure and content of the trace. 2.3 Interaction Indicators In the context of learning, the DPULS project[4] has provided a clearer definition of the concept of an indicator: ”It is a pedagogically significant variable, calculated or established using observed data, that reflects the quality of interaction, activity, and learning.” Indeed, a collaboration indicator is one that provides information on the level of participation, collaboration, and the degree of involvement of learners in collaborative work. 125 According to Djouad[6], each indicator has a name, a textual specification, and a calculation rule. As shown in Figure 2, to arrive at the final value of an indicator, several stages must be considered: from the collection of the necessary traces to their processing, to the formalization of the calculation methods, and finally, to the visualization and interpretation of the data obtained. In all these stages, the central step is the modeling and calculation of indicators. Figure 1: Indicator life cycle 3 Related work and scientific positioning Recently, numerous studies have focused on evaluating the effectiveness of monitoring indicators in collaborative systems[22]. Integrating these evaluation metrics has proven valuable for tracking and enhancing learner engagement in online environments, making them essential for adaptive and personalized learning experiences[16]. In [17], the authors examine how automated analytics can assess collaboration skills by analyzing group speech data. They analyze communication patterns, detect engagement levels, and identify collaboration dynamics in real time. The ICALTS project [12] identified indicators through the analysis of students’ interactions at the metacognitive level, which could help learners self-regulate or evaluate their activity. Similarly, in [3], the authors proposed indicators to assess learning activity based on discussion forum posts. These indicators are also used to validate the quality of asynchronous discussions without requiring an in-depth content analysis. In [5], the authors developed a mechanism allowing students to examine a collaborative task from different perspectives. They proposed a set of metric-based indicators to evaluate the group’s output as the final product of collaborative work. Meanwhile, in [9], researchers focused on calculating collaboration indicators in Moodle by utilizing learning analytics and data mining techniques to define specific collaboration indicators such as participation rate, interaction frequency, response time, message length, and role distribution in group discussions. In [14], specific algorithms were designed to extract and compute collaboration indicators within a structured framework spanning multiple dimensions. However, these indicators are often designed in an ad hoc manner and are typically tailored to a specific platform, with little consideration for reusability. A second category of research focuses on developing methods and tools to facilitate the design and computation of collaboration indicators. The authors of [6] proposed a model-driven engineering approach to simplify the analysis of traces in learning situations. Their method involves saving the transformations applied to the traces to facilitate their reuse, allowing the transition from raw trace data to indicator models. In [10], a computational tool called Genidic was developed to assist users in the development, management, and computation of indicators. It employs a rule-based system where traces serve as facts, and indicator calculation processes are defined as rules. Similarly, [19] introduced Usage Tracking Language (UTL), which allows the definition of indicators in a design pattern-like format to enhance capitalization and reuse. However, UTL initially lacked formal tools to specify how indicators should be computed from collected traces. To address this limitation, [15] proposed a new version called DCL4UTL, which enables indicators to be modeled in a structured way that supports automation and reuse, providing valuable insights for teachers and tutors. 126 Other approaches rely on multi-agent systems. For example, [7] proposed a system based on fuzzy logic techniques to evaluate the level of learner collaboration. The inputs to this system are calculated indicators derived from trace analysis. Similarly, in [14], the authors developed a cloud-based Learning Management System (LMS) that integrates a multi-agent system to collect, analyze, and filter traces, facilitating the computation of interaction indicators that promote collaboration. The below table 1 summarizes and compares the above work with our proposition. Work The suggested method supports the design /calculation of the indicators Indicator type Used approach The implementation system is open or closed [19] The design Indicators in CEHL  Designing of indicator in a form similar to a design pattern opened [6]The calculation Indicators in MOODLE platform  Oriented Model Transformation.  based on MDE.  Using an m-tracebased system(selfdeveloped). Closed [10] The design and calculation Collaboration indicators  Using a collaboration indicator pattern for the design.  Using a Computation method oriented Artificial intelligence and based on rule-logic.  Using a rulebased system(selfdeveloped) Opened [15] The design and calculation Indicators in CEHL  Enrich UTL with formal language to formally describe the calculation method.  Indicator computation using a DCL4UTL interpreter (selfdeveloped) integrated into a trace analysis tool. Opened 127 [1]The calculation Collaboration indicators in collaborative e-learning systems Ad-hoc Manner Closed [14]The calculation Collaboration indicators in e-learning systems  Oriented Model Transformation.  Basing on MDE.  Using A multiagent system (selfdeveloped) and an m-trace based system(KTBS). Closed Our work The design and calculation Collaboration indicators in an e-learning systems  The design is based on the DCIN Model(self proposal)  The calculation is oriented modeltransformation based on MDE.  The sequences of transformations are automatiquelly generated using DCIN-AGSET(self developped). Opened Table 1: The comparaison between related works and our proposition Compared to the above works, our proposition considers the design and computation of collaborative indicators in e-learning systems. Therefore, our approach can be summarized as follows:  The first aspect focuses on the computation of collaboration indicators in e-learning systems, independently of any platform used. For this purpose, we propose to apply a model transformation approach and an MDA-based process to obtain collaboration indicator models.  The second aspect focuses on the design of collaboration indicators in e-learning systems. For this purpose, we propose a formal model that facilitates the design of valid and meaningful collaboration indicators according to the teacher’s observation needs.  Based on the application development process supported by Model-Driven Architecture, the calculation of the collaboration indicator can be seen as a model transformation process, where the trace model is passed through a sequence of transformations to arrive at the collaboration indicator model. For the same previous need of automatic computation of collaboration indicators and the acquisition of specific computer skills that cannot be achieved by a non-computer scientist teacher, we propose to automate the generation of sequences of transformations.  The third aspect focuses on how to obtain sequences of transformations. For this, we will propose a system that ensures the automatic generation of sequences of transformations to be applied in the m-trace base system to arrive at the indicator model. 128 4 Our Contribution Our main objective is to propose an algorithm, a method, and a tool to facilitate the evaluation of learners’ collaborative behavior in a collaborative learning environment. To achieve this goal, our contribution will be divided into three main parts: 4.1 The first contribution The first part of our contribution involves proposing an algorithm for extracting collaborative episodes from a trace. The application of this algorithm will extract collaborative fragments within an interaction trace. This algorithm will be:  The first tool that teachers can use to detect the collaborative behavior of their learners, and  Supported by statistical and mining functions to detect specific aspects of the trace, such as extracting frequent collaborative episodes, the number of frequent sequential episodes, etc. The collaborative fragment extraction algorithm we propose is based on the frequent sequential episode extraction algorithm, with the key difference that the result is a ”collaborative sequential episode.” Let’s take the following example:  Let the following trace be:  Let the collaboration obsels be:  Applying the frequent sequential episode extraction algorithm, we obtain the following frequent episode with a seuil greater than or equal to 2:  Applying our method to propose, we’ll obtain the following episodes with a length greater than or equal to 2:  The calculation of collaboration indicators will be based on these results. 4.2 The Second Contribution The second part of our contribution involves proposing an approach that enables non-computer-scientist teachers to design and calculate their own collaboration indicators to detect and evaluate learners’ collaborative behavior in collaborative e-learning environments. Our approach will be based on a model-driven architecture, where the calculation of indicators can be seen as a series of model transformations. After the design stage, we aim to automate the calculation of the teacher-designed indicators by automatically generating the transformation sequence needed to obtain the corresponding indicator model. 4.3 The Third Contribution The third part consists of proposing a real case study using a learning platform with an online collaborative learning situation and an evaluation grid to implement and adjust the proposed algorithms and methods. This part is organized as follows: 1. Proposing a collaborative learning situation on a learning platform to collect the necessary data interaction traces during collaborative learning sessions. 129 2. Proposing a model for the concept of ’collaborative trace’ within the trace based system’KTBS’, which will serve as an ontology for validating trace models. 3. Developing operators that facilitate the construction of collaborative traces, including a confidence factor. 4. Implementing the evaluation algorithms and integrating several proposed indicators to assess the collaborative learning process (Figure 2 summarizes these steps). Figure 2: Collaboration evaluation stage in a collaborative learning situation. 5 Conclusion Collaborative distance learning represents a valuable approach for enhancing group work and developing collaborative skills. After years of research, the interactions between learners using various distance learning tools can be captured through a computer object called M-Trace. In this research, we focused on evaluating collaborative practices within a learning activity. Our first contribution is the development of an algorithm to detect and extract collaborative fragments from interaction traces. By adapting the principle of frequent sequential episode extraction, we propose a novel algorithm tailored to extract sequential episodes of collaboration. The second contribution lies in evaluating collaborative practices. We propose a Model-Driven Architecture (MDA)-based method to calculate collaboration indicators, coupled with a tool that enables non-computer-scientist teachers to design meaningful collaboration indicators. This will allow teachers to generate the transformation sequence necessary to obtain the corresponding collaboration indicator model. Finally, we plan to conduct a case study to further refine and adjust the evaluation algorithms. This case study will help validate the proposed methods and tools, ensuring their practical applicability in real-world educational settings. References [1] A. Acosta and J. Lee. Multimodal learning analytics for predicting student collaboration satisfaction. In Proceedings of the 17th International Conference on Educational Data Mining (EDM 2024), 2024. [2] M. Arnaud. Current limitations of online collaborative learning (in french). STICEF (Information and Communication Sciences and Technologies for Education and Training), 10:7, 2003. 130 [3] T. Bratitsis and A. Dimitracopoulou. Monitoring and analyzing group interactions in asynchronous discussions with the dias system. In Y. A. Dimitriadis, I. Zigurs, and E. G´omez-S´anchez, editors, Groupware: Design, Implementation, and Use, volume 4154, pages 54–61. Springer Berlin Heidelberg, 2006. [4] C. Choquet and S. Iksal. Modeling and constructing usage traces of a learning activity: A language approach for reengineering a cehl (in french). 2007. [5] C. A. Collazos, L. A. Guerrero, J. A. Pino, S. Renzi, J. Klobas, M. Ortega, M. A. Redondo, and C. Bravo. Evaluating collaborative learning processes using system-based measurement. 18, 2007. [6] T. Djouad, A. Mille, C. Reffay, and M. Benmohamed. Engineering of activity indicators from modeled traces in a computer environment for human learning (in french). 16(1):103–139, 2009. [7] A. El Mhouti, M. Erradi, and N. El Makhfi. A multi-agent system of semantic analysis and filtering of modeled traces to calculate interaction indicators favoring collaboration in lms. In International Conference on Intelligent Systems and Advanced Computing Sciences (ISACS), pages 1–7, 2019. [8] C. Gangloff-Ziegler. The obstacles of collaborative work, steps and organizations. 10(3):95–112, 2009. [9] S. Garc´ıa-Sastre, A. Mart´ınez-Mon´es, and Y. Dimitriadis. Detecting collaboration skills to calculate indicators in moodle. In Proceedings of the 5th International Conference on Technological Ecosystems for Enhancing Multiculturality, pages 1–8. ACM, 2017. [10] ´ E. Gendron. A conceptual framework for the development of collaboration indicators from activity traces (In French). PhD thesis, Claude Bernard UniversityLyon I, 2010. [11] F. Henri. Collaborative learning in virtual mode (fr). 2002. [12] ICALT. Proceedings of the ieee international conference on advanced learning technologies. 2004. [13] D. Laurillard, O. Martin, W. Barbara, and H. Ulrich. Implementing Technology-Enhanced Learning. 2009. [14] I. Matazi, A. Bennane, R. Messoussi, R. Touahni, I. Oumaira, and R. Korchiyne. Multi-agent system based on fuzzy logic for e-learning collaborative system. In International Symposium on Advanced Electrical and Communication Technologies (ISAECT), pages 1–7, 2018. [15] D. P. T. Ngoc. Specification and design of analysis services for the use of a computer environment for human learning (in french). Technical report, 2004. [16] D. Nguyen, S. Yingchareonthawornchai, V. Tekken Valapil, and S. S. Kulkarni. Precision, recall, and sensitivity of monitoring partially synchronous distributed programs. Distributed Computing, 34(4):319–348, 2021. [17] S. PraharaJ, M. Scheffel, and M. et al. Schmitz. Towards automatic collaboration analytics for group speech data using learning analytics. Sensors, 21(9):3156, 2021. [18] G. M. Rafique and M. N. A. Khan. Integrating learning analytics and collaborative learning for improving student’s academic performance. International Journal of Advanced Computer Science and Applications, 12(12), 2021. [19] N. Randriamalaka, S. Iksal, and C. Choquet. Elicitation of indicators for the re-engineering of educational scenarios: A trace-based approach using utl (in french). 2008. [20] S. A. Salloum, M. Al-Emran, K. Shaalan, and A. Tarhini. Factors affecting the e-learning acceptance: A case study from uae. Education and Information Technologies, 2019. [21] L. S. Settouti. Trace-based systems for human learning (in french). 2006. [22] M. V´asquez-Berm´udez, J. Hidalgo-Larrea, F. Orozco Lara, and S. Segura Santana. Effectiveness of monitoring indicators in the architecture of a collaborative system. In Technologies and Innovation, pages 191–202. Springer, Cham, 2022. 131