RECOMMENDATION SYSTEM FOR E-LEARNING STUDENT ORIENTATION BASED ON MACHINE LEARNING ALGORITHMS AND QCM

Journal of Theoretical and Applied Information Technology 15th June 2025. Vol.103. No.11 © Little Lion Scientific ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195 4989 RECOMMENDATION SYSTEM FOR E-LEARNING STUDENT ORIENTATION BASED ON MACHINE LEARNING ALGORITHMS AND QCM MOULAY AMZIL1, AHMED ELGHAZI2, MOHAMED ERRITALI 3 1Data science for sustainable Earth laboratory (DataEarth), Sultan Moulay Slimane University, Beni Mellal 23000, Morocco 2Department of Computer Science FST Beni Mellal, Sultan Moulay Slimane University, Beni Mellal,Morocco 3Data science for sustainable Earth laboratory (DataEarth), Sultan Moulay Slimane University, Beni Mellal 23000, Morocco E-mail: 1 [email protected] , 2 [email protected], 3[email protected] ABSTRACT This article is part of a student guidance project. It involves a recommendation and classification system (E_orientation) based on a real MCQ test. This test is administered to the student and provides random answers that are then used by our recommendation system. We use binary modelling of the answers, which produces a vector of data for each student at the end of the test. For modelling and classification, we use several machine-learning algorithms to optimize the accuracy of the recommendations. The results of the experiment show that Random Forest is the best model (85.93% accuracy), ahead of SVM (80.43%). KNN achieves 76.10%, and the Decision Tree, Logistic Regression and Naive Bayes algorithms have the lowest performance (Accuracy ≤ 67.03%). We can therefore improve the students’ method of orientation by basing it on simple technical questions. This will also improve their contribution to the labor market. Keywords: Recommendation System, School Guidance, Machine Learning, E-Learning Platform, MCQ Test 1. INTRODUCTION For several years now, educational and career guidance mechanisms have been at the heart of educational, political and social debates [36]. At the same time, the worldwide success of artificial learning and intelligence techniques is undeniable. The education sector has been particularly affected by this technological revolution, arousing the interest of numerous researchers [17], [21], [22], [24]. Their work focuses mainly on the issue of predicting students’ university choices.... School and career guidance is a major issue for students, teachers and guidance counsellors, particularly when it comes to making decisions about their future studies [42], [43]; however, thanks to artificial intelligence, which is playing an increasing role in this field, particularly with the application of machine learning, which can analyze large quantities of data on students’ skills, interests and performance to provide more personalized and precise guidance recommendations, the future of education is becoming increasingly complex. In this context, our research will attempt to address the issue of educational and vocational guidance at the decision-making level. To do this, we use machine-learning algorithms group students according to the results of a test administered on an e-learning platform. Each student is invited to take this online MCQ test, which is then transformed into an automatic correction and a vector of binary data. This will be the input for a classification system (E_orientation) of students, after these results, in groups for each specialty. This system will make it easier to recommend courses of study and avoid the problems of choosing the wrong subject. However, several factors can influence a student’s orientation: the family’s social level, the student’s attendance record, his ambitions, etc.... Most of these factors are qualitative and cannot be measured directly, so to collect them you need to conduct a survey of the student’s life. In the literature, the dynamics of educational and vocational guidance have been the subject of number of research projects over the years. On the one hand, most of these studies present good results Journal of Theoretical and Applied Information Technology 15th June 2025. Vol.103. No.11 © Little Lion Scientific ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195 4990 based on artificial learning algorithms. On the other hand, the data used for decision-making is always incomplete and does not in fact reflect the various attributes linked to the student’s life: his or her attendance and technical skills. The contribution of this article is therefore to use machine-learning models to analyze students’ responses and predict the educational paths that correspond to their academic skills. The distinguishing feature of our study is the utilization of machine learning algorithms to provide students with personalised guidance, informed by their performance in an online multiple-choice question (MCQ) test. In contradistinction to conventional academic guidance systems, this system provides personalized recommendations based on students' actual performance, thereby enabling a more dynamic and individualized approach. To develop our research topic, we decided to organize and divide our work into four sections structured as follows:  The first section presents the fundamental concepts and the general context of academic guidance assisted by Machine Learning.  The second section consists of presenting the state of the art in existing work on Machine Learning applied to academic guidance.  The third section will present the methodology adopted, which will focus on the methods used and the work proposed.  The fourth section presents the experimental results obtained. The final section summarizes the results, followed by a discussion of future work prospects. These are aimed at improving educational support, reducing misdirection and fostering the professional development of young people. 2. BACKGROUND 2.1 Machine learning algorithm Machine learning algorithms can process a wide range of input data and producing forecasts, thereby contributing to cost reduction, the enhancement of business procedures, and the improvement of customer service. These algorithms are employed to develop efficient and intelligent systems[1]. Understanding the math principles behind these algorithms is vital for a full comprehension of their methods and interpretations. The selection of the most appropriate algorithm depends on various factors in a study[2]. 2.1.1 Supervised learning Supervised machine learning algorithms are utilized for outcome prediction through training with labeled data. Different supervised learning algorithms include gradient boosting, random forest, K-nearest neighbors, support vector machines(SVM), decision trees, artificial neural networks, naive Bayes, and binary logistic regression. The performance of these algorithms has been assessed based on metrics such as accuracy, area under the curve, sensitivity, specificity, and kappa statistics. Random forest has been found to be the best-performing algorithm in terms of accuracy and precision in multiple studies[3]. 2.1.2 Unsupervised learning Unsupervised machine learning algorithms is a type of algorithm that is designed to create a learning paradigm for the sake of learning, its categorize and group data without labeled training data, and can identify patterns and relationships within the data, leading to the discovery of hidden structures and insights. The motivation of unsupervised learning is that although the data that goes through the unsupervised learning algorithms has a rich inherent structure, the ground truth and the metric used for training are typically sparse. This implies that the majority of what is learned by the algorithm should come from the input data structure rather than applying a specific understanding to a practical task[4]. 2.1.3 Semi-Supervised learning Semi-supervised learning is a pedagogical framework that amalgamates annotated and unannotated data to enhance supervised learning endeavors in situations where annotated data is limited or costly. It is of great interest in machine learning and data mining. Several models and algorithms have been proposed in the field, including self-training, mixture models, co-training, Multiview learning, graph-based methods, and semisupervised support vector machines[5].Figure 1 shows the types of machine learning. Journal of Theoretical and Applied Information Technology 15th June 2025. Vol.103. No.11 © Little Lion Scientific ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195 4991 Figure 1 : Types of machine learning 2.2 Classification algorithms In machine learning, classification is a key aspect of supervised learning, where systems learn from input data to classify new observations. Its importance grows with the complexity and unpredictability of real-world data [6], [26], [27], [28]. This study aims to identify learners’ personalities based on their specific traits. Classification typically involves two phases: a learning phase, where a model is built from training data, and a testing phase, where the model classifies new data. To improve accuracy, fuzzy sets [7] are used to handle uncertainty. The study compares five major algorithms Logistic Regression, Random Forest, KNN, Decision Tree, Naive Bayes, and SVM to determine the most effective approach. 2.2.1 Naive Bayes Algorithm Naive Bayes is a classification algorithm based on Bayes’ theorem of conditional probabilities. This theorem describes the probability of an event occurring based on prior knowledge of conditions related to that event. It states that if we have a hypothesis H and evidence E, it is possible to calculate the probability of H being true given E. This algorithm is intuitive, simple, and robust, making it widely used for predictive modeling, with the main assumption that each feature is treated independently [8], [13]. Figure 2 illustrates a black triangle representing a new sample instance that needs to be classified into either the “star” class or the “square” class, depending on which category achieves the highest posterior probability. To classify this object, a circle is drawn around the triangle to include several neighboring points, allowing the algorithm to evaluate the surrounding instances and determine the most appropriate classification. (Figure 2). Figure 2 : Naive Bayes classification 2.2.2 Decision tree The decision tree constitutes one of the most prevalent classification methods, and also one of the most antiquated. It emerged in the 1960s in the domains of psychology and sociology research [25], [9], [13]. This model is presented in the form of an intuitive tree structure that categorizes data items according to a hierarchical structure. Decision tree nodes generally have several levels, the first of which is called the root node that contains all the data (figure 3 ). Figure 3. Decision tree classification 2.2.3 K-nearest neighbors The k-nearest neighbors (KNN) algorithm is a classification algorithm that is both simple and fundamental in nature. It is one of the oldest Journal of Theoretical and Applied Information Technology 15th June 2025. Vol.103. No.11 © Little Lion Scientific ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195 4992 algorithms of its kind and functions by retaining a set of available cases for the purpose of classifying new examples based on a similarity measure [10],[35]. This non-parametric classification system is notable for the fact that it completely avoids the problem of probability densities. The term “K” in KNN refers to the number of nearest neighbors considered during the classification process, signifying that decisions are made based on the labels of the K nearest samples [11],[12],[13]. As illustrated in Figure 4, the KNN algorithm functions by classifying new objects. For instance, when K = 1, a new object (e.g., a triangle) is classified as a “circle”. Conversely, when K = 4, the same object is classified as a “star” (figure 4). Figure 4: KNN classification 2.2.4 Support vector machine The Support Vector Machine (SVM) is a simple yet powerful algorithm [25], [29],[30],[31] capable of producing accurate results with efficient computing power. It is one of the best-known classification algorithms, relying on linear or nonlinear methods to separate data into different classes. The fundamental principle of SVM is to utilize a hyperplane (or line in the case of two-dimensional space) to differentiate between classes while maximizing the marginal distance between them, thereby minimizing classification errors. As illustrated in Figure 5, this marginal distance corresponds to the space between the hyperplane and the closest instance of each class, termed the support vector [12]. (figure 5 ). Figure 5 : SVM classification 2.2.5 Logistic regression Logistic regression is a robust and wellestablished mathematical model of supervised classification, which is utilized extensively in the field of statistics for the estimation of the probability of an event occurring as a function of a set of input variables [12], [32], [33], [34]. This model operates primarily with binary data, that is to say, it evaluates the probability of an event occurring (P = 1) or not occurring (P = 0). To illustrate this application, consider the context of educational guidance, where the available data comprises two possible classes. In this scenario, logistic regression can be employed to ascertain the probability of receiving favorable guidance (P = 1) or unfavorable guidance (P = 0). (figure 6). Figure 6 :Logistic regression classification 2.2.6 Random Forest (RF) Introduced by Breiman [37], is an ensemblelearning algorithm that combines several decision trees via bagging to reduce overfitting and improve generalization. Its robustness is based on the random selection of data subsets (bootstrap) and variables at each split [36], optimizing predictor diversity. In classification, RF excels in a variety of domains (e.g. medical imaging), thanks to its ability to handle non- Journal of Theoretical and Applied Information Technology 15th June 2025. Vol.103. No.11 © Little Lion Scientific ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195 4993 linear interactions. Its partial interpretability via variable importance [38] makes it a versatile tool for research and industry (figure 7). Figure 7: Random Forest classification 3. RELATED WORKS Machine learning, otherwise termed artificial learning, is currently one of the most extensively practised fields. It has a profound impact on almost every aspect of life, and human beings derive considerable benefit from the advantages offered by this technique. Within the field of education, a significant amount of research has been dedicated to optimising the school learning process, as evidenced by the references cited in [19] and [28]. These references include a number of studies that have explored methods for guiding pupils using artificial learning approaches.In educational research, many researchers have implemented the integration of artificial learning in conjunction with other methodologies, with the aim of generating conclusions that are beneficial for both students and schools. Articles [14], [15], [16], [18] and [13] describe the design of a framework for predicting academic orientation, based on artificial learning. The system developed is based on proven artificial intelligence algorithms, such as the Bayes network and the decision tree. The results obtained by the decision tree are particularly promising. In his article, K. Kivuyirwa [14] presents a system for guiding students towards appropriate studies using the usual machine learning techniques (SVM, Bayes, RNN, decision tree) [20]. The data used is based on questionnaires given to students in order to derive a vector of characteristics for each one. The results obtained are satisfactory, particularly for the SVM technique. In his article [16], F. Ouatik presents a student guidance system based on attendance and the number of absences during lessons. To model the data, the author uses the Big Data technique [18],[23]. The results obtained by neural networks, Bayesian networks and KNNs are then compared. This analysis leads to the conclusion that the best results are obtained by Naive Bayes. In his article [17], F. Ouatik proposes an automatic student guidance system based on machine learning and the processing of large quantities of data [18]. This system uses three parameters relating to the student: the mark obtained in each subject, the number of absences in each subject and the general trend. The Naïve Bayes, SVM, Random Forest and RNN algorithms are then implemented in order to obtain results. A comparison of these algorithms led to the conclusion that Naïve Bayes gives good results. Existing literature has extensively explored the use of artificial intelligence (AI) in educational guidance, but few studies have incorporated a dynamic MCQ test to personalize recommendations. Previous studies, such as those in [13], [14], [15], [16], [17], [18] and [20], have shown the benefits of applying machine learning algorithms in education, but have not used an interactive test to refine recommendations. This study aims to fill this gap by proposing a system based on MCQ answers, enabling academic orientations to be refined in a more personalized way. 4. THE WORK PROPOSES 4.1 Research methodology and contribution Our study proposes an innovative academic guidance system. The system is based on the analysis of answers to a multiple-choice questionnaire (MCQ). The questionnaire covers six subjects and is integrated into an online learning platform. We use the scores obtained as predictive indicators. We combine machine learning classification algorithms with the principles of professional decision-making. Our aim is to optimize students’ academic careers. Journal of Theoretical and Applied Information Technology 15th June 2025. Vol.103. No.11 © Little Lion Scientific ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195 4994 Our approach differs from traditional methodologies, which are based on stable criteria such as average academic grades, in that it is resolutely dynamic. This is based on interactive data from ongoing assessments, enabling the system's predictions to be constantly personalized and optimized. In addition, the integration of an example of online learning means that the analysis of learners' skills and profiles can be progressively refined, guaranteeing more personalized and progressive recommendations. This study has two aims: to create a recommendation system to help students choose their educational path and to find models to suggest the best educational path for each student. To do this, an online test was set up on an e-learning platform to collect data on students. The second objective is to create and evaluate an educational path recommendation system using machine-learning algorithms. To do this, algorithms were used on the data collected on the platform. This makes it possible to create profiles of students and offer them personalized recommendations for their orientation. The conceptual model of this study is predicated on the hypothesis that students' answers to a multiple-choice (MCQ) test can be used to predict their academic aptitudes and preferences. The machine-learning model was designed to classify students into different majors based on their performance on the test. The model under discussion is predicated on the hypothesis that academic aptitude is manifest in the results of the test, thus enabling the generation of recommendations that are tailored to the individual. 4.2 classification In this work, we propose an innovative approach for classifying secondary schools into specialty groups using supervised machine learning. More specifically, we have exploited six algorithms - Decision Tree, Logistic Regression, K-Nearest Neighbors (KNN), Support Vector Machine (SVM), Naïve Bayes and Random Forest - which have demonstrated their effectiveness in this field. To implement and evaluate these models, we used Python, as well as the Anaconda and Jupyter Notebook tools. Anaconda provides an integrated environment for scientific computing and data science, while Jupyter Notebook makes it easy to write, run and share code using a user-friendly interface. The dataset studied comes from a qualifying high school, where each student (Ei, (i=1,..., 6000) is represented by a feature vector (Vi) derived from their performance on an online test. Figure 9 provides a simplified illustration of the classification process and the work carried out. Figure 9. The proposed system 4.3 Data description : The dataset used was collected from a real test administered to teachers in four secondary schools, using an online learning platform. The sample selected was that of the baccalaureate aimed at classifying students for university careers. A database of 6,000 baccalaureate holders was compiled, divided into the different streams shown in table 1 below: Journal of Theoretical and Applied Information Technology 15th June 2025. Vol.103. No.11 © Little Lion Scientific ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195 4995 Table 1: Specialities of the baccalaureate Baccalaureate specialties SH L SVT SP Girls 515 420 1105 1005 Boys 485 58 0 895 995 SP: Physical Sciences SVT: Life and Earth Sciences SH: Human Sciences L: Literary The test proposed for students is given in the form of technical MCQ questions, with each specialty linked to a group of questions. Our test is made up of 60 questions, divided into 10 questions per specialty according to the following table: Table 2: Distribution of questions in the test Geology Q5, Q10,Q16, Q19,Q29, Q36,Q42, Q45,Q52, Q49 Biology Q2, Q11,Q18, Q21,Q26, Q33,Q39, Q48,Q55, Q58 Mathematics Q1, Q7,Q14, Q22,Q30, Q32, Q37, Q43,Q56, Q57 Physics Q3, Q8,Q17, Q23,Q27, Q31, Q41, Q47,Q54, Q59 Chemistry Q4, Q12, Q13, Q20, Q28, Q34, Q40, 46, Q53, Q60 Economics Q6, Q9,Q15, Q24,Q25, Q35,Q38, Q44,Q50, Q51 We used a database containing the answers of 6000 students (boys and girls) from various specialties in the baccalaureate. Each student was invited to take an online test consisting of technical questions on an e-learning platform. The results obtained were then analysed using Machine Learning algorithms to recommend the most appropriate university course for each student. Figures 10 and 11 (see appendix) show respectively the number of correct answers per materials and Correlation matrix between materials 5. RESULTS AND DISCUSSION After collecting the results of the tests carried out on the e-learning platform for six subjects: Mathematics, Economics, Geology, Biology, Chemistry and Physics, an initial classification was made. The results obtained are presented in Table 4. Machine learning algorithms were used to generate personalized recommendations, either for a single subject or for a combination of subjects, according to the following categories: - A = {MIP}: Mathematics, Computer Science, Physics - B = {Economy} - C = {BCG} : Biology, Chemistry, Geology - D = {A, B}: Mathematics, Computer Science, Physics, Economics - E = {B, C} : Economics, Biology, Chemistry, Geology - F = {A, C}: Mathematics, Computer Science, Physics, Biology, Chemistry, Geology - G = {A, B, C}: Mathematics, Computer Science, Physics, Economics, Biology, Chemistry, Geology - Non oriented Figure 11 shows the results of classifying the top five students according to their scores in different subjects (Mathematics, Physics, Biology, Chemistry, Geology, and Economics). To predict their specialty (A, B, C, E, G), Figure 11: Classification results for some students The performance of the different Machine Learning models is summarized in Table 3 below and in Figure 12. They have been evaluated using four key metrics: accuracy, precision, recall and F1score. These measures are used to determine the most effective model for our recommendation system. Journal of Theoretical and Applied Information Technology 15th June 2025. Vol.103. No.11 © Little Lion Scientific ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195 4996 Table 3: The performance of the different Machine Learning models The results show that the Random Forest model performs best, followed by the SVM with a score of 80.43%. KNN scored 76.10%, showing that these two models performed acceptably well. On the other hand, the Decision Tree, Logistic Regression and Naïve Bayes algorithms performed the worst, with an accuracy of 67.03% or less. After applying the Random Forest algorithm, Table 4 below shows the results of our guidance system, which recommends a specialization to baccalaureate students after assessing their skills via a test. The students are classified into different branches: A, B, C, D, E, F, G and “Not oriented”, corresponding to the various categories of specialization. Each column shows the number of students referred to each specialism. These recommendations are generated by analyzing the students’ performances in order to propose the most suitable orientation. Table 4: Results of our proposed guidance system Figure 12: Comparison of models In percentage terms, Table 5. Below are the results. Table 5. Results of our proposed guidance system in percentage Algorith m Accurac y Precisio n Recall F1Score Logistic Regressio n 0.670275 0.636421 0.67027 5 0.63931 5 Decision Tree 0.677769 0.664356 0.67776 9 0.66795 7 Random Forest 0.859284 0.856568 0.85928 4 0.85561 6 SVM 0.804330 0.788851 0.80433 0 0.79170 4 KNN 0.761032 0.756425 0.76103 2 0.75641 0 Naive Bayes 0.661948 0.633411 0.66194 8 0.61845 8 Specialties Number of students recommended A 298 B 1102 C 2691 D 701 E 800 F 503 G 0 Non – oriented 197 Total 6000 specialties % Number of students recommended A 4,97% B 18,37% C 44,85% D 11,68% E 13,33% F 8,38% G 0,00% Non - oriented 3,28% Total 100% Journal of Theoretical and Applied Information Technology 15th June 2025. Vol.103. No.11 © Little Lion Scientific ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195 4997 Figures 13 to 18 show a comparison of the classification results obtained with different machine learning models. Figure13: Classification recommended by Decision Tree Figure14: Classification recommended by logistic Regression Figure15: Classification recommended by naive Bayes Figure16: Classification recommended by Rondom forest Figure17: Classification recommended by SVM Figure18: Classification recommended by KNN