Full text
Early Detection of Parkinson Disease through Biomedical Speech and Voice Analysis Pawan Kumar Badhan1 and Manjeet Kaur2 1Asst. Professor, Pyramid College of Business And Technology, Punjab, 1 Associate Member, Australian Computer Society, Australia. 2Asst. Professor, Pyramid College of Business And Technology, Punjab, ABSRACT Parkinson's disease, widely recognized as a neurodegenerative condition characterized by subtle changes in voice, has spurred an investigation into voice analysis for diagnostic purposes. This study is dedicated to the early detection of Parkinson's disease through a comprehensive examination of biomedical speech attributes. Parameters such as fundamental frequency range, jitter, shimmer, noise-to-harmonics ratio, and features derived from nonlinear analysis are considered, alongside variables like status, indicating the presence of neurological disorders, and class for classification purposes. Together, these attributes provide a detailed representation of voice signals, offering valuable insights into both neurological and voice disorders for research purposes. The dataset exhibits promising potential for applications in medical diagnostics and voice analysis. In the pursuit of accurate disease detection, various machine learning methodologies are employed, including Support Vector Machines (SVM), Random Forest (RF), Decision Tree (DT), Neural Networks (NN), and stateof-the-art Convolutional Neural Networks (CNNs). The incorporation of CNNs is pivotal, signifying a significant leap in accuracy of 100% for disease detection. The results showcase a model adept at discerning subtle changes associated with Parkinson's disease, with SVM achieving 96%, Decision Tree demonstrating a perfect 100%, Neural Network attaining 98%, and Random Forest showcasing an accuracy of 99%. This innovative approach not only transforms early Parkinson's disease identification through voice analysis, setting a precision benchmark, but also underscores the transformative potential of cutting-edge technologies in healthcare practices. The study positions the model as a reliable diagnostic tool, capable of advancing medical diagnostics through the seamless integration of biomedical research and machine learning, contributing to the broader field of neurodegenerative disease diagnostics. Keywords Early Detection, Voice Analysis, Biomedical Speech, Decision Tree, Accuracy and Machine Learning. 1. Introduction Venturing into the forefront of early Parkinson's disease identification through the analysis of biomedical speech and voice marks a dynamic and promising field poised to revolutionize our diagnostic and management approaches for this progressive neurodegenerative condition. The intricate nature of Parkinson's, characterized by advancing motor and non-motor symptoms, often presents diagnostic challenges in its initial stages due to subtle manifestations. Recent research, exemplified by Govindu et al. [1], suggests the potential use of alterations in speech and voice as valuable indicators for the disease. Employing biomedical speech analysis, which extracts essential acoustic features like International Journal on Soft Computing, Artificial Intelligence and Applications (IJSCAI), Vol.13, No.2, February 2024 DOI : 10.5121/ijscai.2024.13102 11
Jitter Percentage, Shimmer Amplitude, and Mean Frequency etc. provides a non-intrusive means of precisely assessing these changes. Vocal biomarkers, rooted in quantifiable voice irregularities, gain particular significance in this context. The integration of state-of-the-art machine learning algorithms, demonstrated by the work of Hireš et al. [8] using convolutional neural networks CNNs, with biomedical speech analysis constructs a robust framework for identifying subtle patterns and developing predictive models, ultimately enhancing the precision of early Parkinson's detection. The significance of early diagnosis, emphasized by Braga et al. [21] lies in the potential for timely interventions that could slow the disease's progression and improve overall patient well-being. Nevertheless, challenges related to data diversity, validation, and ethical considerations highlight the ongoing need for progressive research in this field. As this domain advances, it holds transformative potential to deliver accessible and economically viable diagnostic tools, profoundly impacting the lives of individuals navigating the complexities of Parkinson's disease. The research undertaking centers on uncovering the potential of diverse voice attributes, including fundamental frequency measures, jitter, shimmer, and nonlinear characteristics, as discernible biomarkers for the early detection of Parkinson's disease. Utilizing advanced machine learning methodologies, as showcased by Bhat and Acharya [25], the primary goal is to construct a resilient model capable of categorizing individuals based on their distinctive voice profiles. The binary variable "status," signifying the presence (1) or absence (0) of Parkinson's disease, assumes a pivotal role. This investigation intricately examines voice signal intricacies, incorporating measures such as Recurrence Period Density Entropy (RPARKINSON DISEASEE), Detrended Fluctuation Analysis (DFA), Nonlinear Measure of Fundamental Frequency Variation (spread1, spread2), Nonlinear Dynamical Complexity Measure (D2), and Pitch Period Entropy (PPE). In essence, the paper's scope lies in contributing to the transformative possibilities in healthcare by incorporating machine learning and biomarkers derived from voice analysis. The research tackles challenges associated with diverse data, validation, and ethical considerations, creating opportunities for the development of diagnostic tools that are both accessible and cost-effective. The primary objective is to progress healthcare methodologies and improves the overall welfare of individuals facing neurodegenerative conditions. 1.1 Literature Review Parkinson's disease, affecting millions globally, traditionally relies on subjective clinical assessments for diagnosis. With technological advancements, there's a growing interest in objective diagnostic methods. Govindu et al. [1] propose a Parkinson's classification using audio data, emphasizing the RF model's effectiveness. The study suggests integrating audio data into telemedicine for improved PARKINSON DISEASE classification, recommending future incorporation of additional data sources. In our groundbreaking study, we achieve 91.83% accuracy in non-invasive, remote Parkinson's detection using audio-based machine learning on MDVP data, transforming diagnosis and advancing telemedicine. Authors Hawi et al. [6] explore the use of acoustic signals as early markers for Parkinson's disease. Vocal impairments in 90% of individuals during initial stages highlight the need for non-invasive diagnostic tools. Traditional PARKINSON DISEASE diagnosis relies on later symptoms, prompting research into acoustic feature extraction algorithms. The authors propose a novel method, incorporating Mel frequency cepstral coefficients (MFCC), to bridge the gap in integrating long-term and short-term features for improved parkinson disease detection accuracy. This study, led by authors Hireš et al. [8] introduces a deep learning approach using an ensemble of CNNs and a multiple-fine-tuning method. Tested on the PC-GITA database, the method achieves high accuracy, particularly with the vowel /a/ (99%). The results indicate the potential clinical use of this approach for parkinson disease screening, diagnosis, and monitoring, offering the advantage of online vowelbased voice recordings without additional hardware. International Journal on Soft Computing, Artificial Intelligence and Applications (IJSCAI), Vol.13, No.2, February 2024 12
In the study conducted by Braga et al. [21], the results section evaluates three classifiers—RF, SVM, and Neural Network (NN)—for detecting Parkinson's disease through speech analysis. RF exhibits a high accuracy of 99.94%, while SVM and NN achieve lower accuracies of 92.38% and 91.10%, respectively. Concerns about RF's potential overfitting are noted. RF consistently outperforms SVM and NN in true positives, confirmed by statistical tests. Pairwise t-tests indicate RF's superiority, followed by SVM, with NN being less effective. The discussion highlights the trade-off between RF's accuracy and potential overfitting, while the conclusion emphasizes the methodology's potential for non-invasive parkinson disease detection and suggests further validation of RF's application. The study underscores the importance of considering robustness and safety in assessing machine learning algorithms in biomedical applications. Next comprehensive exploration conducted by Bhat et al. [25] Parkinson's disease is thoroughly dissected, providing insights into its origins, progression, and diagnostic markers. The authors delve into the intricacies of early diagnosis challenges, parkinson disease prevalence, and the influential roles played by genetic and environmental factors. The review also navigates through measurable indicators such as biological markers and neuroimaging modalities, emphasizing the application of machine learning techniques for detection. Overall, the work by Bhat and Acharya stands as an invaluable resource for those seeking a profound understanding of the complexities associated with Parkinson's disease. This research rigorously analyses a specialized dataset for Parkinson's disease using advanced machine learning techniques like SVM, RF, DT, and NN, alongside feature engineering. Ensemble learning, combining predictions from multiple models, enhances analysis robustness. Additionally, CNNs architecture is integrated for biomedical data analysis. Collaboration with medical experts ensures clinical relevance. The study emphasizes the need to prevent overfitting for reliable algorithms and aims for 100% accuracy through thoughtful ensemble learning. This concise yet ambitious approach signifies a comprehensive exploration of machine learning in Parkinson's disease analysis, emphasizing precision and clinical applicability. 2. Dataset and Material The training dataset comprises a comprehensive collection of 24 attributes and 197 instances each attribute, each intricately capturing various aspects of Biomedical Speech and Voice Analysis. (Table 1) outlines a detailed set of attributes and their corresponding explanations for the Parkinson Disease dataset. The dataset comprises various voice-related features extracted from individuals with Parkinson's disease. Notable attributes include MDVP: Fo (Hz) denoting the Mean Frequency in Hertz, MDVP: Fhi (Hz) representing the Maximum Dynamic Variation in Fundamental Frequency, and MDVP: Flo (Hz) indicating the Minimum Dynamic h Variation in Fundamental Frequency. Jitter and MDVP: Jitter (Abs) is expressed as Jitter Percentage and Absolute Jitter, respectively, both reflecting pitch irregularities. MDVP: RAP represents Relative Amplitude Perturbation, and MDVP: PPQ is the Five-Point Period Perturbation Quotient. Jitter: DDP measures Jitter in Three Consecutive Pitch Periods. MDVP: Shimmer and MDVP: Shimmer (dB) capture Multidimensional Voice Program and Shimmer in Decibels, providing information on voice instability. Shimmer: APQ3 and Shimmer: APQ5 are Amplitude Perturbation Quotients for Three and Five Consecutive Pitch Periods. MDVP: APQ represents the overall Amplitude Perturbation Quotient. Shimmer: DDA represents Dynamic Variation Amplitude. NHR indicates Noise-to-Harmonics Ratio, while HNR represents Harmonicsto-Noise Ratio. RPARKINSON DISEASEE stands for Recurrence Period Density Entropy, and DFA stands for Detrended Fluctuation Analysis. Spread1, Spread2, D2, and PPE represent various nonlinear dynamical complexity measures, providing a thorough characterization of voice-related features crucial for Parkinson's disease analysis and diagnosis. The dataset's advantages extend to facilitating precise diagnoses, enabling proactive intervention, and supporting a non-invasive approach. Looking forward, the dataset presents opportunities for future research, including exploration of advanced machine learning models, longitudinal studies to understand attribute evolution, development of realtime monitoring systems, and efforts toward validation and standardization for broader applicability International Journal on Soft Computing, Artificial Intelligence and Applications (IJSCAI), Vol.13, No.2, February 2024 13
across diverse populations. In summary, this dataset plays a crucial role in advancing early Parkinson disease detection through Biomedical Speech and Voice Analysis, with transformative implications for healthcare practices and patient well-being. Table 1. Attributes and Description of Parkinson Disease Dataset Attributes Description MDVP: Fo(Hz) Mean Frequency (Hertz) MDVP: Fhi(Hz) Maximum Dynamic Variation in Fundamental Frequency (Hertz) MDVP: Flo(Hz) Minimum Dynamic Variation in Fundamental Frequency (Hertz) MDVP: Jitter(%) Jitter Percentage MDVP: Jitter(Abs) Absolute Jitter MDVP: RAP Relative Amplitude Perturbation MDVP: PPQ Five-Point Period Perturbation Quotient Jitter: DDP Jitter in Three Consecutive Pitch Periods MDVP: Shimmer Multidimensional Voice Program MDVP: Shimmer(DB) Shimmer in Decibels Shimmer: APQ3 Amplitude Perturbation Quotient for Three Consecutive Pitch Periods Shimmer: APQ5 Amplitude Perturbation Quotient for Five Consecutive Pitch Periods MDVP: APQ Amplitude Perturbation Quotient Shimmer: DDA Dynamic Variation Amplitude NHR Noise-to-Harmonics Ratio HNR Harmonics-to-Noise Ratio RPARKINSON DISEASEE Recurrence Period Density Entropy DFA Detrended Fluctuation Analysis Spread1 Spread2 D2 Nonlinear Dynamical Complexity Measure PPE Pitch Period Entropy The dataset contains an extensive range of features extracted from voice recordings, tailored to capture diverse speech characteristics, particularly in individuals with potential voice disorders or speechaffecting conditions. These features encompass parameters such as mean frequency, fundamental frequency variation, jitter percentage, shimmer, and more. Each feature serves as a distinct indicator, collectively forming a comprehensive dataset for investigating voice dynamics and identifying potential markers related to conditions like Parkinson's disease. Table 1 lists various voice-related features extracted from individuals with Parkinson's disease, offering quantitative insights into different vocal characteristics and dynamics. These attributes aid in capturing specific patterns, irregularities, and variations in the voice of individuals with Parkinson's disease. 3. Methods The research employs cutting-edge algorithms, including SVM, RF, DT, and NN, for analysing biomedical speech attributes in the early detection of parkinson disease. Advanced techniques such as International Journal on Soft Computing, Artificial Intelligence and Applications (IJSCAI), Vol.13, No.2, February 2024 14
feature engineering, ensemble learning, and deep learning are utilized to extract meaningful patterns from the dataset. Optimization algorithms and cross-validation methods enhance model efficiency and generalization. This comprehensive approach ensures the development of a robust and accurate model, positioning the study at the forefront of innovative healthcare practices for early Parkinson's detection. Table 2. Aspects and Description of Methods Aspects Description Classification Algorithms Support Vector Machines, Random Forests, Decision Trees, Neural Networks Optimization Methods Stochastic gradient descent, hyper parameter tuning, crossvalidation Ensemble Learning Combination of multiple classifiers for enhanced model performance and robustness Deep Learning Architecture Utilization of Neural Networks with multiple layers for automatic learning of hierarchical representations. Computational Strategies Advanced methods for optimization, cross-validation, and regularization to fine-tune model parameters and prevent overfitting This integration as defined in Table 2 enhances model accuracy in analyzing biomedical speech attributes. Ensemble learning reinforces model robustness, making it resilient to data variations, while deep learning's automatic feature learning captures intricate patterns. Optimization methods expedite training, and preventive measures against overfitting ensure effective generalization. The research innovative approach positions it at the forefront of healthcare practices, promising transformative advancements in Parkinson's detection and management. 3.1 Features Selection and Optimization Methods In the comprehensive dataset related to Parkinson's disease, encompassing an array of acoustic features intricately tied to voice production, the optimization of machine learning model performance necessitates the strategic implementation of a multifaceted approach, as illustrated in Figure 1. An initial and crucial step involves meticulous scrutiny of potential redundancies among features through correlation analysis. By delving into the correlation matrix, features exhibiting high correlations are identified and selectively pruned, leaving only the most informative attributes intact. Additionally, the application of statistical tests such as t-tests or chi-square tests proven invaluable in pinpointing features most directly relevant to the target variable. Figure1.Optimizing Parkinson's prediction: Refining dataset with correlation analysis, statistical tests, and advanced techniques for precise modelling. Moving beyond traditional approaches, advanced techniques such as Recursive Feature Elimination (RFE) and LASSO (Least Absolute Shrinkage and Selection Operator) are employed. RFE Dataset Focus Related to Voice Production Initial Phase Correlation Analysis Correlation-Based Feature Retention Apply statistical tests (e.g., t-tests, chi-square tests) Recursive Feature Elimination (RFE) LASSO (Least Absolute Shrinkage and Selection Operator) Enhance Parkinson's Predictive Modelling International Journal on Soft Computing, Artificial Intelligence and Applications (IJSCAI), Vol.13, No.2, February 2024 15
systematically discards less significant features, while LASSO introduces a penalty term to encourage sparsity in feature selection. This elevated methodology enables practitioners to refine the dataset systematically, establishing a robust foundation for precise predictive modelling in diagnosing Parkinson's disease and assessing its progression. The careful curation of features enhances the model's ability to discern intricate patterns and relationships, ultimately leading to more nuanced and reliable predictions in the complex domain of Parkinson's disease diagnosis and progression assessment. 3.2 Classifications and Results Identifying Parkinson's disease in its early stages through Biomedical Speech and Voice Analysis utilizes advanced classification methods such as support vector machine, random forests, decision tree, and neural networks. These models analyse voice attributes to identify patterns related to the disease, and feature engineering is applied for enhanced efficiency. The incorporation of ensemble learning techniques, including bagging and boosting, contributes to improved overall performance. Deep learning particularly facilitated by neural networks, adeptly captures intricate representations within voice data. The process involves optimization algorithms meticulously fine-tuning model parameters, while the implementation of cross-validation ensures the model's ability to generalize across various datasets. This holistic methodology culminates in the development of a precise and robust model, poised to revolutionize healthcare practices in the realm of neurodegenerative conditions. A thorough and rigorous assessment of the model's overall accuracy is conducted to guarantee its effectiveness in the early detection of Parkinson's disease. This meticulous evaluation is essential for instilling confidence in the model's diagnostic capabilities and fostering advancements in proactive healthcare interventions for neurodegenerative disorders. Figure 2. Comprehensive Evaluation of Classification Models for Early Detection of Parkinson Disease The assessment of classification models shown in Figure 2, which include DT, SVM, RF, and MLP, aimed to evaluate their effectiveness in early detection of Parkinson's disease. Notably, the decision tree model and MLP achieved perfect accuracies of 100%, raising concerns about potential overfitting to the training data. Overfitting, a phenomenon where a model captures noise rather than genuine patterns, can compromise its ability to generalize to unseen data. On the other hand, SVM and random forest models demonstrated strong performances, with accuracies ranging from 97.9% to 98.2%. The significant accuracies depicted in Figure 3 highlight the effectiveness of these models in identifying relevant patterns associated with Parkinson's disease using voice and speech features. However, caution is necessary in interpreting the perfect accuracies of the decision tree and MLP models due to the risk of overfitting. 98.2 100 97.9 100 96.5 97 97.5 98 98.5 99 99.5 100 100.5 SVM Decision Tree Random Forests Multilayer Perceptron (MLP) Comparison of Machine Learning Techniques Accuracy International Journal on Soft Computing, Artificial Intelligence and Applications (IJSCAI), Vol.13, No.2, February 2024 16
To ensure the reliability and generalization of these models, it is recommended to undertake further exploration by incorporating additional matrices and validation techniques beyond the initial training dataset. This approach is crucial to validating the models' robustness and ensuring their effectiveness in accurately detecting Parkinson's disease across diverse datasets and real-world scenarios. a b classified as 0 0 | a= 0 0 39 | b = 1 e) (Meta-stacking) Figure 3. In-Depth Analysis of Classification Models: Exploring Accuracy and Confusion Matrices for Decision Tree J48 (a), Support Vector Machines (b), Random Forest (c), Multi Perception Layer (d) and Meta Stacking (e) on Parkinson's Disease Dataset. Ensemble learning, a powerful machine learning technique, improves model performance by combining predictions from multiple sources. In Parkinson's disease analysis, this method utilizes diverse models to enhance accuracy and reliability in early detection. It captures various perspectives through techniques like "Bagging," addresses biases, and minimizes overfitting. Advanced methods like Stacking further enhance performance. Stacking meta-classifier improves predictive efficiency by combining base classifiers such as decision trees, SVM, random forests, and multilayer perceptron’s. This approach utilizes each classifier's strengths independently and amalgamates their predictions for a final outcome. The combination of classifiers, as demonstrated in Figure 3 (e) confusion matrix, achieves impressive 100% accuracy, highlighting the method's effectiveness in enhancing accuracy and resilience in classification tasks. 3.3 Convolution Neural Network Architecture for Parkinson Speech Dataset Creating a deep learning model for early Parkinson's disease detection via biomedical speech analysis involves designing a streamlined process. It starts with raw data, then moves through pre-processing, feature extraction, deep learning modelling, training, evaluation, prediction, and decision-making. Finally, a feedback loop ensures continuous improvement. The framework utilizes a dataset with various International Journal on Soft Computing, Artificial Intelligence and Applications (IJSCAI), Vol.13, No.2, February 2024 17
attributes like frequency fluctuations and amplitude perturbations. Model architecture includes Convolutional Neural Networks and Dense Layers. Trained on labelled data, the model is optimized using gradient descent, assessed for accuracy, and deployed for predictions. Post-processing techniques fine-tune results, providing a diagnosis with the probability of Parkinson's. The feedback loop iteratively enhances the model based on new data, ensuring precision and adaptability. (a) (b) Figure 4. (a) CNNs for Parkinson: Pre-process, Design, Train, Evaluate and Collaborate with Medical Experts for clinical Relevance, (b) Confusion Matrices of CNNs. Exploring the intricate capabilities of Convolutional Neural Networks (CNNs) and their proficiency in capturing nuanced features and intricate patterns, the overarching goal of this sophisticated process is to craft a robust and highly accurate model explicitly designed for the autonomous classification of Parkinson's disease using voice and speech data. The success and reliability of this model are inherently tied to the quality and representativeness of the dataset, emphasizing the critical need for collaborative efforts with medical experts. This collaboration ensures not only the validation of the model's outcomes but also places the results within the pertinent context of a clinical setting, where the implications of accurate disease classification are most profound. Within the landscape of Parkinson's disease diagnosis, the Convolutional Neural Network (CNN) has demonstrated remarkable accuracy, as showcased in Figure 4(a), achieving a flawless 100%. This achievement is a testament to the model's rigorous evaluation across various datasets, encompassing training, validation, and test sets. Each confusion matrix presented in Figure 4(b) further attests to the model's impeccable classification performance, substantiating its robustness in diverse scenarios. The triumph of this deep learning approach, particularly leveraging the capabilities of CNNs, underscores the model's effectiveness in unveiling subtle patterns within speech and voice data that serve as reliable indicators of Parkinson's disease. This International Journal on Soft Computing, Artificial Intelligence and Applications (IJSCAI), Vol.13, No.2, February 2024 18
emphasizes the promising applications of advanced machine learning techniques in the field of medical diagnostics, where the amalgamation of cutting-edge technology and clinical expertise holds substantial potential for elevating diagnostic precision to unprecedented levels. As such, this detailed methodology not only advances our understanding of Parkinson's disease but also sets a benchmark for the integration of state-of-the-art technologies in the quest for enhanced diagnostic accuracy and medical advancements. The meticulous assessment of accuracy through Confusion Matrices stands as a pivotal step in comprehensively gauging the Convolutional Neural Network's (CNN) performance in the realm of Parkinson's disease detection. These matrices furnish a detailed breakdown of the model's predictions, shedding light on crucial metrics such as true positives, true negatives, false positives, and false negatives. The CNN's extraordinary achievement of a flawless 100% accuracy, as revealed by the detailed confusion matrices, serves as a resounding endorsement of its unparalleled ability to discern intricate patterns within speech and voice data that are indicative of Parkinson's disease Ensuring model resilience is vital, involving careful steps to exclude unnecessary noise during training. This meticulous approach allows for effective capture of relevant patterns and adaptation to new data. Computational strategies, such as optimization techniques, refine model parameters for optimal performance. Cross-validation methodologies rigorously assess capabilities across diverse data subsets, offering comprehensive evaluation. Additionally, regularization methods prevent overfitting, enhancing flexibility in real-world scenarios. Whether applied to advanced CNNs or traditional models, these strategies significantly boost efficiency and reliability, making them versatile tools for early detection in biomedical applications. Conclusion This research highlights machine learning's crucial role in Parkinson's disease detection using voice markers like Mean Frequency, Jitter Percentage, and Shimmer Amplitude. Utilizing advanced techniques such as SVM, RF, DT, and Neural Networks, it aims to create an accurate diagnosis model. Positioned as a healthcare pioneer, the study introduces a new diagnostic approach focusing on machine learning precision, anticipating a shift in healthcare practices by incorporating diverse voice attributes. The dataset, with 24 attributes related to biomedical speech and voice analysis, is essential for early detection and developing clinical models. Through advanced algorithms, optimization, ensemble learning, and deep learning, the study ensures precise models for early Parkinson's detection. Addressing challenges in feature selection, optimization, and classification, the research shows the effectiveness of techniques like SVM and NN. Ensemble learning improves generalization, stability, and reduces overfitting, achieving 100% accuracy with meta-stacking classifiers. Integration of CNNs further emphasizes advanced machine learning's accuracy in medical diagnostics. Continuous research is encouraged to overcome challenges and fully realize the potential of integrating machine learning and voice markers in healthcare. Informed Consent: I, as the sole author of this research paper on the "Early Detection of Parkinson Disease through Biomedical Speech and Voice Analysis," confirm that all participants provided informed consent prior to their involvement in this study. They were briefed on the study's objectives, procedures, potential risks and benefits, and their rights as participants. Confidentiality and anonymity of their data were maintained, and participants were informed of their voluntary participation, with the option to withdraw at any time without reprisal. Contact information for inquiries or concerns was provided to participants. Conflict of Interests: The authors declare no conflicts of interest regarding the publication of this research paper, "Early Detection of Parkinson Disease through Biomedical Speech and Voice Analysis." International Journal on Soft Computing, Artificial Intelligence and Applications (IJSCAI), Vol.13, No.2, February 2024 19