Comparative Evaluation of Machine Learning Classifiers for Predictive Modelling of Datasets

Data management plan (DMP) Comparative Evaluation of Machine Learning Classifiers for Predictive Modelling of Datasets ML_Class_Exp Version Effective date Description of document/changes 1.0 2/12/2025 First version of the DMP – created for the start of the project 1.1 2/12/2025 Added Zenodo DOI and finalized DMP text, replaced placeholders Level of distribution This DMP is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0). It is publicly available under: 10.5281/zenodo.17785523 2 ML_Class_Exp DMP version 1.0 Project details Project Coordinator Principal Investigator Aman Bhardwaj, ORCID iD: 0009-0002-8215-5243, TU Wien, Project Coordinator Contact person (responsible for data management and DMP) Aman Bhardwaj, ORCID: 0009-0002-8215-5243, TU Wien Contributors Aman Bhardwaj, ORCID: 0009-0002-8215-5243, TU Wien (Project Coordinator, Contact Person) Start date 2025-11-18 End date 2025-11-29 Funder None Funding programme, grant number None Internal project number None List of acronyms DMP data management plan RDM research data management MLP Multi-Layer Perceptron DT Decision Trees GPC Gaussian Process Classifier TU Technische Universität (TU Wien) ARFF Attribute-Relation File Format DOI Digital Object Identifier FAIR Findable, Accessible, Interoperable, Reusable JSON JavaScript Object Notation (metadata file format) CSV Comma Separated Values ML_Class_Exp DMP version 1.0 3 Content INHALTSVER ZEIC HNIS INTRODUCTION 4 Science Europe practical guide, FAIR data 4 Relevant Policies and Guidelines 4 1. DATA DESCRIPTION 5 1a Lists of datasets that will be reused or produced 5 1b Data generation and reuse 6 2. DOCUMENTATION AND DATA QUALITY 6 2a Data organisation, metadata and documentation 6 2b Data quality control 6 3. STORAGE AND BACKUP DURING RESEARCH PROCESS 7 3a Storage and backup facilities 7 3b Data security and protection of sensitive data 7 4. LEGAL AND ETHICAL REQUIREMENTS 7 4a Personal data 7 4b Intellectual property rights and ownership 7 4c Ethical issues 7 5. DATA SHARING AND LONG-TERM PRESERVATION 8 5a Data publication and access conditions 8 5b Long-term preservation and deletion of data 9 6. RDM RESPONSIBILITIES AND RESOURCES 9 6a RDM-roles and responsibilities 9 6b Resources 9 4 ML_Class_Exp DMP version 1.0 Introduction Science Europe practical guide, FAIR data A DMP is a structured document that keeps record of what research data is created and what happens to that data during and after a project. It helps with planning the research process and defining responsibilities in a research project involving several researchers or institutions. For writing this DMP, we followed the recommendations of Science Europe as they reflect the guidelines agreed upon by the major funders in Europe. To make our data FAIR, they generally will be treated according to the following criteria:  We will make our data findable, by uploading it to a data repository that provides a persistent identifier and adding relevant metadata.  We will make our data accessible by providing open access to data, wherever possible. In cases, where open access is not possible, we will provide meaningful metadata plus contact information for access requests.  We will make our data interoperable by providing and describing data in a way that is common within our domain by using the same file formats, schemas and vocabularies. We will provide good documentation for all our datasets.  We will make our data reusable by adding metadata and comprehensive Readme files to all published datasets. The descriptions include details on the methodology used, analytical and procedural information. In case of publication, licenses for code and data will always be assigned and clearly marked. Relevant Policies and Guidelines  European Commission’s document on Ethics and Data Protection: https://ec.europa.eu/info/funding-tenders/opportunities/docs/20212027/horizon/guidance/ethics-and-data-protection_he_en.pdf  Other (e.g. from a project partner) ML_Class_Exp DMP version 1.0 5 1. Data description 1a Lists of datasets that will be reused or produced Produced datasets dataset ID title type format estimated volume contains sensitive data P1 Wine Dataset for Machine Learning Classification Experiment Other .data <100 MB no P2 Colleges Dataset for Machine Learning Classification Experiment Other .arff <100 MB no Description for "Wine Dataset for Machine Learning Classification Experiment": This dataset contains 178 wine samples with 13 chemical analysis features and a categorical class variable indicating wine type. It is sourced from the UCI Machine Learning Repository (DOI: 10.24432/C5PC7J). The dataset is used for supervised classification experiments to compare the performance of Multi-Layer Perceptron, Decision Tree, and Gaussian Process Classifier models. Features include Alcohol, Malic acid, Ash, Alcalinity of ash, Magnesium, Total phenols, Flavanoids, Nonflavanoid phenols, Proanthocyanins, Color intensity, Hue, OD280/OD315 of diluted wines, and Proline. There are no missing values. Description for "Colleges Dataset for Machine Learning Classification Experiment": This dataset contains information for ~7800 US colleges with 50 institutional attributes including demographics, tuition, and post-graduation earnings. The target variable is the type of institution (private or public). It is sourced from OpenML (ID: 42159). It is used for classification experiments comparing MLP, Decision Tree, and Gaussian Process Classifier models. Features include percent_pell_grant, UNITID, school_name, city, state, zip, and school_webpage. Some attributes have missing values which went through the preprocessing steps. Technical resources for "Wine Dataset for Machine Learning Classification Experiment": Any text editor (e.g., Notepad, VS Code) or spreadsheet software (e.g., Excel) can open the file. For analysis in Python, pandas library is recommended.Technical resources for "Colleges Dataset for Machine Learning Classification Experiment": Spreadsheet software (Excel, LibreOffice) or data analysis software in Python (pandas) or R. For ARFF files, Weka software or scipy.io.arff in Python can be used. 6 ML_Class_Exp DMP version 1.0 1b Data generation and reuse Methods and software used for data generation and reuse The research data will be generated by preprocessing and analyzing the Wine and Colleges datasets. Preprocessing includes handling missing values, normalization/scaling, and encoding categorical features. The processed datasets will then be used to train and evaluate three supervised machine learning models: Multi-Layer Perceptron (MLP), Decision Tree (DT), and Gaussian Process Classifier (GPC). Software used includes Python with libraries such as pandas, scikit-learn, and numpy. The datasets themselves are reused from public repositories: the Wine dataset from the UCI Machine Learning Repository and the Colleges dataset from OpenML. 2. Documentation and data quality 2a Data organisation, metadata and documentation The data is organized in a clear structure: original datasets, preprocessed for cleaned and preprocessed datasets, code for scripts, and output files such as plots. Filenames follow a consistent naming convention including dataset name (e.g., wine_preprocessed.csv). Versioning is handled by incrementing version numbers in filenames for each major update and keeping all previous versions for reproducibility. Additionally, a README documents file contents, processing steps, and version history. Metadata will include detailed information about each dataset, including: Title of the dataset (e.g., Wine Dataset for Machine Learning Classification Experiment) Description of the dataset and its contents Creators/Contributors with names and ORCID identifiers Date of creation or collection Version of the dataset File format (e.g., CSV) Variables/features and their types, units, and missing value information License for reuse (e.g., CC BY 4.0) Source/Provenance of the original data (e.g., UCI ML Repository, OpenML) Related works or publications DOI for the dataset itself This metadata will be provided in both the repository entry and accompanying metadata.json and README files, to ensure discoverability, proper citation, and reusability by other researchers. This will help others to identify, discover and reuse our data. Additionally, we will provide common metadata such as title, description or keywords when publishing data in open access repositories. In such a case, we will follow the default template provided by the repository, such as Data Cite Metadata or Dublin Core. A far as possible, we will use controlled vocabularies for our data to allow inter-disciplinary interoperability and machine-actionability. Documentation will be provided through a combination of files included in the repository: README.md – This file will describe the datasets, their features, units, target variables, and any preprocessing steps performed. It will also explain the folder structure, file naming conventions, and how to reproduce the experiment. metadata.json – This file will include structured metadata about datasets, code, and outputs, making it machine-readable and helping others discover and reuse the data. Code documentation – Python scripts/notebooks will include inline comments and function docstrings explaining the logic, preprocessing, model training, evaluation, and visualization steps. Parameters & environment information – Any libraries, versions, or configurations required to run the code will be specified. Sample outputs – Example CSVs of processed datasets, evaluation metrics, and plots will be included to allow validation of results. 2b Data quality control The following data quality checks will be done: repeated samples or measurements, standardised data capture and peer review of data. ML_Class_Exp DMP version 1.0 7 3. Storage and backup during research process 3a Storage and backup facilities For the duration of the project, storage and backup of data will be ensured by Aman Bhardwaj (acting as the person responsible for data management and DMP) in cooperation with the system operator. The data will be stored on the servers of TU Wien. P1 (Wine Dataset for Machine Learning Classification Experiment), P2 (Colleges Dataset for Machine Learning Classification Experiment) will be stored on TUcloud: TUcloud is a sync&share service provided by Campus IT for TU Wien members. It runs on Campus IT servers and offers features known from public cloud systems, such as Dropbox, for example, the exchange of data with authorised persons. Deleted files can be recovered within 180 days. 3b Data security and protection of sensitive data We pay strict attention to compliance with the relevant institutional and national data protection policies listed in the introduction of this document. At this stage, it is not foreseen to process any sensitive data in the project. If this changes, advice will be sought from the data protection specialist at TU Wien, and the DMP will be updated. Access to data during research: dataset ID selected project members all other project members the public P1 reading only reading only reading only P2 reading only reading only reading only Any data security incidents or breaches will be reported and managed according to TU Wien’s IT and data protection policies. 4. Legal and ethical requirements 4a Personal data At this stage, it is not foreseen to process any personal data in the project. If this changes, advice will be sought from the data protection specialist at TU Wien, and the DMP will be updated. 4b Intellectual property rights and rights of use The following individual(s) hold rights and control access to the project data: Aman Bhardwaj (Project Contributor / Contact Person): Has full rights to control access for all datasets, including the Wine dataset and the Colleges dataset. Responsible for determining which users can read or write each dataset. Selected project members: Granted read-only access to both datasets. They can use the data for analysis but cannot modify or publish it. Other project members / collaborators: Granted readonly access if applicable. They can view and use the datasets for reproducibility or related research but cannot alter or redistribute them. This ensures that the project lead retains overall control while allowing appropriate access to collaborators for reproducibility and reuse. 4c Ethical issues No particular ethical issue is foreseen with the data to be used or produced by the project. This section will be updated if issues arise. 8 ML_Class_Exp DMP version 1.0 5. Data sharing and long-term preservation 5a Data publication and access conditions As far as possible, obtained datasets will be published in repositories. Details on access conditions, reuse licenses, reasons for restrictions, etc. are collected in the table below. dataset ID access conditions estimated publication date location for publication (repository) PID license P1 Open 2025-09-28 TU Wien Research Data DOI CC-BY-4.0 P2 Open 2025-09-28 TU Wien Research Data DOI CC-BY-4.0 Repository description: TU Wien Research Data is an institutional repository of TU Wien to enable storing, sharing and publishing of digital objects, in particular research data. It facilitates the funders' requirements for open access to research data and the FAIR principles by making research output findable, accessible, interoperable, and reusable. A DOI is assigned to each dataset published in TU Wien Research Data. This service is developed by the TU Wien Center for Research Data Management and hosted by TU.it. https://researchdata.tuwien.at/ Methods or software needed to access and use data: Potential users will need: A standard text editor or spreadsheet software (e.g., Excel, LibreOffice Calc) to open CSV or tabular files. Python or R environments with common libraries (e.g., pandas, scikit-learn, NumPy) to process, analyze, and model the datasets. Jupyter Notebook or similar tools for running example code and reproducing analyses. ML_Class_Exp DMP version 1.0 9 5b Long-term preservation and deletion of data dataset ID location for long-term storage minimum retention period (≥ 10 years) foreseeable research uses and/or users P1 TU Wien Research Data 10 years Target audience: Researchers, data scientists, and students in the fields of machine learning, data analytics, chemistry, and educational research. Potential reuse: Wine dataset: Useful for testing and benchmarking classification algorithms, chemical analysis studies, and machine learning experiments. Colleges dataset: Useful for predictive modeling of educational outcomes, demographic analysis, institutional studies, and testing machine learning models on real-world tabular data. Reason for reuse: Both datasets provide well-structured, real-world data that can be used to validate models, compare algorithms, or replicate experiments, supporting reproducible research. P2 TU Wien Research Data 10 years 6. RDM responsibilities and resources 6a RDM-roles and responsibilities The Project Coordinator, will direct the data management process overall, with the research assistants responsible for ensuring metadata production, day-to-day cross-checks, back-up and other quality control activities are maintained. 6b Resources There are no costs dedicated to data management and ensuring that data will be FAIR.