HeFDI Data Week: FAIRly Simple – How can generative AI Support FAIR RDM?

HeFDI Data Week 2025 Abstract: FAIR Research Data Management in interdisciplinary large-scale projects is very challenging. Data formats, acquisition processes, and infrastructure are highly heterogeneous. Furthermore, many tasks in FAIR RDM are tedious and complex for the researchers. This talk will discuss the potentials of generative AI to support FAIR RDM on examples from a large-scale project. About the HeFDI Data Week: The HeFDI Data Week 2025 is a multi-day online event series which is offered in the context of the nationwide "Digitaltag”. The series is aimed at researchers, teachers, students and anyone who wants to learn more about data management, FAIR data and code. Over the course of the week, various topics, developments and challenges related to research data will be covered, such as tools and offers for disciplines from NFDI consortia, legal aspects of research data management as well as research data management and artificial intelligence. The HeFDI Data Week is a programme of the federal state initiative HeFDI - Hessian Research Data Infrastructures, which is funded by the Hessian Ministry of Higher Education, Research, Science and the Arts (HMWK). DOI-Link: https://doi.org/10.5281/zenodo.15422345 ; Licence information: Creative Commons Attribution 4.0 International (CC BY 4.0) Date Topic Presenter(s) 27. June 2025 FAIRly Simple – How can generative AI Support FAIR RDM? Prof. Dr. Sandra Geisler (RWTH Aachen University) gefördert durch FAIRly Simple – How can generative AI Support FAIR RDM? Prof. Dr. Sandra Geisler HeFDI Data Week 2025 27.06.2025 Generated with Bing „More than 70% of researchers have tried and failed to reproduce another scientist’s experiments, and more than half have failed to reproduce their own experiments.“ M. Baker, Nature, 2016 FAIRly Simple – How can generative AI support FAIR RDM? | Prof. Dr. Sandra Geisler | DSMA | 27.06.2025 4 EXC "Internet of Production" - Smart, Connected Production https://www.iop.rwth-aachen.de/ “The vision of the Internet of Production (IoP) is to enable a new level of cross-domain collaboration by providing semantically adequate and contextual data from production, development and usage in real time at an appropriate granularity.” © Dr. Martin Riedel FAIRly Simple – How can generative AI support FAIR RDM? | Prof. Dr. Sandra Geisler | DSMA | 27.06.2025 5 Barriers for FAIR Research Data Management Substantial time and effort Heterogeneity in data cultures, processes, infrastructure Missing / too low rewards Missing data (management) literacy Proprietary data types, forms, formats, tools Complexity, e.g., finding & using vocabularies Loss of competitive advantages [Feger et al., 2019; Kim et al., 2023; Jussen et al., 2023] FAIRly Simple – How can generative AI support FAIR RDM? | Prof. Dr. Sandra Geisler | DSMA | 27.06.2025 6 LLMs – Typical tasks •Complete and create text •Summarizing, answering questions, translating, ... •Intelligent assistants •Office applications •Teaching & learning •Programming •Robot control •Science communication https://github.com/features/copilot Robotics in the Age of Generative AI with Vincent Vanhoucke, Google DeepMind | NVIDIA GTC 2024 https://www.youtube.com/watch?v=vOrhfyMe_EQ https://www.microsoft.com/de-DE/microsoft-365/copilot? FAIRly Simple – How can generative AI support FAIR RDM? | Prof. Dr. Sandra Geisler | DSMA | 27.06.2025 7 Adaptation of LLMs – Method 1: Foundation Models & Fine-tuning Foundational Model Unstructured Text Images Speech Signals Structured Data Fine-tuned Model FAIRly Simple – How can generative AI support FAIR RDM? | Prof. Dr. Sandra Geisler | DSMA | 27.06.2025 8 Adaptation of LLMs – Method 2: Retrieval-Augmented Generation (RAG) Support Agent Application LLM (Chat Model) LLM (Embedding Model) Vector Index User Question Vector Similarity Search Implicit Results Explicit & Implicit Results Natural Language Results Queries Knowledge Graph 010…010 011..0011 ..n 10011..00 1….010… 0100101 … 1…. 010….010 01101011 01001101 FAIRly Simple – How can generative AI support FAIR RDM? | Prof. Dr. Sandra Geisler | DSMA | 27.06.2025 9 Research data management cycle Writing proposals •funder-specific text •job profiles, e.g., data stewards Preprocessing and transformation •recommend and implement format changes •suggest and produce pipeline, e.g., as Jupyter notebook •suggest methods for feature selection Search for resources •similar experiments •related publications, datasets •relevant tools, repositories •potential collaborators Data modelling •create model •check for irregularities •suggest standard language Data acquisition •create input masks •declaration of consent text Structure •suggest naming conventions •suggest storage structure Choice of storage •informing about legal conditions for different storage solutions •best practices for established solutions, e.g., Coscine Choice of accessibility •identifying sensitive data •suggesting concepts for data protection Data quality •suggest dimensions and metrics •suggest cleaning methods Writing DMPs •suggest DMP tool, template •suggest standards •automatic creation by abstract •help texts on demand •create data summarization Experiment design •create documentation scheme •suggest algorithms, analysis methods Publication •suggest data market places/repositories •consulting on licenses Archival •create archival guidelines •create archival pipelines Researcher Project management Funder Data stewards Central Services [Geisler & Kim, 2023] FAIRly Simple – How can generative AI support FAIR RDM? | Prof. Dr. Sandra Geisler | DSMA | 27.06.2025 16 Generating Coscine Application Profiles from MitM IoP MitM Data Sharing + Meta data Derived IoP Models-in-theMiddle IoP Models-in-theMiddle Meta data Upload Support IoP MitM Data Application Profile Ontology Slide Credits: István Koren FAIRly Simple – How can generative AI support FAIR RDM? | Prof. Dr. Sandra Geisler | DSMA | 27.06.2025 17 Prompting Techniques – Improve Response Quality Inner Monologue* Chain-of-Thought [Wei et al., 2022] Tree-of-Thought [Yao et al., 2023] Have an inner monologue on how to solve the task first. Only then give a final answer. Solve the task by solving subtask 1, then subtask 2, then give the final answer. Propose multiple solutions for subtask 1. Evaluate the suggestions. Proceed with subtask 2 with the knowledge from the prior step. Derive the final answer. * https://medium.com/contact-research/think-before-you-speak-the-inherent-statelessness-of-large-language-models-ceff8ab20ff1, last accessed 3 June, 2025 See also: [Schulhoff et al., 2024] and https://huggingface.co/docs/transformers/v4.52.3/tasks/prompting FAIRly Simple – How can generative AI support FAIR RDM? | Prof. Dr. Sandra Geisler | DSMA | 27.06.2025 18 Chain-of-thought for ontology creation Slide Credits: Martin Görz FAIRly Simple – How can generative AI support FAIR RDM? | Prof. Dr. Sandra Geisler | DSMA | 27.06.2025 19 KONDA - Overview Slide Credits: Martin Görz FAIRly Simple – How can generative AI support FAIR RDM? | Prof. Dr. Sandra Geisler | DSMA | 27.06.2025 20 KONDA – The User Interface Slide Credits: Martin Görz Prompt or perish – The publication process in times of genAI | Prof. Sandra Geisler | DSMA | 11.02.2025 21 Qualitative Evaluation Study setup: 10 researchers from RWTH Aachen (computer science, biology, economics, materials science) Slide Credits: Martin Görz FAIRly Simple – How can generative AI support FAIR RDM? | Prof. Dr. Sandra Geisler | DSMA | 27.06.2025 22 Conclusion & next steps LLMs helpful to support FAIR RDM Understand and annotate data sets Search for existing ontologies, combination and generation of new ontologies Simple tasks, such as metadata extraction But… Manual checks and some knowledge about ontologie and KGs necessary Understanding the data is limited, but finetuning and RAG can improve this Next steps Intensive analysis with domain experts Benchmarks for qualitative analysis Integration of methodologies for ontology generation and quality checks Graph reuse, versioning, logging Coscine integration FAIRly Simple – How can generative AI support FAIR RDM? | Prof. Dr. Sandra Geisler | DSMA | 27.06.2025 23 Contact us JProf. Dr. Sandra Geisler Principal Investigator IoP Data Stream Management and Analysis (DSMA) Phone: +49 241 80-21501 E-Mail: [email protected] Soo-Yon Kim, M.Sc. Data Steward IoP Data Stream Management and Analysis (DSMA) Phone: +49 241 80-21514 E-Mail: soo-yon.[email protected]achen.de Prof. Dr. Stefan Decker Principal Investigator IoP Databases and Information Systems (DBIS) Phone: +49 1590 4338009 E-Mail: [email protected] FAIRly Simple – How can generative AI support FAIR RDM? | Prof. Dr. Sandra Geisler | DSMA | 27.06.2025 24 References Baker, M. 1,500 scientists lift the lid on reproducibility. Nature 533, 452–454 (2016). https://doi.org/10.1038/533452a Feger, S. S., Dallmeier-Tiessen, S., Schmidt, A., & Woźniak, P. W. (2019, May). Designing for reproducibility: A qualitative study of challenges and opportunities in high energy physics. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (pp. 1-14). https://doi.org/10.1145/3290605.3300685 Jussen, I., Möller, F., Schweihoff, J., Gieß, A., Giussani, G., & Otto, B. (2024). Issues in inter-organizational data sharing: Findings from practice and research challenges. Data & Knowledge Engineering, 102280. https://doi.org/10.1016/j.datak.2024.102280 Kim, S. Y., Hillemacher, S., Decker, S., Rumpe, B., & Geisler, S. (2023). Designing and Implementing Practicable Data Management Plans in Large-Scale Projects. Bausteine Forschungsdatenmanagement, (3), 1-12. https://doi.org/10.17192/bfdm.2023.3.8571 Geisler, S. and Kim, S.-Y., “Unlocking the Potential: LLMs Transforming Research Data Management,” in From Data to Diamonds - Empowering Research with AI and RDM, Aachen, Germany, Nov. 2023, pp. 1–35. https://doi.org/10.18154/RWTH-2023-10498 Moon, J., Gelbich, D., Becker, M., Niemitz, P., & Bergs, T. Predicting fine blanking process signals from sheet metal thickness. Materials Research Proceedings, Vol. 41, pp 1436-1445. https://doi.org/10.21741/9781644903131-159 Schulhoff, S., Ilie, M., Balepur, N., Kahadze, K., Liu, A., Si, C., ... & Resnik, P. (2024). The prompt report: a systematic survey of prompt engineering techniques. arXiv preprint arXiv:2406.06608. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., ... & Zhou, D. (2022). Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35, 24824-24837. Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T., Cao, Y., & Narasimhan, K. (2023). Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems, 36, 11809-11822. FAIRly Simple – How can generative AI support FAIR RDM? | Prof. Dr. Sandra Geisler | DSMA | 27.06.2025 25 Acknowledgment Funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany ` s Excellence Strategy – EXC-2023 Internet of Production – 390621612.