LLM-Assisted Extraction of Mathematical Model Metadata for FAIR Reuse

MaRDI Mathematical Research Data Initiative LLM-Assisted Extraction of Mathematical Model Metadata for FAIR Reuse Jochen Fiedler1, Felix Niclas Kreutz1, Christine Biedinger1, Michael Burger1, Dominik Göddeke2, Thomas Koprucki3, Marco Reidelbach4, Aurela Shehu3, Björn Schembera2, Burkhard Schmidt3, Anita Schöbel1, Jörg Schlötterer5, Marcus Weber4 1Fraunhofer ITWM, 2Universität Stuttgart, 3WIAS Berlin, 4ZIB, 5Universität Marburg 2 •R&D teams often rely on mathematical models and concepts •Models are often scattered across manifold publications and internal documents •Finding the right model for a new research project (including assumptions, scope, required quantities, and references) is timeconsuming •Leads to fragmented knowledge and missed opportunities •Improve discoverability of models (including versions, variants, and application domains) •Provide transparency and thorough documentation of necessary formulations and assumptions •Connect and link research questions and domains to specific models •Enable easy search, maximum synergies and unique identification →Making mathematical models FAIR! Motivation: Create central repository for mathematical models Challenges Goals 3 MaRDI aims and services MaRDI aims •Develop robust Mathematical Research Data Infrastructure •Set standards and confirmable workflows for certified Mathematical Research Data •Provide services to both the mathematical and wider scientific community Services The structure of MathModDB 5 Free Fall models as mathematical research data •Simple model without air drag: ሶ𝑣 = 𝑔 •More complex model with air drag: ሶ𝑣 = 𝑔 − 𝜌𝐶𝐷𝐴𝑣2 2𝑚 Sir Isaac Newton (1666) What and how to store such a model in a meaningful way? →Utilization of ontologies! Introduction: FAIR traded apples in MathModDB 6 MathModDB: Ontology specification Version 1.0.0 published in February 7 Mathematical Model: Free fall with air drag Mathematical Formulation: ሶ𝑣 = 𝑔 − 𝜌𝐶𝐷𝐴𝑣2 2𝑚 Quantities: Free Fall Velocity 𝑣 Gravitational acceleration 𝑔 Density of air 𝜌 Drag coefficient 𝐶𝐷 Cross section 𝐴of the apple Mass of the apple 𝑚 Computational Task: Calculate free fall time Research Field: “Pomology” Research Problem: Gravitational effects on fruit Sir Isaac Newton (1666) The free fall model in our ontology Utilizing LLMs to fill knowledge graph 9 So far: Data needs to be inserted manually MaRDMO Plugin Publication Manual insertion requires effort Requires •Large community •Low barriers 16 Results and Outlook So far: •No complete workflow →no final results •Training and finetuning of models require a fair amount of complex data: Publications with annotated entities, relations and links •Some parts work well (NER), some parts are really difficult (relation extraction due to spurious relations) Future plans: •Build on workflow and finetune models with more data • Implement “human in the loop” approach •Utilize MaRDMO chatbot →chatbot generates proposal of entities and their relations, human can validate them (requires decoder)