Overview of NFDI4DS Shared Tasks

NFDI4DS Shared Tasks Tales from the NFDI4DS Shared Tasks Sonja Schimmler Contact Prof. Dr. Sonja Schimmler Tel. +49 30 3463-7159 [email protected] Fraunhofer FOKUS Kaiserin-Augusta-Allee 31 10589 Berlin www.nfdi4datascience.de Shared Tasks 2025 In 2025, we host the challenges •FoRC: Field of Research Classification (2nd edition), MESD: Metadata Extraction from Scholarly Documents and ReadMe2KG: GitHub ReadMe to Knowledge Graph at ESWC, •ClimateCheck: Scientific Factchecking of Social Media Posts on Climate Change, SciVQA: Scientific Visual Question Answering, SOMD: Software Mention Detection in Scholarly Publications (2nd edition), and LLMs4Subjects: LLM-based Subject Tagging at ACL, and •LLMs4Subjects: LLM-based Subject Tagging (2nd edition) at GermEval. Shared Tasks 2024 In 2024, we hosted the challenges •KGQA: Knowlegde Graph for Question Answering at ACL, •SOTA?:Tracking the State-of-theArt in AI Scholarly Publications at CLEF, •FoRC: Field of Research Classification of Scholarly Publications and SOMD: Software Mention Detection in Scholarly Publications at ESWC, and •LLMsOL: Large Language Models for Ontology Learning and Scholarly QALD: Hybrid Question Answering over Scholarly Knowledge Graphs and Text at ISWC. What are Shared Tasks? The NFDI4DS consortium continuously hosts various shared tasks to tackle problems, usually organized either at conferences or workshops. Shared tasks are scientific competitions in which teams attempt to find efficient solutions to aspecific problem using shared datasets and evaluation measures. The goal is to objectively and directly compare different methods for tackling the same problem by using gold-standard datasets and common performance measures. How does NFDI4DS benefit from Shared Tasks? NFDI4DS aims to utilize shared tasks to activate and serve the community,by defining shared tasks, by providing goldstandard datasets and common performance measures,and by establishing quality standards for both. SOTA? FoRC & SOMD KGQA LLMs4OL & Scholarly QALD The participant-driven solutions, many of them currently based on generative AI, have the potential to lay the foundation for novel and innovative NFDI4DS services.