Increasing FAIRness of FAIRagro data through AI supported metadata enrichment

Training Corpus (331 Documents) Abanoub Abdelmalak (Information Centre for Life Sciences, ZB MED), Juliane Fluck (Information Centre for Life Sciences, ZB MED), Leonard Golz (Julius Kühn-Institut, JKI), Murtuza Husain (Information Centre for Life Sciences, ZB MED), Kristin Meier (Leibniz Centre for Agricultural Landscape Research, ZALF), Heike Riegler (Julius Kühn-Institut, JKI), Gabriel Schneider (Information Centre for Life Sciences, ZB MED), Xenia Specka (Leibniz Centre for Agricultural Landscape Research, ZALF), Nikolai Svoboda (Leibniz Centre for Agricultural Landscape Research, ZALF), on behalf of the FAIRagro consortium Increasing FAIRness of FAIRagro data through AI supported metadata enrichment Background: High quality metadata is essential for FAIR1 data. Method: A training corpus has been generated to enable AI-based extraction of crop and soil metadata from FAIRagro Research Data Infrastructures (RDIs). Outlook: Enhanced findability through integration into the FAIRagro Search Hub2 and RDIs and support researchers in creating high quality metadata during data publication. BonaRes OpenAgrar References •1Wilkinson, M. D., Dumontier, M., Aalbersberg, Ij. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., da Silva Santos, L. B., Bourne, P. E., Bouwman, J., Brookes, A. J., Clark, T., Crosas, M., Dillo, I., Dumon, O., Edmunds, S., Evelo, C. T., Finkers, R., … Mons, B. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data, 3(1). https://doi.org/10.1038/sdata.2016.18 •2https://search-hub.fairagro.net/ •3https://www.flaticon.com/free-icon/ai_8593364?term=ai&page=1&position=2&origin=search&related_id=8593364 RDIs Search Hub Manual annotation based on guidelines image: Flaticon.com3 Method overview Manual curation of results Model evaluation, training and application Result: Annotated corpus Stats •342 documents (titles, abstracts, keywords) •1850 annotations • Inter annotator agreement per class: • Crops: 0.86 • Soil: 0.48 Quality of annotations Outlook Some entities are highly detailed and therefore receive fewer annotations, while soil-related annotations in particular show low inter-annotator agreement (IAA). Therefore we moved forward to group annotations to upper classes (e.g., all soil entities annotated as soil). Despite these challenges, the corpus remains valuable for applications in AI development and evaluation. We have tested it on AI development and evaluation. LLMs Task-specific Language Models Development Evaluation Test Corpus (31 Documents) Power Consumption Accuracy The corpus is used to develop and evaluate AI metadata extraction. The evaluations are done based on output accuracy and energy consumption. • Time statements: 0.82 • Locations: 0.78 Corpus label counts