AI4Science Training @ UFZ Leipzig

AI4Science @UFZ Robert Haase @haesleinhuepf December 2025 1 How language models work under the hood Robert Haase These slides can be reused under the conditions of the CC-BY 4.0 license if not otherwise specified. CENTER FOR SCALABLE DATA ANALYTICS AND ARTIFICIAL INTELLIGENCE Funded by: AI4Science @UFZ Robert Haase @haesleinhuepf December 2025 2 Quiz What is the cut-off […] of a language model? The maximum number of tokens of a prompt The maximum length of generated text The date until when the model knowledge can include information The region that the model can be accessed from AI4Science @UFZ Robert Haase @haesleinhuepf December 2025 3 AI systems / AI agents An AI agent is an AI system acting on behalf of a human . AI system AI model Data Internet E.g. company-internal documents For searching for current results User I need a … Here you go … AI4Science @UFZ Robert Haase @haesleinhuepf December 2025 4 AI systems / AI agents An AI agent is an AI system acting on behalf of a human . AI system AI model Data E.g. company-internal documents User I need a … Error! Too many tokens / documents! Image source: Moore et al. (2024), licensed CC-BY 4.0 https://zenodo.org/records/13168693 AI4Science @UFZ Robert Haase @haesleinhuepf December 2025 5 Large Language Models Text-to-text, Knowledge Extraction Question Answer Hallucination ? https://arxiv.org/abs/2005.11401 AI4Science @UFZ Robert Haase @haesleinhuepf December 2025 6 Short excursion: LLM Embeddings Embeddings are n-dimensional numerical representations of data, such as words, sentences and text. Using dimensionality reduction, we can display them and study distances between words: https://scads.github.io/generative-ainotebooks/60_rag/10_text-embeddings.html AI4Science @UFZ Robert Haase @haesleinhuepf December 2025 7 Embeddings Embeddings represent words, sentences and texts in the form of long vectors of numbers. Example: generated PhD thesis topics for folks @UFZ. Generated using gemma3:12b Source: https://www.ufz.de/export/data/global/98778_UFZOrganigramm_27-10-2025.jpg … AI4Science @UFZ Robert Haase @haesleinhuepf December 2025 8 Exercise: Guessing Groups AI4Science @UFZ Robert Haase @haesleinhuepf December 2025 9 Document index Query-Encoder Generator Retrieval-Augmented Generation Enriching a prompt with context-specific knowledge base Image source: Moore et al. (2024), licensed CC-BY 4.0 https://zenodo.org/records/13168693 Read more: Lewis et al 2020 https://arxiv.org/abs/2005.11401 “What is Dr. Haase responsible for in the NFDI4BioImage project?” “What is Dr. Haase responsible for in the NFDI4BioImage project?” LLM Laut Seite 4 ist Dr. Haase verantwortlich für … Comparis on Relevant documents Embedding AI4Science @UFZ Robert Haase @haesleinhuepf December 2025 16 Information extraction Even incomplete information is often sufficient. Hi Peter and Robert, works for me too! Cheers, Tanja > Peter wrote: > > Hi Robert, > > I'm super busy these days and unsure if next week works. We > still need to finish the presentation slides for the training with the city > and write the two new proposals. So I’d prefer to keep the meeting brief, and I > can only do Thursday. On Friday > I'm on vacation. > > Best regards, > Peter > > >> Robert wrote: >> >> Hi Tanja and Peter, >> >> When would you have time for a meeting? I could do next week on Thursday >> and Friday from 1 pm onward. >> >> Best regards, >> Robert Prompt: When can the meeting take place? {email} Output: The meeting can take place on Thursday, as both Tanja and Robert are available on that day. Tanja has also confirmed that it works for her. However, Peter mentioned that he’d like to keep the meeting short, as he is very busy. The format of the response resembles the format of the prompt. AI4Science @UFZ Robert Haase @haesleinhuepf December 2025 17 Information extraction Even incomplete information is often sufficient. Hi Peter and Robert, works for me too! Cheers, Tanja > Peter wrote: > > Hi Robert, > > I'm super busy these days and unsure if next week works. We > still need to finish the presentation slides for the training with the city > and write the two new proposals. So I’d prefer to keep the meeting brief, and I > can only do Thursday. On Friday > I'm on vacation. > > Best regards, > Peter > > >> Robert wrote: >> >> Hi Tanja and Peter, >> >> When would you have time for a meeting? I could do next week on Thursday >> and Friday from 1 pm onward. >> >> Best regards, >> Robert Prompt: Today is Friday, March 21st. Provide the first possible date for the meeting. Be extremely brief. {email} Output: Thursday, March 27th, from 1 pm onward. AI4Science @UFZ Robert Haase @haesleinhuepf December 2025 18 Further Prompt-Engineering Tips •Reflektion: Dem Chat-Modell seinen eigenen Output zurückgeben um nach Verbesserungsmöglichkeiten zu fragen •Tipp: Starten Sie einen neuen Chat, insbesondere nach langen Diskussionen. Write a document about {Topic X} Which aspects are missing in this document about {Topic X} ? {Document} Write a document about {Topic X} Modifiziere dieses Dokument sodass es in einfacher Sprache geschrieben ist: {Dokument} Start a new chat Start a new chat Manual modifications Manual modifications AI4Science @UFZ Robert Haase @haesleinhuepf December 2025 19 Language models for numerical data anlysis Robert Haase CENTER FOR SCALABLE DATA ANALYTICS AND ARTIFICIAL INTELLIGENCE AI4Science @UFZ Robert Haase @haesleinhuepf December 2025 20 Can LLMs solve real-world GitHub issues? Source: Jimenez et al (2024), licensed CC-BY 4.0 https://arxiv.org/abs/2310.06770 Plot data source: https://www.swebench.com/ (2025-06-21) AI4Science @UFZ Robert Haase @haesleinhuepf December 2025 21 AI-generated Code … appears online more and more Indicator: Share of AIgenerated code to PythonContributions on Github.com (Hint: Presumable not submitted by AI-agents, but by humans using ChatGPT) Source: Daniotti et al (2025), licensed CC-BY 4.0 https://arxiv.org/abs/2506.08945 AI4Science @UFZ Robert Haase @haesleinhuepf December 2025 22 Large-Language Models for Data Analysis Analysing images using ChatGPT Prompt: “Please segment the nuclei in the blue channel of the image” AI4Science @UFZ Robert Haase @haesleinhuepf December 2025 26 How good are LLMs for Bio-image Analysis? Test case: segment the image and measure the average area of objects. https://www.biorxiv.org/content/10.1101/2024.04.19.590278v1 https://github.com/haesleinhuepf/human-eval-bia 858.04 … Unit-test pass-rate (n=10): Large language models AI4Science @UFZ Robert Haase @haesleinhuepf December 2025 27 Benchmarking LLMs for Bio-image Analysis Use-case: compute the correlation matrix Unit-test pass-rate (n=10): https://www.biorxiv.org/content/10.1101/2024.04.19.590278v1 https://github.com/haesleinhuepf/human-eval-bia AI4Science @UFZ Robert Haase @haesleinhuepf December 2025 28 Benchmarking LLMs for Bio-image Analysis Use case: Open a zarr file https://www.biorxiv.org/content/10.1101/2024.04.19.590278v1 https://github.com/haesleinhuepf/human-eval-bia Data Source: https://www.ebi.ac.uk/bioimagearchive/galleries/S-BIAD634-ai.html Unit-test pass-rate (n=10): AI4Science @UFZ Robert Haase @haesleinhuepf December 2025 35 AI systems for coding The number of [agentic] AI systems for coding is exploding https://chatgpt.com/codex https://jupyter-ai.readthedocs.io/ https://github.com/paul-gauthier/aider/ https://www.cursor.com/ https://app.all-hands.dev/ https://www.anthropic.com/claudecode https://github.com/haesleinhuepf/bia-bob AI4Science @UFZ Robert Haase @haesleinhuepf December 2025 36 Large-Language Models for Data Analysis Learn more… [shameless self-advertisement] https://www.nature.com/articles/s43588-025-00781-1 https://www.biorxiv.org/content/10.1101/2024.04.19.590278v3 https://www.youtube.com/watch?v=9dtVlVwk2eg&list=PLZkHu66nUK_5 WEroAgMKrvkD1Ue2XC93w&index=24 AI4Science @UFZ Robert Haase @haesleinhuepf December 2025 37 Exercises Robert Haase CENTER FOR SCALABLE DATA ANALYTICS AND ARTIFICIAL INTELLIGENCE Group work AI4Science @UFZ Robert Haase @haesleinhuepf December 2025 38 Exercise: Knowledge Distillation Create exercise tasks for a psychology field internship •Provide situations in advance •Brainstorm with the system about possible additional situations … https://scads.github.io/ai4psychology2025/session2/lehr_materialien_generieren.html AI4Science @UFZ Robert Haase @haesleinhuepf December 2025 39 Exercise: Create a chatbot Create your own chatbot on a topic of your choice: •Professional ethical guidelines for psychologists •Use of Generative AI •Guidelines for ensuring good scientific practice •Checklist for handling research data •… https://scads.github.io/ai4science-ufz2025/session2/chatbot.html AI4Science @UFZ Robert Haase @haesleinhuepf December 2025 40 Exercise: Search through documents •Search a document for known information •Check the sources. •Compare uploading a PDF to copy&pasting the entire text from the document. https://scads.github.io/ai4science-ufz2025/session2/searching_documents.html AI4Science @UFZ Robert Haase @haesleinhuepf December 2025 41 Exercise: Data analysis Generate code using AI and rerun it on safe infrastructure https://scads.github.io/ai4science-ufz2025/session2/ai_data_analysis.html AI4Science @UFZ Robert Haase @haesleinhuepf December 2025 42 Feedback round What worked well worked? What did not? AI4Science @UFZ Robert Haase @haesleinhuepf December 2025 43 Note The chat bots we developed serve academic purposes. We did not do any quality assurance and did not define terms of service documents etc. AI4Science @UFZ Robert Haase @haesleinhuepf December 2025 44 Note Too many AI-Systems are developed as Prototypes and too fast put in production. Source: Chris Bakke on Twitter https://x.com/ChrisJBakke/status/1736533308849443121