Gene Ontology Curators AI Workshop (Part 1)

Gene Ontology Curators AI Workshop (Part 1) Chris Mungall, Kimberly van Auken Dec 9, 2025 AI access provided by https://go.lbl.gov/go-ai-workshop 10.5281/zenodo.17993529 2 This is part of a multi-part series Module 1: (Dec 2025) Today ●Fundamentals of generative AI ●Practical guidance for generative AI usage ○Chat interfaces, web search, Deep Research [Holiday break!] Module 2: (Feb 2026) Putting agentic AI into practice ●Core agent concepts and why agents are game-changers ●Using agentic curation tools 3 Agenda for today (module 1) Core concepts Training models Chat interfaces Pitfalls Search Deep Research Connectors Prompting Core concepts: what is an LLM How models are trained and evaluated Practical everyday use; chat applications and common tasks Pitfalls: hallucination and other problems How search is incorporated into AI systems How to have AI perform extensive research for you How to prompt and when not to worry about it Using trusted databases in AI systems 4 Learning Objectives for Module 1 What you will learn Foundations: ●✅ Core concepts ●✅ AI fluency Practical how-to guides ●✅ Selecting and navigating AI applications ●✅ Prompting NOT covered ●❌ Underlying mathematical foundations ●❌ Engineering AI applications 5 Interactive exercises Module 1 is oriented around fundamentals, but we will have some exercises later in the session ●Requirements: web browser ●Most use claude.ai, ●Did you sign up for the complementary Pro account? ○If not, don’t worry, you can still follow along! Module 2 (next year) will have more interactive exercises 6 Before we get started, let’s check everything is in working order https://claude.ai/ Pop the sidebar open if it’s closed Your name here! 7 If you signed up in time, you should see “Pro plan” here. If not, don’t worry! Core Concepts: Generative AI and Large Language Models Core concepts Training models Chat interfaces Pitfalls Search Deep Research Connectors Prompting 9 About this section ●We will cover core foundational concepts and terminology ○High level overview, no maths! ●Contextualize genAI relative to ML historically used in biocuration ○But also emphasizing it’s very different ●Useful for building a mental model of what is happening under the hood ○But even without fully understanding these concepts, you can be a successful AI practitioner by treating AI as a ‘black box’ 16 Example: protein sequence generation Generative Model Text generate a cationic antimicrobial peptide of aa length 40 with properties… KKWRKLLKKLGFAKIRGAVKAIKK LAGALKKLGRKALKK Ruffolo, Jeffrey A., and Ali Madani. "Designing proteins with language models." nature biotechnology 42.2 (2024): 200-202. 17 Example: audio generation generation Generative Model Create a podcast about the last GO meeting You can easily do this with: https://notebooklm.google.com/ 18 Example: audio generation generation Generative Model Audio Create a podcast about the last GO meeting You can easily do this with: https://notebooklm.google.com/ GOC: Cambridge 2025 - Featured on everyone’s Spotify 2025 Wrapped 19 Generative AI is a class of deep learning (neural networks) Deep Learning Model = Architecture + Weights Content Input ●Trained on massive amounts of data ●Multiple layers, arranged in blocks 20 A generative model consists of an architecture plus parameters (weights) Deep Learning Model = Architecture + Weights Content Input ●Architecture = Structure of the network ●Parameters = numbers assigned to edges between nodes in network 21 The transformer architecture: the gory details Some technical details (which you don’t need to know!) ●Self-attention ●multi-head attention blocks ●… ●Original was encoder-decoder ●GenAI uses decoder-only Generative Pretrained Model (GPT) architecture The foundation of the AI revolution 22 Different classes of generative model ●Large Language Models (LLMs) ○For text data ○This will be our focus in this module ●Others ○Diffusion models ■For image data https://www.coursera.org/learn/generative-ai-for-everyone/home/module/1 23 LLMs are next token predictors ??? TP53 is a tumor suppressor 24 The first step is tokenization 53 is a tumor suppress orTP 6798 382 261 40770 36382 2677683 ??? Try it yourself! https://tiktokenizer.vercel.app/ } actual input vector (list of numbers) 25 Tokens activate input layers 53 is a tumor suppress orTP 6798 382 261 40770 36382 2677683 ??? LLM 32 TP53 is a ??? tumor DNA … hydrolase … furious fluffy … Given an input context the model calculates the probability of possible next tokens* (based on the training data). Highest probability Lowest probability LLMs are autoregressive next token predictors *for simplicity we don’t show the tokenization here 33 TP53 is a tumor tumor DNA … hydrolase … furious fluffy … Model picks highest probability next token (learned from training data) LLMs are autoregressive next token predictors 34 TP53 is a suppressor negative … activator … kitten peanut … ???tumor Given a new input context the model produces a new probability distribution over the vocabulary for the next token. LLMs are autoregressive next token predictors 35 TP53 is a suppressor negative … activator … kitten peanut … suppressortumor New context LLMs are autoregressive next token predictors 36 TP53 is a protein gene … … kitten peanut … tumor suppressor ??? And the autoregressive process continues… LLMs are autoregressive next token predictors 37 TP53 is a protein gene … … kitten peanut … tumor suppressor protein Model picks highest probability next token LLMs are autoregressive next token predictors 38 TP53 is a tumor suppressor protein Until we arrive at a (hopefully) meaningful, correct, complete sequence of tokens LLMs are autoregressive next token predictors 39 This turns out to be useful for question answering What is TP53?Question: Answer: Prompt Historic note: with early LLMs it was necessary to include markers like “question” and “answer” 40 This turns out to be useful for question answering TP53What is TP53?Question: Answer: 41 This turns out to be useful for question answering TP53 isWhat is TP53?Question: Answer: 48 This turns out to be useful for question answering and many other things LLMs alone can ●Summarize a document ○Upload a PDF and summarize conclusions about gene function ○Summarize a list of GO annotations for a gene ●Review your work ○Check ontology term definitions and synonyms ○Check GO-CAMs ○Check standard annotation against what was said in a paper ●Extract or reshape structured data ○Extract gene lists from papers ○Extract gene-term annotations from papers ○Reorganize a supplementary table to a format for import into a curation form And even more when LLMs are combined with tools (this will be covered in the next module) 49 Exercise: tokenization Visit the following URL ●https://tiktokenizer.vercel.app/ Enter the abstract from a paper ●Try your own, or this one: https://pubmed.ncbi.nlm.nih.gov/41091836/ ●Leave all settings as defaults What is the token count? ●How does it compare with the number of characters? 50 Example https://tiktokenizer.vercel.app/ Clear out any example text here to get a blank form 51 Example: tokenize an abstract https://tiktokenizer.vercel.app/ https://pubmed.ncbi.nlm.nih.go v/41091836/ Characters: 1005 52 Example: tokenize an abstract https://tiktokenizer.vercel.app/ https://pubmed.ncbi.nlm.nih.go v/41091836/ Characters: 1005 53 Recap LLMs are fundamentally next token predictors They are versatile and can be used for a number of different tasks ●Q&A and lookup (with particular caveats – we’ll come to this later) ●Summarization ●Review ●Extraction and reshaping ●Generation of documents How are LLMs trained? Core concepts Training models Chat interfaces Pitfalls Search Deep Research Connectors Prompting 55 LLM Training Recap: Models = Architecture + Weights (parameters) ⇒ Weights are assigned by a process called training ⇒ For LLMs, this is actually multi-step, with the first step called pre-training training Architecture selected in advance Initial weights randomized Architecture unchanged Weights tuned towards task 56 LLMs are pre-trained on massive bodies of text (and images) Much of it derived from internet crawls https://commoncrawl.org/ 57 LLM training is in phases 64 Model knowledge cutoff dates https://platform.openai.com/docs/models/gpt-5 65 Model knowledge cutoff dates https://platform.claude.com/docs/en/about-claude/models/overview 66 How are models evaluated? Classic ML benchmarks (e.g Biocreative) are focused on specific tasks ⇒ e.g. recognizing chemical entity mentions LLMs are generalists, which poses challenges for evaluation 67 Example benchmark: MMLU Hendrycks, Dan, et al. "Measuring massive multitask language understanding." arXiv preprint arXiv:2009.03300 (2020). 68 There are multiple generalist benchmarks Minaee, Shervin, et al. "Large language models: A survey." arXiv preprint arXiv:2402.06196 (2024). 69 Domain-specific benchmarks https://huggingface.co/datasets/MonarchInit/dragon-ai-definition-evals 70 The reality is that the community goes on “vibes” Tried some ad-hoc queries today, looks good to me! AI experts often have their own evals geared towards their own tasks But even experts often go on “vibes” Ni, Jinjie, et al. "Mixeval: Deriving wisdom of the crowd from llm benchmark mixtures." Advances in Neural Information Processing Systems 37 (2024): 98180-98212. 71 Who are the model providers? Provider Category Openness Example Models OpenAI Commercial Company 🔴 Closed-weight GPT-5 Anthropic Commercial Company 🔴 Closed-weight Claude 4.5 Sonnet, Opus Google (Gemini) Commercial Company 🔴 Closed-weight Gemini 3 Pro DeepSeek Commercial Company 🟢 / 🔴 Mixed DeepSeek-V2, DeepSeek-Coder Meta Commercial Company 🟢 Open-weight LLaMA 4 Academic Labs (General) Academic Labs 🟢 Open-weight (usually) BERT (research), T5 (research), domain-specific fine-tunes 72 Why do we train on all this internet junk? Wouldn’t it work better if we trained only on scientific text? 73 People have tried this 80 Querying LLMs via APIs LLM APIs are useful for programmers who want to make applications (e.g. curation tools) that leverage LLMs ●(increasingly developers will be using higher level agentic frameworks and be insulated from the API) The API is usually provided by an LLM provider ●But models can be executed locally too (with enough GPU) Otherwise you don’t need to know about LLM APIs! Curators will love this tool I’m building 81 Using LLMs via applications Specialized applications ●Curation tools enhanced with LLMs ○(see part 2!) ●Knowledgebase chatbots ○E.g. chatbots for Reactome, JBrowse Generalist applications ●Generalist chat interfaces ○E.g. chatgpt.com ●Integration into google/microsoft office tools ●Coding agent tools 82 You don’t need to wait for the right specialized application! Specialized applications ●Curation tools enhanced with LLMs ●Knowledge base chatbots Generalist applications ●Generalist chat interfaces ○E.g. chatgpt.com ●Integration into google/microsoft office tools ●Coding agent tools 83 LLMs are increasingly integrated into existing general purpose applications https://workspace.google.com/solutions/ai/ 84 Gemini is integrated with Google GSuite 85 NotebookLM is useful for searching and organizing docs https://notebooklm.google.com/ 86 Tip: explore the office suite integrations available to you Avoids copy and paste between chat sessions; can do operations directly in the office app Not the focus of today’s lesson though… 87 Chat: the universal interface The generalized abilities of AI are best suited to a freeform interactive interface Natural language is infinitely expressive! (as every curator knows!) Research the latest findings on CHAMP1 Review this spreadsheet of terms Format this in swissprot format Find the passage of text that supports that statement What is a good ontology term to describe this process? 88 How Chat applications work (simplified view) Chat UI LLM user prompt response response Next tokens prediction UI and API calls user prompt 89 Your query is concatenated onto a system prompt Chat UI LLM user prompt response response Next tokens prediction UI and API calls system prompt + user prompt System prompt 96 It depends, but it doesn’t matter that much! Your institute might have a special deal with one Your project might have ways to access certain ones What matters more is the plan (free, pro, max, etc) ●Free plans: ○Less powerful models ○Throttle usage ○Fewer features (including important ones) Today we will be using Claude.ai (PRO) → but everything you learn will be easily generalized! 97 Anatomy of a chat interface: ChatGPT Sometimes annoying welcome Your query Sidebar (click to pop open) 98 Anatomy of a chat interface: ChatGPT Settings and plan details Chat history Search previous chats 99 This is really useful! We’ll cover later Click + for other options Multiple PDFs can be dropped in Anatomy of a chat interface: ChatGPT 100 Personalization and user system prompts System prompt chatgpt.com 101 All chat interfaces basically have the same UI patterns claude.ai 102 Most chat interfaces have a model selector ChatGPT.com 103 Claude.ai Most chat interfaces have a model selector 104 gemini.google.com Most chat interfaces have a model selector 105 Which model should I use? 112 Attach a PDF https://pubmed.ncbi.nlm.nih.gov/21063390/ 113 Ask for a summary of the paper 114 115 Extract structured information 116 117 Ask it to reformat 118 Reformat 119 Ask it to help you plan 120 What we’ve learned ●Summarization ●Knowledge Extraction ●Reshaping ●Planning and co-curation 121 Context windows Is there a limit to the amount of text I can paste into my chat? Current frontier models have a maximum of between 100,000 and 1m ●This is known as the context window size ●This used to be much lower! (4k) I’ll never need to type in that many characters, so it seems this isn’t important to me? But remember, the text the LLM sees is actually a lot more than what you type! ●We’ll return to this later Pitfalls and downsides Or: how I learned to stop worrying and love hallucinations Core concepts Training models Chat interfaces Pitfalls Search Deep Research Connectors Prompting 129 Exercise: Look up a GO term ID For this exercise, we will be using claude.ai First, let’s set things up Even if you didn’t get set up with Pro, you can still do this exercise 130 Switch off web search (just for this section) Ensure switched off here (note for everyday I use we recommend leaving on!) claude.ai 131 Ask for a well known GO term 132 Ask for a well known GO term Unlikely to fail! 133 Ask for a different term 134 Check the result Hmm, looks plausible - it has the right number of digits, the LLM sounds confident… 135 Check it against QuickGO. Was it correct? I got an empty page ⇒ https://www.ebi.ac.uk/QuickGO/term/GO:nnnnnnn FAIL! 136 Another example Create some example GO annotations for the CHAMP1 gene. Return in format: proteinId,proteinName,GOTermID,GOTermName,EvidenceCode,PMID 137 144 Agentic AI provides more opportunities to mitigate hallucination through verification Coming in Module 2 next year… We will get a taste for this in the next section on search https://ai4curation.io/aidocs/how-tos/make-ids-hallucination-resistant/ 145 Sycophancy Incorporating web search Core concepts Training models Chat interfaces Pitfalls Search Deep Research Connectors Prompting 147 About this section We will learn about how most chat applications perform web search and incorporate results This can help mitigate hallucinations (but is not a panacea) 148 Exercise: finding CHAMP1 gene function Note: be sure to start a new conversation 149 For now, switch off web search 150 Step 1: ask for a summary of CHAMP1 gene function 151 Step 1: ask for a summary of CHAMP1 gene function 152 153 Enable web search