The Hubverse: Streamlining Collaborative Infectious Disease Modeling for Public Health Impact

The Hub e se: S eamlining
Collabo a i e In ec ious Disease
Modeling
US-RSE Con e ence 2025
Anna K ys alli
7 Oc obe 2025
[email p o ec ed]
R-RSE SMPC
Conso ium o In ec ious Disease Modeling Hubs
1
Backg ound
3
❌
The p oblem
In ec ious disease modeling has scaled apidly…
Bu he landscape is agmen ed:
Inconsis en o ma s
Redundan o con lic ing o ecas s
Lack o coo dina ion be ween modele s and s akeholde s
“Compa ing he accu acy o o ecas ing applica ions is di icul because o ecas ing
me hods, o ecas ou comes, and epo ed alida ion me ics a ied widely.”
Ch e ien e al., PLOS ONE, 2014
4
✨
The p omise o modeling hubs
Modeling hubs coo dina e collabo a i e o ecas ing:
P o ide cen alised loca ion o e o coo dina ion
De ine da a s anda ds and modeling a ge s
Imp o e anspa ency and compa abili y
Agg ega e o ecas s enabling ensembles
Facili a e imely public heal h decision-making
, Ame ican Jou nal
o Public Heal h Reich, e al.2022
“Collabo a i e Hubs: Making he Mos o P edic i e Epidemic Modeling”
5
🕰
P ojec o igins
P e-COVID: Fo ecas ing code base exis ed o CDC in luenza hubs
Du ing COVID: Tha code was eused o new COVID-19 hubs + demand in e na ionally (e.g.Eu ope) o simila se ups
❗
P oblem: Each hub equi ed manual edi ing o sou ce code
➡
Need o gene alisa ion, modula i y, and con igu abili y
Figu e c edi s: Alex Vespignani and Nicole Samay
6

🌐
En e he hub e se
An open-sou ce so wa e ecosys em o powe modeling hubs:
Gi Hub eposi o ies o cen alising hub ac i i y
Da a s anda ds o in ec ious disease modeling da a
Schema-d i en con igu a ion o modeling asks + hub se up
Modula ools o alida ion, access, e alua ion, ensembling, communica ion and hub
adminis a ion
Suppo s ull li ecycle: om hub se up, da a submission o decision-making
7
Hub e se o e iew
9
☑
S anda dised Da a
Modeling hubs a e buil a ound a sha ed da a s anda d:
Modeling ask de ini ion: a ge s ( esponse a iables), s anda d p edic o s, ou pu ypes
(e.g.
mean
,
quan iles
)
S uc u ed hub layou : consis en ile sys em o o ganizing submissions
S anda d model ou pu o ma : o ile con en and naming
✅
Enables compa abili y, alida ion, and s eamlined da a access
10
⚙
Con ig-d i en hub se up
Hub adminis a o s con igu e hubs using s uc u ed JSON con ig iles:
admin.json
: hub-le el me ada a.
ask.json
: modeling ask speci ica ion:
Task IDs: Ta ge s ( esponse), ho izons, loca ions (p edic o s) e c.
Ou pu ypes: accep ed model ou pu s e.g.
mean
,
median
,
quan iles
,
cd
,
pm
,
samples
.
Con igs a e alida ed agains a sha ed JSON schema
11
Real-wo ld example:

Used by US CDC o
moni o in luenza se e i y
Weekly o ecas s om 40
eams ac oss 70 di e en
models.
Hos ed on Gi Hub + S3
cloud mi o .
Managed using ull
hub e se s ack since
2023/2024 season.
CDC FluSigh Hub
h ps://gi hub.com/cdcepi/FluSigh - o ecas -hub
19

📁
File s uc u e: model ou pu (CDC FluSigh )
Model ou pu s commi ed by eams o e sioned di ec o ies > one di ec o y pe model > one
ile pe modeling ound.
20
✅
Model ou pu alida ion wi h
Model ou pu s submi ed h ough PRs and alida ed h ough Gi Hub Ac ions
hubValida ions
21
📂
Accessing model ou pu ia
Connec o A ow da ase o
o ecas submissions
Que y and collec da a
See mo e in .
Py hon analogue also a ailable.
hubDa a
lib a y(hubDa a)1
2
hub_pa h <- s3_bucke (3
"cdcepi- lusigh - o ecas -hub"4
)5
hub_con <- connec _hub(6
hub_pa h,7
skip_checks = TRUE8
)9
hub_con10
hub_connec ion
9 columns
e e ence_da e: da e32[day]
a ge : s ing
ho izon: in 32
a ge _end_da e: da e32[day]
loca ion: s ing
ou pu _ ype: s ing
ou pu _ ype_id: s ing
alue: double
model_id: s ing
# Fil e o one model and o ecas da e using dply 1
lib a y(dply )2
hub_con |>3
il e (4
model_id == "CADPH-FluCAT_Ensemble",5
a ge _end_da e == "2023-10-28"6
) |>7
collec _hub()8
# A ibble: 92 × 9
model_id e e ence_da e a ge ho izon a ge _end_da e loca ion ou pu _ ype
* <ch > <da e> <ch > <in > <da e> <ch > <ch >
1 CADPH-Flu… 2023-10-14 wk in… 2 2023-10-28 06 quan ile
2 CADPH-Flu… 2023-10-14 wk in… 2 2023-10-28 06 quan ile
3 CADPH-Flu… 2023-10-14 wk in… 2 2023-10-28 06 quan ile
4 CADPH-Flu… 2023-10-14 wk in… 2 2023-10-28 06 quan ile
5 CADPH-Flu… 2023-10-14 wk in… 2 2023-10-28 06 quan ile
6 CADPH-Flu… 2023-10-14 wk in… 2 2023-10-28 06 quan ile
7 CADPH-Flu… 2023-10-14 wk in… 2 2023-10-28 06 quan ile
8 CADPH-Flu… 2023-10-14 wk in… 2 2023-10-28 06 quan ile
9 CADPH-Flu… 2023-10-14 wk in… 2 2023-10-28 06 quan ile
10 CADPH-Flu… 2023-10-14 wk in… 2 2023-10-28 06 quan ile
# ℹ 82 mo e ows
# ℹ 2 mo e a iables: ou pu _ ype_id <ch >, alue <dbl>
Accessing da a igne e
hub-da a
22

🌐
Ensembling wi h
hubEnsembles
Combine models using simple o weigh ed ules
o ecas _d <- hub_con |>1
il e (2
model_id %in%3
c(4
"CADPH-FluCAT_Ensemble",5
"CEPH-R end_ luH",6
"CFA_Py enew-Py enew_HE_Flu"7
),8
ou pu _ ype == "quan ile"9
) |>10
collec _hub()11
12
13
hubEnsembles::simple_ensemble(14
o ecas _d , 15
agg_ un = median,16
model_id = "simple-ensemble-median"17
)18
# A ibble: 282,716 × 9
model_id e e ence_da e a ge ho izon a ge _end_da e loca ion ou pu _ ype
* <ch > <da e> <ch > <in > <da e> <ch > <ch >
1 simple-en… 2023-10-14 wk in… -1 2023-10-07 01 quan ile
2 simple-en… 2023-10-14 wk in… -1 2023-10-07 01 quan ile
3 simple-en… 2023-10-14 wk in… -1 2023-10-07 01 quan ile
4 simple-en… 2023-10-14 wk in… -1 2023-10-07 01 quan ile
5 simple-en… 2023-10-14 wk in… -1 2023-10-07 01 quan ile
6 simple-en… 2023-10-14 wk in… -1 2023-10-07 01 quan ile
7 simple-en… 2023-10-14 wk in… -1 2023-10-07 01 quan ile
8 simple-en… 2023-10-14 wk in… -1 2023-10-07 01 quan ile
9 simple-en… 2023-10-14 wk in… -1 2023-10-07 01 quan ile
10 simple-en… 2023-10-14 wk in… -1 2023-10-07 01 quan ile
# ℹ 282,706 mo e ows
# ℹ 2 mo e a iables: ou pu _ ype_id <ch >, alue <dbl>
23
📈
Dashboa d - o ecas s
24
🩺
Dashboa d -
E alua es o ecas s agains a ge (obse ed) da a.
model e alua ions
25
💡
Lessons & wide ele ance
✅
S anda ds + au oma ion educe
ic ion
🧰
Open sou ce keeps i ee &
accessible
🏥
Collabo a i e in as uc u e
empowe s public heal h
🌍
S anda dised, open da a uels
downs eam use cases like aining,
educa ion, and ep oducible
esea ch
In oduc ion
So a in his cou se we ha e ocused on building, isualising
combining “ oy” o ecas models in somewha syn he ic se
you will wo k wi h eal o ecas s om an exis ing modeling h
many o he challenges in ol ed wi h eal- ime o ecas ing, a
di d d li
E alua ing eal-wo ld ou b eak o
NFIDD SI…
Ge ing
s a ed
Au ho s 
Sessions Re e ence
 h ps://n idd.gi hub.io/sismid/sessions/ eal-wo ld-
o ecas s.h ml
26