Page 1 o 6
Da aTools4Hea
A Eu opean Heal h Da a Toolbox o Enhancing Ca diology Da a In e ope abili y, Reusabili y and P i acy
Miles one 11:
Dedica ed p ojec space o DT4H a ailable a
he OpenEBench benchma king pla o m
Re e ence MS11_ Da aTools4Hea _BSC_30092025
Lead Bene icia y BSC
Au ho (s) Ca les He nandez-Fe e , Laia Codó, Josep Lluís Gelpí
Dissemina ion le el Public
Type -
O icial Deli e y Da e 30/09/2025
Da e o alida ion o he WP leade 30/09/2025
Da e o alida ion by he P ojec
Coo dina o
30/09/2025
P ojec Coo dina o Signa u e
Da aTools4Hea is unded by he Eu opean Union’s Ho izon Eu ope
F amewo k Unde G an Ag eemen No. 101057849.
Page 2 o 6
Ve sion Log
Issue Da e Ve sion In ol ed Commen s
29/09/2025
0.1
Ca les He nandez-Fe e
Ini ial e sion
30/09/2025
0.2
Laia Codó, Josep Lluís
Gelpí
Second e sion
30/09/2025
Final
C is ian Izquie do, Xènia
Puig, Ka im Lekadi
Re ised and co ec ed inal
e sion
Execu i e Summa y
This miles one (MS11) epo s he a ailabili y o a dedica ed p ojec space o Da aTools4Hea
(DT4H) wi hin he ELIXIR OpenEBench benchma king pla o m. The DT4H benchma king hub
will sys ema ically in eg a e benchma king e o s unde aken ac oss he p ojec on ools, scien i ic
ou comes, use expe ience, so wa e quali y, and o e all pe o mance. I p o ides a cen alised
and FAIR-complian en i onmen o anspa en , ep oducible, and communi y-d i en
benchma king o DT4H oolbox componen s (e.g., NLP models, ede a ed lea ning algo i hms,
syn he ic da ase s).
Page 3 o 6
Table o Con en s
Ve sion Log ............................................................................................................................................ 2
Execu i e Summa y ............................................................................................................................... 2
Ac onyms ............................................................................................................................................... 3
Lis o ables ........................................................................................................................................... 3
Lis o igu es .......................................................................................................................................... 3
1 In oduc ion ......................................................................................................................................... 4
2 Benchma king in DT4H ....................................................................................................................... 4
3 In as uc u e ....................................................................................................................................... 5
3.1 DT4H @ OpenEBench .................................................................................................... 5
4 Fu u e Wo k ............................................................................................................................. 6
Ac onyms
OEB: OpenEBench
FL: Fede a ed Lea ning
NLP: Na u al Language P ocessing
FAIR: Findable, Accessible, In e ope able, Rep oducible
Lis o ables
Table 1: O e iew o XXXX (example able cap ion). 4
Lis o igu es
Figu e 1: Illus a ion o XXXX (example igu e cap ion) 5
Page 4 o 6
1 In oduc ion
This documen ou lines he echnical capabili ies and ope a ional amewo k o he benchma king
pla o m designed as pa o he Da aTools4Hea (DT4H) Eu opean p ojec . I se es as a cen alised
hub o eco ding a echnical summa y o p ojec ’s benchma king decisions, goals, ag eemen s, and
capabili ies. C ucially, will p o ide he s uc u ed space o:
● Lis he da ase s used o benchma king (including e e ences o he DT4H's da a ca alogue).
● De ail he comple e os e o benchma ked asse s.
● Comp ehensi e desc ip ion and esul s o all benchma king e en s.
Along he ask, he benchma king hub will be popula ed by a se ies o benchma king e o s pe o med
wi hin he DT4H p ojec ame. The o e a ching goal is o elucida e he igh ool o he igh scena io
by enabling consis en , objec i e e alua ion ac oss di e en echnical a eas. This consis ency is
c i ical o ad ancing he s a e-o - he-a in hea disease da a p ocessing and analysis.
2 Benchma king in DT4H
Benchma king in DT4H is ans e sal, co e ing scien i ic ou comes, use expe ience, so wa e quali y,
and he o e all pe o mance o he pla o m. Asse s o be e alua ed include:
- baseline NLP ools and ans o me -based AI models,
- ede a ed lea ning models ained unde no el schemas (e.g. cen e d opou , unbiased
agg ega ion me hods)
- syn he ic ca diology da ase (Ca dioSyn)
Such benchma king e o s a e ca ied ou by he ele an WPs (see Table 1. Benchma king- ela ed
asks) and la e in eg a ed as s uc u ed e alua ions in o he DT4H OpenEBench po al, ensu ing
anspa en compa ison and long- e m a ailabili y o esul s.
Table 1. Benchma king- ela ed asks
T6.4
De elopmen o an e alua ion en i onmen o DT4H oolbox
T3.6
In eg a ion o clinical NLP models in o ede a ed lea ning amewo k
T4.2
Cen e d opou o e icien ede a ed lea ning
T5.4
Technical E alua ion o Syn he ic Da a Se s o Ca diology
T4.3
Unbiased agg ega ion in ede a ed lea ning
T7.4
Mul i-cen e e alua ion and eedback loop
O hese, T6.4 is esponsible o linking DT4H wi h OpenEBench and adap ing he pla o m o i he
benchma king app oach ollowed by each o he o he asks.
3 In as uc u e
The main achie emen in oduced in his miles one is he DT4H benchma king pla o m buil upon he
exis ing in as uc u e o OpenEBench.
Page 5 o 6
OpenEBench is he ELIXIR Benchma king and Technical Moni o ing pla o m. I is a publicly
a ailable, communi y-d i en pla o m designed o cen alise and s anda dise he esul s o echnical
assessmen s, alida ions, and benchma ks o bioin o ma ics ools and wo k lows. I s co e capabili ies
include:
● S anda disa ion: P o iding s anda dised me ics, me hods, and epo ing schemas o ool
e alua ion.
● Cen alisa ion: Ac ing as a eposi o y o benchma king esul s ac oss di e se domains.
● Rep oducibili y: Facili a ing he c ea ion o ep oducible benchma king e en s, o
challenges.
● Communi y Engagemen : Allowing di e en scien i ic eams o compa e hei ools and
me hods ai ly in a anspa en en i onmen .
OpenEBench p o ides he necessa y amewo k and echnical speci ica ions o ensu e DT4H's
benchma king e o s a e anspa en , consis en , and compa able wi h wide Eu opean li e science
ini ia i es.
3.1 DT4H @ OpenEBench
DT4H has been in eg a ed a OpenEBench as a new dedica ed p ojec space (see Figu e 1.
Da aTools4Hea space a OpenEBench). Accessible a he OpenEBench po al (Home
h ps://openebench.bsc.es/ → P ojec s → Da a ools4Hea ca d), i will in eg a e a comp ehensi e
iew o DT4H assessmen a eas, p oduced me ics, compu a ion pipelines and quali y indica o s.
Pa icipa ing WPs will con ibu e e alua ion ou pu s o his space, ensu ing ha monisa ion ac oss he
conso ium.
Da aTools4Hea p ojec space
h ps://openebench.bsc.es/p ojec s/OEBC014
Fu he mo e, his sec ion o he pla o m ac s as a cen alised hub o links o ex e nal ools, da a
sou ces, and o he pla o ms ha ha e been used o benchma k and compa e ools, ensu ing use s
ha e a single poin o e e ence o all ela ed echnical documen a ion.
Page 6 o 6
Figu e 1. Da aTools4Hea space a OpenEBench
4 Fu u e Wo k
While he benchma king pla o m is al eady ope a ional, he cu en e sion con ains only he skele on
o he u u e hub. Nex s eps include keep wo king in collabo a ion wi h he es o he Benchma king
Wo king G oup o comple ing he lis o e aluable oolbox componen s (da ase s, models, ools) and
bo h:
- O ganising con inuous e alua ion and eedback loops (mul i-cen e, use -d i en)
- P epa ing hema ic benchma king e en s (e.g., NLP model e alua ion, ede a ed lea ning
obus ness, syn he ic da a alida ion)
The esul o his e o would be a cen alised benchma king hub o DT4H ools, o ganised by
hema ic a ea (benchma king e en ) and ask (challenge). E e y benchma king- ela ed ask o he
DT4H p ojec is o collec and submi i s ou come o popula e he cen alised DT4H space in
OpenEBench. Cu en ly, mos o hese asks a e unde p og ess.
