na u e me hods Volume 20 | July 2023 | 973–975 | 973
h ps://doi.o g/10.1038/s41592-023-01929-5
Commen
Using AI in bioimage analysis o ele a e he
a e o scien i ic disco e y as a communi y
Damian Dalle Noga e, Ma hew Ha ley, Jo an Deschamps, Jan Ellenbe g & Flo ian Jug
The u u e o bioimage analysis is inc easingly
de ined by he de elopmen and use o
ools ha ely on deep lea ning and a i icial
in elligence (AI). Fo his end o con inue in
a way mos use ul o s imula ing scien i ic
p og ess, i will equi e ou mul idisciplina y
communi y o wo k oge he , es ablish FAIR
( indable, accessible, in e ope able and
eusable) da a sha ing and deli e usable and
ep oducible analy ical ools.
Bioimage analysis is in he mids o a e olu ion ha will p o oundly
shape he u u e o he ield o decades, spu ed by ecen de elop-
men s in deep lea ning and AI. When hinking abou he u u e o his
change, i is emp ing o imagine one whe e cu en limi a ions and
pain poin s ha e been esol ed. Al hough imagining such a u u e is
easy, i is less easy o imagine he ansi ion om oday’s s a us quo o
ha desi ed u u e s a e: wha i will equi e and how i will be enabled.
He e we discuss how o acili a e scien i ic p og ess in he li e sciences,
iden i y necessa y changes, and p o ide ideas o how hese changes
migh bes be ealized.
O e he pas decade, he use o AI has e olu ionized bioimage
analysis. The numbe o esul s in a PubMed sea ch o he biomedical
li e a u e o he ph ase “deep lea ning” inc eased explosi ely be ween
2012 and 2022, and he numbe o a icles indexed by PubMed ha men-
ion deep lea ning mo e han doubled be ween 2020 and 2022 alone
( om 9,303 in 2020 o 19,650 in 2022). S a e-o - he-a ools o many
common bioimage analysis asks such as segmen a ion and denoising,
which a e c i ical o gene a ing scien i ic insigh om aw image da a,
now employ AI. Fo many use-cases hese ools subs an ially ou pe o m
hei classical compe i o s in speed and accu acy. Such ools a e o ce
mul iplie s o biological disco e y, acili a ing ad ances ha would
be di icul o impossible wi h mo e classical ools ha do no ely on
AI. The e is as po en ial wai ing o be unlocked in many a eas, such
as compu a ional mic oscopy, mul imodal da a analysis, cell acking,
pheno ypic classi ica ion and sma mic oscopy, and ou communi y
mus ind e icien ways o enable his po en ial as quickly as possible.
Challenges o AI in bioimage analysis
O e he nex en yea s, we an icipa e wo majo challenges o he
de elopmen o AI in bioimage analysis, om he pe spec i e o use s
as well as de elope s o AI ools. As we discuss below, hese wo chal-
lenges a e deeply in e wined, as hey bo h s em om he ac ha he
pe o mance o AI-based me hods and ools is closely ied o he da a
ha we e used o ain a model.
F om a me hod de elope pe spec i e, a wide ange o open and
s anda dized da a, me ada a and g ound u h labels need o become
a ailable o ad ance he s a e o he a . These da a should be chosen
o gene a ed in such a way as o enable me hod de elope s o ackle
challenging analyses ha a e cu en impedimen s o scien i ic p o-
g ess in he li e sciences.
F om a use pe spec i e, inding app op ia e models o analyze
a da ase is cu en ly no a s aigh o wa d ask. E en i many models
a e publicly a ailable o use s, choosing a sui able one equi es a way
o e alua e he quali y o model p edic ions on he use ’s own da a.
Indeed, p edic ions gene a ed by a gi en model need o be c i ically
assessed and ca e ully in e p e ed o ensu e he esponsible use o
AI-d i en ools. Me hod de elope s mus enable his by o e ing sui -
able aining oppo uni ies and p o iding ools ha deli e in e p e -
able quali y me ics.
G ound-b eaking AI esea ch equi es a ai amoun o
FAIR da a
The eason ha AI-based me hods ou pe o m classical app oaches in
so many analysis asks is ha hey can dis ill he mos ele an p io s
om a gi en body o aining da a. (A da a p io is a ask-speci ic clue
ex ac ed om p e iously seen examples ha can la e be used o make
be e decisions when new images a e p ocessed.) T ained ne wo ks
a e he e o e p ecisely ailo ed o sol ing a speci ic analysis ask in
he con ex o a speci ic ype o da a; ha is, he kind o da a on which
hey we e ained. These aining da a mus be o su icien quali y
and quan i y and, mo e impo an ly, pai ed wi h high-quali y expe
anno a ions. In addi ion o limi a ions a ound aining da a, he e is
an unme need o e e ence da ase s ha can be used o compa e
and benchma k he pe o mance o he apidly g owing numbe o
ools o common bioimage analysis applica ions. Some benchma k
da ase s exis oday, bu , despi e hei undispu ed u ili y, hey a e
o as ly di e ging quali y, age and p ac ical ele ance. They also do
no sha e a common s anda d o s o age and accessibili y. This lack
o common s anda ds makes i di icul o e alua e compu a ional
ools ac oss mul iple e e ence da ase s, ende ing i much ha de o
de elop widely gene alizable echniques. Be e e e ence da ase s
and ool compa isons will enable li e scien is s o de e mine which o
he apidly g owing zoo o me hods can bes sol e a gi en bioimage
analysis p oblem.
These demands o la ge amoun s o well-anno a ed and s uc-
u ed da a equi e consensus on how da a collec ion, anno a ion,
s o age and access can bes be uni ied and acili a ed. Mo ing o wa d,
we will need o make decisions as a communi y abou how o add ess
hese challenges. Un o una ely, he e is a na u al limi o he ex en
o which indi idual AI esea che s can achie e his goal. While majo
b eak h oughs in he nex en yea s will ce ainly equi e ad ancemen s
in he me hodology and compu a ional amewo ks ha d i e mode n
AI ools, hey will equally equi e be e ways o gene a e, use and sha e
Check o upda es
na u e me hods Volume 20 | July 2023 | 973–975 | 974
Commen
e iew panels. While echnically ela i ely easy o implemen , b oad
accep ance o such ci a ions as ep esen a i e o scien i ic ou pu is a
challenging social p oblem wi hou an easy solu ion5.
Mo e e ec i e ansla ion om me hods o ools can also be
achie ed i ou communi ies hi e and suppo mo e esea ch so -
wa e enginee s and bioimage analys s in a p o essional capaci y. Such
indi iduals a e bes sui ed o inco po a e he la es me hods in o
use - iendly so wa e ools and can media e di ec ly wi h bench sci-
en is s o help hem apply he bes -sui ed analysis ools success ully.
Only close collabo a ion and pa ne ship be ween compu a ional and
li e scien is s will os e he c ea ion o a o wa d-looking sys em ha
mo e apidly and e icien ly acili a es scien i ic disco e ies.
This acili a ion o scien i ic disco e y, hough, equi es ha bioim-
age analysis using AI me hods emains igo ous and ep oducible. To
his end, me hod de elope s, esea ch so wa e enginee s and bioim-
age analys s mus educa e end use s in he li e sciences abou wha AI
models can and canno p edic . Fo example, AI canno p ecisely eco e
ine de ails o s uc u es in di ac ion-limi ed mic oscopy images
ha a e below he di ac ion limi o he mic oscope, as in o ma ion
a hose spa ial scales is los du ing image o ma ion6. A bes , AI can
make p edic ions abou wha such s uc u es migh look like on he
basis o he aw inpu image and a da a p io ha was p e iously dis-
illed om he a ailable body o aining da a. Segmen a ion me hods,
o ins ance, lea n o do a good job on challenging pa s o he image
by inco po a ing a lea ned p io on he ypical shapes o ex u es o
objec s o be segmen ed. Fu he mo e, he ou pu s o gi en models
hemsel es deal wi h unce ain y in di e en ways, and no uni e sally
applicable quali y me ic exis s7. In some cases ( o example, in ecen
denoising app oaches8), wha a e e u ned om an AI model a e sam-
pled ‘in e p e a ions’ o he gi en aw inpu , d awn om a p e iously
lea ned dis ibu ion o easonable da a appea ances. Many app oaches,
howe e , e u n a single ou pu 9, which mos o en is some hing
close o he ‘a e age’ o all possible denoised in e p e a ions.
While hese issues a e known o co e me hod de elope s, hey
migh no be as well unde s ood by use s, and how bes o deal wi h
hem is no i ial and emains a opic o ac i e discussion in he AI
communi y. This unde sco es he impo ance o an open discou se
and consis en aining e o s in his a ea and, in he longe e m, o
b oadly accep ed s anda ds and quali y me ics o p edic ions made
by AI-powe ed analysis ools. The goal mus be o enable he li e science
communi y o iden i y he bes me hod o ool o a gi en job and o
disc imina e no only quali a i ely bu also quan i a i ely o wha
ex en p edic ions can be us ed o in e ac s abou he unde lying
biology. This will e en ually help o alle ia e he use o impene able
AI black boxes wi hin scien i ic da a analysis pipelines.
Gi en he unce ain ies ega ding he quali y o p edic ions and
he dependence on sui able aining da a, a key challenge is alida ion
and ep oducibili y o epo ed esul s. This is an innocen -looking
bu di icul p oblem ha is no easy o sol e in ull gene ali y, bu one
ha will also bene i emendously om FAIR da a esou ces, open
sha ing, s anda dized es da ase s and a simpli ied way o compa e
analysis ools.
The goal-o ien ed collabo a i e u u e o bioimage analysis
is b igh
A collabo a i e pa ne ship be ween li e scien is s and me hod de el-
ope s is a wo-way s ee be ween he biological ques ions being asked
and dedica ed and a ge ed me hodologies being c ea ed. This equi es
bioimage analys s, da a s ewa ds, da a scien is s, and esea ch so wa e
da a. The la e necessi a es no only sol ing he echnical challenges
o s o ing and e ec i ely sha ing la ge da ase s, bu also eaching com-
muni y consensus on ag eeable o ma s o images, image me ada a
and anno a ions such as on ologies and g ound- u h label da a. This
also elies on inding o es ablishing sui able, s able and long- e m
unding sou ces o de elop and main ain he equi ed in as uc u e.
We also need o suppo , encou age, and incen i ize widesp ead
da a anno a ion, sha ing, and euse. The FAIR p inciples we e de el-
oped in la ge pa o add ess hese challenges and a e a co e pa o he
solu ion o image da a1. In he con ex o AI-d i en bioimage analysis,
publicly a ailable FAIR da a allow he communi y o documen he key
analysis needs and enable he c ea ion o be e me hods, me hod
e alua ion and use - acing ools — ul ima ely suppo ing he goal o
ele a ing he a e o scien i ic disco e y.
Li e scien is s and me hod de elope s: be e oge he !
A s eng hened collabo a i e pa ne ship be ween li e scien is s and
me hod de elope s ha add esses he challenges ou lined abo e
should lead o a posi i e eedback loop o accele a ed echnology
de elopmen and success ul applica ion. Howe e , such a pa ne ship
is no wi hou i s challenges. Li e scien is s gene a e la ge amoun s o
aw image da a and a e in many cases he only ones capable o p o id-
ing expe anno a ions. Thus, hey a e key pa ne s in ad ancing he
ield o bioimage analysis. Un o una ely, he e o o anno a ing and
deposi ing new image da a in a FAIR-complian way is subs an ial, i cu -
en ly possible a all2,3, and in many cases la gely un ewa ded. Hence,
we should imp o e da a submission p ocedu es o exis ing o newly
c ea ed image a chi ing in as uc u es such ha da a sha ing becomes
as echnically ic ionless as possible.
A mu ually bene icial pa ne ship, howe e , equi es no only
ha li e scien is s gene a e and deposi FAIR-complian da a o he
use o me hod de elope s, bu also ha hose de elope s in es he
ime and e o equi ed o ans o m hei me hods in o easily usable
ools ha add ess he analysis needs o li e scien is s4. Un o una ely,
such e o s also o en go p o essionally un ewa ded. Al hough he e
migh be indi ec ewa ds o li e scien is s and me hod de elope s
who ope a e his way, he scien i ic communi y mus also s i e o
c ea e mo e incen i es and ewa d s uc u es. The easies and mos
immedia e ac ion would be o s eng hen publica ion and ci a ion
o compu a ional ools and es ablish a concep o da a ci a ions ha
would in eg a e wi h exis ing scien i ic success me ics and could be
used by hi ing, p omo ion and enu e commi ees as well as g an
Bioimage analysis
Bioimage analys s,
acili y s a , li e
scien is s
Tool de elopmen
Resea ch so wa e
enginee s, image
a chi e enginee s
Me hods esea ch
Compu e ision
and machine
lea ning scien is s
Raw da a and
me ada a
Challenges and
alida ion da a
Join da a and anno a ion in as uc u e and s anda ds
Compu a ional sciences Li e sciences
T aining labels
and me ada a
Fig. 1 | To achie e he o e a ching goal o ele a ing he a e o scien i ic
disco e y in he li e sciences, all membe s o ou communi y mus wo k
oge he in mu ually bene icial ways. This in e disciplina y collabo a ion
will bene i om a join da a and anno a ion in as uc u e ha elies on open
s anda ds he communi y can commi o.
na u e me hods Volume 20 | July 2023 | 973–975 | 975
Commen
and da a a chi e enginee s o join o ces o building an open and FAIR
da a in as uc u e (Fig. 1). Da a and da a anno a ions should be o
high quali y and ep esen he di e se ypes o analysis p oblems ha
cu en ly limi he a e o scien i ic p og ess. This will enable me hod
de elope s o pick and wo k on he mos p oduc i e p oblems hei
app oaches can ackle. A he same ime, models need o be sha ed
openly and be su icien ly documen ed, eusable and quan i a i ely
assessable4. This app oach will be key o syne gis ically ele a e he
a e o scien i ic p og ess in he li e sciences and in AI-based bioimage
analysis esea ch.
Damian Dalle Noga e1, Ma hew Ha ley 2, Jo an Deschamps1,
Jan Ellenbe g 3 & Flo ian Jug 1
1Fondazione Human Technopole, Milan, I aly. 2Eu opean Molecula
Biology Labo a o y, Eu opean Bioin o ma ics Ins i u e, Wellcome
Genome Campus, Hinx on, UK. 3Cell Biology and Biophysics Uni ,
Eu opean Molecula Biology Labo a o y, Heidelbe g, Ge many.
e-mail: lo ian.jug@ h .o g
Published online: 11 July 2023
Re e ences
1. Wilkinson, M. D. e al. Sci. Da a 3, 160018 (2016).
2. Ellenbe g, J. e al. Na . Me hods 15, 849–854 (2018).
3. Ha ley, M. e al. J. Mol. Biol. 434, 167505 (2022).
4. Ouyang, W. e al. P ep in a bioRxi h ps://doi.o g/10.1101/2022.06.07.495102
(2022).
5. Way, G. P. e al. PLoS Biol. 19, e3001419 (2021).
6. Mu phy, D. B. Fundamen als o Ligh Mic oscopy and Elec onic Imaging
(Wiley, 2001).
7. Reinke, A. e al. P ep in a a Xi h ps://doi.o g/10.48550/a Xi .2104.05642 (2021).
8. P akash, M, Delb acio, M., Milan a , P. & Jug, F. In In e na ional Con e ence on Lea ning
Rep esen a ions h ps://icl .cc/ i ual/2022/pos e /5977 (2021).
9. Weige , M. e al. Na . Me hods 15, 1090–1097 (2018).
Acknowledgemen s
M.H. and F.J. ecei ed unding by he Eu opean Commission h ough he Ho izon Eu ope
p og am (AI4LIFE p ojec , g an ag eemen 101057970-AI4LIFE).
Compe ing in e es s
The au ho s decla e no compe ing in e es s.
