DeepCon as : Deep Tissue Con as Enhancemen using Syn he ic Da a
Deg ada ions and OOD Model P edic ions
Nuno Pimp˜
ao Ma ins1
[email p o ec ed]
Yannis Kalaidzidis1
[email p o ec ed]
Ma ino Ze ial1
[email p o ec ed]
Flo ian Jug1,2
[email p o ec ed]
1MPI-CBG, D esden, Ge many 2Fundazione Human Technopole, Milano, I aly
Abs ac
Mic oscopy images a e c ucial o li e science esea ch,
allowing de ailed inspec ion and cha ac e iza ion o cel-
lula and issue-le el s uc u es and unc ions. Howe e ,
mic oscopy da a a e una oidably a ec ed by image deg a-
da ions, such as noise, blu , o o he s. Many such deg a-
da ions also con ibu e o a loss o image con as , which
becomes especially p onounced in deepe egions o hick
samples. Today, bes pe o ming me hods o inc ease he
quali y o images a e based on Deep Lea ning app oaches,
which ypically equi e g ound u h (GT) da a du ing ain-
ing. Ou inabili y o coun e ac blu ing and con as loss
when imaging deep in o samples p e en s he acquisi ion
o such clean GT da a. The ac ha he o wa d p ocess
o blu ing and con as loss deep in o issue can be mod-
eled, allowed us o p opose a new me hod ha can ci cum-
en he p oblem o unob ainable GT da a. To his end, we
i s syn he ically deg aded he quali y o mic oscopy im-
ages e en u he by using an app oxima e o wa d model
o deep issue image deg ada ions. Then we ained a
neu al ne wo k ha lea ned he in e se o his deg ada ion
unc ion om ou gene a ed pai s o aw and deg aded im-
ages. We demons a ed ha ne wo ks ained in his way
can be used ou -o -dis ibu ion (OOD) o imp o e he qual-
i y o less se e ely deg aded images, e.g. he aw da a im-
aged in a mic oscope. Since he absolu e le el o deg a-
da ion in such mic oscopy images can be s onge han he
addi ional deg ada ion in oduced by ou o wa d model,
we also explo ed he e ec o i e a i e p edic ions. He e, we
obse ed ha in each i e a ion he measu ed image con as
kep imp o ing while de ailed s uc u es in he images go
inc easingly emo ed. The e o e, dependen on he desi ed
downs eam analysis, a balance be ween con as imp o e-
men and e en ion o image de ails has o be ound.
Figu e 1. P oposed scheme o imp o e deep issue con as .
(1) Pai s o da a o supe ised aining a e gene a ed by deg ad-
ing aw mic oscopy images using a sui able deg ada ion unc ion
d(x)composed o a blu ing and a noising s ep. (2) Du ing supe -
ised ne wo k aining, syn he ically deg aded images a e used as
inpu s and he o iginal images as a ge s. (3) Du ing in e ence, we
eed he o iginal aw mic oscopy images once o i e a i ely in o
he ained ne wo k (see Sec ion 3).
1. Biological Mo i a ion
In his wo k we applied ou me hod (DEEPCONTRAST)
o mic oscopy images o li e issue. The li e is a e-
quen ly s udied sys em in biomedical esea ch, due o i s
i al unc ions in he human body, e.g. blood de oxi ica-
ion and bile p oduc ion. Li e issue is dense and com-
pac , composed o many di e en cell ypes ha display
an in ica e h ee-dimensional a chi ec u e. S ill, many as-
pec s o his s uc u e a e no ully unde s ood, which d i es
biomedical esea ch o use mode n mic oscopy echniques
o image la ge 3D sec ions o li e issue a he highes
achie able quali y and esolu ion.
Da a p esen ed in his wo k was ob ained using a Lase
Scanning Con ocal Mic oscope (LSM). This modali y al-
lowed us o ob ain highly de ailed image da a in la ge and
hick samples wi h sub-cellula esolu ion in all h ee spa-
ial dimensions. Un o una ely, he image quali y ine i ably
deg ades in deepe laye s o he imaged li e issue, mos ly
due o ligh sca e ing. This poses a challenge o be e ou
unde s anding o mesoscale s uc u es ha shape he li e
in i s ull 3D complexi y. The e o e, me hods ha acili a e
he downs eam analysis o la ge 3D image da a a e much
sough a e .
2. Rela ed Wo k
Classical algo i hms o enhance con as in images o -
en ely on he in ensi y his og am, ypically al e ing he
o e all his og am landscape wi h a se o p ede ined ules
o ei he ob ain a mo e uni o mly sampled dis ibu ion o o
ma ch an his og am ob ained om a desi ed e e ence. Ex-
amples o such algo i hms a e known as his og am ma ch-
ing [8] o his og am equaliza ion [20]. These app oaches
a e no con en -awa e, i.e., hey ollow speci ic ules inde-
penden o he s uc u es isible in he image o be mod-
i ied. Con as Limi ed Adap i e His og am Equaliza ion
(CLAHE) [34] is an example o widely used his og am
equaliza ion me hod. We use his me hod as one o ou
baselines me hods due o i s popula i y and widesp ead use
in scien i ic image p ocessing p o ocols.
Ano he popula amily o algo i hms used o imp o e
image quali y and con as a e eed- o wa d decon olu ion
me hods such as he one by Richa dson-Lucy [23,14],
o he popula Huygens so wa e om Scien i ic Volume
Imaging. These a e i e a i e app oaches ha a emp o
undo he blu ing induced by he poin sp ead unc ion
(PSF) o he mic oscope. The main d awback o such ap-
p oaches is he assump ion o spa ial in a iance o he PSF,
which does no hold in hick dense issue mic oscopy.
Deep Lea ning (DL) based applica ions ha e p o en o
pe o m especially well on se e al image es o a ion asks
like denoising [29,11,3,7,22,21], decon olu ion [4,9,
13], and supe - esolu ion [19,18,26,31,32].
Con en Awa e Image Res o a ion (CARE) [29], uses su-
pe ised DL me hods o es o es mic oscopy image quali y
in a ious ways. Howe e , in o de o use CARE, i is nec-
essa y o ob ain low and high quali y e sions o he same
objec s and s uc u es, which is no possible in many eal-
wo ld scena ios, such as he one p esen ed in his wo k. One
in e es ing insigh wi h espec o ou -o -dis ibu ion (OOD)
denoising is p esen ed in [15]. In i , he au ho s show ha
a ne wo k wi hou ainable bias e ms is mo e obus when
applied o inpu s ha con ain le els o noise ha a e incon-
sis en (OOD) wi h espec o he aining da a.
One popula way o sol e he p oblem o GT da a being
equi ed is o syn he ically gene a e he equi ed aining
pai s [28]. In [6], he au ho s used “c api ied” images, as
hey call i , o ob ain said aining pai s o aining supe -
esolu ion ne wo ks and ne wo ks ha inc ease he empo al
consis ency in ime-lapse mo ies. O he s ha e used syn-
he ic da a gene a ion o objec de ec ion [30] o segmen-
a ion [5].
Wo k speci ically conce ned wi h enhancing image con-
as is less common. The FCE-Ne [33] p oposes a ne wo k
a chi ec u e speci ically designed o enhance image con as
in biological image da a. Since his makes FCE-Ne ou
closes compe i o , despi e echnically being a qui e di e -
en app oach, we will always also compa e ou own esul s
o he ones ob ainable wi h he FCE-Ne .
3. Me hods
Inspi ed by [29] and [6], we also se ou o use he ma-
chine y o supe ised lea ning in deep neu al ne wo ks. In
ou case, o he sake o imp o ing image con as in mi-
c oscopy da a o la ge issue samples. To his end, we syn-
he ically gene a e app op ia e aining da a, i.e. pai s o im-
ages ha a e o lowe and highe con as . Na u ally, we
canno syn he ically emo e sca e ed ligh and noise om
aw mic oscopy da a, o he wise he e y ask we a e seek-
ing a solu ion o would be sol ed al eady. Ins ead, we can
add addi ional ligh sca e ing and noise o he a ailable aw
da a, making i e en wo se (see Figu e 1).
Mo e speci ically, ou deg ada ion unc ion is
d(x) = α·x+ (1 −α)·n(b(x)),(1)
whe e xis a aw inpu image, αa hype pa ame e ha con-
ols he blending be ween xand n(b(x)), b is a blu ing
unc ion ha models ligh sca e ing in biological issue,
and n a unc ion adding noise. In line wi h he mos domi-
nan noise in low-ligh luo escen mic oscopy, n is adding
Poisson noise o he blu ed da a.
Ligh sca e ing depends on e ac i e index ansi ions
h oughou he sample and a p ecise o wa d model is no
easy o compu e. Fo ou pu poses, he simple app oxima-
ion in oduced in Equa ion 1leads o con as enhancemen
esul s ha ou pe o m exis ing me hods like he FCE-Ne
(see Sec ion 4).
Once a body o inpu da a X= (x1, x2, . . . , xk)is u -
he deg aded o D= (d(x1), . . . , d(xk)), we use image
pai s (di, xi) o supe ised aining o a con as enhance-
men ne wo k (Model A). The ne wo k was ained as a
bias- ee [15] U-Ne [24]. The eason o no aining he
bias e ms o he ne wo k nodes is ha , once ained, i is
in ended o be applied o images xio simila , which a e
less se e ely a ec ed by deg ada ions, he e o e, OOD wi h
ega ds o di.
I e a i e P edic ions
Since he absolu e le el o deg ada ion in such mi-
c oscopy images can be s onge han he addi ional deg a-
da ion in oduced by ou o wa d model, we also explo ed
he e ec o i e a i e p edic ion I e a i e p edic ions wi h a
ained DEEPCONTRAST ne wo k (DC) a e simply mul i-
ple applica ions o DC o he inpu x. Fo example, he inal
DEEPCONTRAST (3×) p edic ion yis compu ed by
y=DC(DC(DC(x))).(2)
Figu e 2. Quali a i e esul s. Images o li e issue sec ions s ained wi h Phalloidin as a p oxy o cell bo de s, used o compa e ou esul s
(DEEPCONTRAST Model A) o se e al baseline me hods. Rows show image planes a di e en dep hs in he li e issue. Columns show
he aw inpu , esul s ob ained wi h CLAHE [34], Huygens decon olu ion (see Sec ion 4.4), bes FCE-Ne [33] esul s (3×), and ou bes
DEEPCONTRAST esul s (3×), espec i ely. The h ee igh mos columns shows he inse a eas ma ked by dashed boxes and line plo s o
aw in ensi ies, he FCE-Ne , and DEEPCONTRAST (along he g een line in he espec i e images). Scale ba s:20µm in ull size images,
10µm in inse s.
The expe imen s we desc ibe below and he esul s we
show in Figu e 3indica e ha i e a i e p edic ions indeed
keep inc easing image con as . I mus be no ed, ha be e
con as does no necessa ily mean ha he p edic ed image
is be e o downs eam analysis. Image ain de ails migh
ge los a he same ime and he impo ance o such de ails
depends on he downs eam analysis o be conduc ed. We
p esen one se o expe imen s on how o achie e a good
balanced be ween enhancing con as and p ese ing image
de ails in he ollowing sec ions.
4. Expe imen s
4.1. Da a
The imaged samples we e clea ed li e issue sec ions,
as desc ibed in [16], s ained wi h Phalloidin 488 an ibody
o label he Ac in co ex a all cell bo de s. Images we e
acqui ed in a Zeiss LSM 780 con ocal mic oscope, wi h a
Zeiss LCI Plan-Neo lua 63x 1.3 NA Gly/Wa e objec i e,
a 488nm wa eleng h exci a ion lase , an emission window
ange o 489 −551nm, and a pinhole size o 1AU. Images
we e acqui ed wi h an iso opic oxel size o 0.3µm. The
maximum imaging dep h was 100µm .
4.2. Image Deg ada ion Model
To compu e syn he ically deg aded images, as desc ibed
in Eq. 1, we i s blu and noise each 2D slice ( ocal plane)
using a Gaussian il e (σ= 20 pixels) and Poisson noise
a an es ima ed magni ude as desc ibed in [10] (using he
image analysis so wa e Mo ionT acking [17]). Syn he ic
images we e hen me ged wi h he o iginal images using α
alues anging in linea s eps om 0.5 o 0.3, wi h 0.5being
used o he mos supe icial slice.
4.3. Ne wo k A chi ec u e and Hype pa ame e s
Ou ne wo k is a U-Ne [24], using a dep h o i e, 32
ini ial ea u e channels, an MAE loss, and a linea unc ion
as he las ac i a ion laye . Ou models we e ained un il
con e gence wi h an ini ial lea ning a e o 4×10−4 o a
o al o 450 epochs wi h 200 s eps pe epoch. A s ep uses a
ba ch size o 16, o which each pa ch is a 128 ×128 pixels
c op om he body o aining da a. Ne wo ks we e buil
using he CSBDeep oolbox [29] using Tenso low 2.2.1.
By de aul , and i no o he wise s a ed, we would no ain
he bias e ms o each ne wo k node (-bias) o imp o e OOD
p edic ions [15], as also desc ibed in Sec ion 3.
Figu e 3. Quali a i e esul s o i e a i e OOD model applica ion. Con as o he image da a o Figu e 2i e a i ely enhanced using a ained
DEEPCONTRAST ne wo k (Model A). Rows show, analogous o Figu e 2, image planes a di e en dep h in o he imaged issue. Columns
show he aw inpu da a, and he esul s o applying DEEPCONTRAST a single ime, wo, h ee (same as in Figu e 2), and six consecu i e
imes. The h ee igh mos columns shows he inse a eas ma ked by dashed boxes and line plo s along he g een lines in he aw da a, and
along he 1×,3×, and 6×enhanced ou pu s. No e ha , while con as is con inuously enhanced, oo many i e a i e applica ions cause a
no able loss o image de ails. Scale ba s: 20 µm in ull size images, 10 µm in inse s.
Figu e 4. Quan i a i e esul s. Con as quan i ica ion using he
Pe cen ile Con as Index (see Sec ion 4.7) and he Wa ele Con-
as Index [1] (highe alues a e be e ) ep esen ed as a e age and
95% Con idence In e als a each dep h (N= 18). Dashed e i-
cal g ey lines depic dep hs shown in Figu e 2. Mul iple i e a ions
o FCE-Ne [33] and ou DEEPCONTRAST (Model A) app oach
show image con as is u he imp o ed when hese ne wo ks a e
i e a i ely applied.
4.4. Baselines
Baseline me hods, which we used o compa e
DEEPCONTRAST wi h, a e: (i)classical me hods, i.e.
CLAHE and decon olu ion using Huygens, and (ii)DL
based me hods, i.e. he FCE-Ne [33] ained on ou li e
da ase .
CLAHE images we e ob ained using Fiji [25], whe e he
Enhanced Local Con as plugin is an implemen a ion o
he o iginal CLAHE [34] me hod.
Decon ol ed images we e ob ained using he so wa e
Huygens P o essional ( e sion 22.10.0p6) om Scien i ic
Volume Imaging (SVI, The Ne he lands), ollowing p o-
ided pipelines wi h a heo e ical PSF. In e nally, Huygens
is using he CMLE algo i hm, wi h SNR se o 5, o 60 i -
e a ions, and he backg ound alue se o 100. This se up
led o he bes esul s on he da a a hand.
Fo he FCE-Ne [33] we used he code as i is p o ided
by he au ho s o he o iginal pape . I s wo h men ioning
ha he p o ided p e- ained FCE ne wo k led o in e io
esul s, hence, we ained he FCE-Ne om sc a ch un il
con e gence using ou own da a. Please no e ha we ha e
also applied he FCE-Ne i e a i ely, as desc ibed abo e in
Equa ion 2and obse ed ha also FCE-Ne esul s keep im-
p o ing. Fo ai compa ison we a e he e o e epo ing i -
e a i e FCE-Ne esul s whene e hey a e be e han single
p edic ions.
4.5. Double Deg ada ion Expe imen s
Since DEEPCONTRAST, by de ini ion, is applied OOD,
we wonde ed i i e a i e con as enhancemen will make
consis en s eps.
Figu e 5. Quali a i e and quan i a i e esul s o segmen a ion masks c ea ed a mul iple i e a ions o con as enhancemen . Le side
column shows Raw inpu and segmen a ion mask (Sec ion 4.8). Each u he column shows di e en i e a ions o con as enhancemen
and co esponding segmen a ion o cell bo de s. Top ow shows in e ence esul s wi h DEEPCONTRAST and bo om ow shows in e ence
esul s wi h FCE-Ne . Yellow a ow heads in images highligh los o deg aded s uc u es when compa ing DEEPCONTRAST and FCE-Ne .
Violin plo s shows dis ibu ion o IoU alues be ween he di e en con as enhancemen me hods and aw segmen a ion masks used as
e e ence a mul iple i e a ions (N= 2682), showing as e dec ease o IoU alues and mo e abundan mis akes in segmen a ion wi h
FCE-Ne .
Figu e 6. Quali a i e double deg ada ion esul s and p edic ions wi h Model B (see Sec ion 4.5). Each ow shows di e en dep hs in he
sample, as in p e ious igu es. Columns depic , om le o igh , (i)double deg aded images used as inpu s du ing aining, (ii)single
deg aded images used as a ge s du ing aining, (iii)p edic ions o Model B when applied o da a as he one in column wo, (i ) he aw
image da a o compa e Model B (1×) ou pu s agains , ( )Model B (3×) ou pu s, and ( i)Model A (2×) ou pu s o compa e Model B (3×)
ou pu s agains . No e ha hese wo p edic ions should be and a e simila since Model B s a s wi h inpu s ha a e once mo e syn he ically
deg aded. The smalle panels in he igh mos columns show he inse s (ma ked by dashed lines) om he o he columns. S uc u es in all
enhanced images a e consis en wi h s uc u es in he aw da a, which is encou aging. Scale ba s: 20 µm in ull size images, 10 µm in
inse s.
SSIM
MB1× s Raw MB2× s M A1×MB3× s M A2×
Ve y Deep 0.51 ±0.07 0.70 ±0.08 0.80 ±0.07
Deep 0.52 ±0.07 0.72 ±0.09 0.82 ±0.07
In e media e 0.56 ±0.06 0.77 ±0.06 0.84 ±0.03
Shallow 0.60 ±0.06 0.80 ±0.05 0.83 ±0.03
Ve y Shallow 0.59 ±0.08 0.79 ±0.06 0.83 ±0.02
Table 1. Quan i a i e esul s o he double deg ada ion expe i-
men desc ibed in Sec ion 4.5 and shown in Figu e 6. We com-
pa e he ou pu s o h ee i e a ions o Model B (MBk×), which
was ained on double-deg aded and single-deg aded inpu s, o
he closes ma ching images, i.e. aw da a o di ec p edic ions
o Model B (MB1× s Raw), di ec p edic ions o Model A o
wo i e a ions o Model B (MB2× s MA1×), and p edic ions o
wo i e a ions o Model A o h ee i e a ions o Model B (MB3×
s MA2×). Sec ion 3 o de ails.
As a i s e i ica ion o ou app oach we deg aded he
aw image da a wice, acqui ing da a iple s (ei, di, xi),
wi h xi∈X, and di=d(xi)and ei=d(d(xi)). Then
we ained a DEEPCONTRAST ne wo k, Model B, on pai s
(ei, di)and applied he ained ne wo k, in-line wi h he ini-
ially p oposed p ocedu e, o di o inc ease i s con as and
yielding yi=DC(di).
Since we s a ed by double deg ading he o iginal xi,
we can now compa e he p edic ion yiwi h xi, and u -
he i e a ions o Model B wi h co esponding p edic ions
ob ained wi h Model A (see Sec ion 3). I he ained
DEEPCONTRAST ne wo k is indeed a good app oxima ion
o he in e se o ou deg ada ion unc ion d, p edic ions yi
should be simila o he o iginal images xia he i s i e -
a ion, and o he co esponding i e a ion o ou pu images
om Model A.
4.6. Abla ions
In o de o e alua e i bias- ee aining [15] is indeed
leading o be e esul s, we decided o epea model aining
also on ne wo ks ha a e no bias- ee, i.e. ain all weigh s
and biases.
4.7. Con as Quan i ica ion
Wa ele Con as Index
Wi h inc easing con as in an image, we expec backg ound
signal o be educed and, consequen ly, he b igh ness o
biological s uc u es in he image (signal) o be inc eased.
To quan i y image con as when no GT da a is a ailable, we
used he Wa ele Con as Index (WCI) [1]. This measu e
compu es he di e ence be ween coe icien s ob ained om
a wa ele decomposi ion, ollowing he equa ion
WCI(x) = log(W95 h (x)
W50 h (x)),(3)
whe e xis he inpu image o which we wan o e alua e
he con as , W95 h is he 95 h pe cen ile wa ele coe icien
and 50 h is he median wa ele coe icien .
Wa ele decomposi ion was pe o med wi h he Py-
Wa ele s [12] py hon package using a Haa wa ele as he
e e ence unc ion and used coe icien s up o he ou h
le el o decomposi ion.
Pe cen ile Con as Index
We also used he Pe cen ile Con as Index (PCI) o quan i y
in ensi y di e ences be ween image backg ound and image
s uc u es. The PCI is compu ed by
PCI(x) = log(I95 h (x)
I50 h (x)),(4)
whe e xis again he inpu image o e alua e, and I95 h is
he 95 h in ensi y alue in he image being analyzed and
I50 h is he median in ensi y o x. We use he median alue,
assuming ha a leas hal he pixels o any gi en image a e
backg ound pixels.
4.8. Downs eam Segmen a ion a e Con as En-
hancemen
Enhancing con as no necessa ily imp o es down-
s eam p ocess-abili y (in e p e abili y) o a gi en da ase .
While he con as , as measu ed by WCI and/o PCI, migh
s ill imp o e, de ails ele an o biological in e p e a ion o
he da a migh al eady ge los . The e o e, he bes amoun
o con as enhancemen depends on a gi en downs eam
analysis ask. To his end, we in oduced a downs eam seg-
men a ion ask and checked i a ixed segmen a ion pipeline
imp o ed wi h espec o exis ing g ound u h labels. GT
segmen a ion masks we e gene a ed om aw da a using
Labki [2] (a ailable as a Fiji [25] plugin).
Fo simplici y, we segmen ed con as enhanced im-
ages yiby h esholding, op imizing o he bes h eshold
alue, i.e. he one ha maximizes he in e sec ion-o e -
union (IoU) wi h espec o he p e iously gene a ed GT.
Ou easoning was ha enhancing con as would esul in
a be e IoU a e h esholding as long as ele an s uc-
u es in he con as enhanced images yiwe e no los .
As soon as de ails we e ge ing los , he IoU d opped, al-
lowing us o choose he mos sensible i e a ion dep h o
DEEPCONTRAST (o he FCE-Ne ).
5. Resul s
Quali a i e esul s p esen ed in Figu e 2sugges
ha DEEPCONTRAST ou pe o ms all baseline me hods.
DEEPCONTRAST emo es o educes image noise and en-
hances he in ensi y o isible image s uc u es seemingly
wi hou loosing ine de ails (signal) om p edic ed images.
Hence, DEEPCONTRAST is indeed inc easing image con-
as .
Bo h classical me hods, i.e. CLAHE and decon olu ion,
displayed ela i ely poo esul s mainly deep in o he issue.
CLAHE ampli ied image noise a all imaging dep hs and
mos ly ailed o highligh biological s uc u es. Decon olu-
ion, on he o he hand, did educe image noise, bu ailed o
inc ease in ensi ies o o eg ound s uc u es (mos ob ious
deep in o he issue).
The FCE-Ne pe o med much be e , leading o good e-
sul s close o he su ace. Bu he image con as in FCE-Ne
p edic ions decayed wi h inc easing dep h (see inse s in Fig-
u e 2). Quali a i ely, he FCE-Ne also seemed o p oduce
less sha p cell bo de s (as seen in ei he deep and shallow
image egions).
To alida e hese quali a i e obse a ions, we quan i ied
con as wi h he wo measu es WCI and PCI (see Sec-
ion 4.7). As can be seen in Figu e 4, DEEPCONTRAST
achie ed highe image con as a all imaging dep hs and
o e all plo ed i e a i e applica ions (1× o 3×). One no-
able excep ion a e he WCI alues o 3×i e a ions in in-
e media e imaging dep hs. In hese images, despi e he
FCE-Ne showing highe WCI alues, one can see mo e
image s uc u e being los in FCE-Ne p edic ions han in
p edic ions ob ained wi h DEEPCONTRAST (see Figu e 5
o a quali a i e and quan i a i e compa ison).
As in oduced abo e, con as enhancemen can be ap-
plied i e a i ely (Equa ion 2). Resul s o pe o ming mul i-
ple ounds o enhancemen a e shown in Figu e 3. Visually,
he bes esul s we e ob ained wi h h ee ounds o enhance-
men (3×). While con as eadou s using WCI and PCI
would s ill imp o e wi h addi ional i e a ions, image de ails
would s a disappea ing (as can be seen in he 6×column
and he line-plo s in Figu e 3).
Quali a i e esul s o he Double Deg ada ion Expe i-
men s in oduced in Sec ion 4.5, a e shown in Figu e 6.
P edic ions o Model B a i e a ion kshould and a e co -
esponding well o p edic ions o Model A a i e a ion k−1
since Model B is ained on image pai s ha a e one ap-
plica ion o ou o wa d deg ada ion model (d) mo e de-
g aded. We quan i y his ia s uc u e simila i y index mea-
su e (SSIM) [27] in Table 1and allow o a quali a i e com-
pa ison be ween he co esponding columns in Figu e 6.
5.1. Con as Enhancemen s. Segmen a ion
To be e quan i y he undesi ed e ec o loosing ele-
an de ails while simul aneously gaining addi ional con-
as in he p ocessed mic oscopy da a, we in oduced a sim-
ple h eshold based segmen a ion ask (See Sec ion 4.8). A
quali a i e as well as quan i a i e compa ison is shown in
Figu e 5. IoU alues a e ini ially inc easing wi h numbe
o i e a ions, bu hen e en ually d op when oo many im-
age s uc u es a e emo ed. The FCE-Ne gene ally shows
lowe IoU alues, sugges ing ha DEEPCONTRAST is no
only leading o mo e con as ed images, bu is a he same
ime main aining mo e image de ails wi h i e a ions. In ad-
di ion o he IoU quan i ica ion, we highligh ed los de ails
on images wi h yellow a ow heads (see in Figu e 5), poin -
ing di e ences be ween i e a i e in e ences.
5.2. Abla ion: T aining including Bias
As in oduced abo e, DEEPCONTRAST employs bias-
ee [15] ne wo k aining. In Figu e 7we show ep esen-
a i e p edic ions o Model A (3×), as used in Figu e 2,
and compa e hem o p edic ions ob ained wi h an equi -
alen model which was ained wi h bias (+bias). Yellow
a ow heads in he igu e poin a loca ions whe e he bias
ee ne wo k does a be e job e aining image de ails. Em-
pi ically, we did no spo any cases whe e he opposi e is
ue, which ga e us addi ional mo i a ion o use bias- ee
ne wo ks in DEEPCONTRAST.
Figu e 7. Quali a i e esul s o ne wo ks ained wi hou and wi h
bias. Phalloidin s ained images o li e issue sec ions enhanced
3×wi h DEEPCONTRAST models ained wi h (+bias; igh side)
and wi hou bias (-bias; le side). Rows show image planes a
di e en dep hs ela i e o co e -glass. Ne wo k model ained
wi h bias pe o ms wo se when applied OOD, emo ing s uc u es
seen in a model ained wi hou bias (- bias), highligh ed by yellow
a ow-heads. Scale ba s: 20 µm in ull size images, 10 µm in
inse s.
6. Discussion and Conclusion
In his wo k we p opose o use an image deg ada ion
unc ion o app oxima e ligh sca e ing in deep issue imag-
ing and use i o gene a e syn he ically deg aded da a o en-
able supe ised ne wo k aining. Ou esul s show ha he
ela i ely simple deg ada ion model we in oduced is su -
icien o inc ease image con as in eal mic oscopy da a.
Ou me hod can be applied in an i e a i e manne o u -
he inc ease image con as and will e ain de ailed image
s uc u es o mo e i e a ions han he compe i i e baseline
me hods we compa ed agains .
Fo he li e da a a hand, we ha e ound ha he bes
numbe o i e a ions o con as enhancemen is h ee (3×).
This assessmen is based on a combina ion o con as en-
hancemen and e en ion o ine image de ails in he con-
as enhanced p edic ions. A mo e quan i a i e app oach o
he isual assessmen was in oduced by means o a down-
s eam segmen a ion ask, which has indeed con i med ou
ini ial indings.
In gene al, he bes ade-o be ween con as enhance-
men and s uc u al in eg i y o p edic ions depends on he
na u e o he downs eam p ocessing ask o be conduc ed.
Hence, an analysis simila o he one we pe o med o he
segmen a ion ask could be equi ed o e alua e he bes -
pe o ming se up.
Simila ly we ound ha o be e OOD applica ion o
ou ained ne wo ks, he bias ee e sion seems o lead o
be e esul s.
While ou app oach is leading o excellen esul s and
can easily be used by mic oscopis s and li e scien is s o
imp o e olume ic image da a o quan i a i e downs eam
p ocessing, addi ional esea ch will be equi ed o undo im-
age deg ada ions deep in imaged issues in mo e undamen-
al ways.
Acknowledgmen s
The au ho s hank Jos´
e Valenzuela-I u a o acqui ing
image da a. We hank he LMF and SCF acili ies a
MPI-CBG o echnical suppo . We hank Igo Zuba e
o help and eedback wi h he p esen ed expe imen al se-
ups. We hank Ashesh, Ani ban Ray, Sheida Rahnamai
Ko dasiabi, Igo Zuba e , Jo an Deschamps and Damian
Dalle Noga e o help ul discussion and eedback. This
wo k was suppo ed by he Eu opean Resea ch Council
ERC Ad anced Rulli e G an (no. 695646) o M. Ze-
ial. Addi ionally, his wo k was suppo ed by he Eu o-
pean Union h ough he Ho izon Eu ope p og am AI4LIFE
wi h g an ag eemen 101057970-AI4LIFE. Funding was
also p o ided om he Max-Planck Socie y unde p ojec
code M.IF.A.MOZG8106.
Re e ences
[1] Anna Lea Alb igh and Pe e Huybe s. Pain ings by
Tu ne and Mone depic ends in 19 h cen u y ai pollu-
ion. P oceedings o he Na ional Academy o Sciences,
120(6):e2219118120, Feb. 2023. Publishe : P oceedings o
he Na ional Academy o Sciences. 4,6
[2] Ma hias A z , Jo an Deschamps, Ch is ophe Schmied, To-
bias Pie zsch, Debo ah Schmid , Pa el Tomancak, Robe
Haase, and Flo ian Jug. LABKIT: Labeling and Segmen-
a ion Toolki o Big Image Da a. F on ie s in Compu e
Science, 4, 2022. 6
[3] Joshua Ba son and Loic Roye . Noise2Sel : Blind Denoising
by Sel -Supe ision. 36 h In e na ional Con e ence on Ma-
chine Lea ning, ICML 2019, 2019-June:826–835, Jan. 2019.
Publishe : In e na ional Machine Lea ning Socie y (IMLS)
ep in : 1901.11365. 2
[4] Jiji Chen, Hideki Sasaki, Hoyin Lai, Yijun Su, Jiamin Liu,
Yicong Wu, Alexande Zho me , Ch is ian A. Combs, I an
Rey-Sua ez, Hung-Yu Chang, Chi Chou Huang, Xuesong
Li, Min Guo, S ineil Nizambad, A pi a Upadhyaya, Shih-
Jong J. Lee, Luciano A. G. Lucas, and Ha i Sh o . Th ee-
dimensional esidual channel a en ion ne wo ks denoise and
sha pen luo escence mic oscopy image olumes. Na u e
Me hods, 18(6):678–687, June 2021. Numbe : 6 Publishe :
Na u e Publishing G oup. 2
[5] Kenne h W. Dunn, Chichen Fu, Da id Joon Ho, Soonam
Lee, Shuo Han, Paul Salama, and Edwa d J. Delp. Deep-
Syn h: Th ee-dimensional nuclea segmen a ion o biologi-
cal images using neu al ne wo ks ained wi h syn he ic da a.
Scien i ic Repo s, 9(1):18295, Dec. 2019. Numbe : 1 Pub-
lishe : Na u e Publishing G oup. 2
[6] Linjing Fang, F ed Mon oe, Sammy Weise No ak, Lynd-
sey Ki k, Ca a R. Schia on, Seungyoon B. Yu, Tong Zhang,
Melissa Wu, Kyle Kas ne , Alaa Abdel La i , Zijun Lin,
And ew Shaw, Yoshiyuki Kubo a, John Mendenhall, Zhao
Zhang, Gulcin Pekku naz, K is en Ha is, Je emy Howa d,
and U i Mano . Deep lea ning-based poin -scanning supe -
esolu ion imaging. Na u e Me hods, 18(4):406–416, Ap .
2021. Numbe : 4 Publishe : Na u e Publishing G oup. 2
[7] Anna S. Goncha o a, Al Honigmann, Flo ian Jug, and
Alexande K ull. Imp o ing Blind Spo Denoising o Mi-
c oscopy, Aug. 2020. a Xi :2008.08414 [cs, eess]. 2
[8] RC Gonzalez and RE Woods. Digi al image p ocessing hi d
edi ion, s . 653–658, 2006. 2
[9] Min Guo, Yue Li, Yijun Su, Talley Lambe , Damian Dalle
Noga e, Ma k W. Moyle, Leigh on H. Duncan, Richa d
Ikegami, An hony San ella, I an Rey-Sua ez, Daniel G een,
Anas asia Bei ige , Jiji Chen, Ha shad Vishwas ao, Sunda
Ganesan, Vic o ia P ince, Jenni e C. Wa e s, Ch is ina M.
Annunzia a, Ma kus Ha ne , William A. Mohle , Ajay B.
Chi nis, A pi a Upadhyaya, Ted B. Usdin, Zhi ong Bao,
Daniel Col´
on-Ramos, Pa ick La Ri ie e, Hua eng Liu, Yi-
cong Wu, and Ha i Sh o . Rapid image decon olu ion and
mul i iew usion o op ical mic oscopy. Na u e Bio echnol-
ogy, 38(11):1337–1346, No . 2020. Numbe : 11 Publishe :
Na u e Publishing G oup. 2
[10] Yannis Kalaidzidis. Fluo escence Mic oscopy Noise Model :
Es ima ion o Poisson Noise Pa ame e s om Snap-Sho Im-
age. BIOCOMP’17, (1):63–66, 2017. ISBN: 1601324502.
3
[11] Alexande K ull, Tim Oli e Buchholz, and Flo ian Jug.
Noise2 oid-Lea ning denoising om single noisy images.
P oceedings o he IEEE Compu e Socie y Con e ence on
Compu e Vision and Pa e n Recogni ion, 2019-June:2124–
2132, 2019. ISBN: 9781728132938 ep in : 1811.10980. 2
[12] G ego y R. Lee, Ral Gomme s, Filip Waselewski, Kai
Wohl ah , and Aa on O’Lea y. PyWa ele s: A Py hon pack-
age o wa ele analysis. Jou nal o Open Sou ce So wa e,
4(36):1237, Ap . 2019. 6
[13] Yue Li, Yijun Su, Min Guo, Xiao ei Han, Jiamin Liu, Ha -
shad D. Vishwas ao, Xuesong Li, Ryan Ch is ensen, Ti as
Sengup a, Ma k W. Moyle, I an Rey-Sua ez, Jiji Chen,
A pi a Upadhyaya, Ted B. Usdin, Daniel Al onso Col´
on-
Ramos, Hua eng Liu, Yicong Wu, and Ha i Sh o . Inco -
po a ing he image o ma ion p ocess in o deep lea ning im-
p o es ne wo k pe o mance. Na u e Me hods, 19(11):1427–
1437, No . 2022. Numbe : 11 Publishe : Na u e Publishing
G oup. 2
[14] L. B. Lucy. An i e a i e echnique o he ec i ica ion o
obse ed dis ibu ions. The As onomical Jou nal, 79:745,
June 1974. ADS Bibcode: 1974AJ.....79..745L. 2
[15] S eyas Mohan, Zah a Kadkhodaie, Ee o P. Simoncelli, and
Ca los Fe nandez-G anda. Robus and in e p e able blind
image denoising ia bias- ee con olu ional neu al ne wo ks,
Feb. 2020. a Xi :1906.05478 [cs, eess, s a ]. 2,3,6,7
[16] He n´
an Mo ales-Na a e e, Hideno i Nonaka, And ´
e
Scholich, Fabi´
an Sego ia-Mi anda, Wal e de Back, Ki s in
Meye , Roman L Bogo ad, Vic o Ko eliansky, Lu z B usch,
Yannis Kalaidzidis, F ank J¨
uliche , Benjamin M F ied ich,
and Ma ino Ze ial. Liquid-c ys al o ganiza ion o li e
issue. eLi e, 8:1–18, 2019. 3
[17] He n´
an Mo ales-Na a e e, Fabi´
an Sego ia-Mi anda, Pio
Klukowski, Ki s in Meye , Hideno i Nonaka, Gio anni Ma -
sico, Mikhail Che nykh, Alexande Kalaidzidis, Ma ino Ze-
ial, and Yannis Kalaidzidis. A e sa ile pipeline o he
mul i-scale digi al econs uc ion and quan i a i e analysis
o 3D issue a chi ec u e. eLi e, 4(DECEMBER2015):1–29,
2015. 3
[18] Elias Nehme, Lucien E Weiss, Tome Michaeli, and
Yoa Shech man. Deep-STORM: supe - esolu ion single-
molecule mic oscopy by deep lea ning. Op ica, 5(4):458,
Ap . 2018. 2
[19] Wei Ouyang, And ey A is o , Micka¨
el Lelek, Xian Hao, and
Ch is ophe Zimme . Deep lea ning massi ely accele a es
supe - esolu ion localiza ion mic oscopy. Na . Bio echnol.,
27:2672, Ap . 2018. 2
[20] S ephen M. Pize , E. Philip Ambu n, John D. Aus in,
Robe C oma ie, A i Geselowi z, T ey G ee , Ba e
Haa Romeny, John B. Zimme man, and Ka el Zuide eld.
Adap i e his og am equaliza ion and i s a ia ions. Com-
pu e Vision, G aphics, and Image P ocessing, 39(3):355–
368, Sep . 1987. 2
[21] Mangal P akash, Mau icio Delb acio, Peyman Milan a , and
Flo ian Jug. In e p e able Unsupe ised Di e si y Denois-
ing and A e ac Remo al, Feb. 2022. a Xi :2104.01374 [cs,
eess, q-bio]. 2
[22] Mangal P akash, Alexande K ull, and Flo ian Jug. Fully
Unsupe ised Di e si y Denoising wi h Con olu ional Va i-
a ional Au oencode s, Ma . 2021. a Xi :2006.06072 [cs,
eess]. 2
[23] William Hadley Richa dson. Bayesian-Based I e a i e
Me hod o Image Res o a ion*. JOSA, 62(1):55–59, Jan.
1972. Publishe : Op ica Publishing G oup. 2
[24] Ola Ronnebe ge , Philipp Fische , and Thomas B ox. U-ne :
Con olu ional ne wo ks o biomedical image segmen a ion.
In Lec u e No es in Compu e Science (including subse ies
Lec u e No es in A i icial In elligence and Lec u e No es in
Bioin o ma ics), olume 9351, pages 234–241. Sp inge Ve -
lag, 2015. ISSN: 16113349 ep in : 1505.04597. 2,3
[25] Johannes Schindelin, Ignacio A ganda-Ca e as, E win
F ise, Ve ena Kaynig, Ma k Longai , Tobias Pie zsch,
S ephan P eibisch, Cu is Rueden, S ephan Saal eld, Ben-
jamin Schmid, Jean-Y es Tine ez, Daniel James Whi e,
Volke Ha ens ein, Ke in Elicei i, Pa el Tomancak, and Al-
be Ca dona. Fiji: an open-sou ce pla o m o biological-
image analysis. Na u e Me hods, 9(7):676–682, July 2012.
Numbe : 7 Publishe : Na u e Publishing G oup. 4,6
[26] Ying Tai, Jian Yang, and Xiaoming Liu. Image Supe -
Resolu ion ia Deep Recu si e Residual Ne wo k. In 2017
IEEE Con e ence on Compu e Vision and Pa e n Recog-
ni ion (CVPR), pages 2790–2798, Honolulu, HI, July 2017.
IEEE. 2
[27] Zhou Wang, Alan C Bo ik, Hamid R Sheikh, and Ee o P Si-
moncelli. Image Quali y Assessmen : F om E o Visibili y
o S uc u al Simila i y. Technical Repo 4, 2004. Publica-
ion Ti le: IEEE TRANSACTIONS ON IMAGE PROCESS-
ING Volume: 13. 7
[28] Ma in Weige , Loic Roye , Flo ian Jug, and Gene My-
e s. Iso opic econs uc ion o 3D luo escence mic oscopy
images using con olu ional neu al ne wo ks, Ap . 2017.
a Xi :1704.01510 [cs]. 2
[29] Ma in Weige , Uwe Schmid , Tobias Boo he, And eas
M¨
ulle , Alexand Dib o , Akanksha Jain, Benjamin Wil-
helm, Debo ah Schmid , Coleman B oaddus, Siˆ
an Culley,
Mau icio Rocha-Ma ins, Fabi´
an Sego ia-Mi anda, Ca en
No den, Rica do Hen iques, Ma ino Ze ial, Michele Soli-
mena, Jochen Rink, Pa el Tomancak, Loic Roye , Flo ian
Jug, and Eugene W. Mye s. Con en -awa e image es o a-
ion: pushing he limi s o luo escence mic oscopy. Na u e
Me hods, 15(12):1090–1097, Dec. 2018. Publishe : Na u e
Publishing G oup ep in : 236463. 2,3
[30] Ruijie Yao, Jiaqiang Qian, and Qiang Huang. Deep-lea ning
wi h syn he ic da a enables au oma ed picking o c yo-EM
pa icle images o biological mac omolecules. Bioin o ma -
ics, 36(4):1252–1259, Feb. 2020. 2
[31] Yulun Zhang, Kunpeng Li, Kai Li, Lichen Wang, Bineng
Zhong, and Yun Fu. Image Supe -Resolu ion Using Ve y
Deep Residual Channel A en ion Ne wo ks, July 2018.
a Xi :1807.02758 [cs] e sion: 2. 2
[32] Yulun Zhang, Yapeng Tian, Yu Kong, Bineng Zhong,
and Yun Fu. Residual Dense Ne wo k o Image Supe -
