b ain
sciences
A icle
Au oma ic Segmen a ion o he Ol ac o y Bulb
Dmi iy Desse 1,*, F ancisca Assunção2, Xiaoguang Yan 1, Vic o Al es 2, Hen ique M. Fe nandes 3,4,5,†
and Thomas Hummel 1,†


Ci a ion: Desse , D.; Assunção, F.;
Yan, X.; Al es, V.; Fe nandes, H.M.;
Hummel, T. Au oma ic Segmen a ion
o he Ol ac o y Bulb. B ain Sci. 2021,
11, 1141. h ps://doi.o g/10.3390/
b ainsci11091141
Academic Edi o : Tjee d
Olde Schepe
Recei ed: 6 July 2021
Accep ed: 25 Augus 2021
Published: 28 Augus 2021
Publishe ’s No e: MDPI s ays neu al
wi h ega d o ju isdic ional claims in
published maps and ins i u ional a il-
ia ions.
Copy igh : © 2021 by he au ho s.
Licensee MDPI, Basel, Swi ze land.
This a icle is an open access a icle
dis ibu ed unde he e ms and
condi ions o he C ea i e Commons
A ibu ion (CC BY) license (h ps://
c ea i ecommons.o g/licenses/by/
4.0/).
1
Smell & Tas e Clinic, Depa men o O o hinola yngology, Technische Uni e si ä , 01307 D esden, Ge many;
[email p o ec ed] (X.Y.); [email p o ec ed] (T.H.)
2Depa men o In o ma ics, School o Enginee ing, Uni e si y o Minho, 4704-553 B aga, Po ugal;
[email p o ec ed] (F.A.); [email p o ec ed] (V.A.)
3Cen e o Music in he B ain, Depa men o Clinical Medicine, Aa hus Uni e si y, Noe eb ogade 44, 1A,
8000 Aa hus, Denma k; [email p o ec ed]
4Cen e o Func ionally In eg a i e Neu oscience, Aa hus Uni e si y, Noe eb ogade 44, 1A,
8000 Aa hus, Denma k
5Fla ou Ins i u e, Depa men o Clinical Medicine, Aa hus Uni e si y, Noe eb ogade 44, 1A,
8000 Aa hus, Denma k
*Co espondence: dmi iy[email p o ec ed]
† Sha ed las au ho ship.
Abs ac :
The ol ac o y bulb (OB) has an essen ial ole in he human ol ac o y pa hway. A change
in ol ac o y unc ion is associa ed wi h a change o OB olume. I has been shown o p edic he
p ognosis o ol ac o y loss and i s olume is a bioma ke o a ious neu odegene a i e diseases, such
as Alzheime ’s disease. Thus a , ob aining an OB olume o esea ch pu poses has been pe o med
by manual segmen a ion alone; a e y ime-consuming and highly a e -biased p ocess. As such, his
p ocess d ama ically educes he abili y o p oduce ai and eliable compa isons be ween s udies, as
well as he p ocessing o la ge da ase s. Ou s udy aims o sol e his by p oposing a no el me hod-
ological amewo k o he unbiased measu emen o OB olume. In his pape , we p esen a ully
au oma ed ool ha success ully pe o ms such a ask, accu a ely and quickly. In o de o de elop a
s able and e sa ile algo i hm and o ain he neu al ne wo k, we used ou da ase s consis ing o
whole-b ain T1 and high- esolu ion T2 MRI scans, as well as he co esponding clinical in o ma ion
o he subjec ’s smelling abili y. One da ase con ained da a o pa ien s su e ing om anosmia o
hyposmia (N = 79), and he o he h ee da ase s con ained da a o heal hy con ols (N = 91). Fi s ,
he manual segmen a ion labels o he OBs we e c ea ed by wo expe ienced a e s, independen ly
and blinded. The algo i hm consis ed o he ollowing ou di e en s eps: (1) mul imodal da a
co- egis a ion o whole-b ain T1 images and T2 images, (2) empla e-based localiza ion o OBs,
(3) bounding box cons uc ion, and las ly, (4) segmen a ion o he OB using a 3D-U-Ne . The esul s
om he au oma ed segmen a ion algo i hm we e es ed on p e iously unseen da a, achie ing a
mean dice coe icien (DC) o 0.77
±
0.05, which is ema kably con e gen wi h he in e - a e DC
o 0.79
±
0.08 es ima ed o he same coho . Addi ionally, he symme ic su ace dis ance (ASSD)
was 0.43
±
0.10. Fu he mo e, he segmen a ions p oduced using ou algo i hm we e manually
a ed by an independen blinded a e and ha e eached an equi alen a ing sco e o 5.95
±
0.87
compa ed o a a ing sco e o 6.23
±
0.87 o he i s a e ’s segmen a ion and 5.92
±
0.81 o he
second a e ’s manual segmen a ion. Taken oge he , hese esul s suppo he success o ou ool in
p oducing au oma ic as (3–5 min pe subjec ) and eliable segmen a ions o he OB, wi h i ually
ma ching accu acy wi h he cu en gold s anda d echnique o OB segmen a ion. In conclusion,
we p esen a newly de eloped eady- o-use ool ha can pe o m he segmen a ion o OBs based
on mul imodal da a consis ing o T1 whole-b ain images and T2 co onal high- esolu ion images.
The accu acy o he segmen a ions p edic ed by he algo i hm ma ches he manual segmen a ions
made by wo well-expe ienced a e s. This me hod holds po en ial o immedia e implemen a ion in
clinical p ac ice. Fu he mo e, i s abili y o pe o m quick and accu a e p ocessing o la ge da ase s
may p o ide a aluable con ibu ion o ad ancing ou knowledge o he ol ac o y sys em, in heal h
and disease. Speci ically, ou amewo k may in eg a e he use o ol ac o y bulb olume (OBV)
B ain Sci. 2021,11, 1141. h ps://doi.o g/10.3390/b ainsci11091141 h ps://www.mdpi.com/jou nal/b ainsci
B ain Sci. 2021,11, 1141 2 o 12
measu emen s o he diagnosis and ea men o ol ac o y loss and imp o e he p ognosis and
ea men op ions o ol ac o y dys unc ions.
Keywo ds: ol ac o y bulb; ol ac o y loss; deep lea ning; segmen a ion
1. In oduc ion
As one o he i e basic human senses, he sense o smell plays an impo an ole
in ou daily li e. I is essen ial o he de ec ion o dange s, such as gas, i e, smoke, o
haza dous chemicals, and he quali y o ou e e yday social li e. Mo eo e , smelling odo s
makes up a majo pa o ou pleasan expe iences, whe he ea ing a a o i e meal, walking
ou side smelling blooming lowe s and ees, o du ing in imacy wi h one’s pa ne . Las
bu no leas , he sense o smell plays a undamen al ole in some p o essions such as che
(app oxima ely 0.5% o he Ge man wo k o ce), bake , o pe ume [1].
Clinical e iews ha e shown ha 3–20% o he gene al popula ion a e a ec ed by
anosmia (comple e loss o sense o smell) o hyposmia ( educed sense o smell) [
2
,
3
].
Ol ac o y de ici s a e associa ed wi h nume ous neu odegene a i e diso de s such as
Pa kinson’s disease o Alzheime ’s disease and appea as p od omal symp oms. Mo eo e ,
pa ien s wi h comple e o pa ial ol ac o y loss ha e a highe isk o exhibi ing symp oms
o dep ession. In e sely, dep essi e symp oms a e co ela ed wi h lowe ac i a ion in
s uc u es in ol ed in he ol ac ion pe cep ion pa hway o wi h a dec eased ol ac o y bulb
(OB) olume [
4
]. The e o e, he e is a need o unde s and he unc ioning o he human
ol ac o y sys em.
Ol ac o y pe cep ion begins as he ola ile odo molecules inhaled om he ai bind
ol ac o y ecep o p o eins in he cilia o ol ac o y senso y neu ons housed in he neu oep-
i helium o he nasal ca i y. This neu oepi helium con ains 6–10 million neu ons [
5
]. The
axons o hese neu oepi helium cells ascend h ough he c ib i o m pla e as ila ol ac o ia
o he OB loca ed in he ol ac o y ossa o he e hmoid bone. The OB is he i s s age o he
ol ac o y signal p ocessing sys em and, he e o e, an essen ial pa o he ol ac o y pa hway.
Signals om ac i a ed neu oepi helium cells a e ansmi ed o he OB and hen o p ima y
ol ac o y egions o he b ain such as he pi i o m co ex, en o hinal co ex, and amygdala.
The ou pu s om he p ima y ol ac o y a eas a e hen sen o o he b ain s uc u es such as
he o bi o on al co ex (OFC), insula, and hippocampus [6].
I has been shown ha OB olume co ela es wi h ol ac o y sensi i i y and i is
dec eased in pa ien s wi h ol ac o y diso de s [
7
]. Mo eo e , p e ious s udies ha e shown
ha he OB olume co ela es wi h he olume and g ey ma e densi y o he p ima y
ol ac o y egion [
8
]. F om he ange o known causes o p ima y ol ac o y loss, pa ien s wi h
pos - auma ic and pos -in ec ious anosmia o hyposmia consis en ly p esen a educed OB
olume when compa ed o heal hy indi iduals [
9
]. Abou wo hi ds o he pa ien s wi h
congeni al anosmia do no ha e a de ec able OB on magne ic esonance imaging (MRI)
scans, and one hi d p esen hypoplas ic OB [
10
,
11
]. I has been shown ha he dec ease in
OB olume co ela es wi h dec eased ol ac o y sensi i i y. Re e se o his, an inc ease in
OB olume, e.g., a e endoscopic nasal su ge y, leads o inc eased ol ac o y sensi i i y [
12
].
All hese indings sugges ha he olume o he OB is an impo an ma ke o ol ac o y
unc ion and a p edic ing ac o o he ea men o ol ac o y diso de s [13].
The OB is a e y small s uc u e wi hin he human b ain wi h a olume anging
be ween 35–100 mm
3
in no mosmic indi iduals [
13
,
14
]. The e o e, o assess he olume o
such a small s uc u e, specialized high- esolu ion MR sequences a e needed. Howe e ,
he loca ion o he OB wi hin he ol ac o y ossa o he e hmoid bone makes i ulne able
o suscep ibili y a i ac s. Suscep ibili y a i ac s a e dis o ions due o local magne ic ield
inhomogenei ies and o en a ise a in e aces o issues wi h di e en p o on densi ies.
Especially in as sequences wi h lowe spa ial esolu ion, i is ex emely challenging o
achie e a sa is ac o y le el o image quali y necessa y o pe o ming image segmen a ion
B ain Sci. 2021,11, 1141 3 o 12
o small ana omical s uc u es and es ima ing i s olume. In clinical p ac ice and esea ch,
he T2 co onal sequence is mos commonly used because o i s con as be ween he
OB’s issue and he ce eb ospinal luid (CSF) in he su ounding a ea and because i is
less ulne able o suscep ibili y a i ac s. Cu en ly, he gold s anda d s a egy o he
measu emen o OB olumes in ol es he manual segmen a ion o he isible OB in he
co onal plane iew slice by slice. Di e en segmen a ion echniques a e known, such as
manually acing he ou lines o he bulbs o highligh ing he en i e isible a ea slice by
slice [
15
]. S ill, manual segmen a ion is an ex emely ime-consuming p ocess exhibi ing
a ia ions in he deg ee o in e -obse e and in a-obse e eliabili y. I akes a well-
expe ienced a e abou 10–15 min o comple e he OB segmen a ion o a single subjec .
O e he las ew yea s, machine lea ning (ML) algo i hms ha e p o ided e icien
solu ions o au oma ic image segmen a ion. These algo i hms ha e he po en ial o
e icien ly p ocess mo e da a and inc ease he eliabili y and epea abili y o he esul s.
He e, we applied ML o neu oimaging da a o p oduce one o he i s models ha can
au oma ically and accu a ely segmen he OB, a e y small ana omical egion in he human
b ain [16,17].
In sho , ou algo i hm s a s by pe o ming mul imodal da a p ep ocessing o localize
he OB and compu e he bounding box. Subsequen ly, a 3D U-Ne model segmen s he
OB wi hin he bounding box. The pe o mance o he algo i hm was e alua ed using
es ablished me ics [
18
]. Addi ionally, an expe ienced independen and blinded a e a ed
he quali y o he ag eemen be ween he segmen a ions p oduced by bo h he algo i hm
and he human a e s. To p o e ha he algo i hm pe o ms well independen ly o he ield
o iew (FOV), image o ien a ion, angula ion, o o he acquisi ion pa ame e s, we es ed ou
model on andomly selec ed subjec s om neu oimaging da ase s om p e ious s udies.
2. Me hods
2.1. S udy Popula ion
In his s udy, we used da a om ou di e en da ase s [
19
–
21
]. All da a we e ob ained
a he Uni e si y Hospi al Ca l-Gus a Ca us in D esden, Ge many. Wi hin he con ex o
he espec i e s udies, all pa icipan s signed in o med consen on he use o hei da a o
esea ch pu poses. All o hese s udies had been app o ed by he E hics Commi ee a he
Uni e si y Clinic o he TU D esden.
Da a we e sepa a ed in o he ollowing wo sub-da ase s: anosmia and heal hy con-
ols. The i s da ase was collec ed on 79 pa ien s be ween Augus 2015 and July 2017.
Pa ien s we e diagnosed wi h anosmia o hyposmia. Heal hy con ols’ (n= 91) sub-da ase
con ained da a om h ee di e en MRI s udies.
In bo h da ase s, pa icipan s’ smell abili y was measu ed psychophysically using
he Sni in’ S icks es [
22
]. The sco es o he es ( ange: 1–48) we e used o ca ego ize
pa icipan s’ smelling abili y in he ollowing ca ego ies: unc ional anosmia (TDI
≤
16),
hyposmia (16 > TDI < 30.75), o no mosmia (TDI ≥30.75) [23] (Table 1).
Table 1.
Pa icipan ’s TDI Sco es (sum sco e (TDI) o h eshold (T), disc imina ion (D), and iden i i-
ca ion (I) o he odo s; means (M), s anda d de ia ions (SD).
Smell Dys unc ion Pa ien s (N = 79)
TDI T D I
M 18.00 2.67 7.81 7.52
SD 6.72 2.50 3.07 3.01
No mosmic Con ols (N = 91)
TDI T D I
M 36.39 9.97 12.92 13.50
SD 2.10 2.24 1.65 1.27
B ain Sci. 2021,11, 1141 4 o 12
2.2. MRI Image Acquisi ion
All MRI scans we e acqui ed on a 3T Siemens Ve io scanne (Siemens, E langen,
Ge many). All acquisi ions we e made using a 32-channel head coil. The ollowing wo
modali ies we e used in his s udy: T1-weigh ed axial whole-b ain scans and T2-weigh ed
high- esolu ion co onal scans o OB imaging.
T1-weigh ed MPRAGE sequence was acqui ed using he ollowing pa ame e s: epe-
i ion ime: 2300 ms; echo ime 2.98: ms; lip angle: 9
◦
; ield o iew: 240
×
256; acquisi ion
ma ix: 250
×
256 oxel size: 1
×
1
×
1 mm
3
; slice hickness: 1 mm; slices: 176. T2-weigh ed
high- esolu ion sequence was acqui ed using ollowing sequence pa ame e s: epe i ion
ime: 5500 ms; echo ime: 110 ms; lip angle: 150
◦
, ield o iew: 120
×
120; acquisi ion
ma ix: 256 ×256; oxel size: 0.47 ×0.47 ×1.2 mm3; slice hickness: 1.2 mm (no gap).
2.3. Manual Segmen a ions o OB Volume
Fo bo h da ase s, manual segmen a ion o he OBs was pe o med on T2-weigh ed
images in co onal plane iew using ITK-SNAP So wa e . 3.6 [
24
]. Fi s , bo h T1-weigh ed
and T2-weigh ed scans we e con e ed om DICOM o ma o g-zipped NIFTI o ma
(nii.gz) using he dcm2niix con e sion ool [
25
]. All measu emen s we e pe o med by wo
a e s independen ly. Bo h a e s used he p o ocol o pe o ming manual segmen a ions.
Voxels belonging o he le and igh OB we e labeled wi h alues 1 and 2, espec i ely. The
manual measu emen s we e sa ed as bina y segmen a ion masks in g-zipped NIFTI o ma
(nii.gz). To access he in o ma ion o each label, espec i ely, he bina y segmen a ion
masks we e con e ed o a NumPy a ay wi h nibabel py hon lib a y [
26
,
27
]. The olumes
o he labels o le and igh OBs we e calcula ed by mul iplying he numbe o oxels
o he label by oxel dimensions using he NumPy py hon lib a y [
28
]. Fo all manual
segmen a ion masks, he DC was calcula ed using he MedPy py hon lib a y [
29
] o es ima e
he le el o o e lap be ween he wo a e s.
2.4. Au oma ed Localiza ion o he OBs
OBs ha e e y small olumes, especially compa ed o he en i e scanned b ain olume.
This leads o highly imbalanced da a due o class imbalance be ween oxels labeled as
o eg ound ( oxels wi h alues one/ wo o le / igh OB) and oxels labeled as back-
g ound (ze o alue oxels o any o he issue). To sol e his issue, we op ed o using a
empla e-based app oach. To he bes o ou knowledge, no ea lie s udies ha e no malized
indi idual OBs o a s anda d s e eo ac ic Mon eal Neu ological Ins i u e (MNI) [
30
,
31
]
space. He e, we de eloped a pipeline ha allows he au oma ic ans o ma ion o all man-
ual segmen a ions om T2 na i e space o MNI space. Fi s , he T1 whole-b ain image was
co- egis e ed o ICBM 2009c Nonlinea Asymme ic T1 MNI empla e image using ANTs
SyN nonlinea egis a ion ool [
32
]. Subsequen ly, he T2 image was co- egis e ed o he
T1 image by ANTs a ine egis a ion unc ion. Fo bo h s eps, he ans o ma ion ma ices
we e sa ed and applied o he manual segmen a ions in na i e T2 space. Applying he
in e ed egis a ion ma ices allowed a wo-s ep ans o ma ion o manual segmen a ion
masks om T2 na i e space o MNI space. This p ocedu e was ex ended o he manual
segmen a ions p oduced by he wo a e s. The esul ing bina y masks in MNI space
we e ans o med o NumPy a ays using nibabel. The NumPy a ays we e added o one
NumPy a ay and di ided by he highes alue in he a ay.
This p ocess esul ed in a p obabili y map o he OBs in MNI space. To calcula e
he coo dina es o he cen e o g a i y (COG) o he OB, we applied a h eshold o 0.5
o he esul ing OB p obabili y map and calcula ed he coo dina es in MNI space (xyz:
−
4/44/
−
36) using he SciPy py hon lib a y [
33
], and sa ed he esul as a bina y image
(Figu e 1).
B ain Sci. 2021,11, 1141 5 o 12
B ain Sci. 2021, 11, x FOR PEER REVIEW 5 o 12
Figu e 1. OB p obabili y map: backg ound image MNI ICBM 2009c Nonlinea Asymme ic em-
pla e, h eshold 0.5. The coo dina es o he COG a e xyz: −4/44/−36. (x: sagi al, y: on al, z: axial
o ien a ions). The colo map indica es he p obabili y om 0 o 1.
2.5. P ep ocessing Pipeline
The p ep ocessing pipeline o he segmen a ion ool pe o ms all necessa y s eps o
c ea e no malized da a o 3D U-Ne inpu . As desc ibed abo e, he i s s ep consis s in
co- egis e ing he T1 image o MNI empla e image and he T2 image o T1 image o ob ain
he esul ing ans o ma ion ma ices. These will subsequen ly be in e ed, and he esul -
ing in e ed ans o ma ion ma ices applied o he COG bina y map esul ing in he COG
in T2 na i e space. To educe he numbe o ea u es ( oxels) and he da a imbalance o
he inpu images o p ocessing in 3D U-Ne , we cons uc ed a bounding box ex ac ion
algo i hm based on he COG in T2 na i e space. To ensu e ha all images ha e he same
o ien a ion in h ee- idimensional space, all T2 images and he COG bina y mask in T2
na i e space we e eo ien ed using nibabel py hon lib a y o canonical o ien a ion. The
bounding box edges we e de ined as poin s in 3D space shi ed om he COG by +/−10
mm in he x-di ec ion, +/−15 mm in he y-di ec ion, and +/−5 mm in he z-di ec ion. A e -
wa ds, he a ay inside he calcula ed bounding box was ex ac ed om he T2 image and
he co esponding manual segmen a ion. The esul ing images we e esampled o a com-
mon oxel dimension o (0.5, 1, 0.5) and image shape o (4, 32, 32) using cubic in e pola ion
unc ion om slpy py hon lib a y (Figu e 2) [29].
Figu e 2. Au oma ic segmen a ion o he OBs. 1. T1 whole-b ain image o MNI2009casym MNI empla e co- egis a ion.
2. T2 high- esolu ion image o T1 whole-b ain image co- egis a ion. 3. Appling o in e sed ans o ma ion ma ix om
Figu e 1.
OB p obabili y map: backg ound image MNI ICBM 2009c Nonlinea Asymme ic empla e,
h eshold 0.5. The coo dina es o he COG a e xyz:
−
4/44/
−
36. (x: sagi al, y: on al, z: axial
o ien a ions). The colo map indica es he p obabili y om 0 o 1.
2.5. P ep ocessing Pipeline
The p ep ocessing pipeline o he segmen a ion ool pe o ms all necessa y s eps o
c ea e no malized da a o 3D U-Ne inpu . As desc ibed abo e, he i s s ep consis s in co-
egis e ing he T1 image o MNI empla e image and he T2 image o T1 image o ob ain he
esul ing ans o ma ion ma ices. These will subsequen ly be in e ed, and he esul ing
in e ed ans o ma ion ma ices applied o he COG bina y map esul ing in he COG in T2
na i e space. To educe he numbe o ea u es ( oxels) and he da a imbalance o he inpu
images o p ocessing in 3D U-Ne , we cons uc ed a bounding box ex ac ion algo i hm
based on he COG in T2 na i e space. To ensu e ha all images ha e he same o ien a ion
in h ee- idimensional space, all T2 images and he COG bina y mask in T2 na i e space
we e eo ien ed using nibabel py hon lib a y o canonical o ien a ion. The bounding box
edges we e de ined as poin s in 3D space shi ed om he COG by
+/−10 mm
in he
x-di ec ion, +/
−
15 mm in he y-di ec ion, and +/
−
5 mm in he z-di ec ion. A e wa ds,
he a ay inside he calcula ed bounding box was ex ac ed om he T2 image and he
co esponding manual segmen a ion. The esul ing images we e esampled o a common
oxel dimension o (0.5, 1, 0.5) and image shape o (4, 32, 32) using cubic in e pola ion
unc ion om slpy py hon lib a y (Figu e 2) [34].
B ain Sci. 2021, 11, x FOR PEER REVIEW 5 o 12
Figu e 1. OB p obabili y map: backg ound image MNI ICBM 2009c Nonlinea Asymme ic em-
pla e, h eshold 0.5. The coo dina es o he COG a e xyz: −4/44/−36. (x: sagi al, y: on al, z: axial
o ien a ions). The colo map indica es he p obabili y om 0 o 1.
2.5. P ep ocessing Pipeline
The p ep ocessing pipeline o he segmen a ion ool pe o ms all necessa y s eps o
c ea e no malized da a o 3D U-Ne inpu . As desc ibed abo e, he i s s ep consis s in
co- egis e ing he T1 image o MNI empla e image and he T2 image o T1 image o ob ain
he esul ing ans o ma ion ma ices. These will subsequen ly be in e ed, and he esul -
ing in e ed ans o ma ion ma ices applied o he COG bina y map esul ing in he COG
in T2 na i e space. To educe he numbe o ea u es ( oxels) and he da a imbalance o
he inpu images o p ocessing in 3D U-Ne , we cons uc ed a bounding box ex ac ion
algo i hm based on he COG in T2 na i e space. To ensu e ha all images ha e he same
o ien a ion in h ee- idimensional space, all T2 images and he COG bina y mask in T2
na i e space we e eo ien ed using nibabel py hon lib a y o canonical o ien a ion. The
bounding box edges we e de ined as poin s in 3D space shi ed om he COG by +/−10
mm in he x-di ec ion, +/−15 mm in he y-di ec ion, and +/−5 mm in he z-di ec ion. A e -
wa ds, he a ay inside he calcula ed bounding box was ex ac ed om he T2 image and
he co esponding manual segmen a ion. The esul ing images we e esampled o a com-
mon oxel dimension o (0.5, 1, 0.5) and image shape o (4, 32, 32) using cubic in e pola ion
unc ion om slpy py hon lib a y (Figu e 2) [29].
Figu e 2. Au oma ic segmen a ion o he OBs. 1. T1 whole-b ain image o MNI2009casym MNI empla e co- egis a ion.
2. T2 high- esolu ion image o T1 whole-b ain image co- egis a ion. 3. Appling o in e sed ans o ma ion ma ix om
Figu e 2.
Au oma ic segmen a ion o he OBs. 1. T1 whole-b ain image o MNI2009casym MNI empla e
co- egis a ion. 2. T2 high- esolu ion image o T1 whole-b ain image co- egis a ion. 3. Appling o
in e sed ans o ma ion ma ix om s ep 1 o OB COG mask in MNI space. 4. Appling o in e sed
ans o ma ion ma ix om s ep 2 o OB COG mask in T1 space. 5. C ea ing a 3D bounding box based on
COG coo dina es in T2 space om s ep 4. 6. Cu ou he bounding box om he T2 image. 7. Pe o ming
image segmen a ion using ained 3D U-ne .

B ain Sci. 2021,11, 1141 6 o 12
2.6. T aining o he 3D U-Ne Model
The en i e model aining p ocess was ca ied ou using he Monai Py hon Lib a y [
35
].
Model aining was pe o med on he da ase esul ing om p ep ocessing pipeline de-
sc ibed abo e. The da ase included he p ep ocessed da a om 159 T2 images and
318 manual segmen a ions (159 manually c ea ed bina y masks by wo a e s). Se en sub-
jec s om he anosmia da ase we e excluded, due o no ha ing a isible OB. The e o e, he
manual segmen a ions made by bo h a e s we e emp y, con aining only he backg ound.
Fi s , he da ase was andomly spli in o a aining da ase (n= 191 subjec s; 60% o
he da ase ), a alida ion da ase (n= 64 subjec s; 20% o he da ase ,), and a es da ase
(n= 64 subjec s; 20% o he da ase ). Fi s , all da a unde wen in ensi y no maliza ion,
a s anda d s ep o he monai p ep ocessing pipeline. To pe o m da a augmen a ion, a
andom a ine ans o ma ion was applied o he no malized da a om he aining da ase
only. These ans o ma ions we e au oma ically applied by RandA ined unc ion om he
monai py hon package du ing each epoch o he model aining p ocess. Speci ically, he
ans o ma ion ea u es included a ansla ion ange o
−
10 o +10 oxels in each di ec ion
and o a ion o
−
30
◦
o +30
◦
deg ees using bilinea in e pola ion unc ion o T2 images
and nea es -neighbo in e pola ion o bina y label masks.
The U-Ne model was impo ed om monai py hon lib a y and speci ied using
ollowing pa ame e s: dimensions: 3; inpu channels: 1; ou pu channels: 2; channels: 16,
32, 64, 128, 256; s ides: 2, 2, 2, 2; num o esidual uni s = 2.
The model was ained o e 300 epochs. Each epoch con ained 86 i e a ions
o mini ba ches con aining wo co esponding image and bina y mask pai s o size
64 ×32 ×32 oxels
. We op ed o he T e sky unc ion o measu e loss a e sion. Ne wo k
weigh s we e op imized using he Adam op imiza ion unc ion (I = 0.001). One inpu
channel and wo ou pu channels we e de ined o he inpu –ou pu da a s eam o he
model, con aining h ee dimensions (Figu e 3).
B ain Sci. 2021, 11, x FOR PEER REVIEW 6 o 12
s ep 1 o OB COG mask in MNI space. 4. Appling o in e sed ans o ma ion ma ix om s ep 2 o OB COG mask in T1
space. 5. C ea ing a 3D bounding box based on COG coo dina es in T2 space om s ep 4. 6. Cu ou he bounding box
om he T2 image. 7. Pe o ming image segmen a ion using ained 3D U-ne .
2.6. T aining o he 3D U-Ne Model
The en i e model aining p ocess was ca ied ou using he Monai Py hon Lib a y
[30]. Model aining was pe o med on he da ase esul ing om p ep ocessing pipeline
desc ibed abo e. The da ase included he p ep ocessed da a om 159 T2 images and 318
manual segmen a ions (159 manually c ea ed bina y masks by wo a e s). Se en subjec s
om he anosmia da ase we e excluded, due o no ha ing a isible OB. The e o e, he
manual segmen a ions made by bo h a e s we e emp y, con aining only he backg ound.
Fi s , he da ase was andomly spli in o a aining da ase (n = 191 subjec s; 60% o he
da ase ), a alida ion da ase (n = 64 subjec s; 20% o he da ase ,), and a es da ase (n =
64 subjec s; 20% o he da ase ). Fi s , all da a unde wen in ensi y no maliza ion, a s and-
a d s ep o he monai p ep ocessing pipeline. To pe o m da a augmen a ion, a andom
a ine ans o ma ion was applied o he no malized da a om he aining da ase only.
These ans o ma ions we e au oma ically applied by RandA ined unc ion om he
monai py hon package du ing each epoch o he model aining p ocess. Speci ically, he
ans o ma ion ea u es included a ansla ion ange o −10 o +10 oxels in each di ec ion
and o a ion o −30° o +30° deg ees using bilinea in e pola ion unc ion o T2 images
and nea es -neighbo in e pola ion o bina y label masks.
The U-Ne model was impo ed om monai py hon lib a y and speci ied using ol-
lowing pa ame e s: dimensions: 3; inpu channels: 1; ou pu channels: 2; channels: 16, 32,
64, 128, 256; s ides: 2, 2, 2, 2; num o esidual uni s = 2.
The model was ained o e 300 epochs. Each epoch con ained 86 i e a ions o mini
ba ches con aining wo co esponding image and bina y mask pai s o size 64 × 32 × 32
oxels. We op ed o he T e sky unc ion o measu e loss a e sion. Ne wo k weigh s
we e op imized using he Adam op imiza ion unc ion (I = 0.001). One inpu channel and
wo ou pu channels we e de ined o he inpu –ou pu da a s eam o he model, con-
aining h ee dimensions (Figu e 3).
Figu e 3. Model aining p ocess. Le : T aining loss unc ion. Righ : DC plo o aining (g een cu e) and alida ion
( ed cu e) da ase s. The DC and a e age loss alues a e bo h a bi a y uni s. The cu es each a pla eau a epoch 100. The
model a he epoch 297 was selec ed as he highes DC (0.84) o he aining p ocess. The uni (au) means a bi a y uni .
Figu e 3.
Model aining p ocess.
Le
: T aining loss unc ion.
Righ
: DC plo o aining (g een
cu e) and alida ion ( ed cu e) da ase s. The DC and a e age loss alues a e bo h a bi a y uni s.
The cu es each a pla eau a epoch 100. The model a he epoch 297 was selec ed as he highes DC
(0.84) o he aining p ocess. The uni (au) means a bi a y uni .
2.7. Pos p ocessing Pipeline
We designed and implemen ed a no el pos p ocessing pipeline o pe o m da a
cleanup o he p ima y ou pu o he ained 3D U-Ne model and ans o m he ou pu
o he o iginal T2 image’s size, esolu ion, and o ien a ion. As a i s s ep, all clus e s
a e elabeled o unique alues. Secondly, clus e s a e h esholded based on hei size o
elimina e spu ious clus e s. Finally, he esul ing mask is esampled o he esolu ion, oxel
B ain Sci. 2021,11, 1141 7 o 12
size, and o ien a ion o he co esponding T2 image. This pos -p ocessed bina y mask is
hen sa ed in o an emp y NumPy a ay o he same size as he T2 image— he inal bina y
segmen a ion mask.
2.8. E alua ion o T aining and Tes ing
The pe o mance o he au oma ic segmen a ion was e alua ed on he es ing da ase .
This da ase was kep sepa a e be o e he model aining s ep and, he e o e, con ained
only da a “unseen” by he model, consis ing o 64 subjec s and he co esponding manual
segmen a ions o wo a e s. Pe o mance was e alua ed by compu ing an ex ensi e se
o s a is ical me ics— he dice coe icien (DC) and a e age symme ic su ace dis ance
(ASSD)— o e alua e he simila i y be ween he model’s p edic ed segmen a ions and he
manual segmen a ions, o bo h le and igh OB.
DC is an o e lay simila i y index ha e lec s size and localiza ion ag eemen and
anges om 0 (no o e lap) o 1 (comple e o e lap). ASSD ep esen s he mean dis ance o
he bina y objec s in wo images. The e o e, an ASSD o 0 mm ep esen s a pe ec ma ch
be ween bo h segmen a ions.
Fo subjec s wi h diagnosed congeni al anosmia and absence o he OBs, i was no
possible o compu e he DC because emp y segmen a ion masks con ain only he label
co esponding o he image backg ound.
Fu he mo e, o ensu e he quali y o p edic ed segmen a ions, all da a we e andom-
ized and anonymized o ensu e unbiased a ing o he manual and p edic ed segmen a ions.
A well-expe ienced a e , who had no been in ol ed in any s age o he da a segmen a ion
p ocess, was asked o a e he o e lap o bo h manual segmen a ions and p edic ed masks
using a pe o mance scale. The a ing c i e ia we e de ined as a scale: 1: “No cong u-
ency”, 2–3: “Poo cong uency”, 4–5: “good cong uency”, 6–7: “ e y good cong uency”,
8–10: “excellen cong uency”.
3. Resul s
Fi s ly, we compa ed he olumes o he manual segmen a ions made by a e one and
wo and calcula ed he DC o he le and igh OBs indi idually (Table 2). Fo he da ase
con aining he anosmia pa ien s’ da a, he a e age DC was 0.77
±
0.07 and
0.74 ±0.10 mm3
o he le and igh OB, espec i ely. The mean olume o he le OB was 37.82
±
11.48
and 47.82
±
14.78 mm
3
measu ed by a e s one and wo, espec i ely. Fo he igh OB, he
mean olume o he segmen a ions p oduced by a e s one and wo was 34.32
±
10.93 and
46.47 ±15.43 mm3, espec i ely.
Table 2.
The able shows con e gence o manual segmen a ions pe o med by wo a e s, based on
DC and le and igh ol ac o y bulb olume (LOBV and ROBV) o each a e , espec i ely. The uni
(au) means a bi a y uni .
Smell Dys unc ion Pa ien s
Le OB
DC (au)
Righ OB
DC (au)
LOBV
Ra e 1
(mm3)
LOBV
Ra e 2
(mm3)
ROBV
Ra e 1
(mm3)
ROBV
Ra e 2
(mm3)
M 0.77 0.74 37.82 47.64 34.32 46.47
SD 0.07 0.10 11.48 14.78 10.93 15.43
Heal hy Con ols
Le OB
DC (au)
Righ OB
DC (au)
LOBV
Ra e 1
(mm3)
LOBV
Ra e 2
(mm3)
ROBV
Ra e 1
(mm3)
ROBV
Ra e 2
(mm3)
M 0.81 0.80 44.14 56.01 42.47 54.15
SD 0.06 0.05 12.38 16.92 13.54 17.66
B ain Sci. 2021,11, 1141 8 o 12
Fo he da ase con aining he heal hy con ol subjec s’ da a, he a e age DC o he
le OB was 0.81
±
0.06 mm
3
and 0.80
±
0.05 mm
3
o he igh OB. The mean olume o
he le OB was 44.14
±
12.38 mm
3
measu ed by a e one and 56.01
±
16.92 mm
3
measu ed
by a e wo. The mean olume o he igh OB was 42.47
±
13.54 mm
3
measu ed by a e
one and 54.15 ±17.66 mm3measu ed by a e wo (Figu e 4).
B ain Sci. 2021, 11, x FOR PEER REVIEW 9 o 12
SD 0.87 0.81 0.87
Minimum 4.00 4.00 4.00
Maximum 8.00 7.00 7.00
Addi ionally, we es ed he algo i hm on subjec s om he ol ac o y dys unc ion da-
ase wi h diagnosed congeni al anosmia (N = 7). As a o emen ioned, gi en ha hese sub-
jec s do no ha e a isible OB, he esul ing manual segmen a ion masks we e emp y, con-
aining only he backg ound label. Ou model was able o co ec ly iden i y all hese ex-
ao dina y da ase s as non-exis en OB cases, wi h he gene a ed masks only con aining
he backg ound label, as expec ed
Figu e 4. Top: Boxplo showing he OBVs o manual segmen a ions measu ed by wo independen a e s and he 3D U-
Ne model on es da ase . Bo om le and igh : XY-Plo o he le and igh OBV measu ed by wo independen a e s
and 3D U-Ne on es da ase . The end o he g aphs shows a high le el o cong uency o he measu emen s.
Figu e 5. Au oma ic segmen a ion o he le ( ed) and igh (g een) OB in co onal T2 MRI scan pe o med manually by
wo independen a e s and he algo i hm.
In addi ion o he ex ended quali y e alua ion, we de eloped wo in e aces o ou
algo i hm. The ool is ully a ailable ia he pip py hon package managemen sys em, and
he command-line in e ace is accessible in he e minal on UNIX-based ope a ing sys ems
(OS). Fo use in a Mic oso na i e OS, a Linux sub-sys em needs o be u he ins alled.
Figu e 4. Top
: Boxplo showing he OBVs o manual segmen a ions measu ed by wo independen
a e s and he 3D U-Ne model on es da ase .
Bo om le
and
igh
: XY-Plo o he le and igh
OBV measu ed by wo independen a e s and 3D U-Ne on es da ase . The end o he g aphs
shows a high le el o cong uency o he measu emen s.
Fo he e alua ion o he o e lap be ween he p edic ed segmen a ions o he algo i hm
and he manual segmen a ions o he OB o he subjec s in he es da ase (N = 64), a ious
me ics we e calcula ed o he le and igh OBs, sepa a ely, as well as o he en i e
segmen a ion mask. As i can be ound in Table 3, he mean DC was 0.77
±
0.05 (le OB:
0.78
±
0.06, igh OB: 0.75
±
0.08) and he mean symme ic su ace dis ance (ASSD) was
0.43 ±0.10 (le OB: 0.41 ±0.10, igh OB: 0.44 ±0.14) (Table 3).
Table 3.
Model pe o mance me ics o he es da ase . DC (DC), A e age Symme ic Su ace
Dis ance (ASSD). The uni (au) means a bi a y uni .
DC (au) ASSD (au)
Le OB Righ OB Mean Le OB Righ OB Mean
M 0.78 0.75 0.77 0.41 0.44 0.43
SD 0.06 0.08 0.05 0.10 0.14 0.10
Mo eo e , we compa ed he olumes o he manual segmen a ions and he p edic ed
segmen a ions gene a ed by ou algo i hm, o he le and igh OBs indi idually, o
he es da ase . Fo he le hemisphe e, he mean OB olume o he p edic ed mask
was
44.80 ±8.59 mm3
and 46.14
±
12.70 mm
3
o he manually segmen ed mask. Fo he
igh OB, he p edic ed OB olume was 46.73
±
8.86 mm
3
o he p edic ed mask and
42.63 ±14.19 mm3 o he manual masks (Table 4).
B ain Sci. 2021,11, 1141 9 o 12
Table 4.
OB olumes o manual segmen a ions and p edic ed segmen a ions o he algo i hm o he
es da ase .
Manual Segmen a ion 3D U-Ne Segmen a ions
Le OB (mm3) Righ OB (mm3) Le OB (mm3) Righ OB (mm3)
M 46.14 42.63 44.80 46.73
SD 12.70 14.19 8.59 8.86
Subsequen ly, an independen , unbiased, and well-expe ienced a e a ed he manual
segmen a ions p oduced by bo h a e s, as well as he p edic ed segmen a ion o he
algo i hm (Figu e 5) (Table 5). The mean sco e was 6.23
±
0.87 and 5.92
±
0.81 o a e one
and wo, espec i ely, and 5.95 ±0.87 o he segmen a ions gene a ed by ou model.
B ain Sci. 2021, 11, x FOR PEER REVIEW 9 o 12
SD 0.87 0.81 0.87
Minimum 4.00 4.00 4.00
Maximum 8.00 7.00 7.00
Addi ionally, we es ed he algo i hm on subjec s om he ol ac o y dys unc ion da-
ase wi h diagnosed congeni al anosmia (N = 7). As a o emen ioned, gi en ha hese sub-
jec s do no ha e a isible OB, he esul ing manual segmen a ion masks we e emp y, con-
aining only he backg ound label. Ou model was able o co ec ly iden i y all hese ex-
ao dina y da ase s as non-exis en OB cases, wi h he gene a ed masks only con aining
he backg ound label, as expec ed
Figu e 4. Top: Boxplo showing he OBVs o manual segmen a ions measu ed by wo independen a e s and he 3D U-
Ne model on es da ase . Bo om le and igh : XY-Plo o he le and igh OBV measu ed by wo independen a e s
and 3D U-Ne on es da ase . The end o he g aphs shows a high le el o cong uency o he measu emen s.
Figu e 5. Au oma ic segmen a ion o he le ( ed) and igh (g een) OB in co onal T2 MRI scan pe o med manually by
wo independen a e s and he algo i hm.
In addi ion o he ex ended quali y e alua ion, we de eloped wo in e aces o ou
algo i hm. The ool is ully a ailable ia he pip py hon package managemen sys em, and
he command-line in e ace is accessible in he e minal on UNIX-based ope a ing sys ems
(OS). Fo use in a Mic oso na i e OS, a Linux sub-sys em needs o be u he ins alled.
Figu e 5.
Au oma ic segmen a ion o he le ( ed) and igh (g een) OB in co onal T2 MRI scan
pe o med manually by wo independen a e s and he algo i hm.
Table 5.
Human alida ion scale. This able shows he manual a ing c i e ia used by blinded a e
o human alida ion o he segmen a ions. The a ing scale was de ined as: 1: “No cong uency”,
2–3: “Poo cong uency”, 4–5: “good cong uency”, 6–7: “ e y good cong uency”, 8–10: “excellen
cong uency.” The uni (au) means a bi a y uni .
Human Ra ing Resul s
Ra e 1 (au) Ra e 2 (au) U-Ne (au)
M 6.23 5.92 5.95
SD 0.87 0.81 0.87
Minimum 4.00 4.00 4.00
Maximum 8.00 7.00 7.00
Addi ionally, we es ed he algo i hm on subjec s om he ol ac o y dys unc ion
da ase wi h diagnosed congeni al anosmia (N = 7). As a o emen ioned, gi en ha hese
subjec s do no ha e a isible OB, he esul ing manual segmen a ion masks we e emp y,
con aining only he backg ound label. Ou model was able o co ec ly iden i y all hese
ex ao dina y da ase s as non-exis en OB cases, wi h he gene a ed masks only con aining
he backg ound label, as expec ed.
In addi ion o he ex ended quali y e alua ion, we de eloped wo in e aces o ou
algo i hm. The ool is ully a ailable ia he pip py hon package managemen sys em, and
he command-line in e ace is accessible in he e minal on UNIX-based ope a ing sys ems
(OS). Fo use in a Mic oso na i e OS, a Linux sub-sys em needs o be u he ins alled. A
isual in e ace is also a ailable, allowing he use o selec he pa hs o olde s con aining
da a com o ably a he dis ance o only a ew mouse clicks.
4. Discussion
In his s udy, we ha e de eloped a eady- o-use solu ion o he au oma ic segmen a-
ion o human OBs using 3D U-Ne . The algo i hm localizes he COG o OBs, pe o ms he