Automatic segmentation of the olfactory bulb

b ain sciences A icle Au oma ic Segmen a ion o he Ol ac o y Bulb Dmi iy Desse 1,*, F ancisca Assunção2, Xiaoguang Yan 1, Vic o Al es 2, Hen ique M. Fe nandes 3,4,5,† and Thomas Hummel 1,†   Ci a ion: Desse , D.; Assunção, F.; Yan, X.; Al es, V.; Fe nandes, H.M.; Hummel, T. Au oma ic Segmen a ion o he Ol ac o y Bulb. B ain Sci. 2021, 11, 1141. h ps://doi.o g/10.3390/ b ainsci11091141 Academic Edi o : Tjee d Olde Schepe Recei ed: 6 July 2021 Accep ed: 25 Augus 2021 Published: 28 Augus 2021 Publishe ’s No e: MDPI s ays neu al wi h ega d o ju isdic ional claims in published maps and ins i u ional a il- ia ions. Copy igh : © 2021 by he au ho s. Licensee MDPI, Basel, Swi ze land. This a icle is an open access a icle dis ibu ed unde he e ms and condi ions o he C ea i e Commons A ibu ion (CC BY) license (h ps:// c ea i ecommons.o g/licenses/by/ 4.0/). 1 Smell & Tas e Clinic, Depa men o O o hinola yngology, Technische Uni e si ä , 01307 D esden, Ge many; [email p o ec ed] (X.Y.); [email p o ec ed] (T.H.) 2Depa men o In o ma ics, School o Enginee ing, Uni e si y o Minho, 4704-553 B aga, Po ugal; [email p o ec ed] (F.A.); [email p o ec ed] (V.A.) 3Cen e o Music in he B ain, Depa men o Clinical Medicine, Aa hus Uni e si y, Noe eb ogade 44, 1A, 8000 Aa hus, Denma k; [email p o ec ed] 4Cen e o Func ionally In eg a i e Neu oscience, Aa hus Uni e si y, Noe eb ogade 44, 1A, 8000 Aa hus, Denma k 5Fla ou Ins i u e, Depa men o Clinical Medicine, Aa hus Uni e si y, Noe eb ogade 44, 1A, 8000 Aa hus, Denma k *Co espondence: dmi iy[email p o ec ed] † Sha ed las au ho ship. Abs ac : The ol ac o y bulb (OB) has an essen ial ole in he human ol ac o y pa hway. A change in ol ac o y unc ion is associa ed wi h a change o OB olume. I has been shown o p edic he p ognosis o ol ac o y loss and i s olume is a bioma ke o a ious neu odegene a i e diseases, such as Alzheime ’s disease. Thus a , ob aining an OB olume o esea ch pu poses has been pe o med by manual segmen a ion alone; a e y ime-consuming and highly a e -biased p ocess. As such, his p ocess d ama ically educes he abili y o p oduce ai and eliable compa isons be ween s udies, as well as he p ocessing o la ge da ase s. Ou s udy aims o sol e his by p oposing a no el me hod- ological amewo k o he unbiased measu emen o OB olume. In his pape , we p esen a ully au oma ed ool ha success ully pe o ms such a ask, accu a ely and quickly. In o de o de elop a s able and e sa ile algo i hm and o ain he neu al ne wo k, we used ou da ase s consis ing o whole-b ain T1 and high- esolu ion T2 MRI scans, as well as he co esponding clinical in o ma ion o he subjec ’s smelling abili y. One da ase con ained da a o pa ien s su e ing om anosmia o hyposmia (N = 79), and he o he h ee da ase s con ained da a o heal hy con ols (N = 91). Fi s , he manual segmen a ion labels o he OBs we e c ea ed by wo expe ienced a e s, independen ly and blinded. The algo i hm consis ed o he ollowing ou di e en s eps: (1) mul imodal da a co- egis a ion o whole-b ain T1 images and T2 images, (2) empla e-based localiza ion o OBs, (3) bounding box cons uc ion, and las ly, (4) segmen a ion o he OB using a 3D-U-Ne . The esul s om he au oma ed segmen a ion algo i hm we e es ed on p e iously unseen da a, achie ing a mean dice coe icien (DC) o 0.77 ± 0.05, which is ema kably con e gen wi h he in e - a e DC o 0.79 ± 0.08 es ima ed o he same coho . Addi ionally, he symme ic su ace dis ance (ASSD) was 0.43 ± 0.10. Fu he mo e, he segmen a ions p oduced using ou algo i hm we e manually a ed by an independen blinded a e and ha e eached an equi alen a ing sco e o 5.95 ± 0.87 compa ed o a a ing sco e o 6.23 ± 0.87 o he i s a e ’s segmen a ion and 5.92 ± 0.81 o he second a e ’s manual segmen a ion. Taken oge he , hese esul s suppo he success o ou ool in p oducing au oma ic as (3–5 min pe subjec ) and eliable segmen a ions o he OB, wi h i ually ma ching accu acy wi h he cu en gold s anda d echnique o OB segmen a ion. In conclusion, we p esen a newly de eloped eady- o-use ool ha can pe o m he segmen a ion o OBs based on mul imodal da a consis ing o T1 whole-b ain images and T2 co onal high- esolu ion images. The accu acy o he segmen a ions p edic ed by he algo i hm ma ches he manual segmen a ions made by wo well-expe ienced a e s. This me hod holds po en ial o immedia e implemen a ion in clinical p ac ice. Fu he mo e, i s abili y o pe o m quick and accu a e p ocessing o la ge da ase s may p o ide a aluable con ibu ion o ad ancing ou knowledge o he ol ac o y sys em, in heal h and disease. Speci ically, ou amewo k may in eg a e he use o ol ac o y bulb olume (OBV) B ain Sci. 2021,11, 1141. h ps://doi.o g/10.3390/b ainsci11091141 h ps://www.mdpi.com/jou nal/b ainsci B ain Sci. 2021,11, 1141 2 o 12 measu emen s o he diagnosis and ea men o ol ac o y loss and imp o e he p ognosis and ea men op ions o ol ac o y dys unc ions. Keywo ds: ol ac o y bulb; ol ac o y loss; deep lea ning; segmen a ion 1. In oduc ion As one o he i e basic human senses, he sense o smell plays an impo an ole in ou daily li e. I is essen ial o he de ec ion o dange s, such as gas, i e, smoke, o haza dous chemicals, and he quali y o ou e e yday social li e. Mo eo e , smelling odo s makes up a majo pa o ou pleasan expe iences, whe he ea ing a a o i e meal, walking ou side smelling blooming lowe s and ees, o du ing in imacy wi h one’s pa ne . Las bu no leas , he sense o smell plays a undamen al ole in some p o essions such as che (app oxima ely 0.5% o he Ge man wo k o ce), bake , o pe ume [1]. Clinical e iews ha e shown ha 3–20% o he gene al popula ion a e a ec ed by anosmia (comple e loss o sense o smell) o hyposmia ( educed sense o smell) [ 2 , 3 ]. Ol ac o y de ici s a e associa ed wi h nume ous neu odegene a i e diso de s such as Pa kinson’s disease o Alzheime ’s disease and appea as p od omal symp oms. Mo eo e , pa ien s wi h comple e o pa ial ol ac o y loss ha e a highe isk o exhibi ing symp oms o dep ession. In e sely, dep essi e symp oms a e co ela ed wi h lowe ac i a ion in s uc u es in ol ed in he ol ac ion pe cep ion pa hway o wi h a dec eased ol ac o y bulb (OB) olume [ 4 ]. The e o e, he e is a need o unde s and he unc ioning o he human ol ac o y sys em. Ol ac o y pe cep ion begins as he ola ile odo molecules inhaled om he ai bind ol ac o y ecep o p o eins in he cilia o ol ac o y senso y neu ons housed in he neu oep- i helium o he nasal ca i y. This neu oepi helium con ains 6–10 million neu ons [ 5 ]. The axons o hese neu oepi helium cells ascend h ough he c ib i o m pla e as ila ol ac o ia o he OB loca ed in he ol ac o y ossa o he e hmoid bone. The OB is he i s s age o he ol ac o y signal p ocessing sys em and, he e o e, an essen ial pa o he ol ac o y pa hway. Signals om ac i a ed neu oepi helium cells a e ansmi ed o he OB and hen o p ima y ol ac o y egions o he b ain such as he pi i o m co ex, en o hinal co ex, and amygdala. The ou pu s om he p ima y ol ac o y a eas a e hen sen o o he b ain s uc u es such as he o bi o on al co ex (OFC), insula, and hippocampus [6]. I has been shown ha OB olume co ela es wi h ol ac o y sensi i i y and i is dec eased in pa ien s wi h ol ac o y diso de s [ 7 ]. Mo eo e , p e ious s udies ha e shown ha he OB olume co ela es wi h he olume and g ey ma e densi y o he p ima y ol ac o y egion [ 8 ]. F om he ange o known causes o p ima y ol ac o y loss, pa ien s wi h pos - auma ic and pos -in ec ious anosmia o hyposmia consis en ly p esen a educed OB olume when compa ed o heal hy indi iduals [ 9 ]. Abou wo hi ds o he pa ien s wi h congeni al anosmia do no ha e a de ec able OB on magne ic esonance imaging (MRI) scans, and one hi d p esen hypoplas ic OB [ 10 , 11 ]. I has been shown ha he dec ease in OB olume co ela es wi h dec eased ol ac o y sensi i i y. Re e se o his, an inc ease in OB olume, e.g., a e endoscopic nasal su ge y, leads o inc eased ol ac o y sensi i i y [ 12 ]. All hese indings sugges ha he olume o he OB is an impo an ma ke o ol ac o y unc ion and a p edic ing ac o o he ea men o ol ac o y diso de s [13]. The OB is a e y small s uc u e wi hin he human b ain wi h a olume anging be ween 35–100 mm 3 in no mosmic indi iduals [ 13 , 14 ]. The e o e, o assess he olume o such a small s uc u e, specialized high- esolu ion MR sequences a e needed. Howe e , he loca ion o he OB wi hin he ol ac o y ossa o he e hmoid bone makes i ulne able o suscep ibili y a i ac s. Suscep ibili y a i ac s a e dis o ions due o local magne ic ield inhomogenei ies and o en a ise a in e aces o issues wi h di e en p o on densi ies. Especially in as sequences wi h lowe spa ial esolu ion, i is ex emely challenging o achie e a sa is ac o y le el o image quali y necessa y o pe o ming image segmen a ion B ain Sci. 2021,11, 1141 3 o 12 o small ana omical s uc u es and es ima ing i s olume. In clinical p ac ice and esea ch, he T2 co onal sequence is mos commonly used because o i s con as be ween he OB’s issue and he ce eb ospinal luid (CSF) in he su ounding a ea and because i is less ulne able o suscep ibili y a i ac s. Cu en ly, he gold s anda d s a egy o he measu emen o OB olumes in ol es he manual segmen a ion o he isible OB in he co onal plane iew slice by slice. Di e en segmen a ion echniques a e known, such as manually acing he ou lines o he bulbs o highligh ing he en i e isible a ea slice by slice [ 15 ]. S ill, manual segmen a ion is an ex emely ime-consuming p ocess exhibi ing a ia ions in he deg ee o in e -obse e and in a-obse e eliabili y. I akes a well- expe ienced a e abou 10–15 min o comple e he OB segmen a ion o a single subjec . O e he las ew yea s, machine lea ning (ML) algo i hms ha e p o ided e icien solu ions o au oma ic image segmen a ion. These algo i hms ha e he po en ial o e icien ly p ocess mo e da a and inc ease he eliabili y and epea abili y o he esul s. He e, we applied ML o neu oimaging da a o p oduce one o he i s models ha can au oma ically and accu a ely segmen he OB, a e y small ana omical egion in he human b ain [16,17]. In sho , ou algo i hm s a s by pe o ming mul imodal da a p ep ocessing o localize he OB and compu e he bounding box. Subsequen ly, a 3D U-Ne model segmen s he OB wi hin he bounding box. The pe o mance o he algo i hm was e alua ed using es ablished me ics [ 18 ]. Addi ionally, an expe ienced independen and blinded a e a ed he quali y o he ag eemen be ween he segmen a ions p oduced by bo h he algo i hm and he human a e s. To p o e ha he algo i hm pe o ms well independen ly o he ield o iew (FOV), image o ien a ion, angula ion, o o he acquisi ion pa ame e s, we es ed ou model on andomly selec ed subjec s om neu oimaging da ase s om p e ious s udies. 2. Me hods 2.1. S udy Popula ion In his s udy, we used da a om ou di e en da ase s [ 19 – 21 ]. All da a we e ob ained a he Uni e si y Hospi al Ca l-Gus a Ca us in D esden, Ge many. Wi hin he con ex o he espec i e s udies, all pa icipan s signed in o med consen on he use o hei da a o esea ch pu poses. All o hese s udies had been app o ed by he E hics Commi ee a he Uni e si y Clinic o he TU D esden. Da a we e sepa a ed in o he ollowing wo sub-da ase s: anosmia and heal hy con- ols. The i s da ase was collec ed on 79 pa ien s be ween Augus 2015 and July 2017. Pa ien s we e diagnosed wi h anosmia o hyposmia. Heal hy con ols’ (n= 91) sub-da ase con ained da a om h ee di e en MRI s udies. In bo h da ase s, pa icipan s’ smell abili y was measu ed psychophysically using he Sni in’ S icks es [ 22 ]. The sco es o he es ( ange: 1–48) we e used o ca ego ize pa icipan s’ smelling abili y in he ollowing ca ego ies: unc ional anosmia (TDI ≤ 16), hyposmia (16 > TDI < 30.75), o no mosmia (TDI ≥30.75) [23] (Table 1). Table 1. Pa icipan ’s TDI Sco es (sum sco e (TDI) o h eshold (T), disc imina ion (D), and iden i i- ca ion (I) o he odo s; means (M), s anda d de ia ions (SD). Smell Dys unc ion Pa ien s (N = 79) TDI T D I M 18.00 2.67 7.81 7.52 SD 6.72 2.50 3.07 3.01 No mosmic Con ols (N = 91) TDI T D I M 36.39 9.97 12.92 13.50 SD 2.10 2.24 1.65 1.27 B ain Sci. 2021,11, 1141 4 o 12 2.2. MRI Image Acquisi ion All MRI scans we e acqui ed on a 3T Siemens Ve io scanne (Siemens, E langen, Ge many). All acquisi ions we e made using a 32-channel head coil. The ollowing wo modali ies we e used in his s udy: T1-weigh ed axial whole-b ain scans and T2-weigh ed high- esolu ion co onal scans o OB imaging. T1-weigh ed MPRAGE sequence was acqui ed using he ollowing pa ame e s: epe- i ion ime: 2300 ms; echo ime 2.98: ms; lip angle: 9 ◦ ; ield o iew: 240 × 256; acquisi ion ma ix: 250 × 256 oxel size: 1 × 1 × 1 mm 3 ; slice hickness: 1 mm; slices: 176. T2-weigh ed high- esolu ion sequence was acqui ed using ollowing sequence pa ame e s: epe i ion ime: 5500 ms; echo ime: 110 ms; lip angle: 150 ◦ , ield o iew: 120 × 120; acquisi ion ma ix: 256 ×256; oxel size: 0.47 ×0.47 ×1.2 mm3; slice hickness: 1.2 mm (no gap). 2.3. Manual Segmen a ions o OB Volume Fo bo h da ase s, manual segmen a ion o he OBs was pe o med on T2-weigh ed images in co onal plane iew using ITK-SNAP So wa e . 3.6 [ 24 ]. Fi s , bo h T1-weigh ed and T2-weigh ed scans we e con e ed om DICOM o ma o g-zipped NIFTI o ma (nii.gz) using he dcm2niix con e sion ool [ 25 ]. All measu emen s we e pe o med by wo a e s independen ly. Bo h a e s used he p o ocol o pe o ming manual segmen a ions. Voxels belonging o he le and igh OB we e labeled wi h alues 1 and 2, espec i ely. The manual measu emen s we e sa ed as bina y segmen a ion masks in g-zipped NIFTI o ma (nii.gz). To access he in o ma ion o each label, espec i ely, he bina y segmen a ion masks we e con e ed o a NumPy a ay wi h nibabel py hon lib a y [ 26 , 27 ]. The olumes o he labels o le and igh OBs we e calcula ed by mul iplying he numbe o oxels o he label by oxel dimensions using he NumPy py hon lib a y [ 28 ]. Fo all manual segmen a ion masks, he DC was calcula ed using he MedPy py hon lib a y [ 29 ] o es ima e he le el o o e lap be ween he wo a e s. 2.4. Au oma ed Localiza ion o he OBs OBs ha e e y small olumes, especially compa ed o he en i e scanned b ain olume. This leads o highly imbalanced da a due o class imbalance be ween oxels labeled as o eg ound ( oxels wi h alues one/ wo o le / igh OB) and oxels labeled as back- g ound (ze o alue oxels o any o he issue). To sol e his issue, we op ed o using a empla e-based app oach. To he bes o ou knowledge, no ea lie s udies ha e no malized indi idual OBs o a s anda d s e eo ac ic Mon eal Neu ological Ins i u e (MNI) [ 30 , 31 ] space. He e, we de eloped a pipeline ha allows he au oma ic ans o ma ion o all man- ual segmen a ions om T2 na i e space o MNI space. Fi s , he T1 whole-b ain image was co- egis e ed o ICBM 2009c Nonlinea Asymme ic T1 MNI empla e image using ANTs SyN nonlinea egis a ion ool [ 32 ]. Subsequen ly, he T2 image was co- egis e ed o he T1 image by ANTs a ine egis a ion unc ion. Fo bo h s eps, he ans o ma ion ma ices we e sa ed and applied o he manual segmen a ions in na i e T2 space. Applying he in e ed egis a ion ma ices allowed a wo-s ep ans o ma ion o manual segmen a ion masks om T2 na i e space o MNI space. This p ocedu e was ex ended o he manual segmen a ions p oduced by he wo a e s. The esul ing bina y masks in MNI space we e ans o med o NumPy a ays using nibabel. The NumPy a ays we e added o one NumPy a ay and di ided by he highes alue in he a ay. This p ocess esul ed in a p obabili y map o he OBs in MNI space. To calcula e he coo dina es o he cen e o g a i y (COG) o he OB, we applied a h eshold o 0.5 o he esul ing OB p obabili y map and calcula ed he coo dina es in MNI space (xyz: − 4/44/ − 36) using he SciPy py hon lib a y [ 33 ], and sa ed he esul as a bina y image (Figu e 1). B ain Sci. 2021,11, 1141 5 o 12 B ain Sci. 2021, 11, x FOR PEER REVIEW 5 o 12 Figu e 1. OB p obabili y map: backg ound image MNI ICBM 2009c Nonlinea Asymme ic em- pla e, h eshold 0.5. The coo dina es o he COG a e xyz: −4/44/−36. (x: sagi al, y: on al, z: axial o ien a ions). The colo map indica es he p obabili y om 0 o 1. 2.5. P ep ocessing Pipeline The p ep ocessing pipeline o he segmen a ion ool pe o ms all necessa y s eps o c ea e no malized da a o 3D U-Ne inpu . As desc ibed abo e, he i s s ep consis s in co- egis e ing he T1 image o MNI empla e image and he T2 image o T1 image o ob ain he esul ing ans o ma ion ma ices. These will subsequen ly be in e ed, and he esul - ing in e ed ans o ma ion ma ices applied o he COG bina y map esul ing in he COG in T2 na i e space. To educe he numbe o ea u es ( oxels) and he da a imbalance o he inpu images o p ocessing in 3D U-Ne , we cons uc ed a bounding box ex ac ion algo i hm based on he COG in T2 na i e space. To ensu e ha all images ha e he same o ien a ion in h ee- idimensional space, all T2 images and he COG bina y mask in T2 na i e space we e eo ien ed using nibabel py hon lib a y o canonical o ien a ion. The bounding box edges we e de ined as poin s in 3D space shi ed om he COG by +/−10 mm in he x-di ec ion, +/−15 mm in he y-di ec ion, and +/−5 mm in he z-di ec ion. A e - wa ds, he a ay inside he calcula ed bounding box was ex ac ed om he T2 image and he co esponding manual segmen a ion. The esul ing images we e esampled o a com- mon oxel dimension o (0.5, 1, 0.5) and image shape o (4, 32, 32) using cubic in e pola ion unc ion om slpy py hon lib a y (Figu e 2) [29]. Figu e 2. Au oma ic segmen a ion o he OBs. 1. T1 whole-b ain image o MNI2009casym MNI empla e co- egis a ion. 2. T2 high- esolu ion image o T1 whole-b ain image co- egis a ion. 3. Appling o in e sed ans o ma ion ma ix om Figu e 1. OB p obabili y map: backg ound image MNI ICBM 2009c Nonlinea Asymme ic empla e, h eshold 0.5. The coo dina es o he COG a e xyz: − 4/44/ − 36. (x: sagi al, y: on al, z: axial o ien a ions). The colo map indica es he p obabili y om 0 o 1. 2.5. P ep ocessing Pipeline The p ep ocessing pipeline o he segmen a ion ool pe o ms all necessa y s eps o c ea e no malized da a o 3D U-Ne inpu . As desc ibed abo e, he i s s ep consis s in co- egis e ing he T1 image o MNI empla e image and he T2 image o T1 image o ob ain he esul ing ans o ma ion ma ices. These will subsequen ly be in e ed, and he esul ing in e ed ans o ma ion ma ices applied o he COG bina y map esul ing in he COG in T2 na i e space. To educe he numbe o ea u es ( oxels) and he da a imbalance o he inpu images o p ocessing in 3D U-Ne , we cons uc ed a bounding box ex ac ion algo i hm based on he COG in T2 na i e space. To ensu e ha all images ha e he same o ien a ion in h ee- idimensional space, all T2 images and he COG bina y mask in T2 na i e space we e eo ien ed using nibabel py hon lib a y o canonical o ien a ion. The bounding box edges we e de ined as poin s in 3D space shi ed om he COG by +/−10 mm in he x-di ec ion, +/ − 15 mm in he y-di ec ion, and +/ − 5 mm in he z-di ec ion. A e wa ds, he a ay inside he calcula ed bounding box was ex ac ed om he T2 image and he co esponding manual segmen a ion. The esul ing images we e esampled o a common oxel dimension o (0.5, 1, 0.5) and image shape o (4, 32, 32) using cubic in e pola ion unc ion om slpy py hon lib a y (Figu e 2) [34]. B ain Sci. 2021, 11, x FOR PEER REVIEW 5 o 12 Figu e 1. OB p obabili y map: backg ound image MNI ICBM 2009c Nonlinea Asymme ic em- pla e, h eshold 0.5. The coo dina es o he COG a e xyz: −4/44/−36. (x: sagi al, y: on al, z: axial o ien a ions). The colo map indica es he p obabili y om 0 o 1. 2.5. P ep ocessing Pipeline The p ep ocessing pipeline o he segmen a ion ool pe o ms all necessa y s eps o c ea e no malized da a o 3D U-Ne inpu . As desc ibed abo e, he i s s ep consis s in co- egis e ing he T1 image o MNI empla e image and he T2 image o T1 image o ob ain he esul ing ans o ma ion ma ices. These will subsequen ly be in e ed, and he esul - ing in e ed ans o ma ion ma ices applied o he COG bina y map esul ing in he COG in T2 na i e space. To educe he numbe o ea u es ( oxels) and he da a imbalance o he inpu images o p ocessing in 3D U-Ne , we cons uc ed a bounding box ex ac ion algo i hm based on he COG in T2 na i e space. To ensu e ha all images ha e he same o ien a ion in h ee- idimensional space, all T2 images and he COG bina y mask in T2 na i e space we e eo ien ed using nibabel py hon lib a y o canonical o ien a ion. The bounding box edges we e de ined as poin s in 3D space shi ed om he COG by +/−10 mm in he x-di ec ion, +/−15 mm in he y-di ec ion, and +/−5 mm in he z-di ec ion. A e - wa ds, he a ay inside he calcula ed bounding box was ex ac ed om he T2 image and he co esponding manual segmen a ion. The esul ing images we e esampled o a com- mon oxel dimension o (0.5, 1, 0.5) and image shape o (4, 32, 32) using cubic in e pola ion unc ion om slpy py hon lib a y (Figu e 2) [29]. Figu e 2. Au oma ic segmen a ion o he OBs. 1. T1 whole-b ain image o MNI2009casym MNI empla e co- egis a ion. 2. T2 high- esolu ion image o T1 whole-b ain image co- egis a ion. 3. Appling o in e sed ans o ma ion ma ix om Figu e 2. Au oma ic segmen a ion o he OBs. 1. T1 whole-b ain image o MNI2009casym MNI empla e co- egis a ion. 2. T2 high- esolu ion image o T1 whole-b ain image co- egis a ion. 3. Appling o in e sed ans o ma ion ma ix om s ep 1 o OB COG mask in MNI space. 4. Appling o in e sed ans o ma ion ma ix om s ep 2 o OB COG mask in T1 space. 5. C ea ing a 3D bounding box based on COG coo dina es in T2 space om s ep 4. 6. Cu ou he bounding box om he T2 image. 7. Pe o ming image segmen a ion using ained 3D U-ne . B ain Sci. 2021,11, 1141 6 o 12 2.6. T aining o he 3D U-Ne Model The en i e model aining p ocess was ca ied ou using he Monai Py hon Lib a y [ 35 ]. Model aining was pe o med on he da ase esul ing om p ep ocessing pipeline de- sc ibed abo e. The da ase included he p ep ocessed da a om 159 T2 images and 318 manual segmen a ions (159 manually c ea ed bina y masks by wo a e s). Se en sub- jec s om he anosmia da ase we e excluded, due o no ha ing a isible OB. The e o e, he manual segmen a ions made by bo h a e s we e emp y, con aining only he backg ound. Fi s , he da ase was andomly spli in o a aining da ase (n= 191 subjec s; 60% o he da ase ), a alida ion da ase (n= 64 subjec s; 20% o he da ase ,), and a es da ase (n= 64 subjec s; 20% o he da ase ). Fi s , all da a unde wen in ensi y no maliza ion, a s anda d s ep o he monai p ep ocessing pipeline. To pe o m da a augmen a ion, a andom a ine ans o ma ion was applied o he no malized da a om he aining da ase only. These ans o ma ions we e au oma ically applied by RandA ined unc ion om he monai py hon package du ing each epoch o he model aining p ocess. Speci ically, he ans o ma ion ea u es included a ansla ion ange o − 10 o +10 oxels in each di ec ion and o a ion o − 30 ◦ o +30 ◦ deg ees using bilinea in e pola ion unc ion o T2 images and nea es -neighbo in e pola ion o bina y label masks. The U-Ne model was impo ed om monai py hon lib a y and speci ied using ollowing pa ame e s: dimensions: 3; inpu channels: 1; ou pu channels: 2; channels: 16, 32, 64, 128, 256; s ides: 2, 2, 2, 2; num o esidual uni s = 2. The model was ained o e 300 epochs. Each epoch con ained 86 i e a ions o mini ba ches con aining wo co esponding image and bina y mask pai s o size 64 ×32 ×32 oxels . We op ed o he T e sky unc ion o measu e loss a e sion. Ne wo k weigh s we e op imized using he Adam op imiza ion unc ion (I = 0.001). One inpu channel and wo ou pu channels we e de ined o he inpu –ou pu da a s eam o he model, con aining h ee dimensions (Figu e 3). B ain Sci. 2021, 11, x FOR PEER REVIEW 6 o 12 s ep 1 o OB COG mask in MNI space. 4. Appling o in e sed ans o ma ion ma ix om s ep 2 o OB COG mask in T1 space. 5. C ea ing a 3D bounding box based on COG coo dina es in T2 space om s ep 4. 6. Cu ou he bounding box om he T2 image. 7. Pe o ming image segmen a ion using ained 3D U-ne . 2.6. T aining o he 3D U-Ne Model The en i e model aining p ocess was ca ied ou using he Monai Py hon Lib a y [30]. Model aining was pe o med on he da ase esul ing om p ep ocessing pipeline desc ibed abo e. The da ase included he p ep ocessed da a om 159 T2 images and 318 manual segmen a ions (159 manually c ea ed bina y masks by wo a e s). Se en subjec s om he anosmia da ase we e excluded, due o no ha ing a isible OB. The e o e, he manual segmen a ions made by bo h a e s we e emp y, con aining only he backg ound. Fi s , he da ase was andomly spli in o a aining da ase (n = 191 subjec s; 60% o he da ase ), a alida ion da ase (n = 64 subjec s; 20% o he da ase ,), and a es da ase (n = 64 subjec s; 20% o he da ase ). Fi s , all da a unde wen in ensi y no maliza ion, a s and- a d s ep o he monai p ep ocessing pipeline. To pe o m da a augmen a ion, a andom a ine ans o ma ion was applied o he no malized da a om he aining da ase only. These ans o ma ions we e au oma ically applied by RandA ined unc ion om he monai py hon package du ing each epoch o he model aining p ocess. Speci ically, he ans o ma ion ea u es included a ansla ion ange o −10 o +10 oxels in each di ec ion and o a ion o −30° o +30° deg ees using bilinea in e pola ion unc ion o T2 images and nea es -neighbo in e pola ion o bina y label masks. The U-Ne model was impo ed om monai py hon lib a y and speci ied using ol- lowing pa ame e s: dimensions: 3; inpu channels: 1; ou pu channels: 2; channels: 16, 32, 64, 128, 256; s ides: 2, 2, 2, 2; num o esidual uni s = 2. The model was ained o e 300 epochs. Each epoch con ained 86 i e a ions o mini ba ches con aining wo co esponding image and bina y mask pai s o size 64 × 32 × 32 oxels. We op ed o he T e sky unc ion o measu e loss a e sion. Ne wo k weigh s we e op imized using he Adam op imiza ion unc ion (I = 0.001). One inpu channel and wo ou pu channels we e de ined o he inpu –ou pu da a s eam o he model, con- aining h ee dimensions (Figu e 3). Figu e 3. Model aining p ocess. Le : T aining loss unc ion. Righ : DC plo o aining (g een cu e) and alida ion ( ed cu e) da ase s. The DC and a e age loss alues a e bo h a bi a y uni s. The cu es each a pla eau a epoch 100. The model a he epoch 297 was selec ed as he highes DC (0.84) o he aining p ocess. The uni (au) means a bi a y uni . Figu e 3. Model aining p ocess. Le : T aining loss unc ion. Righ : DC plo o aining (g een cu e) and alida ion ( ed cu e) da ase s. The DC and a e age loss alues a e bo h a bi a y uni s. The cu es each a pla eau a epoch 100. The model a he epoch 297 was selec ed as he highes DC (0.84) o he aining p ocess. The uni (au) means a bi a y uni . 2.7. Pos p ocessing Pipeline We designed and implemen ed a no el pos p ocessing pipeline o pe o m da a cleanup o he p ima y ou pu o he ained 3D U-Ne model and ans o m he ou pu o he o iginal T2 image’s size, esolu ion, and o ien a ion. As a i s s ep, all clus e s a e elabeled o unique alues. Secondly, clus e s a e h esholded based on hei size o elimina e spu ious clus e s. Finally, he esul ing mask is esampled o he esolu ion, oxel B ain Sci. 2021,11, 1141 7 o 12 size, and o ien a ion o he co esponding T2 image. This pos -p ocessed bina y mask is hen sa ed in o an emp y NumPy a ay o he same size as he T2 image— he inal bina y segmen a ion mask. 2.8. E alua ion o T aining and Tes ing The pe o mance o he au oma ic segmen a ion was e alua ed on he es ing da ase . This da ase was kep sepa a e be o e he model aining s ep and, he e o e, con ained only da a “unseen” by he model, consis ing o 64 subjec s and he co esponding manual segmen a ions o wo a e s. Pe o mance was e alua ed by compu ing an ex ensi e se o s a is ical me ics— he dice coe icien (DC) and a e age symme ic su ace dis ance (ASSD)— o e alua e he simila i y be ween he model’s p edic ed segmen a ions and he manual segmen a ions, o bo h le and igh OB. DC is an o e lay simila i y index ha e lec s size and localiza ion ag eemen and anges om 0 (no o e lap) o 1 (comple e o e lap). ASSD ep esen s he mean dis ance o he bina y objec s in wo images. The e o e, an ASSD o 0 mm ep esen s a pe ec ma ch be ween bo h segmen a ions. Fo subjec s wi h diagnosed congeni al anosmia and absence o he OBs, i was no possible o compu e he DC because emp y segmen a ion masks con ain only he label co esponding o he image backg ound. Fu he mo e, o ensu e he quali y o p edic ed segmen a ions, all da a we e andom- ized and anonymized o ensu e unbiased a ing o he manual and p edic ed segmen a ions. A well-expe ienced a e , who had no been in ol ed in any s age o he da a segmen a ion p ocess, was asked o a e he o e lap o bo h manual segmen a ions and p edic ed masks using a pe o mance scale. The a ing c i e ia we e de ined as a scale: 1: “No cong u- ency”, 2–3: “Poo cong uency”, 4–5: “good cong uency”, 6–7: “ e y good cong uency”, 8–10: “excellen cong uency”. 3. Resul s Fi s ly, we compa ed he olumes o he manual segmen a ions made by a e one and wo and calcula ed he DC o he le and igh OBs indi idually (Table 2). Fo he da ase con aining he anosmia pa ien s’ da a, he a e age DC was 0.77 ± 0.07 and 0.74 ±0.10 mm3 o he le and igh OB, espec i ely. The mean olume o he le OB was 37.82 ± 11.48 and 47.82 ± 14.78 mm 3 measu ed by a e s one and wo, espec i ely. Fo he igh OB, he mean olume o he segmen a ions p oduced by a e s one and wo was 34.32 ± 10.93 and 46.47 ±15.43 mm3, espec i ely. Table 2. The able shows con e gence o manual segmen a ions pe o med by wo a e s, based on DC and le and igh ol ac o y bulb olume (LOBV and ROBV) o each a e , espec i ely. The uni (au) means a bi a y uni . Smell Dys unc ion Pa ien s Le OB DC (au) Righ OB DC (au) LOBV Ra e 1 (mm3) LOBV Ra e 2 (mm3) ROBV Ra e 1 (mm3) ROBV Ra e 2 (mm3) M 0.77 0.74 37.82 47.64 34.32 46.47 SD 0.07 0.10 11.48 14.78 10.93 15.43 Heal hy Con ols Le OB DC (au) Righ OB DC (au) LOBV Ra e 1 (mm3) LOBV Ra e 2 (mm3) ROBV Ra e 1 (mm3) ROBV Ra e 2 (mm3) M 0.81 0.80 44.14 56.01 42.47 54.15 SD 0.06 0.05 12.38 16.92 13.54 17.66 B ain Sci. 2021,11, 1141 8 o 12 Fo he da ase con aining he heal hy con ol subjec s’ da a, he a e age DC o he le OB was 0.81 ± 0.06 mm 3 and 0.80 ± 0.05 mm 3 o he igh OB. The mean olume o he le OB was 44.14 ± 12.38 mm 3 measu ed by a e one and 56.01 ± 16.92 mm 3 measu ed by a e wo. The mean olume o he igh OB was 42.47 ± 13.54 mm 3 measu ed by a e one and 54.15 ±17.66 mm3measu ed by a e wo (Figu e 4). B ain Sci. 2021, 11, x FOR PEER REVIEW 9 o 12 SD 0.87 0.81 0.87 Minimum 4.00 4.00 4.00 Maximum 8.00 7.00 7.00 Addi ionally, we es ed he algo i hm on subjec s om he ol ac o y dys unc ion da- ase wi h diagnosed congeni al anosmia (N = 7). As a o emen ioned, gi en ha hese sub- jec s do no ha e a isible OB, he esul ing manual segmen a ion masks we e emp y, con- aining only he backg ound label. Ou model was able o co ec ly iden i y all hese ex- ao dina y da ase s as non-exis en OB cases, wi h he gene a ed masks only con aining he backg ound label, as expec ed Figu e 4. Top: Boxplo showing he OBVs o manual segmen a ions measu ed by wo independen a e s and he 3D U- Ne model on es da ase . Bo om le and igh : XY-Plo o he le and igh OBV measu ed by wo independen a e s and 3D U-Ne on es da ase . The end o he g aphs shows a high le el o cong uency o he measu emen s. Figu e 5. Au oma ic segmen a ion o he le ( ed) and igh (g een) OB in co onal T2 MRI scan pe o med manually by wo independen a e s and he algo i hm. In addi ion o he ex ended quali y e alua ion, we de eloped wo in e aces o ou algo i hm. The ool is ully a ailable ia he pip py hon package managemen sys em, and he command-line in e ace is accessible in he e minal on UNIX-based ope a ing sys ems (OS). Fo use in a Mic oso na i e OS, a Linux sub-sys em needs o be u he ins alled. Figu e 4. Top : Boxplo showing he OBVs o manual segmen a ions measu ed by wo independen a e s and he 3D U-Ne model on es da ase . Bo om le and igh : XY-Plo o he le and igh OBV measu ed by wo independen a e s and 3D U-Ne on es da ase . The end o he g aphs shows a high le el o cong uency o he measu emen s. Fo he e alua ion o he o e lap be ween he p edic ed segmen a ions o he algo i hm and he manual segmen a ions o he OB o he subjec s in he es da ase (N = 64), a ious me ics we e calcula ed o he le and igh OBs, sepa a ely, as well as o he en i e segmen a ion mask. As i can be ound in Table 3, he mean DC was 0.77 ± 0.05 (le OB: 0.78 ± 0.06, igh OB: 0.75 ± 0.08) and he mean symme ic su ace dis ance (ASSD) was 0.43 ±0.10 (le OB: 0.41 ±0.10, igh OB: 0.44 ±0.14) (Table 3). Table 3. Model pe o mance me ics o he es da ase . DC (DC), A e age Symme ic Su ace Dis ance (ASSD). The uni (au) means a bi a y uni . DC (au) ASSD (au) Le OB Righ OB Mean Le OB Righ OB Mean M 0.78 0.75 0.77 0.41 0.44 0.43 SD 0.06 0.08 0.05 0.10 0.14 0.10 Mo eo e , we compa ed he olumes o he manual segmen a ions and he p edic ed segmen a ions gene a ed by ou algo i hm, o he le and igh OBs indi idually, o he es da ase . Fo he le hemisphe e, he mean OB olume o he p edic ed mask was 44.80 ±8.59 mm3 and 46.14 ± 12.70 mm 3 o he manually segmen ed mask. Fo he igh OB, he p edic ed OB olume was 46.73 ± 8.86 mm 3 o he p edic ed mask and 42.63 ±14.19 mm3 o he manual masks (Table 4). B ain Sci. 2021,11, 1141 9 o 12 Table 4. OB olumes o manual segmen a ions and p edic ed segmen a ions o he algo i hm o he es da ase . Manual Segmen a ion 3D U-Ne Segmen a ions Le OB (mm3) Righ OB (mm3) Le OB (mm3) Righ OB (mm3) M 46.14 42.63 44.80 46.73 SD 12.70 14.19 8.59 8.86 Subsequen ly, an independen , unbiased, and well-expe ienced a e a ed he manual segmen a ions p oduced by bo h a e s, as well as he p edic ed segmen a ion o he algo i hm (Figu e 5) (Table 5). The mean sco e was 6.23 ± 0.87 and 5.92 ± 0.81 o a e one and wo, espec i ely, and 5.95 ±0.87 o he segmen a ions gene a ed by ou model. B ain Sci. 2021, 11, x FOR PEER REVIEW 9 o 12 SD 0.87 0.81 0.87 Minimum 4.00 4.00 4.00 Maximum 8.00 7.00 7.00 Addi ionally, we es ed he algo i hm on subjec s om he ol ac o y dys unc ion da- ase wi h diagnosed congeni al anosmia (N = 7). As a o emen ioned, gi en ha hese sub- jec s do no ha e a isible OB, he esul ing manual segmen a ion masks we e emp y, con- aining only he backg ound label. Ou model was able o co ec ly iden i y all hese ex- ao dina y da ase s as non-exis en OB cases, wi h he gene a ed masks only con aining he backg ound label, as expec ed Figu e 4. Top: Boxplo showing he OBVs o manual segmen a ions measu ed by wo independen a e s and he 3D U- Ne model on es da ase . Bo om le and igh : XY-Plo o he le and igh OBV measu ed by wo independen a e s and 3D U-Ne on es da ase . The end o he g aphs shows a high le el o cong uency o he measu emen s. Figu e 5. Au oma ic segmen a ion o he le ( ed) and igh (g een) OB in co onal T2 MRI scan pe o med manually by wo independen a e s and he algo i hm. In addi ion o he ex ended quali y e alua ion, we de eloped wo in e aces o ou algo i hm. The ool is ully a ailable ia he pip py hon package managemen sys em, and he command-line in e ace is accessible in he e minal on UNIX-based ope a ing sys ems (OS). Fo use in a Mic oso na i e OS, a Linux sub-sys em needs o be u he ins alled. Figu e 5. Au oma ic segmen a ion o he le ( ed) and igh (g een) OB in co onal T2 MRI scan pe o med manually by wo independen a e s and he algo i hm. Table 5. Human alida ion scale. This able shows he manual a ing c i e ia used by blinded a e o human alida ion o he segmen a ions. The a ing scale was de ined as: 1: “No cong uency”, 2–3: “Poo cong uency”, 4–5: “good cong uency”, 6–7: “ e y good cong uency”, 8–10: “excellen cong uency.” The uni (au) means a bi a y uni . Human Ra ing Resul s Ra e 1 (au) Ra e 2 (au) U-Ne (au) M 6.23 5.92 5.95 SD 0.87 0.81 0.87 Minimum 4.00 4.00 4.00 Maximum 8.00 7.00 7.00 Addi ionally, we es ed he algo i hm on subjec s om he ol ac o y dys unc ion da ase wi h diagnosed congeni al anosmia (N = 7). As a o emen ioned, gi en ha hese subjec s do no ha e a isible OB, he esul ing manual segmen a ion masks we e emp y, con aining only he backg ound label. Ou model was able o co ec ly iden i y all hese ex ao dina y da ase s as non-exis en OB cases, wi h he gene a ed masks only con aining he backg ound label, as expec ed. In addi ion o he ex ended quali y e alua ion, we de eloped wo in e aces o ou algo i hm. The ool is ully a ailable ia he pip py hon package managemen sys em, and he command-line in e ace is accessible in he e minal on UNIX-based ope a ing sys ems (OS). Fo use in a Mic oso na i e OS, a Linux sub-sys em needs o be u he ins alled. A isual in e ace is also a ailable, allowing he use o selec he pa hs o olde s con aining da a com o ably a he dis ance o only a ew mouse clicks. 4. Discussion In his s udy, we ha e de eloped a eady- o-use solu ion o he au oma ic segmen a- ion o human OBs using 3D U-Ne . The algo i hm localizes he COG o OBs, pe o ms he