Uni e si y o Minho
School o Enginee ing
Luís Filipe da Sil a Fe ei a
Fib osis Segmen a ion in Ca diac
Resonance Imaging
oc obe 2024
Uni e si y o Minho
School o Enginee ing
Luís Filipe da Sil a Fe ei a
Fib osis Segmen a ion in Ca diac
Resonance Imaging
Mas e s Disse a ion
Mas e s Deg ee in Bioin o ma ics
Disse a ion supe ised by
P o esso Doc o Ca los Sil a
Co-supe ised by
Doc o Ad iano Pin o
oc obe 2024
Copy igh and Te ms o Use o Thi d Pa y Wo k
This disse a ion epo s on academic wo k ha can be used by hi d pa ies as long as he in e na ionally
accep ed s anda ds and good p ac ices a e espec ed conce ning copy igh and ela ed igh s.
This wo k can he ea e be used unde he e ms es ablished in he license below.
Reade s needing au ho iza ion condi ions no p o ided o in he indica ed licensing should con ac he
au ho h ough he Reposi ó iUM o he Uni e si y o Minho.
CC BY-NC-ND
h ps://c ea i ecommons.o g/licenses/by-nc-nd/4.0/
i
Acknowledgemen s
The academic pa h is illed wi h obs acles and unexpec ed wis s and a ha poin is when encou agemen
om amily, gi l iend, and iends was e y impo an o me o keep mo i a ed in pu suing my aspi a ions.
Na iga ing h ough unknown e i o ies o en b ings choices and momen s o doub ha equi e pa ience
and pe sis ence o ou line hese challenges.
Wi h ha said, I would like o exp ess my g a i ude o all he knowledge, suppo , and dedica ion gi en
by my supe iso P o esso Ca los Sil a, who made his wo k possible o be comple ed, despi e all he
obs acles encoun e ed along he way. Addi ionally, I would like o ex end my app ecia ion o Doc o Ad iano
Sil a o his ole in his p ocess and o Elsa Fe ei a, who conduc ed simila esea ch and gene ously sha ed
he insigh s.
I would like o exp ess my hea el g a i ude o my amily, especially my gi l iend, Ca a ina Nunes, o
always unde s anding and suppo ing me in my lowes momen s and con ibu ing e e y hing wi hin hei
each o help, and inally, o all my iends who we e always he e o help me whene e I needed hem.
None o his p og ess would ha e been possible o achie e alone.
ii

S a emen o In eg i y
I he eby decla e ha ing conduc ed his academic wo k wi h in eg i y.
I con i m ha I ha e no used plagia ism o any o m o undue use o in o ma ion o alsi ica ion o esul s
along he p ocess leading o i s elabo a ion.
I u he decla e ha I ha e ully acknowledged he Code o E hical Conduc o he Uni e si y o Minho.
Uni e si y o Minho, B aga, oc obe 2024
Luís Filipe da Sil a Fe ei a
iii
Abs ac
Hea disease, a leading cause o mo ali y wo ldwide, is associa ed in many cases wi h a condi ion known
as myoca dial ib osis, assuming a c ucial p ognos ic in disease assessmen . Fib osis mani es s in wo
o ms, in e s i ial and eplacemen , bo h a ec he hea issue, howe e , he i s may be e e sible wi h
ea ly iden i ica ion and in e en ion, and he second causes pe manen sca issue. Gi en his, he pa ien
needs a p ecise diagnos ic o e ec i e managemen and ea men , so accu a e ib ous issue localiza ion
is needed, o educe complica ions acco ding o he pa ien ’s condi ion. Manual segmen a ion is he gold
s anda d o accu acy, al hough i ’s ime-consuming and cos ly. Fo his eason, au oma ed me hods ha e
been de eloped o imp o e speed and epea abili y compa ed o manual me hods, howe e , he accu acy
is s ill no eliable, due o di icul ies in segmen ing small and i egula issues, esolu ion challenges, and
ca diac magne ic esonance imaging quali y . Conside ing he main p oblem, his p ojec aims o in oduce
a new a en ion mechanism o imp o e an algo i hm ha can au oma ically segmen he le en icula
myoca dium and dis inguish be ween heal hy and ib ous issue using ad anced deep-lea ning me hods.
Rega ding he me hodology, we will ha e he me hod di ided in o wo phases. In he i s s age, a
2D U-Ne wi h Balanced S eady-S a e F ee P ecession (bSSFP), La e Gadolinium Enhancemen (LGE), and
T2 sequences was used as inpu , o segmen h ee key ana omical s uc u es o he hea : le en icle,
igh en icle, and he le en icula myoca dium, o p edic he myoca dium a ea and iden i y possible
loca ions o edema and sca . In he second s age, he le en icula myoca dium mask om he i s
s age, accompanied by he same h ee magne ic esonance imaging sequences and wo new images,
he LGE-myo and T2-myo, will be used o e ine he segmen a ion and de ec ion o he lesion. In his
s age, a new a en ion mechanism, Bila e al Local A en ion (BLA), was implemen ed wi hin he ans-
o me encode om he ne wo k a ian U-Shape Nes ed T ans o me (UNesT). This ne wo k comp ises
a T ans o me -based encode (Nes ed Hie a chical T ans o me ), and a con olu ion-based decode . The
da ase employed in his app oach was p o ided by he MICCAI2020 MyoPS challenge and consis s o 45
i
se s o ca diac magne ic esonance images wi h h ee sequences (bSSFP, LGE, T2).
Con a y o expec ed, he esul s achie ed did no signi ican ly exceed hose o s a e-o - he-a me hods,
al hough hey emain compe i i e compa ed o he models ha did no employ an ensemble app oach,
i was posi ioned in hi d place among he six me hods ha did no use his s a egy. The a e age Dice
Simila i y Coe icien (DSC) o he me hod was 0.678, in e ms o sca segmen a ion he DSC esul ed was
0.640 ± 0.232, and o he combined sca and edema he DSC was 0.715 ± 0.110.
In conclusion, he esul s sugges ha he implemen ed a en ion mechanism, al hough p omising, did
no b ing signi ican bene i s in cap u ing de ailed in o ma ion om ca diac magne ic esonance images
compa ed o o he wo ks. This may be a ibu ed o challenges in adap ing he model o he speci ic
ana omical a iabili y p esen in he analyzed images o he need o u he in es iga ion o ine- une he
hype pa ame e s o he second s age.
Keywo ds Myoca dial Fib osis, Ca diac Magne ic Resonance Imaging, Deep Lea ning, T ans o me s
Resumo
As doenças ca díacas são a p incipal causa de mo alidade em odo o mundo, sendo associada em
mui os casos a uma condição conhecida como ib ose miocá dica, assumindo um p ognós ico c ucial
na a aliação da doença. A ib ose mani es a-se de duas o mas, in e s icial e de subs i uição, sendo
que ambas a e am o ecido ca díaco, no en an o, a p imei a pode se e e sí el com iden i icação e
in e enção p é ia, e a segunda p o oca cica iz pe manen e no ecido. Pe an e is o, o doen e necessi a
de um diagnós ico p eciso pa a uma ges ão e a amen o e icaz, pelo que é necessá ia uma localização
p ecisa do ecido ib oso de o ma a eduzi as complicações, endo em con a a condição do doen e. A
segmen ação manual é a e e ência pad ão no que oca à p ecisão, no en an o é demo ada e dispendiosa.
Po es e mo i o, o am desen ol idos mé odos au omá icos pa a melho a a elocidade e a epe ibilidade
em compa ação com os mé odos manuais, no en an o, a p ecisão ainda não é iá el, de ido a di iculdades
de segmen ação de ecidos pequenos e i egula es, desa ios de esolução e qualidade das imagem de
essonância magné ica ca díaca. Conside ando o p oblema p incipal, es e p oje o isa in oduzi um
no o mecanismo de a enção, pa a melho a um algo i mo que segmen a au oma icamen e o miocá dio
en icula esque do e dis ingue en e ecido saudá el e ib oso, u ilizando mé odos a ançados de
Deep
Lea ning
.
Quan o à me odologia, e emos o mé odo di idido em duas ases. Na p imei a ase, oi u ilizada
uma U-Ne 2D com as sequências
Balanced S eady-S a e F ee P ecession (bSSFP), La e Gadolinium En-
hancemen (LGE)
e T2 como en ada pa a segmen a ês es u u as ana ómicas p incipais do co ação:
en ículo esque do, en ículo di ei o e miocá dio en icula esque do, pa a p e e a á ea do miocá dio e
iden i ica possí eis locais pa a o edema e a cica iz. Na segunda e apa, a másca a do miocá dio en ic-
ula esque do da p imei a e apa, acompanhada das mesmas ês sequências de essonância magné ica
e de duas no as imagens, a LGE-myo e a T2-myo, se á u ilizada pa a melho a a segmen ação e de eção
da lesão. Nes a ase, um no o mecanismo de a enção,
Bila e al Local A en ion
(BLA), oi implemen-
i
Lis o Tables
1 A e age Dice Simila i y Coe icien alues o segmen a ion in he alida ion da ase a
he i s s age..................................... 55
2 Dice Simila i y Coe icien alues o sca and edema segmen a ion when di e en ans-
o me encode s adop di e en a en ion blocks in he alida ion da ase . . . . . . . . 56
3 Dice Simila i y Coe icien (DSC) coe icien alues o sca and edema wi h di e en
me hods in he MyoPS 2020 con es . In he able below, he DSC alues o he p oposed
model BLA UnesT e lec he esul s ob ained, including case #207, which con ains no
sca . When case #207 is excluded, he a e age DSC changes om 0.640 ± 0.232 o
0.674 ± 0.195. The use o he supe sc ip (**) shows ha he app oach u ilized ensemble
lea ningin heme hod................................. 58
xiii

Ac onyms
AdamW AdamWeigh Decay.
AI A i icial In elligence.
ANN A i icial Neu al Ne wo ks.
BCE Bina y C oss En opy.
BLA Bila e al Local A en ion.
BN Ba ch no maliza ion.
bSSFP Balanced S eady-S a e F ee P ecession.
CE C oss En opy.
CMR Ca diac Magne ic Resonance.
CNN Con olu ional Neu al Ne wo k.
CSWin C oss-Shaped Window.
CT Compu ed Tomog aphy.
DeiT Da a-e icien Image T ans o me s.
DL Deep Lea ning.
DSC Dice Simila i y Coe icien .
ECG Elec oca diog am.
xi
ECMs Ex acellula Ma ix P o eins.
FCN Fully Con olu ional Ne wo k.
FCNN Fully Connec ed Neu al Ne wo ks.
FNN Feed o wa d Neu al Ne wo ks.
FSLA Fea u e-Space Local A en ion.
FWHM Full Wid h a Hal Maximum.
GANs Gene a i e Ad e sa ial Ne wo ks.
GD G adien Descen .
ISLA Image-Space A en ion.
LA Le A ium.
LGE La e Gadolinium Enhancemen .
LV Le Ven icle.
MI Myoca dial In a c ion.
ML Machine Lea ning.
MLP Mul i-Laye Pe cep on.
MN-Ne Mul i-Scale Nes ed Ne wo k.
MRI Magne ic Resonance Imaging.
MSA Mul i-Head Sel -A en ion.
MSED-Ne Mul i-Scale Symme ic Encoding-Decoding Ne wo k.
MyoPS Myoca dial Pa hology Segmen a ion.
NLP Na u al Language P ocessing.
RA Righ A ium.
x
ReLU Rec i ied Linea Uni .
ResNe Residual Ne wo k.
RGB Red G een Blue.
RNN Recu en Neu al Ne wo ks.
RV Righ Ven icle.
SA Sel -a en ion.
SDPA Scaled Do P oduc A en ion.
SGD S ochas ic G adien Descen .
SNR Signal- o-Noise Ra io.
SSIR Single Sho In e sion Reco e y.
Swin-T Swin T ans o me .
TanH Tangen Hype bolic.
UNesT U-Shape Nes ed T ans o me .
ViT Vision T ans o me .
x i
Chap e 1
In oduc ion
This chap e de e mines he mo i a ion and he goals behind his disse a ion, mo e speci ically all he
amewo k o he explo a ion made. I also explains he disse a ion’s s uc u e o, mo e easily, explo e he
a ious chap e s and sec ions.
1.1 Con ex and Mo i a ion
Hea diseases s and ou as a p oblem o eno mous ele ance, ep esen ing he main cause o mo ali y
globally and ha ing a subs an ial impac on heal h sys ems and socie y as a whole. This in ica e scena io is
cha ac e ized by a wide di e si y o clinical con ex s ha igge ca diac condi ions, wi h pa icula emphasis
on pa hologies associa ed wi h he myoca dium.
In he ca dio ascula diseases lies a condi ion known as myoca dial ib osis ha o en ecu s on a his-
ological le el, which assumes a c ucial p ognos ic posi ion. I mani es s in wo o ms: In e s i ial Fib osis,
which sp eads along he hea issue du ing he ini ial s ages o he disease, as myo ib oblas s accumu-
la e collagen and he e ec s migh be e e sible wi h ea ly iden i ica ion and in e en ion; Replacemen
Fib osis, which occu s ollowing cell dea h, in his case, collagen a ec s he cell subsequen ly causing
pe manen al e a ions o he hea issue. I is also impo an o conside ha di e en ca diac diseases
ha e a ied pa e ns and dis ibu ions o ib osis, which complica es diagnosis and ea men e en mo e.
To manage his ype o disease e ec i ely and p edic p ognosis, while unde s anding i s loca ions
in de ail, i ’s necessa y o ha e a p ecise e alua ion and cha ac e iza ion o he p oblem, pa icula ly in
ib osis segmen a ion in Ca diac Magne ic Resonance (CMR) images. This in o ma ion imp o es
managemen and helps in decision-making on how o p oceed wi h he he apy, allowing comp ehensi e
moni o ing o disease p og ession by gi ing exac a eas a ec ed by ib osis in he hea . By doing his,
1
many s a egies can be used, such as p esc ibing speci ic medica ions o de ices o he ib osis loca ion in
Myoca dial In a c ion (MI) on he pa ien ’s hea , possibly esul ing in lowe complica ions depending
on he pa ien ’s condi ion.
Myoca dial ib osis i ’s no jus a p ognosis, i also has a signi ican impac on he quali y o li e om an
indi idual pe spec i e. The e ec s o ib osis can cause many symp oms ha impac an indi idual’s day-
o-day, o example by comp omising hea unc ion, and depending on he p og ession o he pa hology,
many ac i i ies become es ic ed, like hei abili y o exe cise, accompanied by symp oms such as a igue
and dyspnea.
To ha e a p ecise segmen a ion o he ib osis, di e en echniques a e b oadly di ided in o h ee ca -
ego ies: manual, au oma ic, and semi-au oma ic. The gold s anda d is manual segmen a ion, in which
ained p o essionals pe o m an accu a e and i ing segmen a ion. Bu his is equen ly expensi e, slow,
and ope a o -dependen . By con as , au oma ic segmen a ion uses algo i hms o di ide he CMR images
in o segmen s as and e icien ly wi h a some imes lack o p ecision. A leas , he goal o semi-au oma ic
segmen a ion is o imp o e speed and accu acy by using human expe ience wi h au oma ed p ocedu es.
These echniques can be applied in bo h wo and h ee dimensions, depending on he imaging equi e-
men s.
Au oma ion is c i ical in clinical p ac ice, as i o e comes he undamen al limi a ions o doing his
manually, p o iding e iciency alongside consis ency and agili y o assessing ib osis. This de elopmen ,
no only imp o es he accu acy o diagnosis, i also indica es a shi , in he unde s anding and ea men
s a egies conce ning myoca dial ib osis and i s implica ions in pa ien s. Ne e heless, he po ing o
hese me hods o clinical p ac ice has no been ully ecognized ac oss many medical ields, o his mo-
i e, au oma ic segmen a ion emains an impo an esea ch opic. This o en occu s due o insu icien
da a, leading o unde es ima ion in clinical assessmen s, he small size o pa icula s uc u es, which
makes hei esolu ion di icul , and he complex bounda ies o he e ogeneous abno mali ies and s uc-
u es si ua ed be ween di e se ana omical ea u es. On he o he hand, he exis ing echniques ha u ilize
au oma ed segmen a ion solu ions a e limi ed since hey a e sensi i e o imaging quali y, s uc u es wi h
compa able in ensi ies, and di icul ies in accommoda ing he di e se ange o ana omical di e ences
among indi iduals. E en while some me hods in ca diology a e beginning o be p omising, much wo k
needs o be done, especially in segmen ing in ica e ca diac s uc u es and anomalies. Au oma ed seg-
men a ion echniques ha e also been ad anced in a ious domains, including pulmonology, hepa ology,
neu ology, and o he s.
2

1.2 Objec i es
This p ojec aims o imp o e an algo i hm ha can au oma ically segmen he le en icula myoca dium
by dis inguishing be ween heal hy and ib ous issue using ad anced deep-lea ning me hods. The main
goal o his s udy consis s o in oducing a new a en ion mechanism wi hin an a chi ec u e ha inco -
po a es a nes ed hie a chical ans o me as an encode and comp ises a con olu ion-based decode ,
a emp ing o imp o e he ex ac ing o ea u es in he da a and consequen ly imp o e he model. The al-
go i hm’s pe o mance is assessed me hodically by u ilizing accessible images o mul i-sequenced CMR
based on he Myoca dial Pa hology Segmen a ion (MyoPS) challenge da ase , p o ided by MIC-
CAI2020.
Addi ionally, imp o ing ou skills in he Py hon p og amming language and di e se Machine Lea n-
ing (ML) lib a ies such, as Py o ch, Monai, and PyTo ch Ligh ning. Such assignmen equi es combining
ields o expe ise, like image p ocessing and p og amming along wi h ocusing on ML and heal h sci-
ences, o ad ance ca diac segmen a ion echniques e ec i ely by analyzing ca diac images, speci ically,
de ec ing myoca dial ib osis accu a ely.
1.3 S uc u e o he Disse a ion
The s uc u e o his disse a ion is made o six dis inc chap e s. The in oduc o y chap e add esses
he con ex , mo i a ion, and objec i es unde lying he de elopmen o his wo k. The second chap e
co esponding o he heo e ical p inciples, explo es wo impo an keys in his wo k, he biological e ms
and he A i icial In elligence (AI) e ms. Rela i ely o he biological e ms, i desc ibes he ana omy
and physiology o he myoca dium, along wi h he pa hophysiology o myoca dial ib osis, and pe mi s an
unde s anding o how he p esence o ib osis impac s he pa ien . Addi ionally, speci ic aspec s o he
magne ic esonance echnique along wi h he di e en ypes o sequences ha can be used, depending
on he objec i e, a e explo ed. A he p inciples o AI, essen ial heo e ical ounda ions will be examined
o suppo he de elopmen o he disse a ion, co e ing concep s ela ed o ML,Deep Lea ning (DL),
and mo e impo an ly, ans o me s, os e ing amilia i y wi h hese concep s and hei e olu ion o e ime.
In he hi d chap e , a b ie in oduc ion o exis ing me hods used in he segmen a ion o CMR images, o
iden i y ca diac s uc u es such as he Le Ven icle (LV) myoca dium, LV, and Righ Ven icle (RV),
and he pa hologies (edema and sca ), highligh ing a ious algo i hms (au oma ic, semi-au oma ic) and
s a egies. In he ou h chap e , all he speci ic in o ma ion has been me iculously explained o lay ou
3
he g oundwo k o he p ojec , wha was ini ially p o ided, and wha was modi ied ac oss bo h s ages o
he p ojec . Finally, he six h chap e p esen s he conclusions and u u e pe spec i es, conside ing all he
wo k and he esul s ob ained, add essing he p oblem p esen ed, he p oposed me hod, i s ad an ages
o disad an ages, and wha could be done o imp o e he esul s.
4
Chap e 2
Theo e ical P inciples
2.1 Medical Pe spec i e
In his chap e , he medical condi ion b ough o ligh is explained, emphasizing he unde s anding o
ib osis pa hophysiology, speci ically he ana omy, and physiology, desc ibing some o he consequences
associa ed wi h i s p esence. The ad anced CMR imaging echniques and some sequences associa ed,
like La e Gadolinium Enhancemen (LGE), will be discussed in de ail h oughou he chap e .
2.1.1 Hea Ana omy and Physiology
Almos e e y li ing being needs an o gan o keep he ci culan luid lowing h oughou he body, and o
humans, he hea makes he blood low and se es as one o he mos i al o gans in he body. I se es
as he cen al componen o he ca dio ascula sys em, si ua ed in he uppe pa o he ches , sligh ly
behind and o he le , shielded by he s e num and he ib cage. The muscle o he hea is highligh ed
wi h ou s uc u es, as we can see in Figu e 1, speci ically wo a ia ( igh and le sides each) in he uppe
pa o he s uc u e and wo en icles ( igh and le ) in he lowe pa . Each chambe has a unique
unc ion in he pumping p ocess, and ci cula ing blood, which is essen ial o he ca diac cycle. The hea ’s
chambe s p oceed in wo s ages, con ac ing and elaxing, which p o ide con ol o he blood low inside
he hea . Du ing sys ole, he a ia and en icles con ac , which p opels blood in o essels, and hen in
he dias ole, he chambe s elax and a e illed again wi h blood in an icipa ion o ano he pump [1].
To ensu e ha oxygen and nu ien s a e dis ibu ed o e e y cell in he body, he ca dio ascula cycle
pumps blood wi h low oxygen sa u a ion om he body o he Righ A ium (RA), whe e i ’s pumped
o he RV and hen o he lungs ia he pulmona y a e y. On he o he hand, i pumps blood wi h high
oxygen sa u a ion om he lungs o he Le A ium (LA), whe e i is pumped o he LV and hen expelled
5
o he body h ough he ao a [2].
Figu e 1: Hea ana omy. Adap ed om [2].
The p ope unc ioning o he hea depends on i s muscle walls. The walls a e made o h ee laye s
which wo k oge he o con ac he hea ’s muscles. The endoca dium (inne mos laye ) gua an ees a
smoo h blood low su ace. The epica dium (ou e laye ) p o ec s and p o ides s uc u al suppo . The
myoca dium is in be ween he epica dium and endoca dium, he hick middle laye ha makes up he
hea muscle [2].
Ca diomyocy es a e he specialized cells ha compose his muscle, he blood low and p essu e in he
ci cula o y sys em a e main ained by hem. Ca diomyocy es a e ubula s uc u es composed o chains o
myo ib ils, od-like uni s wi hin he cell. These myo ib ils co espond o epea ed sec ions o sa come es,
which a e he undamen al con ac ile ac o s o muscle cells. Myosin and ac in a e p o eins ha o m
myo ilamen s, hick and hin ibe s ha a e con ained wi hin he sa come es. The sliding o myo ilamen s
du ing muscle comp ession and elaxa ion is in e media ed by he elease o calcium om he sa coplasmic
e iculum, igge ing exci a ion-comp ession coupling [3].
Oxygen and nu ien s acili a e blood low o he myoca dium ia he co ona y a e ies showing ha
he ca diac cycle is c ucial. Any al e a ion in co ona y ascula iza ion o hea muscle unc ion can induce
a ange o clinical condi ions, simila o ischemia o in lamma o y ca diomyopa hy, dec easing ca diac
unc ion and po en ially leading o hea ailu e [3].
2.1.2 Pa hophysiology o Fib osis
A signi ican his ological aspec , such as ca diac ib osis is equen ly obse ed in hea condi ions, whe e
he e is an ini ial build-up o Ex acellula Ma ix P o eins (ECMs). This p ocess a ec s he hea ’s
6
se ings using medical images ha a e usually de ined by pixel in ensi y alues. In his scena io, he goal
is o gene a e a nume ic code as he ou pu , which signi ies he iden i ica ion and classi ica ion o di e en
issues, speci ic pa hological ea u es, o condi ions wi hin he medical images [14, 17].
2.2.1.3 C oss-Valida ion
To assess and compa e lea ning algo i hms i ’s used a echnique called c oss- alida ion, o spli he da ase
in o mul iple subse s: one o he pa s is mean o aining and he o he o alida ing i . The main concep
is ha du ing ounds o c oss- alida ion i e a ions, all subse s a e used o bo h aining and alida ion a
di e en i e a ions, ensu ing each da a poin is included in a alida ion se a leas once. Valida ion h ough
c oss- alida ion becomes pa icula ly c ucial in si ua ions whe e he da ase is limi ed in size and he e’s
a isk o o e i ing since i o e s an assessmen o a model’s e ec i eness by gua an eeing e e y da a
poin is used, in bo h aining and alida ion phases [18, 19].
Va ious echniques can be used in alida ion, bu he ocus will be cen e ed on he mos common one,
K-Fold C oss-Valida ion, whose pu pose is o segmen a da ase in o ksec ions, known as olds o simila
sizes, o aining and alida ion pu poses. Fo example, conside ing a 3- old alida ion p ocess, he da a
is di ided in o h ee pa s, his way he model is ained h ee imes in o al. One po ion is se aside o
alida ion pu poses and he wo le ou a e used o aining sessions, in each aining session as pa o
his p ocess. When in each i e a ion, a old o he da a is held ou o alida ion, he o he emaining (k-1
olds) a e used o lea ning. Once all he ounds a e inished and done wi h he model e alua ion p ocess
o each segmen , he model is moni o ed using a p ede ined measu e like an accu acy sco e. By aking
an a e age o gene a e a gene al pe o mance analysis o he model he pe o mance indica o s om
e e y segmen a e combined. This me hod assis s in gua an eeing ha he model’s e iciency emains
s able and is no excessi ely elian on one speci ic pa o he da ase [18]. C oss- alida ion se es no
only he pu pose o assessing he e ec i eness and applicabili y o a ained model wi h one algo i hm bu
also e alua ing di e en lea ning algo i hms o a ia ions o a pa ame e ized model o make compa isons
easie . These compa isons a e done by analyzing pe o mance me ics ac oss a ious olds in he da ase ,
o iden i y he mos op imal algo i hm o model a ia ion o a pa icula da ase [18, 19].
2.2.2 Deep Lea ning
Wi hin he b oade ca ego y o DL,ML exempli ies a ans o ma i e app oach in he e a o he Fou h In-
dus ial Re olu ion. Deep Neu al Ne wo ks a e he basic a chi ec u e inside DL, which ep esen s complex
a chi ec u es ha can ans o m aw da a in o abs ac and e en hie a chical ep esen a ions, allowing ma-
13

chines o pe o m in insically complex asks such as de ec ion and classi ica ion wi h comple e au onomy.
Unlike adi ional p og amming pa adigms, in which algo i hms a e hand-coded o wo k wi h speci ic da a
s uc u es, DL shines wi h i s abili y o ep esen a ion lea ning a mul iple le els. A a low le el, we ha e
hings like pa ches o edges and ex u es and so on, all he way o class labels in an image a a highe le el,
whe e each ”neu on” compu es sophis ica ed non-linea unc ions, allowing i o coope a e wi h neighbo -
ing neu ons o make up simple dis o ions coded by simple pa e ns. DL signi icance lies in combining
hese changes, in e lea ing hem, and eaching he machine o lea n mons ously non-linea ope a ions
la gely by i sel , gi en a sui able skele on. This au oma ion is pa icula ly use ul when i comes o pa e n
ecogni ion in high dimensions, as we would ha e wi h speech ecogni ion o image classi ica ion [20, 14].
Thus, DL has played a i al and i eplaceable ole in ad ancing algo i hms o p ocessing medical im-
ages, e idenced by signi ican success in se e al asks, mainly in he de ec ion and diagnosis o diseases.
The main applica ions o his ield in medical image analysis include classi ica ion, de ec ion, segmen-
a ion, and o he asks associa ed wi h image p ocessing [21]. To be mo e p ecise, his p ocess (up an
image ha has meaning in o meaning ul objec s) is called image segmen a ion. Gene ally speaking, i
is di ided in o wo ypes: seman ic segmen a ion and ins ance segmen a ion. The o me aims o ha e
a ca ego y label class o each pixel; The la e in ends o ca e ou exac en i ies, o example, lesions.
In medical image analysis, seman ic segmen a ion is usually e ec i e, in mos cases using powe ul DL
algo i hms like CNN [22].
In addi ion o he p e iously men ioned model, se e al ad ances in he ield o medical image analysis
include app oaches such as T ans o me s, U-Ne , FCN, 3D CNN, and ANN, among o he s. Each o hese
ep esen a ions cons i u es a unique app oach o speci ic challenges [14].
2.2.2.1 A i icial Neu al Ne wo k
The compu e models o DL, also known as ANN, illus a ed in Figu e 5, is an abs ac model ha mimics
he s uc u e and unc ion o he human b ain, consis ing o a se o a i icial neu ons connec ed by
weigh s. They ha e inpu nodes ha deli e in o ma ion and analogous nodes in an ou pu laye ha
p ocess his in o ma ion, modi ied by an ac i a ion unc ion. ANN a e s uc u ally designed in laye s and
can ha e in e media y laye s (an addi ional laye called a hidden laye ) depending on he complexi y o
he asks. These models a e ained o adjus he weigh s o connec ions be ween neu ons o op imized
pe o mance, usually by compa ing he ne wo k ou pu wi h he ue esul s and upda ing he weigh s ia
an op imiza ion algo i hm [23, 14].
14
Feed o wa d Neu al Ne wo k In eg a ing he componen s discussed ea lie , Feed o wa d Neu al
Ne wo ks (FNN) may ei he be classi ied as simple (i hey don´ ha e hidden/in e media e laye s) o
deep (wi h one o mo e hidden laye s). The idea o hese hidden laye s is ha hey lea n mo e complex,
hie a chical ep esen a ions o da a which hen allows he model o cap u e much mo e in ica e non-linea
ela ionships. In FNN, as he name sugges s, he in o ma ion mo es only in one di ec ion, om he inpu
laye h ough any hidden laye s o he ou pu laye , wi hou ha ing eedback connec ions. When eedback
connec ions a e inco po a ed, he a chi ec u e is called Recu en Neu al Ne wo ks (RNN).Mul i-
Laye Pe cep on (MLP) is one o he mos common deep FNN a chi ec u es and can be used o sol e
complex p oblems and lea n abs ac da a ep esen a ions [14].
Figu e 5: Illus a ion o an a i icial neu al ne wo k example, speci ically a Deep Feed o wa d Ne wo k,
cha ac e ized by he p esence o a hidden laye . Adap ed om [24]
To un a el he in ica e ope a ion o an ANN, is impo an o explo e he undamen al concep o he
Pe cep on. This singula compu a ional model ep esen s he mos elemen a y o m o a FNN, composed
o a single laye . As depic ed in Figu e 6, he pe cep on will ecei e minpu s, symbolized by a ec o
x∈Rn. Subsequen ly, i weigh s each inpu indi idually, using a ec o o weigh s w0, w1, w2, . . . , wn,
adding hem oge he and submi ing he sum o a non-linea unc ion known as he ac i a ion unc ion
σ. The esul o his p ocess is he gene a ion o he ou pu y∈R. I his sum exceeds a de ined limi ,
he neu on is ac i a ed; o he wise, i emains inac i e. Fu he mo e, an addi ional elemen , known as
bias θ, can be inco po a ed as a cons an added o he weigh ed sum o he inpu s, p o iding an a ine
ans o ma ion p ope y o he model [25, 14]. In gene al, he model is exp essed by
y=σ(n
∑
i=1
wixi+θ)
15
Figu e 6: Rep esen a ion o basic componen s o a pe cep on: inpu nodes, weigh s, summa ion, and an
ac i a ion unc ion. Adap ed om [26].
Backp opaga ion Algo i hm Deep FNN uses he backp opaga ion algo i hm, which is in ima ely
ela ed o a loss unc ion and op imiza ion me hod. They do his by compu ing he g adien s necessa y
o adjus ne wo k weigh s h ough a di ec ie-in be ween he ac ual loss unc ion and ake ca e o he
occu ence o how e o p opaga es om he ou pu laye o hidden laye s. The loss unc ion measu es
he di e ence, be ween he model p edic ions and he eal labels o help adjus he weigh s. A he ime
he op imiza ion echnique de e mines how hese g adien s a e used o upda e he ne wo k pa ame e s
du ing aining. This calcula ion o g adien s happens locally o make su e each node only conside s i s
connec ions[15].
Ac i a ion Func ion Ac i a ion unc ions, σ, in oduce nonlinea i ies in he ou pu s o neu ons by
applying hem o he sum o inpu s and hei espec i e weigh s wi hin each neu on’s p oduc e ms. I
allows ne wo ks o be lexible and unde s and pa e ns e icien ly by using well-known ac i a ion unc ions
such, as he sigmoid unc ion (2.1), Tangen Hype bolic (TanH) (2.2)) and Rec i ied Linea Uni
(ReLU) (2.3)).
σ(x) = 1
1 + e−x,(2.1)
σ(x) = ex−e−x
ex+e−x,(2.2)
σ(x) = max(0, x),(2.3)
In Figu e 7 i is possible o obse e a compa ison be ween he di e en ac i a ion unc ions discussed.
ReLU, e icien and simple, e u ns ze o o nega i e alues and he espec i e inpu o posi i e alues.
16
Sigmoid p oduces ou pu s be ween 0 and 1, use ul in bina y classi ica ion, and TanH gene a es ou pu s
be ween -1 and 1, being use ul when he a e age o he inpu s is ele an , al hough he la e wo ace
g adien ading challenges [27, 14].
Figu e 7: Compa a i e analysis o h ee ac i a ion unc ions, namely Sigmoid (2.1), TanH (2.2) and ReLU
(2.3). Adap ed om [28].
2.2.2.2 Con olu ional Neu al Ne wo ks
CNN in he cons an ly changing wo ld o DL is ai ly new and has been implemen ed as a concep o un-
de s and isual da a. The success ul i s appea ance o ANN in demons a ing hei lea ning capabili ies
d o e u he explo a ion in he analysis o complex isual da a which inally ga e ise o CNN o ackle
hese asks. CNN a e cha ac e ized as being a speci ic kind o eed o wa d ne wo k, inco po a ing ze o
o mo e con olu ional laye s and succeeding pooling laye s. These no el cons uc ional modules enable
CNN models o ha e local insigh in o he da a and s uc u es o he ac ual inpu . This p ope y makes
hem sui able o such applica ions ilms o images. Usually, as illus a ed in Figu e 8 he s uc u e o
CNN can be buil s ep by s ep. The ini ial s age ocuses on ea u e ex ac ion, accomplished h ough
con olu ional and pooling laye s. Subsequen ly, he second s age, co esponding o he concluding laye s
o he CNN, is cons i u ed o ully connec ed laye s, also known as dense laye s.
17
Figu e 8: Illus a ion o a con olu ional neu al ne wo k ea u ing wo con olu ional laye s, each succeeded
by pooling, and wo ully connec ed laye s. Typically, each con olu ional laye is pai ed wi h ba ch no mal-
iza ion and an ac i a ion unc ion o enhanced pe o mance
Con olu ion Laye s Con olu ional laye s mo e il e s, also known as ke nels, ac oss he image da a
piece by piece, ex ac ing ea u es in o ano he ma ix a each posi ion o he inpu ma ix (Figu e 9). The
inpu image can be g ayscale wi h black and whi e pixels, o i may ha e mul iple channels o colo ed
images, wi h each g id elemen ep esen ing a ea u e ec o . A ke nel impac s only 1 channel o ea u e
map in ou pu and has bias associa ed wi h i . A each poin , he ea u e map is gene a ed by compu ing
he do p oduc be ween he il e and he co esponding image pa ch, based on hei dimension This
build p ocess con inues un il a il e spans o e an image so ha mul iple ea u e maps can be buil a
once. The ini ial inpu could be he image i sel , o he i s laye , and i could be ea u e maps o p e ious
con olu ion laye s o subsequen deepe laye s [25, 14].
Applying linea ans o ma ions like ReLU, Sigmoid, and TanH unc ion is essen ial in making he
ne wo k mo e complex a e each con olu ion s ep in o de o i o lea n e ec i ely om he inpu da a
and cap u e in ica e pa e ns au onomously o imp o ed isual ep esen a ion, wi hin he ne wo k.
Figu e 9: Con ol ing a 3 × 3 ke nel o e a single channel inpu . Adap ed om [29].
18

An indispensable cha ac e is ic in he implemen a ion o any CNN is i s capaci y o implici ly ze o-pad
he inpu o expand i s wid h. This ea u e becomes c ucial as, wi hou i , he wid h o he ep esen a ion
would dec ease by one pixel less han he ke nel wid h a each laye . Ze o-padding he inpu p o ides con ol
o e he ke nel wid h and he ou pu size independen ly. In he absence o ze o-padding, he choices a e
cons ained o ei he apidly sh inking he spa ial ex en o he ne wo k o eso ing o small ke nels—bo h
scena ios se e ely es ic ing he exp essi e powe o he ne wo k [14]. Th ee common padding op ions
a e alid, same, and ull, as can be obse ed in Figu e 10.
”Valid” con olu ion, compu a ions occu only whe e he inpu ully con ains he ke nel, esul ing in a
sh inking ou pu a each laye . I he inpu image has a wid h o mand he ke nel wid h is k, he ou pu
wid h becomes m−k+ 1. Each ou pu pixel is a ec ed by he same numbe o inpu pixels, esul ing in a
mo e uni o m beha io . The same con olu ion ype applies ze o padding uni o mly on e e y side so ha
he ou pu con olu ion ea u e map size will be he same as he inpu ea u e map dimensions. This does
no es ic he numbe o con olu ional laye s bu adds some so o a i icial in o ma ion a ound bo de s.
Fu he , inpu pixels close o he bo de ha e less e ec on he ou pu pixel compa ed o hose ha a e
nea he cen e , and hence hese may no be ep esen ed adequa ely in he case o bo de pixels. On he
o he hand, a ully con olu ional laye includes enough padding o allow each pixel o be isi ed k imes
in each di ec ion, so he esul ing wid h o he ou pu is m+k−1. Pixels nea he bo de ecei e inpu
om ewe pixels compa ed wi h hose nea he cen e , making i di icul o ain a ke nel ha wo ks well
o e e yone. The bes amoun o ze o-padding, as measu ed by imp o emen s in es se classi ica ion
accu acy, ypically lies somewhe e be ween alid and same [14].
Polling Laye s The pooling laye s sha ed in CNN models igh a e he con olu ion laye s help o
adjus he ea u es ob ained. These laye s o m a ype o spa ial agg ega ion, which eplaces each pa
o he ea u e map wi h one s a is ic calcula ed in he pooling egion. In Figu e 11 i can be no iced ha
me hods like max pooling o a e age pooling a e used o dec ease he size o he da a dimensions by
summa izing and downsampling he in o ma ion e ec i ely h ough spa ial agg ega ion echniques such,
as selec ing he maximum o a e age alue om a se o adjacen pixels. Doing so makes he ne wo k’s
wo kload ligh e and enhances i s capaci y o adap mo e b oadly o a ious si ua ions. In he i s laye s
o con olu ional p ocessing lies a skill in iden i ying edges and colo - ela ed cha ac e is ics while when
mo ing in o he ne wo k’s laye s, he e is a endency o combine hese ini ial ai s o o m mo e de ailed
quali ies, like designs and in ica e objec s [25, 14].
19
Figu e 10: Compa ing padding echniques in con olu ional neu al ne wo ks: isualizing ”Valid,” ”Same,”
and ”Full” padding. Adap ed om [30].
Figu e 11: Compa ing pooling ope a ions in con olu ional neu al ne wo ks: Max Pooling ( op) and a e age
pooling (bo om). Adap ed om [31].
Regula iza ion O Neu al Ne wo ks E o s o imp o e e iciency and gene aliza ion in neu al ne -
wo ks in ol e implemen ing undamen al op imiza ion s a egies, wo no able ones being Ba ch no mal-
iza ion (BN) and D opou .
BN is loca ed be ween he con olu ional laye s and he ac i a ion unc ions, and i no malizes he
ea u e maps ha a e gene a ed du ing aining. This me hod s anda dizes he inpu dis ibu ion o ha e
a mean o 0 and uni a iance. This will speed up he lea ning p ocess since he model easily eaches a
local minimum. BN help o s abilize aining, leading o imp o ed pe o mance on he ask. Meanwhile,
as an opposi e s a egy o a oid o e i ing, when he model is oo adjus ed o he aining da a and
20
pe o ms wo se in new da a, he e is a d opou . D opou andomly masks neu ons du ing aining. The
pa icula o m o andomness in oduced educes he dependence be ween neu ons which ac s as a o m
o egula iza ion. D opou assis s he neu al ne wo k o expand mo e by no allowing i o lean owa ds any
speci ic inpu aining se cha ac e is ics. This will also esul in a s onge gene aliza ion p o iding be e
capabili y o he model when classi ying unseen da a [25, 14].
Fully-connec ed Laye The mos impo an kind o laye in a CNN du ing he classi ica ion p ocess is
he dense laye s also called ully connec ed laye s. These laye s eme ge a he end o CNN a chi ec u e,
a e he ea u e ex ac ion om con olu ion laye s and pooling laye s. A his poin in he p ocess, he
ou pu ea u e maps a e la ened in o a 1D a ay o ec o o numbe s and hen combined wi h dense
laye s. Then his one-dimensional a ay is a ached o one o mo e dense laye s which apply a linea ans-
o ma ion. E e y elemen in he ec o co esponds o a neu on in he dense laye , and each connec ion
om he ec o o he neu on in he dense laye has a lea nable weigh du ing he aining o he ne wo k.
A his s age, he linea ans o ma ion is ollowed by a non-linea ac i a ion unc ion [32, 14].
2.2.2.3 Fully Con olu ional Ne wo ks
Used in ML, seman ic segmen a ion is a compu a ionally expensi e ask ha wo ks owa ds de eloping
DL models o compu e ision. CNN ha e been es ablished as a e y powe ul ool o pe o m image
classi ica ion due o hei capabili y o lea ning hie a chical ea u es om da a, howe e , when pixel-le el
p edic ions and seman ic segmen a ion a e conce ned, he inhe en design limi a ions s ill lie he e, o
example, p ese a ion o spa ial in o ma ion.
The FCN in Figu e 12 explici ly deno es a new a chi ec u e pa adigm beyond he adi ional con ines
o CNN and can o e o push seman ic segmen a ion o new heigh s wi h inc edibly simple builds. FCN
di e om CNN by no con aining any o he ully connec ed laye s as hei las laye . FCN ake ad an age
o con olu ional laye s in he en i e ne wo k o he smoo h handling o spa ial in o ma ion and o gene a e
pixel-le el p edic ions. As a esul , his new ype o me hodology which de eloped FCN is g oundb eaking
as i allows he ne wo k o ake in inpu s o di e en sizes which means ha hese ne wo ks can be lexible
and adap ed o many ypes o image size scales. The FCN a chi ec u e inco po a es a downsampling
pa h (encode ) o decoding ich con ex ollowed by an upsampling pa h (decode ) o he eco e y o
he o iginal image esolu ion. An FCN is no hing mo e han a CNN wi h a e y special s uc u e. In
pa icula , he ne wo k connec s he encode and decode using skip connec ions, allowing in o ma ion o
be ans e ed om one pa h o ano he , he eby signi ican ly imp o ing he spa ial localiza ion o a ce ain
21
ea u e a he global map le el. Since he ou pu s o hese ne wo ks a e pixel-le el p edic ions, hey a e
ideal o segmen a ion asks [32].
Figu e 12: A chi ec u e o a ully con olu ional ne wo k. Adap ed om [32].
2.2.2.4 U-Ne A chi ec u e
The U-Ne a chi ec u e was i s p oposed by Ronnebe ge e al. [33]. This o me was cus omized o
biomedical image segmen a ion. He e, he a chi ec u e consis s o h ee p incipal pa s: an encode , a
bo leneck module, and a decode . The mos ob ious cha ac e is ic is i s “U” shape a chi ec u e which
consis s o a se o skip connec ions be ween he encode and decode pa hs o ensu e he p ese a ion
o spa ial de ails [34].
A ne wo k s uc u e is gi en in de ail in Figu e 13, which consis s o a con ac ing pa h (le side) and an
expansi e pa h ( igh side). The con ac ion pa h ollows he ypical a chi ec u e o a con olu ional ne wo k
which consis s o epe i ions con olu ions 3x3 (wi h padding) and ollowed by ReLU, ending in a 2x2 max-
pooling ope a ion ha downsamples he ea u e maps o hal i s esolu ion. In each downsampling s ep,
he numbe o ea u e channels g ows wice.
The expansi e pa h is cha ac e ized by an upsampling o he ea u e map, ollowed by a 2x2 con-
olu ion (”up-con olu ion”) ha educes he numbe o ea u e channels by hal . This is succeeded by
a conca ena ion wi h he co espondingly c opped ea u e map om he con ac ing pa h and wo 3x3
con olu ions, each ollowed by a ReLU. C opping is a echnique o make up o he loss o bo de pixels
du ing con olu ions. This is ed in o a sequen ial neu al ne wo k, whe e he las laye con ol es a 1x1
con olu ion wi h an ou pu leng h equal o he numbe o classes [33].
22
mensions o he p ojec ed que ies, keys, and alues o he k h head a e all deno ed by dk. Subsequen ly,
he esul o he ho izon al s ipes SA o he k h head is explained as:
X= [X1, X2, . . . , XM],
Yk
i=A en ion(XiWk
Q, XiWk
K, XiWk
V),
H-A en ionk(X) = [Yk
1, Y k
2, . . . , Y k
M]
Whe e Xi∈R(sw×W)×CeM=H
sw, i = 1, . . . , M.WQ
k∈RC×dk, WK
k∈RC×dk, WV
k∈
RC×dk ep esen he ma ices ha p ojec he que ies, keys, and alues o he k h head, espec i ely, and
dkis de ined as C
K. The SA mechanism wi h e ical s ipes can also be de i ed in a simila manne . I s
esul ing alue, o he k h head, is ep esen ed as V-A en ionk(X).
When he na u al images do no ha e any di ec ional bias p esen , he k-heads a e di ided in o wo
sepa a e g oups unning pa allel o each o he (wi h each g oup ha ing K/2 heads, mos imes K is an
e en numbe ). The i s se o heads ocuses on ho izon al s ipes SA while he second se concen a es
on e ical s ipes SA. In he end, he esul s om hese wo g oups a e conca ena ed back oge he o a
combined ou pu .
CSWin-A en ion(X) = Conca (head1, . . . , headK)WO(2.6)
headk=




H-A en ionk(X) o k= 1, . . . , K/2
V-A en ionk(X) o k=K/2 + 1, . . . , K
(2.7)
The ma ix WO, wi h dimensions RC×C, is ypically used o ans o m he SA ou comes in o he
desi ed ou pu dimension (de aul se as C). The CSWin mechanism imp o es e iciency wi h pa allel
MSA g ouping and dynamic s ipe wid hs, balancing accu acy and compu a ional cos .
Me hodologies in 3D Image Segmen a ion Two impo an a icles a e in e linked o lay he g ound-
wo k o he de elopmen o he echnique known as 3D medical image segmen a ion by using he U-
Shape Nes ed T ans o me (UNesT) app oach. This echnique builds upon he app oach de ailed
in a s udy w i en by Yu e al. [43] wi h an emphasis on imp o ing he iden i ica ion and sepa a ion o
ana omical componen s in he b ain, kidneys, and o he o gans om 3D medical scans, by in eg a ing
in o he a chi ec u e he nes ed hie a chical ans o me om Zhang e al. [44]. The main objec i e is o
p opose and alida e an app oach ha can pe o m his segmen a ion accu a ely and eliably, adap ing o
di e en da ase s, which include bo h adul s and child en and co e ing a ious imaging modali ies such
29

as Magne ic Resonance Imaging (MRI) and Compu ed Tomog aphy (CT). The UNesT model
aims o c ea e an a chi ec u e ha e icien ly cap u es in o ma ion bo h locally and globally in olume ic
medical images, acili a ing he p ecise segmen a ion o mul iple complex ana omical s uc u es. This is
made aking in o accoun wo main elemen s, which o m a hyb id design model. The i s , men ioned
in he a icle au ho ed by Zhang e . al. [44], consis s in he nes ed hie a chical ans o me cons uc ion
which se es as an encode , and he second elemen comp ises a con olu ion-based decode componen .
The mos in-dep h explana ion o he neu al ne wo k will be gi en in he me hodology sec ion since his
a chi ec u e will be inco po a ed in o he basis o he wo k.
Wi h ha said, se e al ana omical s uc u es we e segmen ed, anging om la ge o gans such as he
b ain and li e , o small s uc u es, such as glands o ce ain blood essels, ob aining be e esul s in
ce ain segmen a ions compa a i ely o he s a e-o - he-a . As o he limi a ions, as expec ed, smalle
s uc u es a e always mo e challenging o segmen and he e a e also di icul ies wi h i egula s uc u es
ac oss di e en pa ien s. Also, he Dice Simila i y Coe icien (DSC) alue migh no adequa ely
e lec segmen a ion accu acy due o he minimal con ibu ion o he small segmen s o he inal me ic.
2.2.3 Lea ning P ocess
2.2.3.1 Ini ializa ion
In DL se ups he e is an impo an s ep called ini ializa ion, whe e he ini ial pa ame e s o a neu al ne wo k
a e se up o make aining p ocesses smoo h and e ec i e. The p ima y objec i e is o b eak symme y
wi hin he ne wo k componen s so ha each pa can pe o m i s unc ion e ec i ely. To achie e his goal
a ious ini ializa ion echniques can be used o assign weigh s alues om dis ibu ions ha co e a ange
o possibili ies. One common echnique is called andom ini ializa ion, whe e weigh s ecei e alues om
dis ibu ions, wi h high en opy. In oducing andomness helps a oid epe i ion du ing he lea ning phase
by making su e ha each uni begins wi h se ings [14].
Selec ing he igh weigh ini ial scale is a e y impo an s ep in he ini ializa ion p ocess. In neu al
ne wo ks, usually ini ialize biases as ze o alongside weigh ini ializa ion. This is o p o ide a smoo h low o
in o ma ion ac oss he ne wo k by ying o o e come some common issues such as anishing o exploding
g adien s
Glo o and Bengio’s Ini ializa ion Glo o and Bengio’s no malized ini ializa ion aims o s ike a bal-
ance be ween he ac i a ion and g adien a iances ac oss laye s. The unde lying idea is o ini ialize
weigh s in such a way ha he signal nei he diminishes oo quickly no g ows oo apidly as i passes
30
h ough he ne wo k du ing o wa d and backwa d passes. Main aining a balance is c ucial, o ensu ing
e icien weigh adjus men s du ing he aining phase.
The p ocess o no maliza ion en ails modi ying he weigh scale a he s a acco ding o he inpu and
ou pu uni s in he laye s uc u e o he ne wo k o main ain a iance in ac i a ions and g adien s, and o
a oid p oblems linked wi h anishing o exploding g adien s based on ne wo k a chi ec u e conside a ions
[14].
2.2.3.2 Regula iza ion
To enhance he use ulness and adjus abili y o ne wo ks comes he ques ion o applying op imiza ion
echniques. One majo aspec o his is called egula ion, which ies o p e en o e i ing. Tha is he
si ua ion in which a model has been pa icula ly uncons ained o i he aining da a, meaning ha i can
no do well wi h new da a. Di e en egula ion me hods such, as L1 and L2 no ms, D opou , BN, and
Da a Augmen a ion play oles in a aining his objec i e.
L1 and L2 No ms The L1 no m o Lasso no m adds up he alues o weigh s, o he model’s loss
unc ion o penalize indi idual weigh s and p omo e some o each ze o p ecisely. This spa si y-inducing
ea u e makes L1 egula iza ion pa icula ly use ul when ea u e selec ion is desi ed, e ec i ely leading o
a simple model. On he side o hings, idge egula iza ion also known as he L2 no m, includes he o al
o squa ed weigh s, in he loss unc ion. This me hod punishes weigh s wi hou educing hem o ze o. I
a he encou ages weigh s o be smalle and mo e e enly dis ibu ed ac oss he model ea u es by u ilizing
L2 egula iza ion o p e en any ea u e, om o e powe ing he model.
The main in ui ion behind bo h L1 and L2 egula iza ion is o minimize he no ms. When ei he he
L1 o L2 no m is added o he loss unc ion in he aining phase, we in o m he op imiza ion p ocess no
jus o i da a bu also o keep model pa ame e s wi hin a speci ic ange. This egula iza ion e ec means
ha he model will gene alize well o unseen da a and helps p e en o e i ing om happening [45].
D opou D opou shown in Figu e 18 is a egula iza ion echnique in oduced by Hin on e al [46] and
is widely used o ackle o e i ing in Neu al Ne wo k Models. In p ac ice, i is done o p e en he model
om ge ing oo well adap ed on he aining da ase p e en ing ails o gene alize well on new, unseen
samples. D opou laye s injec andomness by s ochas ically deac i a ing neu ons in a laye o he aining
o any gi en ins ance. This me hod dis u bs he coo dina ion o ea u e de ec o s since he uni s ha a e
d opped ou do no ha e an impac , on he uni s ha a e e ained. Essen ially, d opou ac s as a o m
o egula iza ion by educing dependence be ween neu ons. Ano he way o looking a d opou is as an
31
e icien implemen a ion o model a e aging, whe e he ained models sha e all laye s excep he uni s
being d opped ou . En o cing his leads o he model no being oo g eedy o ce ain ea u es ou o he
aining se , esul ing in a mo e eliable gene aliza ion, which imp o e he classi ying o unseen da a [25,
14, 46].
Figu e 18: D opou isualiza ion: On he le , a egula neu al ne wo k wi hou d opou . On he igh ,
d opou is in ac ion. Adap ed om [47].
D op pa h D op pa h is a echnique o deep neu al ne wo ks, especially Residual Ne wo k (ResNe ),
ha in oduces andomness in o he ne wo k’s dep h du ing aining. Ins ead o using he ull dep h o
he ne wo k in e e y aining i e a ion, d op pa h andomly ac i a es only a subse o laye s, esul ing in
a shallowe ne wo k o each i e a ion. A each laye is calcula ed a p obabili y de e mined by Be noulli
andom a iables wi h a su i al p obabili y pl ha ep esen s ha laye . Thus, in each i e a ion, a an-
domly selec ed subse o laye s is ac i a ed, educing he dep h o he ne wo k o ha speci ic i e a ion.
This app oach means he ne wo k is ained wi h a andomly educed e sion o i sel , which lowe s com-
pu a ional cos s since only he ac i e laye s a e used o o wa d p opaga ion and g adien compu a ion.
Du ing in e ence, howe e , he ne wo k ope a es a i s ull dep h, u ilizing he comple e model’s capaci y.
In d op pa h, aining ime is dec eased since he e a e ewe compu a ions caused by he ac ion o inac-
i e laye s du ing aining. In doing so, his egula iza ion will also make he lea ning be e by imp o ing
i s gene aliza ion, he eby boos ing he pe o mance o his model. I also o e comes p oblems such as
anishing g adien and in o ma ion loss, making aining mo e s able [48, 49].
The p obabili ies o laye ac i a ion can be adjus ed uni o mly ac oss all laye s o in a dec easing
manne along he ne wo k’s dep h. In he linea decay mode, hese p obabili ies s a high o he ini ial
laye s and dec ease o deepe laye s. Du ing aining, he ne wo k is sampled based on hese su i al
p obabili ies. Du ing es ing, all laye s a e u ilized, and he ou pu s o he laye s a e calib a ed acco ding
o he su i al p obabili ies used du ing aining [48].
32
Ba ch No maliza ion Be ween laye s and ac i a ion unc ions, si BN, which is designed o s anda d-
ize he p oduced ea u e maps while aining is ongoing wi h he main goal o speeding up and s eadying
neu al ne wo ks h ough s anda dizing inpu s, wi hin each mini-ba ch o ackle p oblems linked o in e nal
co a ia e shi [50].
Du ing he aining o a ne wo k in a usual se ing da a is adjus ed o achie e consis en scales, o
be e adap abili y. BN akes i a s ep u he by wo king on ba ches as a whole o indi idual da a poin s.
This me hod ollows a wo-pa s a egy; ini ially s anda dizing inpu ea u es o ha e an a e age o ze o and
a de ia ion o one and hen adjus ing he scaled and o se no malized esul using lea ned alues known as
gamma (γ) and be a (β). When no malizing ne wo ks i is impo an o calcula e he a e age and a ia ion
o each ea u e in a ba ch o ensu e ha he hidden ac i a ions a e consis en and uni o m h oughou he
ne wo k laye s. Rescaling and adjus ing s eps a e included o gi e he ne wo k he abili y o handle pa e ns
while s ill bene i ing om he s abilizing impac o no maliza ion echniques. By enhancing con e gence
a es and educing eliance, on weigh ini ializa ions h ough BN echniques deep neu al ne wo ks can
achie e g ea e aining s abili y and imp o ed o e all pe o mance [51, 50].
BN has also been disco e ed o dec ease he necessi y o d opou . Can se e as a ype o egula -
iza ion making i an essen ial elemen , in con empo a y neu al ne wo k designs [52].
Laye no m Laye no maliza ion is a me hod employed o s abilize and speed up aining in ne wo ks
such as ans o me s. In con as o BN ha no malizes ac oss he ba ch dimension, laye no m no mal-
izes each da a poin indi idually by adjus ing i s ea u es. I ensu es ha he ea u es ha e an a e age o
ze o and a uni o m a iance ac oss he ea u e dimension, o e e y da a poin . This app oach wo ks well
o models ha ha e ba ch sizes o sequence leng hs. Laye no maliza ion helps by no malizing ea u es
wi hin each da a poin and hence inc eases an explo a o y ac o . This will help in making he lea ning
p ocedu e mo e consis en as well and also makes pe o mance o same model cons an ac oss di e en
sizes. This me hod is especially use ul, in ans o me models o handle inpu scaling and imp o e aining
s abili y and con e gence [53].
Da a Augmen a ion Da a augmen a ion is abou expanding da ase s a i icially by making mo e da a
poin s based on he ones we al eady ha e in hand ei he by weaking he exis ing da a a bi o using
ad anced ools, like Gene a i e Ad e sa ial Ne wo ks (GANs) and au oencode s, o p oduce esh
da a poin s [54].
Many echniques all in o he ca ego y o da a augmen a ion. These me hods include changing he
shape and he size o objec s o images, al e ing colo ones and using ke nel il e s. Mo e common
33
p ac ices such as o a ions o lipping, c opping along wi h noise in oduc ion also can also be used.
The use o da a augmen a ion in me hods ha in eg a e gene a i e models is c ucial in da ase s such as
medical images, whe e ob aining la ge and a ied da ase s can be challenging. Da a augmen a ion no
only helps o enla ge he da ase s du ing aining bu hey a e also conside ed powe ul me hods agains
o e i ing, which imp o e ML models in gene al. I can also apply da a augmen a ion echniques du ing
aining, u ilizing ools like PyTo ch, which enables easy modi ica ions wi hou he need o physically expand
he da ase [55].
To e ine da ase s, echniques such as GANs and au oencode s, which a e p ima ily used in medical
imaging, c ea e da a poin s by syn hesizing images ha closely esemble he ac ual da ase . Howe e ,
GANs encoun e challenges such as aining di icul ies and mode collapse, which can es ic he a ie y
o gene a ed samples. On he o he hand, Va ia ional Au oencode s (VAEs) p oduce a wide a ay o
ou comes, al hough his can occasionally lead o a dec ease in image quali y [56, 55].
Op imiza ion In essence, op imiza ion is he p ocess o slowly adjus ing neu al ne wo k pa ame e s
(weigh s and biases) o minimize ou loss unc ion as much as possible. We need o y o ind which
weigh s cause he a ious neu ons wi hin ou neu al ne wo k cell (laye s) o pe o m op imally conce ning
some ask we ca e abou , such as classi ica ion o eg ession. Fo his p oblem, op imiza ion algo i hms
a e esponsible o deciding how hese pa ame e s should be upda ed du ing he aining phase [57].
To make he neu al ne wo k aining mos e ec i e, we should be me iculous and speci ic abou wha
equi emen s a e depending on which op imize s. The e o e, many op imize s y o upda e he model
pa ame e s in di e en ways when i ains such as s ochas ic g adien (Adam) and s ochas ic G adien
Descen (GD) wi h momen um. The e icacy o some pa icula me hod can a y wi h he da a, size o
aining se and cons uc ed ne wo k a chi ec u e.
Adam The Adam op imize algo i hm is a well-known me hod o aining ML models. By adjus ing
lea ning a es, i makes aining mo e e icien and smoo he han i has e e been be o e. I is supe io o
s anda d S ochas ic G adien Descen (SGD) in se e al aspec s. One key imp o emen ha I would
like o men ion is his. I uses di e en lea ning a es o indi idual pa ame e s a he han one single a e
ac oss he en i e ne wo k, which leads o some g ea di idends. I combines ideas om AdaG ad and
RMSP op [58] whe e AdaG ad adjus s a es based on pa ame e g adien s, o spa se da a handling while
RMSP op adjus s a es by a e aging g adien s o adap o changing condi ions. Adams’s e ec i eness
comes om using bo h subsequen momen calcula ions, he o me o ack ends o e ime and he
la e o adjus he lea ning a e i sel , imp o ing compu a ional speed and educing memo y usage in
34

eal-wo ld scena ios. Adam se s pa ame e s and momen es ima ions a he s a o aining and hen
upda es hese es ima ions based on g adien s as aining p og esses o co ec any biases [59].
Al hough Adam adjus s he lea ning a e o each pa ame e based on g adien s, i s pe o mance can
be in luenced by he ini ial lea ning a e. In his case, a wa m-up s a egy can be used, whe e a lowe
lea ning a e is de ined a he s a and i will be inc eased linea ly o a highe alue, a e eaching his
peak, is applied a decay s a egy. This s a egy mi iga es he p oblem o inding a good minimum and he
conce n o s a ing wi h a low o high lea ning a e [60].
AdamW The AdamWeigh Decay (AdamW) is a e sion o he Adam op imize ha inco po a es
sepa a ed weigh decay. This me hod sepa a es L2 egula iza ion om he op imiza ion p ocess in ela ion
o he loss unc ion, so ha weigh decay is uni o mly applied ega dless o he loss unc ion o op imiza ion
s ep aken. In con as o Adam in which L2 egula iza ion di ec ly a ec s g adien upda es, AdamW
applies weigh decay o he ne wo k weigh s ega dless o whe he a g adien upda e was pe o med
o no . This adjus men pe mi s Adam o e ain i s p ope ies and adds he abili y o egula ize. The
e iciency o he op imize depends on how i ocuses on eaching a balance be ween con e gence and
gene aliza ion by egula ing he minimiza ion o loss a he han di ec ly inco po a ing egula iza ion in o
pa ame e upda es, as Adam does [61].
S ochas ic G adien Descen wi h Momen um GD is a e y impo an pa in he ML, bu his
me hod o calcula ing g adien s o e he whole da ase can ge qui e compu a ionally hea y, so o ec i y
ha p oblem, SGD was in oduced. This inno a i e me hod ede ines how we upda e he weigh s. Unlike
he GD echnique, SGD upda es he weigh s e e y ime a aining example is used. Wi h his me hod,
no only can da a be p ocessed mo e quickly, bu he g adien es ima e will also ha e some andom noise
added o i . This noise may a imes, u n he aining in o a chao ic expe ience, unhelp ul o p ac ical
easons, and uns able a empha ically o he imes.
SGD wi h momen um imp o es he algo i hms by adop ing he physics momen um concep and com-
p ises wo impo an componen s: eloci y and ic ion. Since eloci y is a mo ing a e age o g adien s
o e ime, i can be concep ually hough o as an indica o poin ing in he di ec ion whe e he g adien
should go. F ic ion, on he o he hand, is a a e-limi ing ac o o how as you can decay eloci y. SGD
wi h momen um has one main ad an age, i wo ks o elimina e he oscilla ions ypically seen wi h pu e
SGD. This algo i hm encou ages model con e gence by educing apid a e luc ua ions and main aining a
consis en di ec ion. The physical cha ac e is ics o momen um in oduced he e no only a oid undesi able
oscilla ions bu also esul in as e and mo e s able con e gence. [62].
35
2.2.3.3 Lea ning Ra e Schedule s
Lea ning Ra e Schedule s a e elemen s in he aining o ML models ocusing p ima ily on op imiza ion
algo i hms like GD. Thei main pu pose is o adap he lea ning a e h oughou aining o imp o e model
con e gence and pe o mance o e all by egula ing he size o pa ame e upda es in algo i hms such, as
descen deno ed by he symbol η. I can be qui e challenging o choose he lea ning ac o as se ing i
oo high may cause o e shoo ing and con e gence p oblems while se ing i oo low can esul in slow
con e gence o ge ing s uck in local minimum poin s on he lea ning cu e. This Hype pa ame e helps
ackle hese di icul ies by inco po a ing adap abili y and enabling aining, h oughou di e en phases.
Di e en me hods exis o adjus ing his hype pa ame e in ML models, one app oach in ol es using
Lea ning Ra e Decay schedules such as ime-based decay, s ep decay, and exponen ial decay o educe
he lea ning ac o based upon a se schedule and he o he me hod is adap i e lea ning a e me hods like
adam and RMSP op which adjus he hype pa ame e based upon g adien in o ma ion o op imiza ion
p ocesses [63, 60, 64].
36
Chap e 3
S a e o he A
3.1 Segmen a ion o Medical Diagnos ics
MRI image segmen a ion, is a undamen al ea u e in e alua ing a pa ien ’s heal h s a us, no only be-
cause i ecognizes s uc u es like o gans and issues bu also anomalies, because hey a e essen ial
o heal hca e p o essionals o ack a pa ien ’s si ua ion accu a ely and plan ea men s and diagnoses
e ec i ely. In he science wo ld and medical p ac ice, he e a e ypically h ee ypes o segmen a ion
me hods known and used. Au oma ic, semi-au oma ic, and manual app oaches. These me hods a e o -
en employed in wo o h ee dimensions o segmen images. While 2D segmen a ion is widely a o ed
and aluable, o pinpoin ing ana omical ai s, 3D segmen a ion o e s u he dep h pa icula ly when i
comes o analyzing olumes and g asping he spa ial ela ionships among s uc u es. The choice be ween
2D and 3D segmen a ion depends on se e al ac o s such as he in icacy o he ana omy in ol ed in he
analysis p ocess and he speci ic needs o analysis along wi h he compu a ional capabili ies a hand.
Bo h segmen a ion, in 2D and 3D space, can be pe o med ei he manually o , h ough compu a ional
me hods based on he indi idual scena ios and p e e ences o medical p o essionals [65].
P o essionals wi h app op ia e aining, like adiologis s o specialized physicians, will manually pe -
o m his segmen ing wo k. This ca ies a high ope a ion cos [66]. This can be done ei he in 2D in he
x–y plane, o slice by slice. Al hough manual segmen a ion is ime-consuming and subjec o obse e
a iabili y, i does allow expe in o ma ion o be inco po a ed. While manual segmen a ion is conside ed
he me hod used in his ield o s udy, i can cause inconsis encies be ween di e en obse e s o en
esul ing in signi ican disc epancies, in he da a ob ained. Au oma ed segmen a ion in ol es using algo-
i hms o di ide MRI images in o segmen s consis en ly, howe e , i may lack he nuanced unde s anding
ha human specialis s o e in his p ocess o c ea ing and adap ing algo i hms ac oss di e en imaging
37
modali ies which poses a challenge, in au oma ed segmen a ion [67]. The semi-au oma ic segmen a ion
sys em is he las segmen a ion me hod ha combines he inpu o humans collec i ely wi h au oma ed
p ocesses o imp o e speed and epea abili y compa ed o manual me hods. I o en equi es human
assis ance o se ing segmen a ion pa ame e s o e alua ion, p omo ing accu acy and e iciency, which
is c ucial in heal hca e se ings [68].
3.2 Segmen a ion in Ca diac Magne ic Resonance Images
Ca diac segmen a ion is c i ical in ecognizing nec o ic a eas o he hea , such as p e-abla ion ib osis in
he LA, pos -abla ion sca ing, and LV in a c ion. Depending on hei loca ion and size, hese egions migh
subs an ially impac he pa ien ’s ca dio ascula heal h. Fo ins ance, en icula sca s indica e p e ious
MI. Fib osis and sca localiza ion and quan i ica ion can assis wi h he apy s a i ica ion in condi ions
such as a ial ib illa ion o en icula achyca dia, help di ec su ge y o abla ion p ocedu es, and assess
ea men esponse.
A la ge numbe o algo i hms ha e been p oposed o ca diac sca ing segmen a ion, some o which
eached conside able pe o mance le els in challenges. O me hods used o sca segmen a ion, he 2-SD
[69] and Full Wid h a Hal Maximum (FWHM) [70] echniques a e ecommended because o hei
accu acy and ep oducibili y. The e is an inc eased in e es in au oma ing he segmen a ion o LA and LV
ana omy om LGE CMR images. The de elopmen o a i icial in elligence, in pa icula DL algo i hms
such as Fully Connec ed Neu al Ne wo ks (FCNN) and unc ional models like U-Ne has b ough a lo
o hope o he segmen a ion o ca diac subs uc u e. This makes i pe ec ly sui ed o sca segmen a ion
on LGE CMR images, making i possible o c ea e ully au oma ed solu ions. In eg a ion o LGE CMR
in o co- egis e ed images om o he imaging modali ies may also imp o e segmen a ion [71].
3.2.1 Au oma ic Algo i hm
Au oma ed echniques, pa icula ly hose based on DL, ha e eme ged as he gold s anda d o au oma -
ing ca diac a ea segmen a ion, ou pe o ming ea lie me hods and demons a ing he abili y o duplica e
expe analyses. While hese compu a ional me hods, pa icula ly hose based on DL, con inue o show
p omise o segmen a ion, o e ing a po en ially accu a e and e icien al e na i e o manual p ocesses,
ully au oma ed me hods o ca diac segmen a ion ha e ye o achie e he equi ed le el o accu acy o
compe e wi h expe segmen a ion.
In a s udy done by Radau e al.[72], hey examined he Sunnyb ook Ca diac Da a [73] o analyze a
38
4.2.3 Neu al Ne wo k
As we can see in Figu e 19, he neu al ne wo k model selec ed o segmen he h ee ana omical com-
ponen s (LV,RV,LV myoca dium) was a U-ne 2D. Fo inpu , as desc ibed ea lie , pa ches 2D wi h
co- egis e ed MRI image sequences (bSSFP,LGE and T2) we e gi en, and in he end, a segmen a ion
mask was ob ained, accu a ely delinea ing he egions o in e es o each ana omical componen .
Figu e 19: U-Ne 2D a chi ec u e used o segmen a ion ask
Following his, a pos -p ocessing s ep was applied o e ain only he la ges connec ed componen o
each o he h ee segmen ed classes, e ec i ely emo ing small isola ed ma kings and mino connec ed
componen s. Tha said, he numbe o ou pu channels (Cou ) was 3, co esponding o he LV,RV, and
LV myoca dium.
4.3 Second S age: Segmen a ion o Ana omical S uc u es
In he second s age, he inpu is mo e complex. We use he MRI image sequences (bSSFP,LGE e T2),
he LV myoca dium mask ob ained om he i s s age, and wo new images, he LGE-myo and T2-myo,
all conca ena ed in one o o m he inal inpu . The mask om s age one helps di ec he model’s a en ion
o speci ic egions by limi ing he sea ch ield o sca s and edema, ensu ing ha he segmen a ion o his
45

pa hology is mo e p ecise by minimizing hese possible loca ions. Addi ionally, new images we e c ea ed
by mul iplying he p obabili y o he LV in he myoca dium and he espec i e sequences, o ming he
LGE-myo and T2-myo, in oducing c ucial de ails since hese sequences p o ide impo an in o ma ion on
myoca dial, sca and edema espec i ely.
4.3.1 P e-P ocessing
En e ing he second s age, simila o he app oach aken in he i s phase o imp o e he aining model,
we s a wi h c opping, his ime aking in o ca e he segmen a ion esul s om he i s phase and ocusing
on educing alse posi i es. The model inpu beyond he h ee MRI image sequences (bSSFP,LGE, T2),
now also includes he segmen a ion p obabili y mask o he LV (P ob_1) in he myoca dium and wo new
images, LGE-myo and T2-myo, o ming a 6-channel inpu . Since he model equi es a ixed inpu size
ha is di isible by 16, so he images a e ze o-padded o a size o 6×256×256×N, whe e N ep esen s he
numbe o slices, which a ies depending on he subjec .
Conside ing ha he e a e s ill a ia ions, among subjec s, no maliza ion will be in oduced a each
channel, esul ing in a ze o a e age and a s anda d de ia ion o one, which imp o es he model as de-
sc ibed ea ly in phase one. Ano he me hod used o imp o e he images was augmen a ion. This echnique
acili a es he expansion o he da ase and in oduces a iabili y by inco po a ing lips and o a ions. The
o a ions we e pe o med in wo ways: i s , by se ing he angle o 90°, 180°, o 270°, and second, by
using angles anging om -30° o 30°.
To conclude, du ing he aining phase o his p ocess we ex ac 2D slices, om he 3D images o use
as inpu and when i comes o es ing he ne wo k, i ecei es all N slices om each image a e e e sing
he esizing ope a ion.
4.3.2 Neu al Ne wo k
The ne wo k a chi ec u e used in his wo k is based on he hyb id UNesT model, which combines wo
ypes o s uc u es: he Hie a chical T ans o me as an encode and a con olu ional ne wo k as a decode .
In he encode s uc u e, he e a e h ee s ages, wi h each le el con aining wo blocks and wo p ocesses.
The blocki y p ocess demons a ed in Figu e 21b comes i s , hen is ollowed by he ans o me encode ,
hen he deblocki y p ocess, and inally he pooling block. In he las phase o he hie a chy, he pooling
block is no p esen anymo e. Ins ead, he ou pu o he deblocki y unde goes no maliza ion, ollowed by
a con olu ion, esul ing in he bo leneck ou pu . F om he bo leneck, he decode , based on he U-Ne
s uc u e, p ocesses he ea u es h ough con olu ional laye s, p og essi ely es o ing he spa ial esolu ion
46
and combining mul i-scale in o ma ion o gene a e he inal ou pu wi h 3 channels (Cou ), co esponding
o he edema, myoca dium sca , and backg ound.
Figu e 20: Re iewing he ou line o he sugges ed UNesT, wi h he hie a chical ans o me encode and
a con olu ion-based decode .
4.3.2.1 UnesT Encode
As o he encode in he neu al ne wo k in Figu e 21a, he inpu o ou ne wo k will consis o images
o size H×W×C(heigh H, wid h W, and channels C) ha will be ans o med in o 2D pa ches
o size S×S. A e his ans o ma ion, each one o he pa ches will be eshaped and hen p ojec ed
in o embeddings in RC0using a pa ch p ojec ion laye . The ollowing s age, known as blocki y, o ganizes
he embeddings in o non-o e lapping blocks (each block ep esen s a subse o he 2D pa ches), which
a e hen la ened, esul ing in a enso X∈RB×Tn×n×C0, whe e Bis he ba ch size, Tnis he o al
numbe o blocks in he cu en hie a chy, nis he sequence leng h (numbe o embeddings) in each block,
and C0is he embedding dimension. No ably, Tn×n=H×W
S2. The model employs h ee hie a chical
le els, a each hie a chical le el, 2D blocks a e p ocessed sepa a ely h ough ans o me laye s, allowing
o mul i-scale p ocessing, whe e each le el o he hie a chy handles blocks o di e en esolu ions. The
ans o me laye comp ises a no maliza ion block wi h connec ions, a CSWin SA mechanism, and an
MLP.
The nex s ep, called Deblocki y, uses an agg ega ion unc ion esponsible o conca ena ing he ou -
47
comes om he non-o e lapping 2D blocks, o c ea e he ou pu o he ans o me ( e e ed o as Hie a chy
Ou pu ), esul ing in a seamless olume ic ea u e map. A e his, he ou pu goes h ough a pooling block
which in ol es a max pooling, a 3x3 con olu ion laye wi h a s ide o 1, and a no maliza ion laye . The max
pooling laye wi h a ke nel size o 3×3 downsamples ac oss he dimensions by a ac o o 2×2 (s ide o
2). The ea u e map ha is p oduced is hen ans e ed o he succeeding hie a chical le el, o addi ional
p ocessing, which enables he model o e ec i ely ga he and combine in o ma ion om a ious scales
and dep hs, allowing i o manage in ica e ela ionships, in he da a e ec i ely.
Mo e in de ail, he ea u es maps s a a dimensions H
S×W
S×C0and a e educed by hal in size a
he s a o each new hie a chy le el. The ea u e map unde goes pooling by a 2 ac o in bo h di ec ions
(2×2) wi hou al e ing he sequence leng h o he embeddings (n) esul ing in a dec ease in he numbe
o blocks (Tn) a e e y hie a chy le el by 4. The s uc u e consis s o h ee hie a chical le els in o al, wi h
16 blocks in he i s , 4 blocks in he second le el, and a single block in he hi d le el. The numbe o
ans o me laye s (dep h) a e de ined as 2,2,8 espec i ely om he la ges o smalles esolu ions, and
he dimensions o espec i e embeddings (wid h) a e se up by 64,128,256 wi h sizes gene a ed a e e y
s age om W
2×2i×W
2×2i×Cwhe e i= 0,1,2and C= 64,128,256. A he encode o ou model,
we u ilized a pa ch size o 2×2and o he embedding dimension was de ined a alue o 64, leading o
ea u e maps size o H
2×W
2×64 a e pa ch p ojec ion.
48
Figu e 21: Visualiza ion o he UNesT a chi ec u e wi h he hie a chical ans o me encode (a) and he
a chi ec u e o he Nes ed T ans o me (b).
4.3.2.2 UNesT Decode
Upon eaching he inal ou pu s age, he Decode s eps in o ca y ou i s asks wi hin a U-shaped a chi-
ec u e. The ea u e map om he las hie a chical le el unde goes a no maliza ion and passes h ough a
3×3 con olu ional laye o c ea e an enhanced ans o me encode ou pu . Subsequen ly, his ou pu is
u ilized o gene a e a bo leneck ea u e map ha is upsampled h ough he u iliza ion o a anspose con o-
lu ional laye . In he U-shaped design layou , he upsampled bo leneck ea u e map is conca ena ed wi h
ea u e maps om he espec i e le els o he encode using skip connec ions. These connec ions help
inco po a e in o ma ion om di e en scales, acili a ing he econs uc ion and ob aining a segmen ed
and de ailed esul . In he i s applied skip connec ion, he e is an excep ion, gi en ha ou a chi ec u e
bo leneck conce ns he ans o ma ion o he laye ’s ou pu h ough a no maliza ion laye and a con olu-
ional laye . I is an icipa ed ha he seman ic signi icance will be simila o he las hie a chy ou pu . Tha
said, he esidual block will no be inco po a ed in he skip connec ion o he las hie a chy. Following he
conca ena ion, he combined ea u e maps go h ough a block denomina ed ResBlocks, which includes a
3×3 con olu ional laye and a no maliza ion applied he ea e . This block imp o es he combined ea u es
by add essing he di e ences be ween he encode and decode ep esen a ions. Once he ResBlocks a e
49
a e sed h ough, he cycle con inues and he combined ea u e map unde goes upsampling o he nex
le el, h ough he anspose con olu ion laye . Be o e p oceeding o he nex s age, om now on, he
ou pu om he co esponding encode hie a chy passes h ough a skip connec ion ha inco po a es a
ResBlock, jus a e ha , he ea u e map om he encode is conca ena ed wi h he ea u es om he
decode . A e he usion, unde goes u he e inemen wi h ResBlocks and he ea u e map is upsam-
pled again. This happens a a ious le els inside he Decode . A leas , he e ined and p ocessed ea u e
map, om he las ResBlock, now en iched wi h di e se mul i-scale and mul i-dep h de ails is employed o
p oduce he inal segmen a ion mask. To comple e his ask success ully, a con olu ional laye wi h a size
o 1x1 and a so max ac i a ion unc ion is used on he ea u e map o p oduce he inal segmen a ion
ou come. This ou pu allows us o ob ain he da a ou lined wi h he di e en s uc u es and issues ele an
o he image.
4.3.3 New App oach in he T ans o me Encode
ViT, like ViT and DeiT use global a en ion bu i can be compu a ionally in ensi e because i scales
quad a ically wi h he numbe o pa ches. Recen models, like he Swin-T ha e implemen ed local a en-
ion mechanisms based on windows o imp o e e iciency. The downside in ol es he lack o connec ions,
be ween dis an pa ches in he image. Simila ly, he CSWin T ans o me , which employs CSWin, com-
pu es local SA in ho izon al and e ical s ipes simul aneously, lacking a en ion o ea u e-space locali y.
To o e come his limi a ion, a new app oach was in oduced by Yu e al. [86], he Bila e al Local A en-
ion (BLA) block a en ion, which combines local a en ion in he image space wi h local a en ion in he
ea u e space. Taking his in o accoun , o imp o e ou model pe o mance, he ans o me encode will
be adjus ed o u ilize he BLA block. This block combines wo modules, he Image-Space A en ion
(ISLA) adap ed wi h he CSWin SA and a de eloped Fea u e-Space Local A en ion (FSLA) mod-
ule. In addi ion, i also includes an MLP module and se e al laye no maliza ion (LN) modules wi hin he
block.
In his mechanism, he inpu se o okens will be deno ed as Tin ={ i}N
i=1, whe e i∈RC,Cis
he numbe o channels, and Nis he numbe o okens. A i s , he inpu okens a e p ocessed h ough
a no maliza ion laye and nex by an ISLA,), a sho cu connec ion is applied as ollows:
TISLA =Tin +ISLA(Laye No maliza ion(Tin))
Local a en ion in image space only calcula es SA be ween okens wi hin he same pa ch. The esul om
he ISLA componen , TISLA, is inpu ed in o a no maliza ion laye be o e en e ing a ea u e space local
50

a en ion module known as FSLA. This module includes a connec ion whe e:
TFSLA =TISLA +FSLA(LN(TISLA)).
The FSLA module calcula es SA be ween okens ha a e p oxima e in he ea u e space and wo ks
alongside he ISLA module. Fo las , he ou pu o he FSLA module, TFSLA, is p ocessed h ough ano he
no maliza ion laye and an MLP module o p oduce he inal esul o he BLA Block:
Tou =TFSLA +MLP(LN(TFSLA)).
The MLP componen includes wo ully connec ed laye s. The i s laye inc eases he ea u e dimension
om C o ×C, while he second laye educes i back o C. By de aul , is se o 4.
Now del ing deepe in o he explana ion o how a en ion wo ks, beginning wi h he ea u e-space local
a en ion. FSLA in ol es wo key elemen s. One is he balanced hie a chical clus e ing algo i hm (Figu e
22), an app oach ha o e comes he imbalance p oblem o K-means clus e ing. This me hod pe o ms
Kle els o clus e ing on he inpu okens se T={ i}N
i=1. In he i s s age o clus e ing he N okens,
he se Tis di ided in o wo subse s con aining N/2 okens each. A he k- h s age o clus e ing, he
N/2k−1 okens ha ha e been alloca ed o he subse in he p eceding le el a e spli in o wo subse s
wi h a size o N/2keach. In he end, he esul is 2Ke enly sized subse s, {Ti}2K
i=1, and each subse
size |Ti|is equal o N/2K.
Figu e 22: Hie a chical clus e ing scena io whe e he e a e a o al o 3 hie a chical le els wi h 23= 8
clus e s, a he lowes le el. Adap ed om [86].
The o he key elemen , execu ed a each le el o clus e ing, is balanced bina y clus e ing. The ISLA
di ides a se o 2m okens, { i}2m
i=1, in o wo clus e s, each o size m. I equi es bo h g oups o okens o
ha e he same size. The cen oids o each one a e labeled as c1and c2. Each oken iis e alua ed using
51
a dis ance a io, i, o de e mine i s clus e membe ship:
i=s( i, c1)
s( i, c2),∀i∈[1,2m].
The o mula s(x, y)is used by he algo i hm o de e mine he cosine simila i y, be ween okens xand
y. A e so ing he 2m okens in dec easing o de based on hei dis ance a ios { i}2m
i=1, clus e C1is
comp ised o he i s hal o he lis , and clus e C2consis s o he second hal , whe e he size o each
clus e is ep esen ed as m. The a e age o okens in each clus e is u ilized o upda e hei cen oids
and clus e membe ship o e e y sample esul ing in upda es owa ds op imized clus e ing simila , o K
means.
In adi ional balanced bina y clus e ing, he esul ing clus e s, C1and C2, do no sha e any okens,
indica ed by C1∩C2=∅. The lack o o e lap in his s uc u e could cause issues o okens posi ioned
in he middle o he o ganized lis . Fo example, a oken wi hin his sec ion migh ha e ea u e-space
neighbo s, in one g oup clus e being placed in ano he clus e i sel making a en ion calcula ions less
e icien . To add ess his p oblem e ec i ely, he esea che s ha e in oduced a me hod called o e lapping
bina y clus e ing. This app oach in ol es assigning he i s m+n okens om he so ed lis o one clus e ,
ˆ
C1={ ji}m+n
i=1 , and he las m+n okens o he second clus e , ˆ
C2={ ji}2m
i=m−n+1. This c ea es an
o e lap o 2n okens, ep esen ed as ˆ
C1∩ˆ
C2={ ji}m+n
i=m−n+1. The o e lap enables cen al okens o s ay
connec ed wi h some o hei ea u e-space neighbo s, which imp o es communica ion when compu ing
a en ion sco es. The sugges ed hie a chical clus e ing app oach adop s o e lapping bina y clus e ing a
he las le el o he p oposed balanced hie a chical clus e ing and uses he non-o e lapping e sion a he
o he le els. In he a chi ec u e ou lined, n alue is de e mined based on 40 pe cen o he pa ch size
(block size) which enhances he da a adap abili y in o e lapping bina y clus e ing scena ios.
Wi h his balanced hie a chical clus e ing om abo e, he se o okens Tis di ided o be he ensemble
o 2Ksubse s {Ti}2K
i=1, whe e he size o each subse is deno ed by |Ti|=N/2K. Wi hin each subse ,
i is applied he s anda d SA:
ˆ
Tk=SA(Tk),∀k∈[1,2K].
The ou pu , ˆ
Tis de ined as he union o all a ended subse s:
ˆ
T=∪
k∈[1,K]
ˆ
Tk
In his app oach, he MSA con igu a ion is adop ed, which is a undamen al ea u e o ans o me
models, o implemen FSLA. A his con igu a ion, is pe o med s anda d SA bu also applied balanced
52
hie a chical clus e ing independen ly in each a en ion head. This s a egy allows di e en heads o ocus
on di e en okens o a gi en inpu oken.
To in oduce he BLA in o he p oposed algo i hm, as we can see in Figu e 23, he adap ed T ans o me
Encode o he UNesT a chi ec u e (23a) was changed o he BLA T ans o me Encode (23b).
(a)
(b)
Figu e 23: T ans o me encode used in UNesT adap ed wi h CSWin block a en ion (a) and ans o me
encode adap ed wi h BLA block a en ion (b).
53
Chap e 5
Resul s and Discussion
This chap e p esen s he esul s o he segmen a ion o ana omical s uc u es ob ained in he i s phase
and he segmen a ion o sca s and edema in he second phase. The esul s we e e alua ed by he DSC
and al hough he i s s age did no unde go changes om he baseline wo k, his ou pu may in luence he
pe o mance o he model in he second s age. The esul s we e compa ed wi h me hodologies add essed
in he s a e o he a , allowing o unde s and he posi ion o he implemen ed model and e i y whe he i
ob ained compe i i e esul s in he con ex o he segmen a ion o ca diac lesions.
5.1 Model pe o mance
To assess he p ecision o he segmen a ion esul s, o sca s and edema sepa a ely, he DSC was com-
pu ed as ollows:
Dice(V eal, VGD) = 2|V eal ∩VGD|
|V eal|+|VGD|(5.1)
whe e VGD indica es he gold s anda d segmen a ion while V eal ep esen s he segmen a ion ob ained
using he algo i hm.
5.2 P o ocol adop ed
Du ing he esea ch conduc ed o his p ojec , we used a 5 c oss- alida ion app oach and calcula ed he
a e age DSC. Ou o he 25 pa ien s, in ou aining g oup, 20 cases we e designa ed o aining and 5
cases o alida ion, in each old.
A e comple ing he aining p ocedu e o each old, he c i e ia used o ob ain he inal ne wo k o
each old consis ed o selec ing he epoch whose pe o mance me ic o alida ion was he maximum.
54
mask ob ained om he i s s age, and wo new images, he LGE-myo and T2-myo. These wo channels
allow he model o ob ain mo e p ecise in o ma ion abou he edema, in he T1 sequence, and he ib ous
issue, in he LGE sequence. The a chi ec u e used in his phase was a UNesT a ian which inco po a es
a nes ed hie a chical ans o me as an encode adap ed wi h BLA and comp ises a con olu ion-based
decode .
Rega ding he esul s ob ained, he pe o mance achie ed did no ully mee ini ial expec a ions, al-
hough he model managed o posi ion i sel in hi d place, among he six p esen ed me hods ha didn’
use an ensemble in he segmen a ion ask. This sugges s ha he implemen ed a en ion mechanism,
al hough p omising, did no b ing signi ican bene i s in cap u ing de ailed in o ma ion om CMR images
compa ed o he CSWin model, possibly due o limi a ions in adap ing he model o he speci ic ype o
ana omical a iabili y p esen in he analyzed images.
6.2 P ospec o Fu u e Wo k
To comple e his wo k, a ious skills we e equi ed. Howe e , gi en ime cons ain s, i was no possible o
co e all aspec s needed o achie e op imal ou comes. As a esul o his, he e a e s ill se e al s a egies
a ailable o u he imp o e he p oposed app oach.
In u u e de elopmen s, an aspec ha can be in oduced consis s o ensemble s a egies implemen-
a ion o combine p edic ions om he bes models, he eby imp o ing he accu acy o ca diac pa hologies
segmen a ion and c ea ing a mo e eliable and e icien sys em. This combined p edic ion s a egy en-
hances pe o mance compa ed o using a single model.
The in oduc ion o he new a en ion mechanism did no s ee he model owa d mo e compe i i e
esul s, in ac , i had he opposi e e ec . The e o e, i is necessa y a deepe s udy o he a chi ec u e
pa ame e s o his mechanism o analyze all he de ails ha can be modi ied o imp o emen . Mo eo e ,
conside ing al e na i e a en ion mechanisms migh be ano he op ion i he adjus men s made o he
p oposed mechanism a e insu icien .
Fu he imp o emen s could be achie ed by e ining ne wo k pa ame e s o , mo e impo an ly, by al-
e ing he base a chi ec u e. This includes ying ou new loss unc ions, explo ing new ac i a ion unc ions,
and de ining al e na i e da a p e-p ocessing me hods.
Las ly, he amoun o da a p esen ed in he MyoPS da ase is limi ed, which in luences he esul s
ob ained by he model. In an a emp o sol e his p oblem, he p o ision o new da a would be highly bene-
icial, as i could a oid possible o e i ing issues, imp o e he dis ibu ion be ween he aining, alida ion,
61

and es se s, and p oduce be e segmen a ions.
62
Bibliog aphy
[1] Raheel Chaudh y, Julia H. Miao, and A zal Rehman. Physiology, Ca dio ascula (Oc . 2022).
[2]
Hea : Ana omy and Func ion
.
[3] An hony Sax on, Muhammad Ali Ta iq, and B uno Bo doni. Ana omy, Tho ax, Ca diac Muscle (Aug.
2023).
[4] Ping Kong, Panagio a Ch is ia, and Nikolaos G. F angogiannis. The pa hogenesis o ca diac ib osis.
71.4 (May 2013), pp. 549–574.
[5]
Myoca dial ib osis. a Heal hy myoca dium consis s o a compac ... | Download Scien i ic Diag am
.
[6] Yiling Si u e al. Ca dio ascula magne ic esonance imaging o s uc u al hea disease. 10.2 (Ap .
2020), pp. 36175–36375.
[7] Au elio Ma amo os and Ka hleen M. Schmele . Tumo s o he U e ine Co pus In oduc ion. (June
2012), pp. 423–439.
[8] R. J.M. Van Geuns e al. Basic p inciples o magne ic esonance imaging. 42.2 (Sep . 1999),
pp. 149–156.
[9] Yasuo Amano e al. Th ee-dimensional Ca diac MR Imaging: Rela ed Techniques and Clinical Appli-
ca ions. 16.3 (2017), p. 183.
[10] Daniel T. Gina e al. Ca diac imaging: Pa 1, MR pulse sequences, imaging planes, and basic
ana omy. 197.4 (Oc . 2011), pp. 808–815.
[11] Elizabe h R. Jenis a e al. Re isi ing how we pe o m la e gadolinium enhancemen CMR: insigh s
gleaned o e 25 yea s o clinical p ac ice. 25.1 (Dec. 2023), pp. 1–16.
[12]
home | MS-CMR Segmen a ion
.
63
[13] James A. Nichols, Hsien W. He be Chan, and Ma hew A. B. Bake . Machine lea ning: applica ions
o a i icial in elligence o imaging and diagnosis. 11.1 (Feb. 2019), pp. 111–118.
[14] Ian Good ellow, Yoshua Bengio, and Aa on Cou ille.
Deep lea ning
. MIT P ess, 2016.
[15] Ke in P Mu phy.
Machine Lea ning A P obabilis ic Pe spec i e
. MIT P ess, 2012.
[16] Fa had Maleki e al. Machine Lea ning Algo i hm Valida ion. 30 (No . 2020), pp. 433–445.
[17] Home o Schiabel Ana Ma ia Ma ques da Sil a Ana Cláudia Pa ocínio. P ocessamen o e Análise de
Imagens Médicas (2019).
[18] Payam Re aeilzadeh, Lei Tang, and Huan Liu. C oss-Valida ion (2009), pp. 532–538.
[19] D ew Wilimi is and Colin G Walsh. P ac ical Conside a ions and Applied Examples o C oss-Valida ion
o Model De elopmen and E alua ion in Heal h Ca e: Tu o ial. 2 (Dec. 2023), e49023.
[20] Rene Y. Choi e al. In oduc ion o Machine Lea ning, Neu al Ne wo ks, and Deep Lea ning. 9.2
(2020).
[21] Xuxin Chen e al. Recen ad ances and clinical applica ions o deep lea ning in medical image
analysis. 79 (July 2022), p. 102444.
[22] Dan Luo e al. Deep Lea ning o Au oma ic Image Segmen a ion in S oma ology and I s Clinical
Applica ion. 3 (Dec. 2021), p. 767836.
[23] Zhongheng Zhang. A gen le in oduc ion o a i icial neu al ne wo ks. 4.19 (Oc . 2016), pp. 370–
370.
[24] Mohammad Faqe Hussein, Azad Saeed, and So an Mohamad. Compa ison Ma ko Chain and
Neu al Ne wo k Models o o ecas ing Popula ion g ow h da a in I aq. 13 (Dec. 2023), pp. 1–14.
[25] Ma ibel To es-Velazquez e al. Applica ion and Cons uc ion o Deep Lea ning Ne wo ks in Medical
Imaging. 5.2 (Ma . 2021), p. 137.
[26] Danial Ka imi e al. Ene gy Managemen and Op imiza ion o La ge-Scale Elec ic Vehicle Cha ging
on he G id. 14.4 (Ap . 2023), p. 95.
[27]
Ac i a ion Func ions in Neu al Ne wo ks | by SAGAR SHARMA | Towa ds Da a Science
.
[28]
Hidden Laye Ac i a ion Func ions | by Kinde Chen | Medium
.
[29]
Anh H. Reynolds
.
[30]
Classi ying Images wi h Deep Con olu ional
.
64
[31]
Pooling laye ope a ion. Le a 4x4 ma ix inpu . Righ 2x2 pooling... | Download Scien i ic Diag am
.
[32] Rikiya Yamashi a e al. Con olu ional neu al ne wo ks: an o e iew and applica ion in adiology. 9.4
(Aug. 2018), pp. 611–629.
[33] Ola Ronnebe ge , Philipp Fische , and Thomas B ox. U-ne : Con olu ional ne wo ks o biomedical
image segmen a ion. 9351 (2015), pp. 234–241.
[34] Xiao Xia Yin e al. U-Ne -Based Medical Image Segmen a ion. 2022 (2022).
[35] Ch is ophe D. Hollande , S ee upa Das, and Su aiya Suliman. Au oma ing isual inspec ion wi h
con olu ional neu al ne wo ks. 11.1 (Sep . 2019).
[36] Ashish Vaswani e al. A en ion Is All You Need (Aug. 2023).
[37] Fahad Shamshad e al. T ans o me s in medical imaging: A su ey. 88 (Aug. 2023), p. 102802.
[38] Xiangxiang Chu e al. CONDITIONAL POSITIONAL ENCODINGS FOR VISION TRANSFORMERS (Feb.
2021).
[39] Li Yuan e al. Tokens- o-Token ViT: T aining Vision T ans o me s om Sc a ch on ImageNe (Jan.
2021), pp. 538–547.
[40] Wuyang Chen e al. A Simple Single-Scale Vision T ans o me o Objec De ec ion and Ins ance
Segmen a ion (Oc . 2022).
[41] Ze Liu e al. Swin T ans o me : Hie a chical Vision T ans o me using Shi ed Windows (Aug. 2021).
[42] Xiaoyi Dong e al.
CSWin T ans o me : A Gene al Vision T ans o me Backbone wi h C oss-Shaped
Windows
. 2022.
[43] Xin Yu e al.
UNesT: Local Spa ial Rep esen a ion Lea ning wi h Hie a chical T ans o me o E icien
Medical Segmen a ion
. 2023.
[44] Zizhao Zhang e al.
Nes ed Hie a chical T ans o me : Towa ds Accu a e, Da a-E icien and In e -
p e able Visual Unde s anding
. 2021.
[45]
Visualizing egula iza ion and he L1 and L2 no ms | by Chia a Campagnola | Towa ds Da a Sci-
ence
.
[46] G E Hin on e al. Imp o ing neu al ne wo ks by p e en ing co-adap a ion o ea u e de ec o s (July
2012).
[47]
D opou in (Deep) Machine lea ning | by Ama Budhi aja | Medium
.
[48] Gao Huang e al.
Deep Ne wo ks wi h S ochas ic Dep h
. 2016.
65
[49] Sou iane Hayou and Fadhel Ayed.
Regula iza ion in ResNe wi h S ochas ic Dep h
. 2021.
[50]
In oduc ion o Ba ch No maliza ion: Unde s anding he Basics
.
[51] Youngeun Kim and P iyada shini Panda. Re isi ing Ba ch No maliza ion o T aining Low-La ency
Deep Spiking Neu al Ne wo ks F om Sc a ch. 15 (Dec. 2021), p. 773954.
[52]
Ba ch No maliza ion and i s Ad an ages | by Ramji Balasub amanian | Medium
.
[53] Jimmy Lei Ba, Jamie Ryan Ki os, and Geo ey E. Hin on.
Laye No maliza ion
. 2016.
[54] Jakub Nalepa, Michal Ma cinkiewicz, and Michal Kawulok. Da a Augmen a ion o B ain-Tumo Seg-
men a ion: A Re iew. 13 (Dec. 2019).
[55] Cecilia Di Rube o e al. Deep Lea ning App oaches o Da a Augmen a ion in Medical Imaging: A
Re iew. 9.4 (Ap . 2023), p. 81.
[56] Conno Sho en and Taghi M. Khoshgo aa . A su ey on Image Da a Augmen a ion o Deep Lea n-
ing. 6.1 (Dec. 2019), pp. 1–48.
[57] Ume Waqas e al. Op imiza ion o neu al-ne wo k model using a me a-heu is ic algo i hm o he
es ima ion o dynamic Poisson’s a io o selec ed ock ypes. 13.1 (Dec. 2023), p. 11089.
[58] R. Elshamy e al. Imp o ing he e iciency o RMSP op op imize by u ilizing Nes e o in deep lea n-
ing. 13.1 (May 2023), p. 8814.
[59] Miaomiao Liu e al. An Imp o ed Adam Op imiza ion Algo i hm Combining Adap i e Coe icien s
and Composi e G adien s Based on Randomized Block Coo dina e Descen . 2023 (Jan. 2023),
pp. 1–14.
[60] Yichuan Shao e al. An Imp o emen o Adam Based on a Cyclic Exponen ial Decay Lea ning Ra e
and G adien No m Cons ain s. 13.9 (May 2024), p. 1778.
[61] Towa ds Unde s anding Con e gence and Gene aliza ion o AdamW (2024).
[62] Es aa Hassan e al. The e ec o choosing op imize algo i hms o imp o e compu e ision asks:
a compa a i e s udy. 82.11 (May 2023), pp. 16591–16633.
[63]
Lea ning Ra e Schedules and Adap i e Lea ning Ra e Me hods o Deep Lea ning | by Suki Lau |
Towa ds Da a Science
.
[64] Hideaki Iiduka. App op ia e Lea ning Ra es o Adap i e Lea ning Ra e Op imiza ion Algo i hms o
T aining Deep Neu al Ne wo ks. 52.12 (Dec. 2022), pp. 13250–13261.
[65] Ahmed Elnakib e al. Medical Image Segmen a ion: A B ie Su ey (2011), pp. 1–39.
66

[66] Loïc Muhi wa.
Image Segmen a ion in Medical Imaging
. 2022.
[67] Ma ijn P.A. S a mans e al. Radiomics (Jan. 2019), pp. 429–456.
[68] Ha i McG a h e al. Manual segmen a ion e sus semi-au oma ed segmen a ion o quan i ying
es ibula schwannoma olume on MRI. 15.9 (Sep . 2020), pp. 1445–1455.
[69] Ch is ophe M K ame e al. S anda dized ca dio ascula magne ic esonance (CMR) p o ocols
2013 upda e. 15.1 (2013), p. 91.
[70] And ew S. Fle e al. E alua ion o Techniques o he Quan i ica ion o Myoca dial Sca o Di e ing
E iology Using Ca diac Magne ic Resonance. 4.2 (2011), pp. 150–156.
[71] Yinzhe Wu e al. Recen Ad ances in Fib osis and Sca Segmen a ion F om Ca diac MRI: A S a e-
o - he-A Re iew and Fu u e Pe spec i es. 12 (Aug. 2021), p. 709230.
[72] Anupama Bhan, Pa hasa a hi Mangipudi, and Ayush Goyal. Deep Lea ning App oach o Au oma ic
Segmen a ion and Func ional Assessmen o LV in Ca diac MRI. 11.21 (No . 2022), p. 3594.
[73] P. Radau e al. E alua ion F amewo k o Algo i hms Segmen ing Sho Axis Ca diac MRI (2020).
[74] Seyed Sadegh Mohseni Salehi, Deniz E dogmus, and Ali Gholipou . T e sky loss unc ion o image
segmen a ion using 3D ully con olu ional deep ne wo ks. abs/1706.05721 (2017).
[75] Sa a Moccia e al. De elopmen and es ing o a deep lea ning-based s a egy o sca segmen a ion
on CMR-LGE images. 32.2 (Ap . 2019), pp. 187–195.
[76] Shangqi Gao e al.
BayeSeg: Bayesian Modeling o Medical Image Segmen a ion wi h In e p e able
Gene alizabili y
. 2023.
[77] Xinzhe Luo and Xiahai Zhuang.
X-Me ic: An N-Dimensional In o ma ion-Theo e ic F amewo k o
G oupwise Regis a ion and Deep Combined Compu ing
. No . 2022.
[78] Kai-Ni Wang e al.
AWSne : An Au o-weigh ed Supe ision A en ion Ne wo k o Myoca dial Sca
and Edema Segmen a ion in Mul i-sequence Ca diac Magne ic Resonance Images
. 2022.
[79] Junyi Qiu e al. MyoPS-Ne : Myoca dial pa hology segmen a ion wi h lexible combina ion o mul i-
sequence CMR images. 84 (2023), p. 102694.
[80] Weisheng Li, Linhong Wang, and Sheng Qin. “CMS-UNe : Ca diac mul i- ask segmen a ion in MRI
wi h a u-shaped ne wo k”.
Myoca dial Pa hology Segmen a ion Combining Mul i-Sequence Ca diac
Magne ic Resonance Images: Fi s Challenge, MyoPS 2020, Held in Conjunc ion wi h MICCAI 2020,
Lima, Pe u, Oc obe 4, 2020, P oceedings 1
. Sp inge . 2020, pp. 92–101.
67
[81] Weisheng Li, Linhong Wang, and Sheng Qin. “CMS-UNe : Ca diac Mul i- ask Segmen a ion in MRI
wi h a U-Shaped Ne wo k”.
Myoca dial Pa hology Segmen a ion Combining Mul i-Sequence Ca -
diac Magne ic Resonance Images
. Ed. by Xiahai Zhuang and Lei Li. Cham: Sp inge In e na ional
Publishing, 2020, pp. 92–101.
[82] Jianpeng Zhang e al. “E icien Seg: A Simple Bu E icien Solu ion o Myoca dial Pa hology Segmen-
a ion Challenge”.
Myoca dial Pa hology Segmen a ion Combining Mul i-Sequence Ca diac Mag-
ne ic Resonance Images
. Ed. by Xiahai Zhuang and Lei Li. Cham: Sp inge In e na ional Publishing,
2020, pp. 17–25.
[83] Feiyan Li and Weisheng Li. “Dual-pa h ea u e agg ega ion ne wo k combined mul i-laye usion
o myoca dial pa hology segmen a ion wi h mul i-sequence ca diac MR”.
Myoca dial Pa hology
Segmen a ion Combining Mul i-Sequence Ca diac Magne ic Resonance Images: Fi s Challenge,
MyoPS 2020, Held in Conjunc ion wi h MICCAI 2020, Lima, Pe u, Oc obe 4, 2020, P oceedings
1
. Sp inge . 2020, pp. 146–158.
[84] Elsa Fe ei a. “Segmen ação de Fib ose em Imagens de Ressonância Ca díaca”. Mes ado In e-
g ado em Engenha ia Biomédica, Ramo de Ele ónica Médica. T abalho o ien ado pelo P o esso
Dou o Ca los Albe o Sil a e Dou o Ad iano Pin o. Disse ação de Mes ado. B aga, Po ugal:
Uni e sidade do Minho, Dec. 2023.
[85] Richa d Conno e al. Co ela ions o C oss-En opy Loss in Machine Lea ning. 26.6 (2024).
[86] Tan Yu e al.
BOAT: Bila e al Local A en ion Vision T ans o me
. 2022.
68