Compu e s in Biology and Medicine 135 (2021) 104533
A ailable online 15 June 2021
0010-4825/© 2021 The Au ho (s). Published by Else ie L d. This is an open access a icle unde he CC BY-NC-ND license
(h p://c ea i ecommons.o g/licenses/by-nc-nd/4.0/).
Deep lea ning me hod o ao ic oo de ec ion
Pablo G. Tahoces
a
,
*
, Ra ael Va ela
b
, Jose M. Ca ei a
b
a
Depa men o Elec onics and Compu e Science, Uni e sidad de San iago de Compos ela, San iago de Compos ela, Spain
b
Complejo Hospi ala io Uni e si a io de San iago (CHUS), San iago de Compos ela, Spain
ARTICLE INFO
Keywo ds:
Compu ed omog aphy angiog aphy (CTA)
Ao ic oo
Vascula imaging
De ec ion
Landma ks
ABSTRACT
Backg ound: Compu ed omog aphy angiog aphy (CTA) is a p e e ed imaging echnique o a wide ange o
ascula diseases. Howe e , ex ensi e manual analysis is equi ed o de ec and iden i y se e al ana omical
landma ks o clinical applica ion. This s udy demons a es he easibili y o a ully au oma ic me hod o
de ec ing he ao ic oo , which is a key ana omical landma k in his ype o p ocedu e. The app oach is based on
he use o deep lea ning echniques ha a emp o mimic expe beha io .
Me hods: A o al o 69 CTA scans (39 o aining and 30 o alida ion) wi h di e en pa hology ypes we e
selec ed o ain he ne wo k. Fu he mo e, a o al o 71 CTA scans we e selec ed independen ly and applied as
he es se o assess hei pe o mance.
Resul s: The accu acy was e alua ed by compa ing he loca ions ma ked by he me hod wi h benchma k loca ions
(which we e manually ma ked by wo expe s). The in e obse e e o was 4.6 ±2.3 mm. On an a e age, he
di e ences be ween he loca ions ma ked by he wo expe s and hose de ec ed by he compu e we e 6.6 ±3.0
mm and 6.8 ±3.3 mm, espec i ely, when calcula ed using he es se .
Conclusions: F om an analysis o hese esul s, we can conclude ha he p oposed me hod based on p e- ained
CNN models can accu a ely de ec he ao ic oo in CTA images wi hou p io segmen a ion.
1. In oduc ion
Compu ed omog aphy (CT) is a well-es ablished 3D imaging mo-
dali y ha p o ides a map o a pa ien ’s ana omy. Key ad an ages o he
modali y a e he high spa ial esolu ion and high ela i e con as o he
images ob ained, he sho acquisi ion ime, he wide ield-o - iew, and
he easibili y o ob aining high-quali y h ee-dimensional mul iplana
econs uc ions. These ea u es make his modali y pa icula ly e ec i e
o p o iding accu a e in o ma ion on diseases ela ed o he
mo phology o mos ana omical s uc u es.
I is easible o ex ac p ecise and eliable in o ma ion on cha ac-
e is ics ela ed o leng h, diame e , olume, and o he ana omical pa-
ame e s om he slices acqui ed by his modali y [1]. These ea u es
a e quan i iable and can help in e alua ing no mali y o se e i y, deg ee
o a ia ion, o s a e o a disease o an inju y [2]. In addi ion, his
quan i a i e in o ma ion has se ed as a basis o he publica ion o
se e al clinical guidelines ela ed o di e en ypes o diseases including
ca dio ascula diseases [3]. The e o e, he de elopmen o new algo-
i hms ha can p o ide accu a e and ep oducible alues o ex ac ing
hese quan i a i e cha ac e is ics has become an impo an issue.
Wi h ega d o he ho acic ao a, CTA images ha e been widely
applied o es ablish he p esence o aneu ysms, dissec ions, mu al
h ombi, o elonga ions. These a e conside ed as main anomalies o be
ea ed in his pa o he ao a [4]. Howe e , i is di icul o de elop
au oma ic compu e ools ha use in o ma ion ex ac ed om CTA
images, o add ess such diseases. I is necessa y o de elop se e al
p eceding asks o 1) isola e he oxels o he ao a om he emainde
o he olume ha cons i u e he CTA (ao a segmen a ion) and 2) de ec
and ag se e al ana omical landma ks o egula use o ex ac ing
quan i a i e in o ma ion [5].
Di e en algo i hms ha e been p oposed o au oma ed ao ic seg-
men a ion based on CTA olume. In gene al, he segmen a ion p ocess
s a s wi h he au oma ic de ec ion o he ao a in one o he CTA slices.
The conside a ion o he i s slice ha is ela ed o an ana omical
landma k (such as he Ca ina [6] o he pulmona y unk [7]) o is
iden i iable by i s shape using he Hough ans o m [8,9], has been
es ablished o be a eliable solu ion o his ask. Ei he a s a ing poin
o an ini ial con ou is ex ac ed om his i s slice, and he segmen-
a ion p ocess s a s and con inues un il a p e-es ablished c i e ion e -
mina es i . Howe e , he implemen a ion o a e mina ion c i e ion ha
* Co esponding au ho .
E-mail add ess: [email p o ec ed] (P.G. Tahoces).
Con en s lis s a ailable a ScienceDi ec
Compu e s in Biology and Medicine
jou nal homepage: www.else ie .com/loca e/compbiomed
h ps://doi.o g/10.1016/j.compbiomed.2021.104533
Recei ed 12 Ma ch 2021; Recei ed in e ised o m 25 May 2021; Accep ed 25 May 2021
Compu e s in Biology and Medicine 135 (2021) 104533
2
is sui able o all ci cums ances is challenging. Mo e ecen ly, deep
lea ning solu ions ha e also been p oposed [10]. Ne e heless, in many
cases, he segmen a ion p ocess ails because leaks cause he p ocess o
ad ance inside he hea . Thus, an accu a e de ec ion o he ao ic oo
be o e segmen a ion can imp o e he pe o mance o segmen a ion
algo i hms.
Meanwhile, he de ec ion o he ao ic oo has been he mo i a ion
o he de elopmen o di e en me hods ela ed o i) he cha ac e -
iza ion o he ao a o assess he ela i e loca ion o a possible aneu ysm
[11,12] o analyze he s i ness o he ao ic a ch [13,14], ii) he
de ec ion and cha ac e iza ion o he co ona y a e ies o assess he
p esence o calcium deposi s [15,16], and iii) he ex ac ion o he di-
mensions o he ao ic oo p io o in e en ions [17]. The e o e, he
de ec ion o he ao ic oo is a undamen al issue o be conside ed while
de eloping compu e applica ions o ao ic diseases ei he o imp o e
segmen a ion o o asks ela ed o he ex ac ion o measu emen s o
clinical in e es .
2. Rela ed wo k
Manual and au oma ic de ec ion schemes ha e been p oposed in
di e en s udies o de ec ing ao ic oo s [18]. Howe e , he imple-
men a ion o au oma ic me hods has been gaining inc easing in e es in
he pas ew yea s [19]. The mo i a ion o his is he elimina ion o he
human ac o , which is ime-consuming and can yield esul s ha
in oduce in e obse e a ia ions [20].
In gene al, ECG-ga ed CT angiog aphy has been employed o de ec
he ao ic oo in p eope a i e scena ios ela ed o planning o ans-
ca he e ao ic al e in e en ion (TAVI) [21]. This modali y is highly
ecommended o use when p ecise measu emen s a e equi ed. This is
because i educes he p esence o mo ion a i ac s ha inc ease he
a iabili y o he measu emen s ob ained [22]. F om his pe spec i e,
Ela a e al. [5] p oposed a me hod o de ec ing he sino ubula junc-
ion and wo co ona y os ia in CTA images. Lalys e al. [23] p oposed a
me hod o au oma ically de ec ing ao ic lea le s and co ona y os ia
loca ions o de i e ana omical measu emen s om hese landma ks.
Meanwhile, he applica ion o he ao ic oo as a e e ence poin has
been p oposed o ex ac quan i a i e in o ma ion ela ed o he shape
and size o he ao a. Con en ional CTA (non-ECG-ga ed) o MRI is used
o his ask. Ku ugol e al. [24] and Tahoces e al. [25] p oposed ully
au oma ed pipelines o calcula ing he ao ic mo phology in la ge co-
ho s o CTA scans. He ein, se e al landma ks we e de ec ed au oma -
ically o ex ac ing ea u es ela ed o ao ic mo phology.
This s udy aimed o de elop a me hod o au oma ically de ec he
ao ic oo om images acqui ed om CTA s udies. Th ee main con-
s ain s we e imposed. Fi s , we conside ed cases in ol ing only he
ho acic ao a as well as hose in ol ing bo h ho acic and abdominal
ao a in he same scan olume. Thus, ECG non-ga ed cases, cons i u ed
he main a ge o ou app oach. Second, we did no pe o m p io
segmen a ion o he ao a. Thi d, he numbe o cases used o aining
was ela i ely small.
3. Da ase and me hods
3.1. Da ase
CTA scans o 140 cases wi h app oxima ely 79, 000 images we e used
o ain and es he p oposed scheme. Fo compa ison wi h p e ious
s udies [25], only 39 cases (16, 353 images) we e used o aining,
whe eas 30 cases (18, 747 images) we e used o alida ion. Fu he -
mo e, a e he aining was comple ed, he pe o mance o he p oposed
algo i hm was es ed on he emaining 71 cases (43, 501 images).
All he CTA examina ions we e pe o med on pa ien s om he
Complejo Hospi ala io Uni e si a io de San iago (CHUS) who we e
e e ed o CTA examina ion conside ing he clinical indica ions ha
hey displayed. The indica ions we e suspec ed ho acic ao ic disease o
ho acic ao ic disease wi h ex ension o he abdominal ao a, ollow-up
con ols o pa ien s diagnosed ea lie , and con ols o ea men s pe -
o med on pa ien s diagnosed and ea ed wi h endo ascula o open
su ge y. In all he cases, he ho acic ao a was included in he exami-
na ion. The abdominal ao a was also included when necessa y. The
da ase included 37 women and 103 men wi h an a e age age o 66
yea s ( anging om 27 o 89 yea s). The ollowing condi ions we e
imposed o he selec ion o cases: i) he ao ic oo should be isible and
co ec ly opaci ied by he con as agen in he CTA olume, and ii) he
slice hickness should be 0.625 mm.
The numbe o slices pe scan anged om 280 (minimum) o 1, 128
(maximum). This implies ha in ce ain cases, only he sec ion co e-
sponding o he ho acic ao a was scanned du ing he acquisi ion p o-
cess, whe eas in se e al o he cases, bo h he ho acic and abdominal
sec ions o he ao a we e scanned. In all he case, he ao ic sec ions
included in a s udy depended on he s udy’s clinical indica ions.
All he cases we e anno a ed by wo expe s, who independen ly
ma ked wo poin s on he image: i) he cen e o he ao ic oo , which is
loca ed in he sinus o Valsal a (SOV) and ii) a poin wi hin he
ascending ao a ha p o ides he o ien a ion o i s cen e line. The
anno a ion p ocess was pe o med using a able equipped wi h a ouch
sc een and ins alled wi h he ITK-SNAP p og am, and a digi al pen.
Anno a ion was pe o med in he co onal plane, al hough he axial and
sagi al iews we e simul aneously a ailable o he expe . The SOV
commissu e was conside ed as a e e ence o he posi ion o he cen e
o he ao a. (Fig. 1).
3.2. P oposed de ec ion scheme
Ou scheme a emp s o mimic he manne in which adiologis s
pe o m ao ic oo de ec ion. Fi s , he CTA axial slices we e ans-
o med in o co onal slices (Fig. 2a). This s ep makes i mo e con enien
o dis inguish he loca ion o he SOV. Then, he adiologis analyzed he
egion loca ed wi hin he ibcage and na iga ed h ough he co onal
slices un il he hea was isible (Fig. 2b). We call he Hea Box (HB) o
his ibcage sec ion. I includes he co onal slices whe e he hea is
isible. Then, he slices ha belonged o his olume we e analyzed in
de ail o loca e he cen e o he ao ic oo (Fig. 2c). Once loca ed, he
o ien a ion o he ascending ao a was de e mined. We call he plane
ha includes he cen e o he ao ic oo as SOV plane. I is no mal o he
ec o ollowing he cen e line o he ascending ao a.
The e o e, wi h ega d o he algo i hm’s design, he e a e h ee
main asks in he en i e p ocess: classi ica ion, de ec ion, and
Fig. 1. Co onal iew o a CT scan. The black c osses indica e he wo ma ks
inse ed by he expe , i.e., he commissu e o he SOV loca ion (x
0
, y
0
, z
0
) and a
poin on he ao a’s cen e line (x
1
, y
1
, z
1
). The dashed black lines indica e he
slopes o bo h he ascending ao a and he plane whe e he SOV is loca ed ha
a e calcula ed om he ma ks inse ed by he expe .
P.G. Tahoces e al.
Compu e s in Biology and Medicine 135 (2021) 104533
3
cha ac e iza ion. i) A classi ica ion ask is necessa y o ob ain axial
sec ions in which he p esence o he lungs is de ec ed. The eby, he
olume con aining he ibcage is isola ed ( ibcage bounding ex ac ion).
This elimina es he likelihood ha he axial slices belonging o he
emaining olume in e e e wi h he calcula ion in he subsequen s eps.
ii) A classi ica ion ask is also ca ied ou o ex ac he co onal slices
ha include he HB om he ibcage olume (HB bounding ex ac ion).
iii) A de ec ion p ocess is pe o med o de e mine he ao ic oo ’s
loca ion wi hin he HB (ao ic oo de ec ion). i ) Finally, p incipal
componen analysis (PCA) is pe o med o de e mine he o ien a ion o
he ascending ao a. The eby, he SOV plane can be calcula ed (ao ic
oo cha ac e iza ion). Fig. 3 shows he o e all scheme o he p oposed
me hod.
3.3. Classi ica ion: ex ac ion o ibcage and HB
We ained models based on con olu ional neu al ne wo k (CNN)
a chi ec u es o pe o m his ask. We applied h ee a chi ec u es
equen ly used in medical imaging o cons uc hese models: VGG,
ResNe , and Incep ion. To summa ize, each CNN is composed o wo
essen ial elemen s, namely, a con olu ional base (whe e a succession o
con olu ional il e s and pooling laye s a e de ined) and a classi ie
block (whe e all he nodes a e in e connec ed), which a e gene ally
cons uc ed h ough dense laye s. The con olu ional il e s ex ac he
undamen al cha ac e is ics o he images in e ms o shape and o ien-
a ion. The pooling laye s al e he size o he ex ac ed ea u es. An
addi ional laye ( la ened laye ) unc ions as an in e ace o adap he
o ma o he da a ob ained om he con olu ional base o he classi ie .
Fo a con olu ional base, each model de ines i s opology. VGG consis s
o one o mo e blocks ha a e composed o a succession o con olu ional
il e s and e mina ed by a pooling laye (Fig. 4a). ResNe is composed o
blocks o esidual connec ions, which p o ides he ou pu o a laye as an
inpu o a subsequen laye (Fig. 4b). The incep ion block consis s o
se e al il e s placed in o di e en pa allel b anches ha con e ge in o a
conca ena e laye (Fig. 4c).
We demons a ed h ee scena ios: i) aining om sc a ch (VGG16,
ResNe 50, and Incep ionV3), ii) use o p e- ained alues while main-
aining he con olu ional base and e aining he classi ie (VGG16
(p e), ResNe 50 (p e), and Incep ionV3 (p e)), and iii) use o p e- ained
alues while pa ially main aining he con olu ional base and e aining
he classi ie (VGG16 (p e, blk), ResNe 50 (p e, blk), and Incep ionV3
(p e, blk)). Models p e- ained on he ImageNe da ase we e used o
ans e lea ning in all he cases. We also applied h ee VGG a chi ec-
u es wi h di e en numbe s o con olu ional blocks (VGG1, VGG2, and
VGG3) ha we e ained om sc a ch.
To homogenize he inpu o he p oposed models, black and whi e
images o 128 ×128 we e used when he aining p ocess was pe -
o med om sc a ch. Howe e , o e ec i ely u ilize p e- ained models,
i is necessa y o accommoda e he inpu o each model’s equi emen s.
The e o e, we ans o med he inpu images o 224 ×224 colo images
o VGG16 and ResNe 50, and o 299 ×299 colo images o Incep-
ionV3. This was pe o med by adap ing he inpu laye s o he di e en
models o he equi emen s o each case.
We used he gene al me hod o ain he models. Fi s , we ained on
a aining da ase using he alida ion se o ine- une he pa ame e s.
Then, we used he es se o e alua e he pe o mance o he ained
model independen ly. The esul s ob ained om he es se we e used o
compa e and selec he bes models. The models we e cons uc ed in
Py hon using he Ke as lib a y wi h Tenso Flow (GPU e sion). Bina y
c oss-en opy was used o he loss unc ion in all he cases. A g adien
descen (wi h momen um) op imize was used o minimize he loss
unc ion using accu acy as a me ic. All he in e media e ac i a ion
laye s used ReLU as an ac i a ion unc ion. The ou pu laye used a
sigmoid unc ion o deli e alues in he ange [0, 1]. A o al o 20
epochs we e pe o med o each model, and he model ha achie ed he
highes accu acy in he alida ion se was sa ed o es ing. The lea ning
a e was main ained cons an h oughou he aining p ocess.
As men ioned abo e, classi ica ion was used o wo asks in he
o e all scheme: 1) Ribcage bounding ex ac ion o selec he slices ha
belong o he ibcage om he ull se o axial slices o he CTA scan and
2) HB bounding ex ac ion o selec he co onal slices ha include he
hea , om he se o co onal slices. The e o e, wo CNN ne wo ks we e
ained o pe o m hese asks.
To ain and e alua e he i s classi ica ion model ( ibcage bounding
ex ac ion), he ull se o axial slices was spli in o h ee da ase s
( aining, alida ion, and es ). Then, a spa se selec ion p ocess was
Fig. 2. Schema ic ep esen a ion o ao ic oo de ec ion by adiologis s: (a) Among he co onal iews o he ini ial CT olume, he adiologis ocuses on he ibcage
a ea. (b) Wi hin he ibcage olume, he selec s co onal slices in which he hea is isible. (c) Finally, he ao ic oo is loca ed be ween hese slices.
Fig. 3. Flowcha o he me hod.
P.G. Tahoces e al.
Compu e s in Biology and Medicine 135 (2021) 104533
4
designed o main ain a balanced numbe o cases (wi h and wi hou
lungs). The eby, a simila numbe o axial slices wi h and wi hou lungs
we e selec ed o each CTA olume. Howe e , he c i e ia o de e -
mining which slices include o does no include he lungs a e no
s ingen . Thus, only he axial slices whe ein he p esence o he lungs
was e lec ed in a high pe cen age o oxels we e labeled as lung slices.
The emaining samples we e labeled as “wi hou a lung.”
To ain and e alua e he second classi ie (HB bounding ex ac ion),
he se o co onal slices ob ained om he ibcage olume was also spli
in o h ee da ase s ( aining, alida ion, and es ). Slices whe ein bo h
he ao a and hea we e isible we e agged as Hea , hose whe ein
nei he he ao a no he hea was isible we e labeled as No Hea , and
he emaining ones (whe ein he hea may ha e been isible whe eas
he ao a was no ) we e no labeled. As a esul , a highe a iabili y can
be obse ed in co onal slices ha do no con ain he ao a. In con as ,
he slices con aining he ao a a e highly simila . The e o e, we decided
o cons uc unbalanced da ase s because he No Hea slice da ase s
we e signi ican ly la ge han he Hea da ase s.
3.4. Ao ic oo de ec ion
The HB classi ie ’s ou pu is a numbe ha ep esen s he p obabili y
ha a co onal iew would include he ao ic oo in he image. The e-
o e, he classi ie would assign a ela i ely high p obabili y o all co -
onal iews ha include he le en icle connec ed o he ascending
ao a. Howe e , o ob ain he posi ion o he ao ic oo , we need o
calcula e i s coo dina es (xSOV,ySOV,zSOV) wi hin he en i e olume o he
CTA. The e o e, ySOV would be he esul o selec ion om he se o
co onal iews, which is closes o he cen e o he SOV. To ob ain his
alue, we i s calcula e ySOVp e using he ollowing exp ession:
ySOVp e =∑kk⋅pTH (k)
pTH (k)(1)
whe e pTH(k)is he ou pu o he HB classi ie o co onal iew k. I has a
alue abo e he h eshold TH
1
. Hence, only co onal slices wi h a high
p obabili y o including he ao ic oo (TH
1
=0.9, in ou case) we e
included in his compu a ion.
Fig. 4. Fundamen al opologies o he con olu ional base o he CNN a chi ec u es used. (a) VGG, (b) ResNe , and (c) Incep ion.
Fig. 5. Diag am o he a chi ec u e o a Fas e R–CNN.
P.G. Tahoces e al.
Compu e s in Biology and Medicine 135 (2021) 104533
5
To calcula e he inal alues o he ao ic oo coo dina es (xSOV,ySOV,
zSOV), we ained a new model based on he as e egion-based con-
olu ional neu al ne wo k (Fas e R–CNN) [26]. This ne wo k consis s o
h ee main s ages (Fig. 5). In he i s s age, a CNN ans o ms he inpu
image in o ea u e maps. P e- ained CNN models can be used o his
ask. In ou case, a ResNe 101 p e- ained on he MSCOCO objec
de ec ion da ase was used as a p e- ained ne wo k. In he second s age,
a egion p oposal ne wo k (RPN) p oposes bounding boxes o candida e
objec s ela ed o objec s con ained in he image. Se e al squa e egions
o di e en sizes and aspec a ios (2D ancho s) a e used o pe o m his.
The ROI pooling s ep accommoda es he di e en sizes o he ROIs
p oposed in he second s age by ma ching hese. The inal s ep pe o ms
classi ica ion and bounding-box eg ession o each candida e ROI
selec ed in he second s age by using ully connec ed laye s (FC
ne wo k). The ea u es used by hese FC laye s o igina e om he ea u e
maps ob ained du ing he i s s age. Finally, he ou pu is o ked in o
wo b anches: (i) one associa ed wi h a so max laye ha p oduces an
es ima e o he p obabili y o belonging o a class and (ii) ano he
associa ed wi h a linea eg esso ha gene a es ou numbe s. These
numbe s code he posi ions o he bounding boxes o he objec de ec ed
in he inpu image.
In ou case, he ou pu o his ne wo k was a ec angula ROI
bounded by ou numbe s (x
1
, x
2
, z
1
, z
2
) (see Fig. 6) calcula ed by he
eg esso o he ne wo k and he p obabili y (q) ha his ROI con ains
he ao ic oo . To calcula e ySOV, we applied he as e R–CNN model o
he 2 ×W +1 co onal slices cen e ed a he ySOVp e compu ed p e iously
(W =c e). Then, we calcula ed ySOV as
ySOV =∑kk⋅qTH(k)
qTH (k)(2)
whe e (again) qTH(k)is he p obabili y deli e ed by he classi ie o he
Fas e R–CNN ne wo k o co onal iew k. I has a alue abo e he
h eshold (TH
2
). Thus, he inal alue o ySOV was selec ed by he Fas e
R–CNN model. Mo eo e , xSOV and zSOV can be calcula ed as ollows:
xSOV =x1+x2
2(3)
zSOV =z1+z2
2(4)
3.5. Ao ic oo cha ac e iza ion
The esul o he de ec ion s ep was a 2D ROI, which includes a
po ion o he ascending ao a, he SOV, and a po ion o he le
en icle (Fig. 6a). To ob ain an image o he ao ic oo displaying he
SOV, we calcula ed he SOV plane as de ined abo e. To ob ain his plane
and he di ec ion o he ascending ao a’s cen e line, we used p incipal
componen analysis (PCA) on he 2D ROI. Thus, we ob ained he
co a iance ma ix o he coo dina e ec o o he pixels included in he
ROI and calcula ed hei eigen ec o s and eigen alues. The eigen ec o
wi h a highe eigen alue co esponds o he di ec ion o he ascending
ao a in he co onal plane, a leas in he icini y o he hea . The
ollowing is a b ie desc ip ion o he p ocedu e:
i) The 2D ROI ob ained du ing he de ec ion s ep is p e-p ocessed by
ans o ming he pixel alues ha belong o i s uppe igh and he
le bo om co ne s o ze o (Fig. 6b). Thus, he con ibu ion o hese
egions o he calcula ion o he co a iance ma ix is ze o. This s ep is
necessa y o p e en he p esence o a i ac s in he compu a ion o
an op imal h eshold (TH
3
). I is pe o med as ollows:
ROIp e(x,z) =
⎧
⎪
⎪
⎪
⎪
⎪
⎨
⎪
⎪
⎪
⎪
⎪
⎩
0 i x<1
4xSOV and z <1
4zSOV
0 i x>3
4xSOV and z >3
4zSOV
ROI(x,z)any o he case
(5)
ii) To ob ain he co a iance ma ix o pixel loca ions (x
i
, z
i
), we bina ize
he pixels o he ROI (Fig. 6c). Thus, only he posi ions o he pixels
whose alue is one become pa o he co a iance ma ix. A h esh-
olding algo i hm is applied o he pixels o he ROI o pe o m his
ask. Thus, pixels below a h eshold (TH
3
) become ze o and hose
abo e i become one:
Fig. 6. (a) Co onal iew supe imposed wi h he ao ic oo de ec ed. (b) and (c) p ep ocessing s eps, (d) and (e) axis o he SOV plane and he ao a cen e line
compu ed using PCA.
P.G. Tahoces e al.
Compu e s in Biology and Medicine 135 (2021) 104533
6
ROIp e(x,z) = {0 i ROIp e(x,z)<=TH3
1 i ROIp e(x,z)>TH3
(6)
The alue o TH
3
was ob ained om a smoo hed e sion o he his-
og am o each ROI by iden i ying he i s la zone be o e he peak
ep esen ing bo h he ascending ao a and le en icle.
iii) We collec he coo dina es o he pixels o ROI
p e
ha a e non-ze o
and s o e hese in he ec o s x
coo d
and z
coo d
(Fig. 6d). Thus, he
k-elemen o hese ec o s is
xcoo d(k) = xi
zcoo d(k) = zi}i ROIp e(xi,zi) ∕= 0,∀i∈ROIp e (7)
whe e 0 <k <n and n is he numbe o non-ze o pixels o ROI
p e
.
i ) We compu e he co a iance ma ix co
coo d
be ween he wo co-
o dina e ec o s x
coo d
and z
coo d
.
co coo d(xcoo d ,ycoo d ) = ∑k
i=1(xcoo di−xcoo d)(zcoo di−zcoo d)
k−1(8)
) We compu e he eigen ec o s (A) and eigen alues (B) o he
co a iance ma ix.
A=(A00 A01
A10 A11 )(9)
B=(B0
B1)(10)
i) We iden i y he highes eigen alue (B
k
) and selec i s associa ed
eigen ec o (A
0k
, A
1k
) ha would be he main componen o he
2D ROI. Tha is, i would ollow he di ec ion o he ascending
ao a in he p oximi y o he hea (Fig. 6e). I s slope m is
m=A1k
A0k
(11)
Thus, he s aigh line
z=zSOV +m(x−xSOV )(12)
co esponds o he compu ed cen e line o he ascending ao a ( he
black line in Fig. 7a) and i s co esponding o hogonal (whi e line in
Fig. 7a).
z=zSOV −1
m(x−xSOV )(13)
belongs o he SOV plane, i.e., he plane de ined by he no mal ec o
n
(A0k,A1k).
ii) Based on his no mal ec o n
→we can de ine he plane:
A0kx+A1kz+C=0 wi h C= − A0kxSOV −A1kzSOV (14)
whe e he SOVs a e isible (Fig. 7b).
3.6. Pe o mance e alua ion
To de e mine he pe o mance o he wo-ca ego y classi ica ion
asks, we cons uc ed he con usion ma ix and hen compu ed he
me ics p ecision and ecall:
P ecision =TP
TP +FP (15)
Recall =TP
TP +FN (16)
In he las wo exp essions, T ue Posi i es (TP) a e he slices ha
include he ibcage/HB and we e classi ied as con aining he ibcage/HB,
TN (T ue Nega i es) a e he slices ha do no include he ibcage/HB and
we e classi ied as no con aining he ibcage/HB, FP (False Posi i es) a e
he slices ha do no include he ibcage/HB and we e classi ied as
con aining he ibcage/HB (False Posi i es), and FN (False Nega i es) a e
he slices ha include he ibcage/HB and we e classi ied as no con-
aining he ibcage/HB.
In addi ion, o ob ain a single alue ela ed o he pe o mance, we
compu ed he accu acy by
Accu acy =TP +TN
TP +TN +FP +FN (17)
and he F1 sco e:
F1sco e =2×P ecision ×Recall
P ecision +Recall (18)
The accu acy is pa icula ly e ec i e o analyzing he pe o mance
o ibcage de ec ion because he numbe o posi i es is balanced well
wi h he numbe o nega i es. In con as , he F1 sco e is e ec i e o
analyzing he balance be ween p ecision and ecall whe e he e is a non-
uni o m dis ibu ion. This is he case o HB de ec ion, as men ioned
abo e.
The pe o mance achie ed by he Fas e R–CNN ne wo k o ao ic
oo de ec ion was calcula ed in e ms o he mean Euclidean dis ance
be ween he posi ions calcula ed by he algo i hm (xSOV, ySOV, zSOV) and
Fig. 7. Example o he esul o he me hod. (a) Co onal iew wi h he box o he ao ic oo calcula ed, including he (x
SOV,ySOV ,zSOV)do (black do ). The wo lines
ep esen ing he cen e line o he ascending ao a (black s aigh line) and he SOV plane (whi e s aigh line) a e supe imposed. (b) Re o ma ed image ollowing he
SOV plane.
P.G. Tahoces e al.
Compu e s in Biology and Medicine 135 (2021) 104533
7
hose ma ked by he wo expe s who pa icipa ed in he s udy ((x0i, y0i,
z0i), i =1, 2). Finally, o analyze he pe o mance o he cha ac e iza ion
s ep, we calcula ed he di e ence be ween he il angles o he SOV
plane as de e mined by he algo i hm (θSOVc) and ha de e mined om
he wo ma ks en e ed manually by he expe s (θSOV0i, i =1, 2).
4. Expe imen s and esul s
Fig. 7a shows he esul o applying ou me hod o he example case.
The s aigh lines co esponding o he SOV plane (Fig. 7b) and he
ascending ao a’s cen e line calcula ed by ou me hod a e also included.
Fig. 8 shows he esul s ob ained applying ou me hod o di e en cases
included in ou es se . The ma ks in oduced by one o he expe s
(o ange colo ) and he ao ic oo ’s loca ion calcula ed by he compu e
Fig. 8. Resul s ob ained by ou me hod unde di e en assump ions. F om uppe o lowe : (a) no mal case. (b) low-con as case. (c) case wi h a p os he ic hea
al e inse ed. (d) noisy case. The images depic a co onal slice wi h he ao ic oo (le ), he SOV plane ( igh ), and he ROIs wi h i) he loca ion o he wo poin s
ma ked by he wo expe s (o ange do s), ii) he oo posi ion compu ed by he algo i hm (g een do ), and iii) he axes ep esen ing bo h he cen al line o he ao a
and he o ien a ion o he SOV calcula ed by he compu e (black lines) (cen e ). The do s ma ked by Expe
1
a e in he uppe ROI, and hose ma ked by Expe
2
a e in
he bo om ROI.
P.G. Tahoces e al.
Compu e s in Biology and Medicine 135 (2021) 104533
8
(g een colo ) a e supe imposed. The cases selec ed e eal he beha io
o ou algo i hm in ou scena ios: i) no mal case, ii) low con as case,
iii) case wi h p os he ic hea al e inse ed, and i ) noisy case. The
images include he loca ion o he wo poin s ma ked by he expe and
he ao ic oo posi ion calcula ed by ou me hod.
4.1. Ribcage and HB bounding ex ac ion
The ibcage and HB bounding ex ac ions we e implemen ed in
e ms o a bina y classi ica ion ask as explained abo e. The e o e, wo
models we e specially ained o his pu pose. Axial slices we e used as
inpu s o he model o ibcage bounding ex ac ion, whe eas co onal
slices we e used o HB bounding ex ac ion.
The pe o mances and con igu a ions o he h ee CNN a chi ec u es
we e compu ed. Tables 1 and 2 p esen a compa ison o he esul s ob-
ained om hese app oaches a e hei applica ion o he cases
included in he es se . We can obse e ha he complex a chi ec u es
VGG16, ResNe 50, and Incep ionV3 achie ed hei bes esul s o p e-
ained models when bo h he inal laye s and he classi ica ion s age
we e e ained ((p e, blk) models, see Sec ion 3.3). I is impo an o
no e ha he accu acy achie ed o he models VGG16, ResNe 50, and
Incep ionV3 when ained om sc a ch was educed o app oxima ely
50%.
Acco ding o he esul s depic ed in Table 1, we can conclude ha he
VGG16 (p e, blk) model achie ed he highes classi ica ion accu acy
(98.0%). Simila esul s we e achie ed o he HB bounding ex ac ion
(Table 2), whe e he 1 sco e was 0.72 o his con igu a ion. The e o e,
he models based on VGG16 (p e, blk) we e inally employed o isola e
he HB olume, om which he ao ic oo was ex ac ed in he nex
s ep.
4.2. Ao ic oo de ec ion
The di e ences be ween he posi ions ma ked by he expe s and
hose ob ained using he compu e we e calcula ed. The di e ences
be ween he posi ions ma ked by he wo expe s we e also calcula ed o
es ima e he in e obse e e o . The di e ences be ween he expe s
se ed as a e e ence o compa ison wi h he esul s o he algo i hm.
The QQ plo was analyzed o e i y he no mal dis ibu ion o he
calcula ed dis ances o he es se . We had obse ed ha mos o he
poin s a e loca ed close o he s aigh line co esponding o he diagonal
o he plo . Simila esul s we e ob ained in he aining and alida ion
cases. The e o e, we can assume a no mal dis ibu ion o hese dis-
ances. Based on hese esul s, he mean alue and s anda d de ia ion o
he dis ance we e calcula ed o es ima e he e o o ou me hod
(Table 3).
Nex , we analyzed he ou lie s in he uppe igh co ne o he QQ-
plo s. These we e conside ed he mos ele an because hese co e-
spond o cases in which he di e ence be ween he posi ion de e mined
by he expe s and ha iden i ied by he compu e is mo e signi ican .
We concluded ha in gene al, hese cases a e ela ed o he p esence o
a i icial hea al es o an inco ec loca ion o he oo posi ion
because o he selec ion o an inaccu a ely de e mined co onal slice.
A de ailed analysis o he alues ob ained o he es se and he
calcula ion o he di e ences (sepa a ed by iews) e ealed ha he
wo s esul s we e ob ained o he co onal iew. This is ue bo h in
e ms o he mean dis ance and dispe sion o he alues ob ained
(Table 4). These esul s a e consis en wi h hose ob ained when he
compa ison was made wi h he expe s’ ma ks. In addi ion, in his case,
he wo s esul s we e ob ained wi h he selec ion o he co onal iew,
o bo h a e age alue (2.8 o he co onal iew s. 2.1 and 1.9 o he
sagi al and axial iews, espec i ely) and dispe sion (2.3, s. 1.7 and
1.3, espec i ely).
An analysis o he numbe o cases in he es se e ealed ha he
dis ance be ween he loca ion ma ked by he expe and ha calcula ed
by he algo i hm exceeds 10 mm o some pa ien s (Table 5). We can
obse e ha his scena io occu s only in he sagi al and co onal iews.
Tha is, he co onal iew has he wo s esul in his case as well.
QQ-plo analysis was also pe o med o e i y he no mal dis ibu-
ion o he di e ences be ween he slope o he SOV plane compu ed
om he loca ions ma ked by he expe s and he slope calcula ed by he
algo i hm. Bo h mean alue and s anda d de ia ion o hese di e ences
we e calcula ed based on hese esul s. The esul s o he di e en
da ase s a e shown in Table 6.
Bland—Al man plo s (Figs. 9 and 10) we e plo ed o analyze he
ag eemen be ween he wo p ocedu es (manual and au oma ic) p o-
posed o quan i y he SOV slope. As is e iden , biases exis be ween he
wo me hods. I we compa e hese biases (−7.2◦ o Expe
1
s. Compu e ,
and −10.4◦ o Expe
2
s. Compu e ), he alues a e nega i e and close
in bo h cases. Mo eo e , he esul s ob ained o mos cases we e wi hin
he 95% con idence in e al. This indica es he easibili y o in oducing
a co ec ion ac o o co ec o bias.
5. Discussion
The au oma ic ex ac ion o quan i a i e in o ma ion om medical
images is an impo an challenge ha has been add essed wi h a ce ain
deg ee o success in ecen yea s. Howe e , o achie e op imal esul s, i
is necessa y o ob ain e ec i e segmen a ion o he o gan/s uc u e o
be analyzed and he p ecise loca ion o se e al e e ence poin s used o
ex ac such in o ma ion.
In his s udy, we ocused on he au oma ic de ec ion o he ao ic
oo . The il o he SOV plane was also ob ained a e he de ec ion. The
de eloped me hod consis s o h ee main s eps: classi ica ion, de ec ion,
and slope calcula ion. Fo classi ica ion and de ec ion, we used models
based on di e en CNN a chi ec u es. A PCA-based me hod was p o-
posed o he calcula ion o he slope.
Fo classi ica ion, models based on ne wo ks wi h complex a chi-
ec u es (namely, as VGG16, ResNe 50, o Incep ionV3) we e demon-
s a ed o be in alid when ained om sc a ch, owing o o e aining.
In his case, accu acies close o 50% we e ob ained, which implies ha
hey could no dis inguish be ween he g oups. The la ge numbe o
pa ame e s o be uned (o e 20 million) and he limi ed size o he
da ase s hinde ed he ob ainmen o op imal solu ions o such complex
ne wo ks. Howe e , he esul s imp o ed subs an ially when p e- ained
models we e used on hese a chi ec u es. We ob ained eliable classi ie s
Table 1
Summa y o esul s o di e en CNN a chi ec u es used o ibcage ex ac ion.
CNN NPTT Acc (%) P ec Rec 1 TP TN FP FN
VGG1 16787617 97.3 0.96 0.99 0.97 21019 21805 963 203
VGG2 8454433 97.4 0.97 0.97 0.97 20680 22146 622 542
VGG3 4629153 97.4 0.96 0.98 0.97 20864 21999 769 358
VGG16 (p e) 1048833 96.5 0.95 0.97 0.96 20672 21784 984 550
VGG16 (p e, blk) 3408641 98.0 0.97 0.98 0.98 20886 22209 559 336
ResNe 50 (p e) 1312257 95.8 0.92 0.99 0.96 21064 21060 1708 158
ResNe 50 (p e, blk) 5777921 97.5 0.97 0.98 0.97 20696 21194 574 526
Incep ionV3 (p e) 1312257 50.9 0.49 0.39 0.43 8255 14149 8619 12967
Incep ionV3 (p e, blk) 7385793 95.9 0.97 0.94 0.96 20016 22162 606 1196
P.G. Tahoces e al.
Compu e s in Biology and Medicine 135 (2021) 104533
9
e en when da ase s o limi ed size we e employed o aining.
Compa ing he esul s achie ed o bo h he classi ica ion asks (see
Table 1 and 2) and ocusing on he esul s achie ed o he di e en
models, we obse e ha in gene al, alues abo e 90% we e ob ained
only o a ew o he p e- ained model con igu a ions and only in hose
cases whe e he classi ie and speci ic laye s o he con olu ional pa
we e e ained ((p e, blk) models). This indica es ha he second clas-
si ica ion is mo e complex, which is consis en wi h he ac ha i was
necessa y o cons uc an unbalanced da ase o aining. I is likely ha
a highe numbe o cases in he aining se would imp o e hese
numbe s. Howe e , he model ained wi h VGG16 (p e, blk) a ained
accu acies close o 95.9% (0.72 o F1 sco e) and ecall and p ecision o
0.95 and 0.59, espec i ely.
Wi h ega d o he de ec ion o he ao ic oo , we e i ied ha o
he es se , he di e ences be ween he esul s ob ained using he al-
go i hm and he loca ion ma ked by he expe s ollow a no mal dis-
ibu ion. Acco dingly, we can conclude ha in 95% o he cases, he
di e ence be ween he loca ion ma ked by Expe
1
and he posi ion
calcula ed by he compu e (see Table 3) is less han 12.6 mm (13.4 mm
o Expe
2
). Following his a gumen , he di e ences be ween he po-
si ions ma ked by he expe s (Expe
1
s. Expe
2
) a e less han 9.2 mm
in 95% o he es se cases. Thus, he in e obse e ag eemen was
ela i ely simila ega dless o whe he he obse e s we e humans o
compu e s. I is impo an o no e ha he cases used in ou s udy we e
acqui ed wi h non ECG-ga ed CT scanne s. The e o e, an implici e o
occu s while a emp ing o ma k he exac loca ion o he ao ic oo .
Meanwhile, we know ha he mean ascending ao a diame e is
app oxima ely 35 mm o men and 33 mm o women [27]. Thus, he
e o can be ega ded as easonable conside ing bo h in e obse e
ag eemen e o and size o he ao a.
The e a e ew pape s in he li e a u e on he au oma ic de ec ion o
ao ic oo in CTA images. Mos o hese a e ela ed o he de ec ion o
di e en landma ks du ing TAVI p ocedu e planning. Thus, Ela a e al.
[5] epo ed ha au oma ic landma k de ec ion had a mean e o o
2.66 ±1.63 mm and 2.96 ±2.52 mm when compa ed wi h Obse e s I
and II, espec i ely. The mean dis ance o he ma ched obse e s was
2.38 mm. As udillo e al. [28] used Euclidean dis ance and epo ed a
median di e ence o 1.5 (1.1, 2.0) mm o all landma ks combined. This
was smalle han he di e ence o 2.0 (1.3, 2.8) mm achie ed o he
manually de ec ed landma ks by he i s and second obse e s. Fo Al
e al. [29], he o e all landma k e o o pa ien s wi hou and wi h
TAVI was 1.94 ±0.93 mm and 2.74 ±1.78 mm, espec i ely. Finally, in
Lalys e al. [23], he esul s o he de ec ion o he wo os ia e ealed
dis ance e o s o 1.80 ±0.74 mm and 1.96 ±0.87 mm.
To summa ize, all he p e ious esul s show di e ences o app oxi-
ma ely 2.0 mm be ween compu e -anno a ed and hand-anno a ed
landma ks, which is simila o hose ob ained while compa ing ob-
se e s. Howe e , in all hese cases, he p oblem o be sol ed is ela ed
o p eope a i e TAVI p ocedu es. Consequen ly, he images used we e
ECG-ga ed CTAs. The e o e, a compa ison wi h ou me hod would be
biased. In ou me hod, he mean dis ance be ween he compu e and he
human obse e was 6.7 mm, which is close o he in e obse e e o o
4.6 mm (see Table 3). In ou p e ious wo k [25], he esul s ob ained o
ao ic oo de ec ion on non-ECG-ga ed CTA a e simila (5.7 ±7.3 mm)
o hose epo ed he e. Howe e , in ha case, he ao a was segmen ed
p e iously and i s cen e line calcula ed. Conside ing he wo ac s ( he
use o non-ECG ga ed cases and no p e ious segmen a ion o he ao a),
he esul s ob ained by ou new me hod a e signi ican .
Table 2
Summa y o esul s o di e en CNN a chi ec u es used o HB ex ac ion.
CNN NPTT Acc (%) P ec Rec 1 TP TN FP FN
VGG1 16787617 92.0 0.42 0.97 0.58 2049 31877 2877 61
VGG2 8454433 89.5 0.35 1.00 0.52 2110 30888 3866 0
VGG3 4629153 92.9 0.44 0.97 0.61 2039 32200 2554 71
VGG16 (p e) 1048833 92.6 0.44 0.98 0.60 2065 32089 2665 45
VGG16 (p e, blk) 3408641 95.9 0.59 0.95 0.72 2001 33335 1419 109
ResNe 50 (p e) 1312257 76.6 0.19 0.96 0.32 2036 26204 8550 74
ResNe 50 (p e, blk) 5777921 94.1 0.49 0.86 0.62 1819 32862 1892 291
Incep ionV3 (p e) 1312257 76.6 0.17 0.80 0.28 1681 26558 8196 429
Incep ionV3 (p e, blk) 7385793 87.7 0.31 0.94 0.47 1977 30341 4413 133
Table 3
Dis ance be ween he loca ion o he ao ic oo calcula ed by ou me hod and
ha de e mined by wo expe s.
Da a Se Cases Expe 1 s
Compu e
Expe 2 s
Compu e
Expe 1 s
Expe 2
(mm) (mm) (mm)
T aining se 39 4.9 ±2.5 6.8 ±3.3 5.5 ±2.8
Valida ing
Se
30 7.5 ±2.7 7.1 ±3.1 5.7 ±3.1
Tes Se 71 6.6 ±3.0 6.8 ±3.3 4.6 ±2.3
Table 4
Dis ance be ween he loca ion o he ao ic oo as calcula ed by ou me hod and
ha de e mined by he wo expe s o he es se , sepa a ed by iew.
View Expe 1 s Compu e Expe 2 s Compu e Expe 1 s Expe 2
(mm) (mm) (mm)
Sagi al 3.6 ±2.5 3.6 ±2.7 2.1 ±1.7
Co onal 3.9 ±3.0 3.9 ±3.2 2.8 ±2.3
Axial 2.5 ±1.8 2.7 ±1.8 1.9 ±1.3
Table 5
De ailed analysis o he alues ob ained o he es se . The maximum dis ance
be ween he ao ic oo loca ions de e mined by ou me hod and ha ma ked by
each o he wo expe s is depic ed in he Max Di column. The numbe o cases
whe e he dis ance be ween he alue calcula ed by he algo i hm and ha by
each o he expe s exceeds 10 mm is depic ed in he NC column.
View Expe 1 s Compu e Expe 2 s Compu e
Max Di NC ≥10 Max Di NC ≥10
Sagi al 11.0 1 13.8 2
Co onal 15.6 2 17.6 2
Axial 7.5 0 6.9 0
Table 6
Di e ences be ween he SOV plane slope compu ed om he loca ions ma ked
by he expe s and ha compu ed by he algo i hm.
Da a Se Expe 1 s
Compu e
Expe 2 s
Compu e
Expe 1 s Expe
2
(deg ees) (deg ees) (deg ees)
T aining se −10.9 ±11.4 −10.4 ±9.2 −0.5 ±5.3
Valida ing
Se
−5.5 ±12.9 −10.7 ±9.9 5.2 ±13.3
Tes Se −7.2 ±11.4 −10.4 ±9.3 3.1 ±6.5
P.G. Tahoces e al.
