Deep Learning-Based Algorithms for Real-Time Lung Ultrasound Assisted Diagnosis

Ci a ion: Muñoz, M.; Rubio, A.;
Cosa insky, G.; C uza, J.F.; Camacho, J.
Deep Lea ning-Based Algo i hms o
Real-Time Lung Ul asound Assis ed
Diagnosis. Appl. Sci. 2024,14, 11930.
h ps://doi.o g/10.3390/
app142411930
Academic Edi o s: Giacomo Cappon
and Ma ina Ve o e i
Recei ed: 13 No embe 2024
Re ised: 11 Decembe 2024
Accep ed: 14 Decembe 2024
Published: 20 Decembe 2024
Copy igh : © 2024 by he au ho s.
Licensee MDPI, Basel, Swi ze land.
This a icle is an open access a icle
dis ibu ed unde he e ms and
condi ions o he C ea i e Commons
A ibu ion (CC BY) license (h ps://
c ea i ecommons.o g/licenses/by/
4.0/).
A icle
Deep Lea ning-Based Algo i hms o Real-Time Lung
Ul asound Assis ed Diagnosis
Ma io Muñoz 1,2,* , Ad ián Rubio 1,2 , Guille mo Cosa insky 1, Jo ge F. C uza 1and Jo ge Camacho 1
1
Ins i u e o Physical and In o ma ion Technologies, Spanish Na ional Resea ch Council, 28006 Mad id, Spain;
[email p o ec ed] (A.R.); [email p o ec ed] (G.C.); jo [email p o ec ed] (J.F.C.); [email p o ec ed] (J.C.)
2Elec onic Depa men , Uni e sidad de Alcalá, 28805 Alcaláde Hena es, Spain
*Co espondence: [email p o ec ed]
Abs ac : Lung ul asound is an inc easingly u ilized non-in asi e imaging modali y o assessing
lung condi ion bu in e p e ing i can be challenging and depends on he ope a o ’s expe ience.
To add ess hese challenges, his wo k p oposes an app oach ha combines a i icial in elligence
(AI) wi h ea u e-based signal p ocessing algo i hms. We in oduce a specialized deep lea ning
model designed and ained o acili a e he analysis and in e p e a ion o lung ul asound images
by au oma ing he de ec ion and loca ion o pulmona y ea u es, including he pleu a, A-lines,
B-lines, and consolida ions. Employing Con olu ional Neu al Ne wo ks (CNNs) ained on a semi-
au oma ically anno a ed da ase , he model delinea es hese pulmona y pa e ns wi h he objec i e
o enhancing diagnos ic p ecision. Real- ime pos -p ocessing algo i hms u he e ine p edic ion
accu acy by educing alse-posi i es and alse-nega i es, augmen ing in e p e a ional cla i y and
ob aining a inal p ocessing a e o up o 20 ames pe second wi h accu acy le els o 89% o
consolida ion, 92% o B-lines, 66% o A-lines, and 92% o de ec ing no mal lungs compa ed wi h
an expe opinion.
Keywo ds: lung ul asound (LUS); a i icial in elligence (AI); con olu ional neu al ne wo k; deep
lea ning; pleu a; B-line; A-line; consolida ions; assis ed diagnosis; eal- ime
1. In oduc ion
Wi hin he ield o diagnos ic imaging, lung ul asound (LUS) has become signi i-
can ly mo e p ominen in ecen yea s, o e ing a non-in asi e, adia ion- ee app oach
o dynamic assessmen o lung condi ion in a ious espi a o y diseases [
1
]. Ye , despi e
i s g owing impo ance, he in e p e a ion challenges inhe en o pulmona y ul asound
images pe sis , which, combined wi h a sho age o skilled p o essionals ained in his
echnique, limi s i s applica ion in clinical p ac ice [2].
LUS images do no p o ide an ana omical iew o he lung; his is because ul asound
wa es ha dly pene a e in o he lung pa enchyma due o he ai p esen in he al eoli.
Ins ead, mos o he acous ic ene gy is e lec ed by he pleu a, which is he mos easily
iden i iable s uc u e, appea ing like a b igh ho izon al line. Fu he mo e, eplicas o
he pleu a appea a egula in e als below his line, gene a ed by he e e be a ion o
he inciden wa e be ween he ansduce and he pleu a. This a i ac , named A-Line, is
indica i e o good lung condi ion.
In he p esence o pneumonia and o he in e s i ial synd ome diseases, he ai inside
he al eoli is p og essi ely subs i u ed by liquid, which modi ies he acous ic impedance o
he lung pa enchyma, making i mo e simila o he impedance o muscle and a abo e he
pleu a. As he acous ic impedance di e ence be ween bo h issues educes, mo e ene gy o
he inciden wa e passes h ough he pleu a o he lung pa enchyma. Bu i ai emains
in some al eoli, a local e e be a ion phenomenon occu s (acous ic ap), and a b igh
e ical line appea s in he image because o he mul iple echoes backsca e ed inside he
Appl. Sci. 2024,14, 11930. h ps://doi.o g/10.3390/app142411930 h ps://www.mdpi.com/jou nal/applsci
Appl. Sci. 2024,14, 11930 2 o 23
lesion [
3
]. This e ical a i ac is named B-Line, and i is indica i e o he p esence o
pneumonia. Figu e 1shows some examples o be e unde s and he mo phology o he
a i ac s explained.
Appl. Sci. 2024, 14, x FOR PEER REVIEW 2 o 25
b igh e ical line appea s in he image because o he mul iple echoes backsca e ed in-
side he lesion [3]. This e ical a i ac is named B-Line, and i is indica i e o he p esence
o pneumonia. Figu e 1 shows some examples o be e unde s and he mo phology o he
a i ac s explained.
Figu e 1. Typical LUS a i ac s: Pleu a (blue), A-lines (g een), B-lines (o ange), Consolida ion ( ed).
As he disease ad ances, mo e ai is subs i u ed by liquid and hen consolida es in o
solid ma e ial. These consolida ions appea in he image as hypoechoic egions and con-
s i u e he hi d a i ac usually looked o when imaging he lung. In sum, a i ac s like
A-lines, B-lines, and consolida ions a e indica i e o lung condi ion and can be used o
diagnose a pulmona y pa hology [4]. De ec ing and in e p e ing hese a i ac s is he key
o a co ec e alua ion o lung images, which has been he p incipal bo leneck in he
dissemina ion o his echnique.
Mo eo e , he shock caused by he COVID-19 pandemic ampli ied he ele ance o
pulmona y ul asound as a i al ool o e alua ing pulmona y condi ions. As he pan-
demic dissemina ed wo ldwide, LUS eme ged as a on line modali y o assessing lung
in ol emen and moni o ing pulmona y condi ion, pa icula ly in pa ien s affec ed wi h
COVID-19 pneumonia. I s eal- ime, bedside capabili ies offe ed apid insigh s in o lung
pa hology, especially in eme gency, c i ical ca e uni s, and domicilia y a en ion, whe e
o he imaging modali ies like TAC and magne ic esonance encoun e ed limi a ions o
we e less accessible [1,5–8].
In his con ex , he in eg a ion o diagnos ic aid algo i hms in o ul asound scanne s
could help educe he lea ning cu e o he echnique, as well as educing he e alua ion
ime and possibly inc easing he diagnosis success. A i icial in elligence (AI) has b ough
abou signi ican ad ancemen s ac oss di e se scien i ic and medical ields, showcasing
i s po en ial in eshaping he landscape o diagnosis and pa ien wel a e. Wi hin he ealm
o heal hca e, AI is e olu ionizing he ield, showcasing p omising ou comes h ough he
p o ision o sophis ica ed ools ha aid heal hca e p o ide s in making clinical decisions.
This in eg a ion has led o imp o emen s in p ecision and effec i eness, enhancing he
p ocess o diagnosing and ea ing a wide ange o medical condi ions [9].
The machine lea ning app oach o de ec ing and classi ying lesions is no exclusi e
o ul asound imaging. In [10], he au ho s p opose a Vision T ans o me (ViT) based a -
chi ec u e able o classi y melanoma e sus non-cance ous lesions, which was es ed on
public skin cance da a wi h good esul s. In he X-Ray ield, se e al machine lea ning
(ML) app oaches a e p oposed in [11] o dis inguish be ween benign and malign umo s
in mammog aphy images, whe e he bes esul s we e ob ained wi h a Nai e Bayes algo-
i hm. In [12], a e iew is p o ided o ML me hods o lung cance de ec ion and classi i-
ca ion wi h diffe en imaging modali ies (X- ays, compu e omog aphy, and magne ic
esonance), epo ing a sensi i i y be ween 0.81 and 0.99, a speci ici y be ween 0.46 and
1.00, and an accu acy om 77.8% o 100%, which e eals he po en ial o hese echniques.
Figu e 1. Typical LUS a i ac s: Pleu a (blue), A-lines (g een), B-lines (o ange), Consolida ion ( ed).
As he disease ad ances, mo e ai is subs i u ed by liquid and hen consolida es
in o solid ma e ial. These consolida ions appea in he image as hypoechoic egions and
cons i u e he hi d a i ac usually looked o when imaging he lung. In sum, a i ac s
like A-lines, B-lines, and consolida ions a e indica i e o lung condi ion and can be used
o diagnose a pulmona y pa hology [
4
]. De ec ing and in e p e ing hese a i ac s is he
key o a co ec e alua ion o lung images, which has been he p incipal bo leneck in he
dissemina ion o his echnique.
Mo eo e , he shock caused by he COVID-19 pandemic ampli ied he ele ance
o pulmona y ul asound as a i al ool o e alua ing pulmona y condi ions. As he
pandemic dissemina ed wo ldwide, LUS eme ged as a on line modali y o assessing
lung in ol emen and moni o ing pulmona y condi ion, pa icula ly in pa ien s a ec ed
wi h COVID-19 pneumonia. I s eal- ime, bedside capabili ies o e ed apid insigh s in o
lung pa hology, especially in eme gency, c i ical ca e uni s, and domicilia y a en ion, whe e
o he imaging modali ies like TAC and magne ic esonance encoun e ed limi a ions o
we e less accessible [1,5–8].
In his con ex , he in eg a ion o diagnos ic aid algo i hms in o ul asound scanne s
could help educe he lea ning cu e o he echnique, as well as educing he e alua ion
ime and possibly inc easing he diagnosis success. A i icial in elligence (AI) has b ough
abou signi ican ad ancemen s ac oss di e se scien i ic and medical ields, showcasing i s
po en ial in eshaping he landscape o diagnosis and pa ien wel a e. Wi hin he ealm o
heal hca e, AI is e olu ionizing he ield, showcasing p omising ou comes h ough he
p o ision o sophis ica ed ools ha aid heal hca e p o ide s in making clinical decisions.
This in eg a ion has led o imp o emen s in p ecision and e ec i eness, enhancing he
p ocess o diagnosing and ea ing a wide ange o medical condi ions [9].
The machine lea ning app oach o de ec ing and classi ying lesions is no exclusi e
o ul asound imaging. In [
10
], he au ho s p opose a Vision T ans o me (ViT) based
a chi ec u e able o classi y melanoma e sus non-cance ous lesions, which was es ed on
public skin cance da a wi h good esul s. In he X-Ray ield, se e al machine lea ning
(ML) app oaches a e p oposed in [
11
] o dis inguish be ween benign and malign umo s in
mammog aphy images, whe e he bes esul s we e ob ained wi h a Nai e Bayes algo i hm.
In [
12
], a e iew is p o ided o ML me hods o lung cance de ec ion and classi ica ion
wi h di e en imaging modali ies (X- ays, compu e omog aphy, and magne ic esonance),
epo ing a sensi i i y be ween 0.81 and 0.99, a speci ici y be ween 0.46 and 1.00, and an
accu acy om 77.8% o 100%, which e eals he po en ial o hese echniques.
Se e al au ho s p opose he applica ion o AI-based algo i hms wi h ul asound lung
images o add ess he challenges ela ed o a i ac s iden i ica ion and quan i ica ion. The e
a e usually wo di e en app oaches o his p oblem: p ocessing a whole ul asound ideo
Appl. Sci. 2024,14, 11930 3 o 23
and classi ying i acco ding o he disease se e i y, o p ocessing he ideo ame-by- ame,
highligh ing he a i ac s ha a e ound by labeling he image. The i s app oach means a
classi ica ion p oblem, whe e he inpu is a bu e o images, and he ou pu is a se e i y
sco e. The second one is a segmen a ion p oblem, whe e isual eedback is gi en o
he clinician abou he ype, posi ion, and ex en o he imaging a i ac . Segmen a ion
means o de ec and isola e a egion o in e es inside an image, in his case, he A-Line,
B-Line and consolida ion a i ac s, usually highligh ing hem wi h a colo o e lay on he
con en ional g ay-scale image. This eedback would be use ul o e alua ing he image,
bu also imp o es he in e p e abili y o he model, as he physician can con i m ha he
deep lea ning model is paying a en ion o he co ec zone o he image. In his wo k, we
p opose o combine bo h app oaches, p o iding ame-by- ame eal- ime iden i ica ion and
enhancemen o a i ac s, along wi h a sco e o he whole ideo based on he a i ac s ound.
In [
13
], he au ho s p opose a deep lea ning solu ion capable o assigning a se e i y
sco e o a lung ul asound ideo. They pose he p oblem as a classi ica ion p oblem whe e
he ne wo k lea ns o assign a sco e based on he a i ac s ound in he inpu o he ne wo k.
In [
14
], he au ho s design an algo i hm o ex ac se e al images ha se e as a summa y o
he ul asound examina ion om a lung ul asound ideo. These images a e subsequen ly
in oduced o a neu al classi ica ion ne wo k o assign a sco e. The ne wo k goes on
o e alua e he classi ica ion p oblem on an image-by-image basis. In [
15
], he au ho s
combine CNN wi h Long Sho -Te m Memo y (LSTM) ne wo ks o ex ac special and
empo al ea u es o classi y be ween heal hy and non-heal hy a ideo le el.
I we alk abou image-by-image p ocessing, in [
16
], he au ho s p opose a deep lea n-
ing model o B-line de ec ion ained on images om Dengue pa ien s as a classi ica ion
p oblem. In [
17
], di e en solu ions o B-line de ec ion and localiza ion a e p oposed: as
a ideo-le el and ame-le el classi ica ion p oblem and as a ame-le el segmen a ion
p oblem. The au ho s conclude ha ame-by- ame segmen a ion has an ad an age o e
he o he solu ions because i esul s in less a iabili y in image in e p e abili y by di e en
physicians. O he wo ks, such as [
18
], ocus on he de ec ion o B lines by isualizing
he ac i a ion map on he ou pu o a bina y classi ie con olu ional ne wo k. He e hey
con empla e he possibili y o using such solu ions in eal ime.
The abo e-men ioned wo ks ocus on he de ec ion o a single a i ac , he B line, which
means ha o he solu ions a e needed o he o he a i ac s ypical o pneumonia and lung
ul asound. In [
19
], a mul iclass segmen a ion model ha dis inguishes mul iple a i ac s is
p oposed ins ead, bu unlike ou wo k, he model is ained wi h images ob ained om a
lung phan om.
In his wo k, we p opose a no el comple e wo k low o compu ed assis ed diagnosis
in lung ul asound imaging, aimed o be implemen ed in eal- ime. A specialized deep
lea ning model was designed and ained o de ec pulmona y ea u es, including he
pleu a, A-lines, B-lines, and consolida ions. Fu he mo e, a se o p e- and pos -p ocessing
algo i hms a e p oposed o no malize images o a common ep esen a ion space and
ake in o accoun a p io i knowledge abou he p oblem o enhance obus ness. Ano he
con ibu ion o he wo k is a semi-au oma ed labeling ool ha could possibly con ibu e o
ex ending da ase s o u he aining o new deep lea ning models. Al oge he , his wo k
aims o ake a u he s ep owa ds he implemen a ion o diagnos ic aids in lung imaging
on clinical scanne s.
2. Me hods
2.1. Neu al Ne wo k A chi ec u e
Con olu ional Neu al Ne wo ks (CNNs) ind ex ensi e usage in eal- ime segmen a-
ion asks due o hei capabili y o ecognizing local pa e ns and cha ac e is ics wi hin
images. They p o ide p ocessing e iciency and a hie a chical ea u e s uc u e enabling
he iden i ica ion o bo h in ica e de ails and b oade pa e ns. Speci ically, he U-Ne
a chi ec u e s ands ou o i s specializa ion in seman ic and medical segmen a ion [
20
]. I s
skip connec ions acili a e he usion o in o ma ion ac oss a ious spa ial scales, esul ing
Appl. Sci. 2024,14, 11930 4 o 23
in he c ea ion o highly de ailed segmen ed masks. This is pa icula ly ad an ageous in
medical imaging, like lung consolida ion segmen a ion.
Wi h he aim o imp o ing he beha io and esponse o ou model, a ious U-Ne
ne wo k modi ica ions we e s udied, including he s anda d U-Ne a chi ec u e. A s udy
was pe o med using he classical U-Ne ; he esul s can be ound in [
21
], whe e i was
shown ha he pe o mance was in e io compa ed o he a chi ec u e wi h a en ion blocks
p esen ed in his pape . Fo he modi ica ions, we used he Py hon lib a y Ke as-une ,
de eloped by K. Zak and published on Gi Hub [
22
]. Finally, he chosen a chi ec u e is
A en ion U-Ne , de eloped by O. Ok ay in [
23
], whe e a en ion ga es il e s a e added
in he decode pa , which au oma ically lea ns o ocus on main pa e ns wi h di e en
shapes and sizes, imp o ing he esul o he p edic ion. To ind a comp omise be ween
compu a ional cos , pe o mance, and ne wo k complexi y, we chose o use 16, 32, 64, and
128 il e s on he encoding laye s and he in e se o he decoding laye s. Highe il e
coun s inc ease he model’s complexi y and esou ce needs signi ican ly, which may no
ansla e in o be e pe o mance o ou ask.
Two implemen a ion op ions we e explo ed wi h he aim o seeing wha is mo e sui -
able o he desc ibed p oblem: A di e en model o each pneumonia pa e n, and a single
model capable o p edic ing and segmen ing all main lung ul asound pa e ns (Pleu a,
A-line, B-line, Consolida ion). Bo h solu ions p esen ad an ages and disad an ages ha
we e aken in o accoun . The use o se e al models o e s a mo e speci ic esponse and he
possibili y o une he ne wo k acco ding o each pa e n o ob ain he bes beha io , bu ,
on he o he hand, when using se e al models, he in e ence ime inc eases, making he
eal- ime implemen a ion mo e di icul . While using one global ne wo k wi h mul iple
ou pu s, he esponse could be less p ecise i we compa e each ou pu wi h a ne wo k
speci ically ained o ha pa e n, bu he main ad an age is ha he in e ence ime is
educed, he eby allowing eal- ime implemen a ion.
2.2. Da ase
The da a used o pe o m his s udy we e ob ained in he clinical ial ULTRACOV [
24
].
They we e gene a ed wi h a 128-channel ul asound elec onic equipmen de eloped
in-house and a medical g ade 3.5 MHz con ex p obe. A o al o 689 ideos we e ac-
qui ed, co esponding o 30 pa ien s and ollowing a s anda dized scanning p ocedu e o
12 ho acic egions [25].
The ideos we e manually anno a ed by an expe physician, indica ing he p esence
o absence o each a i ac . The labeling p ocess was pe o med in wo s eps: i s , he
physician ma ked he p esence o absence o each a i ac in he scanne so wa e du ing
he examina ion, educing he bias ha could be in oduced when e iewing he ideos o
all he pa ien s in a single session. Then, a e iew p ocess was pe o med o -line o disca d
e o s and inally alida e he da ase labeling.
Then, a sco e o he pa ien was calcula ed, aking in o accoun he ideos o he
12 scanned egions. Because he ideos we e no labeled ame-by- ame by he expe , a
semi-au oma ed anno a ion algo i hm was de eloped (Sec ion 2.2.2).
2.2.1. Neu al Ne wo k Inpu Da a
The o ma o he inpu da a has a g ea impac on he deep lea ning model de ini ion
and pe o mance. A ypical ul asound image wi h a cu ed a ay (like ha used in his
s udy) is a ci cle sec o , de ined by an ape u e angle
α
, an ini ial ange
1
, and a inal ange
2
, which is usually amed in o a ec angula image wi h size W
×
H pixels (Figu e 2Le ).
Bu , in ac , a sec o ul asound image is o iginally o med by N scan lines usually equally
dis ibu ed inside he sec o a ea. These lines, con aining M samples each, a e he ou pu o
he beam o ming algo i hm, and could be in e p e ed hemsel es as a ec angula image
(Figu e 2Righ ).
The algo i hm o ob ain he sec o image om he B-Scan da a is usually called scan-
con e e , and i is ypically implemen ed by bi-linea in e pola ion o he acqui ed samples
Appl. Sci. 2024,14, 11930 5 o 23
o e he pixel g id. This p ocess is ca ied ou by he scanne , which gi es he use he
sec o image in W
×
H o ma . The e o e, a ques ion a ises abou which image o ma is
mo e app op ia e o implemen ing he deep lea ning algo i hm aimed a in his wo k.
Appl. Sci. 2024, 14, x FOR PEER REVIEW 5 o 25
2.2.1. Neu al Ne wo k Inpu Da a
The o ma o he inpu da a has a g ea impac on he deep lea ning model de ini ion
and pe o mance. A ypical ul asound image wi h a cu ed a ay (like ha used in his
s udy) is a ci cle sec o , de ined by an ape u e angle α, an ini ial ange
1
, and a inal ange
2
, which is usually amed in o a ec angula image wi h size W × H pixels (Figu e 2 Le ).
Bu , in ac , a sec o ul asound image is o iginally o med by N scan lines usually equally
dis ibu ed inside he sec o a ea. These lines, con aining M samples each, a e he ou pu
o he beam o ming algo i hm, and could be in e p e ed hemsel es as a ec angula im-
age (Figu e 2 Righ ).
Figu e 2. Possible ep esen a ions o an ul asound sec o image: (le ) con en ional pixel-based
image gi en by ul asound scanne s; ( igh ) B-Scan ec angula image o med by ul asound
samples only, wi hou geome ical in o ma ion o he p obe.
The algo i hm o ob ain he sec o image om he B-Scan da a is usually called scan-
con e e , and i is ypically implemen ed by bi-linea in e pola ion o he acqui ed sam-
ples o e he pixel g id. This p ocess is ca ied ou by he scanne , which gi es he use
he sec o image in W × H o ma . The e o e, a ques ion a ises abou which image o ma
is mo e app op ia e o implemen ing he deep lea ning algo i hm aimed a in his wo k.
Sec o image has he ad an age o being mo e easily accessible, because i is he yp-
ical ou pu o ma o mos ul asound equipmen . The e o e, i main ains he aspec a io
o he s uc u es o be imaged, which eases in e p e a ion by medical p o essionals. How-
e e , accommoda ing a sec o image inside a ec angula g id gene a es black ma gins
a ound i , which, besides adding pixels wi h no in o ma ion, could po en ially in oduce
a bias in he au oma ed image analysis. Fu he mo e, he shape and ex ension o hese
zones depends on he scanne model and he con igu a ion, hinde ing he ansla ion o
he esul an model be ween diffe en scanne s.
On he o he hand, ec angula B-scan images ha e he ad an age o p o iding only
use ul in o ma ion (no black ma gins), while i s ec angula o ma is highly sui able as
inpu o segmen a ion models. Ano he impo an ad an age is ha each e ical line
ep esen s a physical p opaga ion di ec ion o he beam inside he issue, which is pa ic-
ula ly ele an o a i ac s like B-Lines, ha appea p ecisely on hose di ec ions. The e-
o e, a B-Line will be always seen in he B-Scan as a e ical a i ac , independen o i s
Figu e 2. Possible ep esen a ions o an ul asound sec o image: (le ) con en ional pixel-based
image gi en by ul asound scanne s; ( igh ) B-Scan ec angula image o med by ul asound samples
only, wi hou geome ical in o ma ion o he p obe.
Sec o image has he ad an age o being mo e easily accessible, because i is he ypical
ou pu o ma o mos ul asound equipmen . The e o e, i main ains he aspec a io o
he s uc u es o be imaged, which eases in e p e a ion by medical p o essionals. Howe e ,
accommoda ing a sec o image inside a ec angula g id gene a es black ma gins a ound i ,
which, besides adding pixels wi h no in o ma ion, could po en ially in oduce a bias in he
au oma ed image analysis. Fu he mo e, he shape and ex ension o hese zones depends
on he scanne model and he con igu a ion, hinde ing he ansla ion o he esul an
model be ween di e en scanne s.
On he o he hand, ec angula B-scan images ha e he ad an age o p o iding only
use ul in o ma ion (no black ma gins), while i s ec angula o ma is highly sui able
as inpu o segmen a ion models. Ano he impo an ad an age is ha each e ical
line ep esen s a physical p opaga ion di ec ion o he beam inside he issue, which is
pa icula ly ele an o a i ac s like B-Lines, ha appea p ecisely on hose di ec ions.
The e o e, a B-Line will be always seen in he B-Scan as a e ical a i ac , independen
o i s posi ion on he pleu a and o he scanne con igu a ion. Fu he mo e, scanne
con igu a ion pa ame e s like he ape u e angle
α
o he ini ial and inal ange
1
and
2
only a ec he size M
×
N o he image, which simpli ies adap ing images acqui ed wi h
di e en con igu a ions o he same ne wo k, only by e ical and ho izon al scaling.
On he o he hand, hese images a e no sui able o isual in e p e a ion, as hey
p esen a dis o ed iew o he issue ana omy. Because no all ul asound equipmen
p o ides di ec access o B-Scan da a, a Sec o -Image- o-B-Scan con e sion algo i hm
would be needed. Wi h a simila app oach han scan-con e e algo i hms om B-Scan o
sec o image, i could be based on a simple bilinea in e pola ion algo i hm a e de ining a
se o beam lines ha co e he use ul a ea o he sec o image (g een lines in Figu e 2Le ).
In his wo k, we had access o he B-Scan aw da a gene a ed by ou sys em, so no Sec o -
Image- o-B-Scan p ocess was needed.

Appl. Sci. 2024,14, 11930 6 o 23
Fu he mo e, o possible u u e ha dwa e implemen a ions o he p oposed models,
using B-scan da a would be op imal in he sense ha i educes he amoun o in o ma ion
o be p ocessed, and uses da a in a aw o ma a ailable a low ha dwa e le el.
Fo de ining he size o he B-Scan images, he ypical numbe o beams in an ul a-
sound scanne mus be aken in o accoun . Fo con ex p obes like he one used in his
wo k, an ac i e sub-ape u e is used o gene a ing each beam, ypically wi h be ween 32
and 64 elemen s. Tha sub-ape u e is mo ed along he a ay in one o se e al elemen s
s ep, o ming he B-Scan. Fo example, a 128 elemen s a ay wi h a 32 elemen ape u e can
gene a e up o 96 scan lines, while a 196 elemen s a ay wi h an ape u e o 64 elemen s
gene a es 132 scan lines. Based on hese ypical numbe s, an inpu wid h o 128 lines was
selec ed, being powe o 2 o ope a ions op imiza ion.
The size o he e ical di ec ion is ela ed o he equency con en o he signal and
he sampling a e. Fo an ideal 100% bandwid h a ay, he maximum equency con en o
he signal en elope is equal o hal he a ay cen e equency. To econs uc he en elope
wi hou aliasing, he pixel densi y in he e ical di ec ion should be, a leas , able o sample
he signal a double o ha equency (Nyquis c i e ia). Fo he a ay used in his wo k wi h
3.5 MHz cen e equency and 70% bandwid h, he numbe o pixels equi ed o sampling
up o 70 mm and 90 mm is 210 and 294, espec i ely. Based on hese numbe s, a heigh o
256 was selec ed o he ne wo k, which imposes a ade-o be ween aining and in e ence
cos and image quali y. In cases when la ge images a e used, hey should be scaled down
using comp ession algo i hms ha p ese e he a i ac s’ in o ma ion. Fo example, in he
scanne used in his wo k, a da a educ ion algo i hm wi hou peak in o ma ion losses is
a ailable [
26
] and was used o accommoda e he B-Scan heigh o he ne wo k size. In Figu e 3,
a compa ison example be ween sec o ial and B-scan images is shown.
Appl. Sci. 2024, 14, x FOR PEER REVIEW 6 o 25
posi ion on he pleu a and o he scanne con igu a ion. Fu he mo e, scanne con igu a-
ion pa ame e s like he ape u e angle α o he ini ial and inal ange
1
and
2
only affec
he size M × N o he image, which simpli ies adap ing images acqui ed wi h diffe en
con igu a ions o he same ne wo k, only by e ical and ho izon al scaling.
On he o he hand, hese images a e no sui able o isual in e p e a ion, as hey
p esen a dis o ed iew o he issue ana omy. Because no all ul asound equipmen p o-
ides di ec access o B-Scan da a, a Sec o -Image- o-B-Scan con e sion algo i hm would
be needed. Wi h a simila app oach han scan-con e e algo i hms om B-Scan o sec o
image, i could be based on a simple bilinea in e pola ion algo i hm a e de ining a se
o beam lines ha co e he use ul a ea o he sec o image (g een lines in Figu e 2 Le ).
In his wo k, we had access o he B-Scan aw da a gene a ed by ou sys em, so no Sec o -
Image- o-B-Scan p ocess was needed.
Fu he mo e, o possible u u e ha dwa e implemen a ions o he p oposed models,
using B-scan da a would be op imal in he sense ha i educes he amoun o in o ma ion
o be p ocessed, and uses da a in a aw o ma a ailable a low ha dwa e le el.
Fo de ining he size o he B-Scan images, he ypical numbe o beams in an ul a-
sound scanne mus be aken in o accoun . Fo con ex p obes like he one used in his
wo k, an ac i e sub-ape u e is used o gene a ing each beam, ypically wi h be ween 32
and 64 elemen s. Tha sub-ape u e is mo ed along he a ay in one o se e al elemen s
s ep, o ming he B-Scan. Fo example, a 128 elemen s a ay wi h a 32 elemen ape u e
can gene a e up o 96 scan lines, while a 196 elemen s a ay wi h an ape u e o 64 ele-
men s gene a es 132 scan lines. Based on hese ypical numbe s, an inpu wid h o 128
lines was selec ed, being powe o 2 o ope a ions op imiza ion.
The size o he e ical di ec ion is ela ed o he equency con en o he signal and
he sampling a e. Fo an ideal 100% bandwid h a ay, he maximum equency con en
o he signal en elope is equal o hal he a ay cen e equency. To econs uc he en e-
lope wi hou aliasing, he pixel densi y in he e ical di ec ion should be, a leas , able o
sample he signal a double o ha equency (Nyquis c i e ia). Fo he a ay used in his
wo k wi h 3.5 MHz cen e equency and 70% bandwid h, he numbe o pixels equi ed
o sampling up o 70 mm and 90 mm is 210 and 294, espec i ely. Based on hese num-
be s, a heigh o 256 was selec ed o he ne wo k, which imposes a ade-off be ween
aining and in e ence cos and image quali y. In cases when la ge images a e used, hey
should be scaled down using comp ession algo i hms ha p ese e he a i ac s’ in o -
ma ion. Fo example, in he scanne used in his wo k, a da a educ ion algo i hm wi hou
peak in o ma ion losses is a ailable [26] and was used o accommoda e he B-Scan heigh
o he ne wo k size. In Figu e 3, a compa ison example be ween sec o ial and B-scan im-
ages is shown.
(a)
(b)
Figu e 3. Compa ison be ween sec o ial image (a) and B-Scan (b).
Figu e 3. Compa ison be ween sec o ial image (a) and B-Scan (b).
2.2.2. Neu al Ne wo k Ou pu Da a
Labelling Tools
One o he limi a ions o he used da ase [
24
] is ha i was labeled by an expe
physician a ideo le el, while o aining a segmen a ion ne wo k, i mus be labeled
a ame le el. While asking a physician o label such a la ge da ase ame-by- ame is
usually imp ac ical, a semi-au oma ed algo i hm was de eloped o his ask, based on
he ini ial label o he p esence o each a i ac in he ideo. Depending on he ype o
a i ac , his p ocess is ully au oma ed o equi es o iden i y a key ame whe e he a i ac
is isible and manually segmen i , being hen au oma ically ollowed in he subsequen
ames in he ideo. These algo i hms a e explained in he ollowing subsec ions.
Pleu al Line Labelling
One o he mos impo an indica ions o iden i y in a lung ul asound image is
he pleu a, because i de ines he zone whe e he a i ac s mus be looked o . Co ec ly
iden i ying he pleu a helps o disca d zones ha do no ha e o be analyzed, like he
Appl. Sci. 2024,14, 11930 7 o 23
ibs and hei shadows and he muscle and a a ea abo e i . Gi ing his in o ma ion
o he neu al ne wo k could imp o e he esul s, as i will lea n ha he B-Lines and
consolida ions a e loca ed bellow he pleu a, and no in o he a eas o he image.
The algo i hm o au oma ically segmen ing he pleu a is based on he ac ha when
he ul asound p obe emains s a iona y, he image abo e he pleu a ( a and muscle)
emains basically unchanged be ween ames, while a ia ions in he image occu below
he pleu a due o he espi a o y cycle. Then, by sub ac ing consecu i e ames, an
auxilia y image is o med, whe e he uppe egion is almos black and he lowe pa is
b igh . Fu he mo e, i se e al o hese images a e a e aged, he andom na u e o he
speckle in he lowe pa gene a es a qui e homogeneous egion ha can be mo e easily
dis inguished om he uppe egion. The on ie be ween hese egions can be conside ed
a i s app oxima ion o he pleu a line.
Gi en a ideo V, o med by K ames wi h Bde ined by:
V={B1,B2, . . . , BK}(1)
whe e
Bi=[bi(m,n)] ∈RMxN o 1≤i≤K, 1 ≤m≤M and 1≤n≤N(2)
and b
i
(m,n) is he pixel in ensi y o image ia posi ion (m,n), he ou pu o he p oposed
algo i hm is:
e
V=ne
B1,e
B2, . . . , e
BK−Lo(3)
whe e
e
Bi=1
L
k=i+L−1
∑
k=i
|Bi+1−Bi|(4)
wi h Lbeing he leng h o he a e aging il e . Figu e 4shows an example ou pu o he
algo i hm. On he le is he o iginal B
i
image o a ame, and on he igh is i s il e ed
e sion V
i
, showing he wo o me ly men ioned da k and b igh zones. Fo de ec ing
hei bounda y, a dynamic h eshold was applied o each e ical line o he il e ed image,
calcula ed om he a e age ampli ude o he uppe pa o he image:
e
Pj=mini|Aj(i)> hj(5)
whe e Ajis he e ical line o he il e ed image a column j,e
Pjis he e ical index o he
pleu a guess o ha line, and h
j
he applied h eshold. Figu e 4b shows, wi h g een do s,
an example o his se o ini ial pleu a aw guess poin s
e
Pn
, which a e hen used o i a
second o de polynomial o ob ain a smoo h ep esen a ion
Pn
o he ini ial guess ( ed line
in Figu e 4b).
Appl. Sci. 2024, 14, x FOR PEER REVIEW 8 o 25
i a second o de polynomial o ob ain a smoo h ep esen a ion 𝑃 o he ini ial guess
( ed line in Figu e 4b).
I is wo h men ioning ha di ec ly applying a h eshold o he o iginal image (Figu e
4a) is e y p one o misde ec ions, as o he ho izon al b igh lines in he uppe egion o
he image a e p esen , wi h pixel in ensi y alues e en la ge han hose o he pleu a. On
he o he hand, he auxilia y image il e ed by he p oposed me hod is e y sui able o a
simple h eshold de ec ion because o i s s epped na u e.
(a) (b)
Figu e 4. Fi s pleu a app oxima ion example. (a) o iginal B-scan image; (b) il e ed image.
This i s app oxima ion does no ake in o accoun ha a b igh line has o be seen
in he image o be conside ed pa o he pleu a line. Then, he P
n
indexes a e used as he
cen e o a e ical window wi h W samples, whe e o look o he maximum ampli ude
o he o iginal image. I his ampli ude is la ge han a global h eshold T, i is conside ed
ha he pleu a is p esen in he window. Then, a −3 dB signal d op c i e ia is used o ind
he high ampli ude zone a ound he maximum, and hose pixels a e labeled as belonging
o he pleu a in a bina y mask called M. The ma hema ical o mula ion o his pa o he
algo i hm is as ollows:
𝑀󰇛𝑚,𝑛󰇜=󰇫1, 𝑖𝑓 𝑏󰇛𝑚,𝑛󰇜
𝐴
󰇛𝑚󰇜⋅10 
 ;
𝑓
𝑜𝑟 𝑛 ∈󰇟𝑃
−𝑊, 𝑃
𝑊󰇠
0, 𝑜𝑡ℎ𝑒𝑟 (6)
Keeping he p e ious nomencla u e, his condi ion checks i he ampli ude a posi-
ion (m, n) emains wi hin 3 dB o he maximum ampli ude a 𝑃
 (𝐴󰇜, which co e-
sponds o an app oxima e 29% d op in ampli ude. I his condi ion is me , he pixel is
conside ed pa o he pleu a and is ma ked as 1 in he bina y mask. O he wise, i is
ma ked as 0. Fo his pa icula da ase , he app oxima ion is applied in a sea ching win-
dow W = 20 samples a ound he pleu a maximum ampli ude iden i ied in each scan line,
bu W should be adjus ed acco ding o he image esolu ion o include a egion o abou
6 mm a ound he pleu a.
Finally, mo phological opening and closing il e s sized a 3 × 3 a e applied o e ine
and smoo h he pleu a labeling mask (Figu e 5c), elimina ing ou lie s and gi ing a mo e
p ecise ep esen a ion o he highe ampli ude line. Fo each ideo, a se o K-L pleu a
bina y mask images a e gene a ed, o be used du ing he aining p ocess o he ne wo k.
Figu e 4. Fi s pleu a app oxima ion example. (a) o iginal B-scan image; (b) il e ed image.
Appl. Sci. 2024,14, 11930 8 o 23
I is wo h men ioning ha di ec ly applying a h eshold o he o iginal image
(
Figu e 4a
) is e y p one o misde ec ions, as o he ho izon al b igh lines in he uppe
egion o he image a e p esen , wi h pixel in ensi y alues e en la ge han hose o he
pleu a. On he o he hand, he auxilia y image il e ed by he p oposed me hod is e y
sui able o a simple h eshold de ec ion because o i s s epped na u e.
This i s app oxima ion does no ake in o accoun ha a b igh line has o be seen
in he image o be conside ed pa o he pleu a line. Then, he P
n
indexes a e used as he
cen e o a e ical window wi h Wsamples, whe e o look o he maximum ampli ude
o he o iginal image. I his ampli ude is la ge han a global h eshold T, i is conside ed
ha he pleu a is p esen in he window. Then, a
−
3 dB signal d op c i e ia is used o ind
he high ampli ude zone a ound he maximum, and hose pixels a e labeled as belonging
o he pleu a in a bina y mask called M. The ma hema ical o mula ion o his pa o he
algo i hm is as ollows:
Mi(m,n)=(1, i bi(m,n)≥Amax(m)·10−3
20 ; o n ∈he
Pj−W,e
Pj+Wi
0, o he (6)
Keeping he p e ious nomencla u e, his condi ion checks i he ampli ude a posi ion
(m,n) emains wi hin 3 dB o he maximum ampli ude a
e
Pj(Amax)
, which co esponds o an
app oxima e 29% d op in ampli ude. I his condi ion is me , he pixel is conside ed pa o he
pleu a and is ma ked as 1 in he bina y mask. O he wise, i is ma ked as 0. Fo his pa icula
da ase , he app oxima ion is applied in a sea ching window W = 20 samples a ound he
pleu a maximum ampli ude iden i ied in each scan line, bu W should be adjus ed acco ding
o he image esolu ion o include a egion o abou 6 mm a ound he pleu a.
Finally, mo phological opening and closing il e s sized a 3
×
3 a e applied o e ine
and smoo h he pleu a labeling mask (Figu e 5c), elimina ing ou lie s and gi ing a mo e
p ecise ep esen a ion o he highe ampli ude line. Fo each ideo, a se o K-L pleu a
bina y mask images a e gene a ed, o be used du ing he aining p ocess o he ne wo k.
Appl. Sci. 2024, 14, x FOR PEER REVIEW 9 o 25
(a) (b) (c)
Figu e 5. Pos -p ocessing adjus men o au oma ic pleu al anno a ion. (a) B-scan image; (b) ini ial
pleu al anno a ion app oxima ion; (c) inal pleu a anno a ion.
Figu e 6 summa izes he wo k low o he pleu al line au oma ic anno a ion algo-
i hm.
Figu e 6. Au oma ic pleu al line anno a ion lowcha .
A-Line Labelling
In case o A-lines, he de eloped algo i hm de ec s hese pa e ns as echoes o he
pleu a occu ing a dep hs ha a e mul iples o he p obe- o-pleu a dis ance. A sea ch
window W 󰇟𝑤,𝑤󰇠 is applied a hose dep hs o ind signal peaks ha c oss a h eshold,
and hen e i y hei alidi y by compa ing wi h he a e age ampli ude in consecu i e
lines: 𝑎=a g max
∈󰇟

,

󰇠
𝐴
󰇛𝑥󰇜 (7)
whe e 𝑎 co esponds o he index in he A-scan signal 𝐴 whe e he maximum ampli-
ude is loca ed wi hin he window 󰇟𝑤,𝑤󰇠, which co esponds wi h an ini ial guess o he
A-Line posi ion.
As in he case o he pleu a line, o de ine he g ound u h mask, a −3 db signal d op
c i e ia equi alen o he Fo mula (6) is used, ollowed by he applica ion o opening and
closing mo phological il e s sized a 3 × 3 o e ine he mask and elimina ing ou lie s.
Figu e 7 summa izes he wo k low o he A-lines au oma ic anno a ion algo i hm.
Figu e 7. Au oma ic A-line anno a ion lowcha .
Figu e 5. Pos -p ocessing adjus men o au oma ic pleu al anno a ion. (a) B-scan image; (b) ini ial
pleu al anno a ion app oxima ion; (c) inal pleu a anno a ion.
Figu e 6summa izes he wo k low o he pleu al line au oma ic anno a ion algo i hm.
Appl. Sci. 2024, 14, x FOR PEER REVIEW 9 o 25
(a) (b) (c)
Figu e 5. Pos -p ocessing adjus men o au oma ic pleu al anno a ion. (a) B-scan image; (b) ini ial
pleu al anno a ion app oxima ion; (c) inal pleu a anno a ion.
Figu e 6 summa izes he wo k low o he pleu al line au oma ic anno a ion algo-
i hm.
Figu e 6. Au oma ic pleu al line anno a ion lowcha .
A-Line Labelling
In case o A-lines, he de eloped algo i hm de ec s hese pa e ns as echoes o he
pleu a occu ing a dep hs ha a e mul iples o he p obe- o-pleu a dis ance. A sea ch
window W 󰇟𝑤,𝑤󰇠 is applied a hose dep hs o ind signal peaks ha c oss a h eshold,
and hen e i y hei alidi y by compa ing wi h he a e age ampli ude in consecu i e
lines: 𝑎=a g max
∈󰇟

,

󰇠
𝐴
󰇛𝑥󰇜 (7)
whe e 𝑎 co esponds o he index in he A-scan signal 𝐴 whe e he maximum ampli-
ude is loca ed wi hin he window 󰇟𝑤,𝑤󰇠, which co esponds wi h an ini ial guess o he
A-Line posi ion.
As in he case o he pleu a line, o de ine he g ound u h mask, a −3 db signal d op
c i e ia equi alen o he Fo mula (6) is used, ollowed by he applica ion o opening and
closing mo phological il e s sized a 3 × 3 o e ine he mask and elimina ing ou lie s.
Figu e 7 summa izes he wo k low o he A-lines au oma ic anno a ion algo i hm.
Figu e 7. Au oma ic A-line anno a ion lowcha .
Figu e 6. Au oma ic pleu al line anno a ion lowcha .
Appl. Sci. 2024,14, 11930 9 o 23
A-Line Labelling
In case o A-lines, he de eloped algo i hm de ec s hese pa e ns as echoes o he
pleu a occu ing a dep hs ha a e mul iples o he p obe- o-pleu a dis ance. A sea ch
window W
[w1,w2]
is applied a hose dep hs o ind signal peaks ha c oss a h eshold, and
hen e i y hei alidi y by compa ing wi h he a e age ampli ude in consecu i e lines:
aj=a g max
x∈[w1,w2]Aj(x)(7)
whe e
aj
co esponds o he index in he A-scan signal
Aj
whe e he maximum ampli ude
is loca ed wi hin he window
[w1,w2]
, which co esponds wi h an ini ial guess o he
A-Line posi ion.
As in he case o he pleu a line, o de ine he g ound u h mask, a
−
3 db signal d op
c i e ia equi alen o he Fo mula (6) is used, ollowed by he applica ion o opening and
closing mo phological il e s sized a 3 ×3 o e ine he mask and elimina ing ou lie s.
Figu e 7summa izes he wo k low o he A-lines au oma ic anno a ion algo i hm.
Appl. Sci. 2024, 14, x FOR PEER REVIEW 9 o 25
(a) (b) (c)
Figu e 5. Pos -p ocessing adjus men o au oma ic pleu al anno a ion. (a) B-scan image; (b) ini ial
pleu al anno a ion app oxima ion; (c) inal pleu a anno a ion.
Figu e 6 summa izes he wo k low o he pleu al line au oma ic anno a ion algo-
i hm.
Figu e 6. Au oma ic pleu al line anno a ion lowcha .
A-Line Labelling
In case o A-lines, he de eloped algo i hm de ec s hese pa e ns as echoes o he
pleu a occu ing a dep hs ha a e mul iples o he p obe- o-pleu a dis ance. A sea ch
window W 󰇟𝑤,𝑤󰇠 is applied a hose dep hs o ind signal peaks ha c oss a h eshold,
and hen e i y hei alidi y by compa ing wi h he a e age ampli ude in consecu i e
lines: 𝑎=a g max
∈󰇟

,

󰇠
𝐴
󰇛𝑥󰇜 (7)
whe e 𝑎 co esponds o he index in he A-scan signal 𝐴 whe e he maximum ampli-
ude is loca ed wi hin he window 󰇟𝑤,𝑤󰇠, which co esponds wi h an ini ial guess o he
A-Line posi ion.
As in he case o he pleu a line, o de ine he g ound u h mask, a −3 db signal d op
c i e ia equi alen o he Fo mula (6) is used, ollowed by he applica ion o opening and
closing mo phological il e s sized a 3 × 3 o e ine he mask and elimina ing ou lie s.
Figu e 7 summa izes he wo k low o he A-lines au oma ic anno a ion algo i hm.
Figu e 7. Au oma ic A-line anno a ion lowcha .
Figu e 7. Au oma ic A-line anno a ion lowcha .
B-Lines Labelling
The app oach employed o ecognize hese pneumonia indica o s in ol es i ing each
A-scan line
Aj
wi hin he image o a linea unc ion
y(x)
ha ini ia es om he iden i ied
pleu al line ep esen ing he ampli ude o he signal a a dep h x:
y(x)=m·x+d(8)
whe e
m
is he slope and
d
is he in e cep . Then, se e al h esholds a e es ablished o
con i m he de ec ion o he B-line based on a maximum slope
S h
, s anda d de ia ion
σ h
,
and a e age ampli ude o he signal
Aa g h
a e he pleu al line de ec ed. The e o e, he
lines conside ed as B-line
Blj
a e calcula ed as ollows, as a boolean ec o wi h leng h n
(numbe o lines):
Blj=T ue,i m >S h and σ<σ h and Aa g >Aa g h
False,o he s (9)
Once he A-scans wi h B-lines a e known, he anno a ed masks a e gene a ed ma king
B-lines egion below he pleu a line ame by ame.
Figu e 8summa izes he wo k low o he B-lines au oma ic anno a ion algo i hm.
Appl. Sci. 2024, 14, x FOR PEER REVIEW 10 o 25
B-Lines Labelling
The app oach employed o ecognize hese pneumonia indica o s in ol es i ing
each A-scan line 𝐴 wi hin he image o a linea unc ion 𝑦󰇛𝑥󰇜 ha ini ia es om he
iden i ied pleu al line ep esen ing he ampli ude o he signal a a dep h 𝑥:
𝑦󰇛𝑥󰇜=𝑚⋅𝑥𝑑 (8)
whe e 𝑚 is he slope and 𝑑 is he in e cep . Then, se e al h esholds a e es ablished o
con i m he de ec ion o he B-line based on a maximum slope 𝑆, s anda d de ia ion
𝜎, and a e age ampli ude o he signal 𝐴  a e he pleu al line de ec ed. The e o e,
he lines conside ed as B-line 𝐵𝑙 a e calcula ed as ollows, as a boolean ec o wi h leng h
n (numbe o lines):
𝐵𝑙=𝑇𝑟𝑢𝑒, 𝑖𝑓 𝑚>𝑆 𝑎𝑛𝑑 𝜎𝜎 𝑎𝑛𝑑
𝐴
 >
𝐴
 
𝐹𝑎𝑙𝑠𝑒, 𝑜𝑡ℎ𝑒𝑟𝑠 (9)
Once he A-scans wi h B-lines a e known, he anno a ed masks a e gene a ed ma k-
ing B-lines egion below he pleu a line ame by ame.
Figu e 8 summa izes he wo k low o he B-lines au oma ic anno a ion algo i hm.
Figu e 8. Au oma ic B-line anno a ion lowcha .
Consolida ion Labelling
While pleu a, A-lines, and B-lines can be obus ly de ec ed by qui e simple algo-
i hms applied along each scan line, he consolida ions canno be ackled wi h he same
app oach. As hey a e in insically wo-dimensional s uc u es, i is no possible o de ec
hem on a line-by-line basis, and image algo i hms a e needed.
The app oach ollowed in his wo k was ha , gi en a ideo labeled by he expe as
con aining a consolida ion, a key ame whe e his consolida ion is seen is i s iden i ied.
Then, i is manually delinea ed using a cus om de eloped in e ac i e ool (Figu e 9). Fi-
nally, an op ical low algo i hm [27,28] acks he mo emen o he consolida ion in he
subsequen ames, au oma ically gene a ing he g ound u h masks o he en i e ideo.
The op ical low algo i hm is based on he p inciple o selec ing a se o e e ence poin s
and acking hem h ough he ideo. This algo i hm is pa icula ly sui ed o ul asound
images because he ex u e gene a ed by he image speckle can be used o his pu pose.
The implemen ed applica ion eques s ame-by- ame alida ion om he use o ensu e
co ec labelling. In case he algo i hm ails in he de ec ion, he poin s o in e es will be
e-selec ed again.
These semi-au oma ic algo i hms aim o dec ease he ime equi ed o labelling id-
eos a ame le el, while hey only equi e manual in e en ion in he case o consolida-
ions and o segmen ing a unique ame. Ne e heless, because hey can also ail in de-
ec ing he indica ions, supe ision o he whole ideo a e i is labeled is equi ed, wi h
he possibili y o elimina ing hose ames whe e he labeling is conside ed w ong.
Figu e 8. Au oma ic B-line anno a ion lowcha .
Consolida ion Labelling
While pleu a, A-lines, and B-lines can be obus ly de ec ed by qui e simple algo i hms
applied along each scan line, he consolida ions canno be ackled wi h he same app oach.
As hey a e in insically wo-dimensional s uc u es, i is no possible o de ec hem on a
line-by-line basis, and image algo i hms a e needed.
Appl. Sci. 2024,14, 11930 16 o 23
3.1. Model Resul s
As explained in he alida ion sec ion, a s udy was conduc ed o alida e he model’s
esponse ame by ame. The esul s a e p esen ed in Tables 1–4, whe e he Dice coe icien s,
In e sec ion o e Union (IoU), F1-sco e, ecall, and p ecision alues a e calcula ed. Figu e 14
shows he beha iou o he model applying di e en h esholds o he ne wo k ou pu
showing he mean alues o he me ics in he es da ase . Gi en he s abili y o hese
esul s, i makes sense o apply 0.5 as a gene ic h eshold o he ne wo k ou pu .
Appl. Sci. 2024, 14, x FOR PEER REVIEW 18 o 25
P ecision 0.81 0.11 0.97 0.14 0.88 0.28 0.69 0.39
Table 2. Table o F1-sco e o he model a a ame le el in es da ase .
A i ac Pleu a Consolida ion B-Line A-Line
F1-sco e 0.83 0.97 0.91 0.73
Table 3. Table o s a is ic me ics o model a a ame le el in ou o aining pa ien s.
Me ic
A i ac
Pleu a Consolida ion B-Line A-Line
Mean S d Mean S d Mean S d Mean S d
Dice coe icien 0.77 0.13 0.85 0.34 0.62 0.41 0.48 0.41
IoU 0.64 0.15 0.84 0.35 0.58 0.42 0.42 0.40
Recall 0.78 0.17 0.88 0.31 0.81 0.31 0.59 0.41
P ecision 0.79 0.15 0.95 0.20 0.70 0.40 0.64 0.40
Table 4. Table o F1-sco e o he model a a ame le el in ou o aining pa ien s.
A i ac Pleu a Consolida ion B-Line A-Line
F1-sco e 0.78 0.91 0.75 0.61
Figu e 14. Th eshold compa ison in CNN ou pu applied o es da ase .
Figu e 15 shows diffe en ne wo k ou pu examples modi ying he sa u a ion o he
image a he inpu o he ne wo k, o simula e he gain a ia ion usually applied on an
ul asound scanne . Segmen a ion and de ec ion o a i ac s emains s able, gua an eeing
co ec ope a ion in spi e o signal sa u a ion.
Figu e 15a shows he lea ning and gene aliza ion capaci y o he ained model. The
labelling algo i hms o he A-line only conside s a single A-line a double he dis ance
Figu e 14. Th eshold compa ison in CNN ou pu applied o es da ase .
Table 1. Table o s a is ic me ics o model a a ame le el in es da ase .
Me ic
A i ac
Pleu a Consolida ion B-Line A-Line
Mean S d Mean S d Mean S d Mean S d
Dice coe icien 0.83 0.08 0.96 0.17 0.87 0.30 0.62 0.40
IoU 0.72 0.11 0.95 0.18 0.84 0.30 0.56 0.40
Recall 0.86 0.10 0.97 0.12 0.94 0.17 0.78 0.33
P ecision 0.81 0.11 0.97 0.14 0.88 0.28 0.69 0.39
Table 2. Table o F1-sco e o he model a a ame le el in es da ase .
A i ac Pleu a Consolida ion B-Line A-Line
F1-sco e 0.83 0.97 0.91 0.73
I is in e es ing o no e in Tables 1–4how he esul s, be ween pa ien s included in he
aining and no , p esen a simila beha iou . The ne wo k, despi e he ac ha in bo h
cases a e images ha ha e ne e been seen, is able o ma ch mo e accu a ely in da a om
known pa ien s (Tables 1and 2). Howe e , he esul s shown in Tables 3and 4sugges ha
he neu al ne wo k has been able o gene alize he p oblem and main ain high success a es.

Appl. Sci. 2024,14, 11930 17 o 23
Table 3. Table o s a is ic me ics o model a a ame le el in ou o aining pa ien s.
Me ic
A i ac
Pleu a Consolida ion B-Line A-Line
Mean S d Mean S d Mean S d Mean S d
Dice coe icien 0.77 0.13 0.85 0.34 0.62 0.41 0.48 0.41
IoU 0.64 0.15 0.84 0.35 0.58 0.42 0.42 0.40
Recall 0.78 0.17 0.88 0.31 0.81 0.31 0.59 0.41
P ecision 0.79 0.15 0.95 0.20 0.70 0.40 0.64 0.40
Table 4. Table o F1-sco e o he model a a ame le el in ou o aining pa ien s.
A i ac Pleu a Consolida ion B-Line A-Line
F1-sco e 0.78 0.91 0.75 0.61
Figu e 15 shows di e en ne wo k ou pu examples modi ying he sa u a ion o he
image a he inpu o he ne wo k, o simula e he gain a ia ion usually applied on an
ul asound scanne . Segmen a ion and de ec ion o a i ac s emains s able, gua an eeing
co ec ope a ion in spi e o signal sa u a ion.
Appl. Sci. 2024, 14, x FOR PEER REVIEW 19 o 25
be ween he p obe and he pleu a; howe e , he model can de ec he second and e en he
hi d echo in some pa icula cases, demons a ing he capaci y o he model in gene aliz-
ing and lea ning he p oblem. I is wo h men ioning ha his ac nega i ely affec s he
alida ion me ics calcula ed in Tables 2 and 4, whe e he alse posi i e alues o A-lines
a e highe . This is because he second and hi d occu ences o he A-lines a e no labelled
in he inpu da ase , which is a limi a ion o he labelling algo i hm a he han o he
model i sel .
(a)
(b)
Figu e 15. Con .
Appl. Sci. 2024,14, 11930 18 o 23
Appl. Sci. 2024, 14, x FOR PEER REVIEW 20 o 25
(c)
Figu e 15. Resul s compa a i e wi h diffe en image gain: (a) heal hy lung; (b) pa hological lung; (c)
g a e lung.
3.2. Real-Time Implemen a ion Resul s
As al eady men ioned, he implemen ed so wa e ( e sion 1.0) is capable o p o-
cessing and displaying in eal- ime he pleu al line, B-lines, consolida ions, and A-lines.
The algo i hm can calcula e and ob ain he pe cen age o pleu a affec ed by B-lines, which
could be use ul o physicians when classi ying and de e mining he se e i y o a pa ien ’s
condi ion.
The implemen a ion p oposed in his s udy achie es a p ocessing a e o up o 20
p edic ions and image upda e pe second using an oc a-co e i5 CPU p ocesso and a
NVIDIA GeFo ce RTX 2060 GPU, which is a medium- ange ha dwa e se -up. This igu e
coincides wi h he ame a e gi en by he ul asound scanne , so we can s a e ha he
solu ion ope a es in s ic eal- ime. In e ms o compu a ional cos in he diffe en p o-
cesses, he main p ocess akes abou 10 ms, which gi es a luid and la ency- ee eeling
du ing he scan, while he eal- ime compu a ion p ocess akes abou 50 ms o which 20–
25 ms is o model in e ence on he GPU. I is wo h men ioning ha , in he cu en im-
plemen a ion, he inal image e esh a e depends on he numbe o a i ac s de ec ed,
due o he espec i e pos -p ocessing and pain ing algo i hms ha a e no ye op imized.
The e o e, he effec i e ame a e ob ained was be ween 17 and 20 FPS, which should be
imp o ed by pa allelizing he las s ages o he algo i hm.
Rega ding p edic ion capabili y a he ideo le el, he esul s gi en by he model a e
compa ed wi h he opinion o an expe physician. Two diffe en calcula ions a e pe -
o med: i s , i s pe o mance wi h he whole da ase is assessed (Table 5), and second, a
speci ic compa ison is made o he se o pa ien s excluded om aining (Table 6). Wi h
hese esul s, i is wo h no ing ha he numbe o alse posi i es and alse nega i es a e
balanced in he case o de ec ion o consolida ions and B-lines, which is posi i e since i is
a sign ha he e is no bias in he aining. Addi ionally, o ensu e ha he model pe o ms
well o no mal cases, speci ic me ics such as accu acy, alse posi i es, and alse nega i es
we e calcula ed o ideos showing only A-lines o no a i ac s a all. These me ics a e
included in Tables 5 and 6 unde he “No mal lung” column. The esul s demons a e ha
he model achie es balanced pe o mance, e en o heal hy lungs, despi e he da ase o-
cusing mo e on pa hological indings.
Figu e 15. Resul s compa a i e wi h di e en image gain: (a) heal hy lung; (b) pa hological lung;
(c) g a e lung.
Figu e 15a shows he lea ning and gene aliza ion capaci y o he ained model. The
labelling algo i hms o he A-line only conside s a single A-line a double he dis ance
be ween he p obe and he pleu a; howe e , he model can de ec he second and e en he
hi d echo in some pa icula cases, demons a ing he capaci y o he model in gene alizing
and lea ning he p oblem. I is wo h men ioning ha his ac nega i ely a ec s he
alida ion me ics calcula ed in Tables 2and 4, whe e he alse posi i e alues o A-lines
a e highe . This is because he second and hi d occu ences o he A-lines a e no labelled
in he inpu da ase , which is a limi a ion o he labelling algo i hm a he han o he
model i sel .
3.2. Real-Time Implemen a ion Resul s
As al eady men ioned, he implemen ed so wa e ( e sion 1.0) is capable o p ocessing
and displaying in eal- ime he pleu al line, B-lines, consolida ions, and A-lines. The algo-
i hm can calcula e and ob ain he pe cen age o pleu a a ec ed by B-lines, which could be
use ul o physicians when classi ying and de e mining he se e i y o a pa ien ’s condi ion.
The implemen a ion p oposed in his s udy achie es a p ocessing a e o up o 20 p e-
dic ions and image upda e pe second using an oc a-co e i5 CPU p ocesso and a NVIDIA
GeFo ce RTX 2060 GPU, which is a medium- ange ha dwa e se -up. This igu e coincides
wi h he ame a e gi en by he ul asound scanne , so we can s a e ha he solu ion
ope a es in s ic eal- ime. In e ms o compu a ional cos in he di e en p ocesses, he
main p ocess akes abou 10 ms, which gi es a luid and la ency- ee eeling du ing he
scan, while he eal- ime compu a ion p ocess akes abou 50 ms o which 20–25 ms is o
model in e ence on he GPU. I is wo h men ioning ha , in he cu en implemen a ion, he
inal image e esh a e depends on he numbe o a i ac s de ec ed, due o he espec i e
pos -p ocessing and pain ing algo i hms ha a e no ye op imized. The e o e, he e ec i e
ame a e ob ained was be ween 17 and 20 FPS, which should be imp o ed by pa allelizing
he las s ages o he algo i hm.
Rega ding p edic ion capabili y a he ideo le el, he esul s gi en by he model
a e compa ed wi h he opinion o an expe physician. Two di e en calcula ions a e
pe o med: i s , i s pe o mance wi h he whole da ase is assessed (Table 5), and second, a
speci ic compa ison is made o he se o pa ien s excluded om aining (Table 6). Wi h
hese esul s, i is wo h no ing ha he numbe o alse posi i es and alse nega i es a e
balanced in he case o de ec ion o consolida ions and B-lines, which is posi i e since i is a
sign ha he e is no bias in he aining. Addi ionally, o ensu e ha he model pe o ms
well o no mal cases, speci ic me ics such as accu acy, alse posi i es, and alse nega i es
Appl. Sci. 2024,14, 11930 19 o 23
we e calcula ed o ideos showing only A-lines o no a i ac s a all. These me ics a e
included in Tables 5and 6unde he “No mal lung” column. The esul s demons a e
ha he model achie es balanced pe o mance, e en o heal hy lungs, despi e he da ase
ocusing mo e on pa hological indings.
Table 5. Table o accu acy a ideo le el in whole da ase .
Consolida ions
B-Lines A-Lines No mal Lung
Accu acy (%) 97.81 88.74 65.79 88.74
False posi i es (%) 0.44 4.09 23.54 7.16
False nega i es (%) 1.75 7.16 10.67 4.09
Table 6. Table o accu acy a ideo le el in ou o aining da ase .
Consolida ions
B-Lines A-Lines No mal Lung
Accu acy (%) 89.29 92.86 66.07 92.86
False posi i es (%) 3.57 1.79 26.79 5.36
False nega i es (%) 7.14 5.36 7.14 1.79
I can be obse ed ha he accu acy o he p oposed solu ion a he ideo le el is qui e
high, and i beha es s ably e en wi h pa ien s ou o aining. In he case o he de ec ion
o A lines, he esul s a e no ably wo se han o consolida ions and B-Lines, and his
because, in he labelling p ocess ca ied ou by he physician in [
24
], labelling he A lines
was no manda o y, gi en ha hey a e indica i e o heal hy lungs. The e o e, he da ase
labelling con ains alse nega i es in hose ideos whe e he A-Line is p esen , bu i was no
ma ked by he physician. This limi a ion migh pa ially explain he lowe pe o mance
in de ec ing A-lines compa ed o o he a i ac s. Howe e , he inclusion o no mal lung
me ics demons a es ha he solu ion emains obus and accu a e when dis inguishing
heal hy om pa hological cases. This is a limi a ion o his s udy, bu i does no imply ha
he pe o mance o he solu ion is ne e heless p omising.
Figu e 16 shows se e al examples o he sc een o he implemen ed so wa e in
ope a ion explained in Sec ion 2.5.4.
Appl. Sci. 2024, 14, x FOR PEER REVIEW 21 o 25
I can be obse ed ha he accu acy o he p oposed solu ion a he ideo le el is
qui e high, and i beha es s ably e en wi h pa ien s ou o aining. In he case o he de-
ec ion o A lines, he esul s a e no ably wo se han o consolida ions and B-Lines, and
his because, in he labelling p ocess ca ied ou by he physician in [24], labelling he A
lines was no manda o y, gi en ha hey a e indica i e o heal hy lungs. The e o e, he
da ase labelling con ains alse nega i es in hose ideos whe e he A-Line is p esen , bu
i was no ma ked by he physician. This limi a ion migh pa ially explain he lowe pe -
o mance in de ec ing A-lines compa ed o o he a i ac s. Howe e , he inclusion o no -
mal lung me ics demons a es ha he solu ion emains obus and accu a e when dis-
inguishing heal hy om pa hological cases. This is a limi a ion o his s udy, bu i does
no imply ha he pe o mance o he solu ion is ne e heless p omising.
Table 5. Table o accu acy a ideo le el in whole da ase .
Consolida ions B-Lines A-Lines No mal Lung
Accu acy (%) 97.81 88.74 65.79 88.74
False posi i es (%) 0.44 4.09 23.54 7.16
False nega i es (%) 1.75 7.16 10.67 4.09
Table 6. Table o accu acy a ideo le el in ou o aining da ase .
Consolida ions B-Lines A-Lines No mal Lung
Accu acy (%) 89.29 92.86 66.07 92.86
False posi i es (%) 3.57 1.79 26.79 5.36
False nega i es (%) 7.14 5.36 7.14 1.79
Figu e 16 shows se e al examples o he sc een o he implemen ed so wa e in op-
e a ion explained in Sec ion 2.5.4.
(a) (b)
(c) (d)
Figu e 16. Con .
Appl. Sci. 2024,14, 11930 20 o 23
Appl. Sci. 2024, 14, x FOR PEER REVIEW 21 o 25
I can be obse ed ha he accu acy o he p oposed solu ion a he ideo le el is
qui e high, and i beha es s ably e en wi h pa ien s ou o aining. In he case o he de-
ec ion o A lines, he esul s a e no ably wo se han o consolida ions and B-Lines, and
his because, in he labelling p ocess ca ied ou by he physician in [24], labelling he A
lines was no manda o y, gi en ha hey a e indica i e o heal hy lungs. The e o e, he
da ase labelling con ains alse nega i es in hose ideos whe e he A-Line is p esen , bu
i was no ma ked by he physician. This limi a ion migh pa ially explain he lowe pe -
o mance in de ec ing A-lines compa ed o o he a i ac s. Howe e , he inclusion o no -
mal lung me ics demons a es ha he solu ion emains obus and accu a e when dis-
inguishing heal hy om pa hological cases. This is a limi a ion o his s udy, bu i does
no imply ha he pe o mance o he solu ion is ne e heless p omising.
Table 5. Table o accu acy a ideo le el in whole da ase .
Consolida ions B-Lines A-Lines No mal Lung
Accu acy (%) 97.81 88.74 65.79 88.74
False posi i es (%) 0.44 4.09 23.54 7.16
False nega i es (%) 1.75 7.16 10.67 4.09
Table 6. Table o accu acy a ideo le el in ou o aining da ase .
Consolida ions B-Lines A-Lines No mal Lung
Accu acy (%) 89.29 92.86 66.07 92.86
False posi i es (%) 3.57 1.79 26.79 5.36
False nega i es (%) 7.14 5.36 7.14 1.79
Figu e 16 shows se e al examples o he sc een o he implemen ed so wa e in op-
e a ion explained in Sec ion 2.5.4.
(a) (b)
(c) (d)
Figu e 16. Applica ion isualiza ion sample: (a) B-lines (o ange) and consolida ion ( ed) de ec ion;
(b) no mal lung wi h A-lines (g een); (c) p obe mo emen de ec ed; (d) B line (o ange) and A-line
(g een) de eccion on a Lung phan om. On he igh o each image he C-scan image is shown.
4. Discussion
The implemen a ion o so wa e solu ions using deep lea ning algo i hms is a s ep
owa ds assis ed diagnosis in lung ul asound. Bo h he model and he comple e solu ion
demons a e solid pe o mance, alida ed by he esul s shown in he p e ious sec ion.
Based on s udies such as [
34
], which demons a e he a iabili y in physicians’ opinions,
and, aking in o accoun ha he p oposed me hod wo ks in eal- ime (up o 20 ps), i could
be a use ul ool o clinical p ac ice, helping physicians o quickly add ess lung condi ions
and educing he lea ning cu e o less expe ienced heal hca e pe sonnel in he ield.
Despi e he p omising esul s, he e a e some limi a ions in his wo k ha equi e
u he esea ch. One o he main ones is he limi ed ex en o he da abase used o ain
he model. Only 689 ideos om 30 pa ien s a e a ailable, which a e insu icien o ob ain
a ully ained model capable o gene alizing he en i e p oblem ac oss he ou a i ac s
sough . Fo example, in he case o ideos wi h consolida ions, only 58 o he ideos exhibi
ha a i ac , which also explains he high alues shown in Table 1. Ano he limi a ion o
he da ase is ha A-Lines we e no consis en ly labeled by he physician, as hey a e no
pa hological indings.
E en hough A-Lines, B-Lines, and consolida ions a e common a i ac s ound in
pa ien s wi h pulmona y a ec a ion o di e en o igins, a limi a ion o he da ase used is
ha only pa ien s wi h COVID-19 a e p esen . This would impose a bias when e alua ing
pa ien s wi h o he pulmona y diseases, which should be u he es ed in u u e s udies.
I should also be highligh ed ha he images used in his s udy belong o a single
scanne , which is likely o be a limi a ion, gi en he unique image cha ac e is ics o each
equipmen in e ms o noise, sampling equency, e c., which may in oduce bias o he
ne wo k aining and limi he c oss-pla o m applicabili y o he solu ion.
Ano he limi a ion lies in he semi-au oma ed labelling app oach used o consoli-
da ions. While he au oma ic labelling o pleu a, B-lines, and A-lines is suppo ed by a
alida ed algo i hm om a p e ious s udy [
24
], he labelling o consolida ions equi ed
manual in e en ion guided by physician anno a ions. This in oduces some a iabili y o
he da ase , as he iden i ica ion o consolida ions is inhe en ly subjec i e and dependen
on he physician’s expe ience. Al hough his me hod acili a ed he c ea ion o a clinically
meaning ul da ase , i ep esen s a limi a ion in e ms o ensu ing comple e consis ency
ac oss all anno a ions and highligh s he need o imp o ed au oma ed labelling app oaches
in u u e wo k.
Addi ionally, he implemen ed model shows an inabili y o always segmen he com-
ple e B-line. In some cases, only pa o he B-line is segmen ed, as is shown in Figu e 17.
This beha iou is co ec ed by he pos -p ocessing algo i hm, which ma ks he whole scan
line as a ec ed by B-line ega dless he e ical ex ension o he ou pu mask. The e o e,
he esul s shown in Tables 5and 6a e no a ec ed by his phenomenon, bu i indica es
Appl. Sci. 2024,14, 11930 21 o 23
ha he e is al eady some imp o emen ma gin in he ne wo k de ini ion and/o in he
aining p ocess.
Appl. Sci. 2024, 14, x FOR PEER REVIEW 23 o 25
he segmen a ion esul , in eal ime, in a seconda y sc een. While i is a lexible app oach
ha could ake ad an age o cu en ly a ailable scanne s, i would p obably ha e limi a-
ions ela ed o he limi ed quali y o he ecei ed image, he lack o con igu a ion in o -
ma ion, and he need o con e he sec o -scan image o a B-Mode image be o e p o-
cessing i . Al e na i ely, he solu ion could be deployed on he scanne ha dwa e, which
would be ideal in e ms o image quali y, accessibili y o aw da a (B-Scan), ease o usage,
e c., bu depends on he in e es o he scanne s manu ac u e s o in oducing hese ea-
u es in hei sys ems.
Figu e 17. B-line segmen a ion e o examples.
Rega ding egula o y aspec s, a solu ion like his will likely need o comply wi h
“So wa e as a Medical De ice” s anda ds ha de ine he me hodological and implemen-
a ion aspec s o wa an y pa ien sa e y and diagnosis quali y. Complying wi h hese
s anda ds should be pa o he indus ializa ion p ocess o he solu ion.
5. Conclusions
In his s udy, we ha e shown he effec i eness o employing deep lea ning models
along wi h signal p ocessing algo i hms o accu a e de ec ion o pleu a, A-lines, B-lines,
and consolida ions in lung ul asound images. The p oposed model has been de eloped
as a help o assis ed diagnosis, demons a ing p omising capabili ies in accu a ely seg-
men ing and de ec ing key ea u es in he images.
The p oposed implemen a ion exhibi s efficien eal- ime p ocessing capabili ies wi h
mode a e ha dwa e esou ces, achie ing a a e o up o 20 p edic ions pe second in a
mid- ange compu e . This makes i sui able o applica ion in clinical se ings, whe e
imely diagnosis is c ucial o pa ien ca e. While he esul s a e p omising, his solu ion
p esen s oppo uni ies o u u e esea ch and imp o emen s in model and a chi ec u e
design.
The use o semi-au oma ic labelling ools could be help ul when wo king wi h la ge
amoun s o da a. F ame-by- ame labelling is a ime-consuming ask, which canno be al-
ways pe o med by expe physicians in a ully manual way. In e ms o segmen a ion
accu acy, he ained model ob ained DICE alues o 83% o pleu a, 96% o consolida-
ions, 87% o B lines, and 62% o A lines. Addi ionally, he p oposed solu ion achie ed
92% o accu acy o no mal lung de ec ion. These esul s sugges ha solu ions o his ype
could be used in clinical en i onmen s, o educe subjec i i y in he in e p e a ion o im-
ages, and e en as a ool o help less expe ienced physicians o pe o m lung echog aphy
and educe he lea ning cu e o he echnique.
Au ho Con ibu ions: Concep ualiza ion, M.M., G.C., J.F.C. and J.C.; me hodology, M.M. and J.C.;
so wa e, M.M. and J.C.; alida ion, M.M., A.R., G.C., J.F.C. and J.C.; o mal analysis, M.M.; in es i-
ga ion, M.M. and J.C.; esou ces, J.F.C. and J.C.; da a cu a ion, M.M.; w i ing—o iginal d a p epa-
a ion, M.M. and J.C.; w i ing— e iew and edi ing, M.M., A.R., G.C., J.F.C. and J.C.; isualiza ion,
Figu e 17. B-line segmen a ion e o examples.
Fo u u e wo k, hese limi a ions could be add essed by including new da a in o
he aining o applying ans e lea ning echnics o adap he esul o o he scanne s
o ge a mo e scalable solu ion. New modi ica ions o he ne wo k a chi ec u e could be
also s udied o imp o e he beha iou o he model, as well as implemen ing in FPGAs
using p ope engines adap ed o ha ha dwa e a chi ec u e [
35
]. This can enhance i s
adap abili y o b oade clinical scena ios [36].
Wi h ega d o he implemen a ion o his me hod in clinical p ac ice, we o esee wo
app oaches. One is o deploy he algo i hm in a dedica ed ha dwa e ha is connec ed o a
con en ional scanne h ough an a ailable ou pu po (E he ne , HDMI, e c), showing he
segmen a ion esul , in eal ime, in a seconda y sc een. While i is a lexible app oach ha
could ake ad an age o cu en ly a ailable scanne s, i would p obably ha e limi a ions
ela ed o he limi ed quali y o he ecei ed image, he lack o con igu a ion in o ma ion,
and he need o con e he sec o -scan image o a B-Mode image be o e p ocessing i .
Al e na i ely, he solu ion could be deployed on he scanne ha dwa e, which would be
ideal in e ms o image quali y, accessibili y o aw da a (B-Scan), ease o usage, e c., bu
depends on he in e es o he scanne s manu ac u e s o in oducing hese ea u es in
hei sys ems.
Rega ding egula o y aspec s, a solu ion like his will likely need o comply wi h “So -
wa e as a Medical De ice” s anda ds ha de ine he me hodological and implemen a ion
aspec s o wa an y pa ien sa e y and diagnosis quali y. Complying wi h hese s anda ds
should be pa o he indus ializa ion p ocess o he solu ion.
5. Conclusions
In his s udy, we ha e shown he e ec i eness o employing deep lea ning models
along wi h signal p ocessing algo i hms o accu a e de ec ion o pleu a, A-lines, B-lines,
and consolida ions in lung ul asound images. The p oposed model has been de eloped as
a help o assis ed diagnosis, demons a ing p omising capabili ies in accu a ely segmen -
ing and de ec ing key ea u es in he images.
The p oposed implemen a ion exhibi s e icien eal- ime p ocessing capabili ies wi h
mode a e ha dwa e esou ces, achie ing a a e o up o 20 p edic ions pe second in a
mid- ange compu e . This makes i sui able o applica ion in clinical se ings, whe e imely
diagnosis is c ucial o pa ien ca e. While he esul s a e p omising, his solu ion p esen s
oppo uni ies o u u e esea ch and imp o emen s in model and a chi ec u e design.
The use o semi-au oma ic labelling ools could be help ul when wo king wi h la ge
amoun s o da a. F ame-by- ame labelling is a ime-consuming ask, which canno be
always pe o med by expe physicians in a ully manual way. In e ms o segmen a ion
accu acy, he ained model ob ained DICE alues o 83% o pleu a, 96% o consolida ions,
87% o B lines, and 62% o A lines. Addi ionally, he p oposed solu ion achie ed 92% o
accu acy o no mal lung de ec ion. These esul s sugges ha solu ions o his ype could
be used in clinical en i onmen s, o educe subjec i i y in he in e p e a ion o images, and

Appl. Sci. 2024,14, 11930 22 o 23
e en as a ool o help less expe ienced physicians o pe o m lung echog aphy and educe
he lea ning cu e o he echnique.
Au ho Con ibu ions: Concep ualiza ion, M.M., G.C., J.F.C. and J.C.; me hodology, M.M. and J.C.;
so wa e, M.M. and J.C.; alida ion, M.M., A.R., G.C., J.F.C. and J.C.; o mal analysis, M.M.; in es iga-
ion, M.M. and J.C.; esou ces, J.F.C. and J.C.; da a cu a ion, M.M.; w i ing—o iginal d a p epa a ion,
M.M. and J.C.; w i ing— e iew and edi ing, M.M., A.R., G.C., J.F.C. and J.C.; isualiza ion, M.M.;
supe ision, J.F.C. and J.C.; p ojec adminis a ion, J.C.; unding acquisi ion, J.C. All au ho s ha e
ead and ag eed o he published e sion o he manusc ip .
Funding: This esea ch was pa ially suppo ed by he p ojec PID2022-143271OB-I00, ounded
MCIN/AEI/10.13039/501100011033/FEDER, UE and by he ellowship PRE2019-088602 ounded by
MCIU (Spain).
Ins i u ional Re iew Boa d S a emen : The s udy was conduc ed acco ding o he guidelines o he
Decla a ion o Helsinki and app o ed by he Ins i u ional Re iew Boa d o Hospi al Uni e si a io
Pue a de Hie o (app o al code PI47-21, p ocol e sion 3.0 and da e o app o al 5 Ap il 2021).
In o med Consen S a emen : In o med consen was ob ained om all subjec s in ol ed in he s udy.
Da a A ailabili y S a emen : The da a p esen ed in his s udy a e a ailable on eques om he
co esponding au ho .
Acknowledgmen s: We would like o hank Yale Tung and Angela T ueba-Vicen e o hei suppo
and ad ice in he in e p e a ion o images and labelling p ocess.
Con lic s o In e es : The au ho s decla e no con lic o in e es .
Re e ences
1.
Fe ei o Gómez, M.; Dominguez Pazos, S.J. The Role o Lung Ul asound in he Managemen o Respi a o y Eme gencies. Open
Respi . A ch. 2022,4, 100206. [C ossRe ] [PubMed] [PubMed Cen al]
2.
Hansell, L.; Mil oss, M.; Delaney, A.; Tian, D.H.; Rajamani, A.; N oumenopoulos, G. Ba ie s and acili a o s o achie ing
compe ence in lung ul asound: A su ey o physio he apis s ollowing a lung ul asound aining cou se. Aus . C i . Ca e 2023,
36, 573–578. [C ossRe ] [PubMed]
3.
Solda i, G.; Sma giassi, A.; Demi, L.; Inchingolo, R. A i ac ual lung ul asonog aphy: I is a ma e o aps, o de , and diso de .
Appl. Sci. 2020,10, 1570. [C ossRe ]
4.
Bhoil, R.; Ahluwalia, A.; Chop a, R.; Su ya, M.; Bhoil, S. Signs and lines in lung ul asound. J. Ul ason. 2021,21, e225–e233.
[C ossRe ] [PubMed] [PubMed Cen al]
5.
Tung-Chen, Y.; Ma íde G acia, M.; Díez-Tascón, A.; Alonso-González, R.; Agudo-Fe nández, S.; Pa a-Go do, M.L.; Ossaba-Vélez,
S.; Rod íguez-Fue es, P.; Llamas-Fuen es, R. Co ela ion be ween ches compu ed omog aphy and lung ul asonog aphy in
pa ien s wi h co ona i us disease 2019 (COVID-19). Ul asound. Med. Biol. 2020,46, 2918–2926. [C ossRe ]
6.
Vieilla d-Ba on, A.; Go i, A.; Mayo, P. Lung ul asonog aphy as an al e na i e o ches compu ed omog aphy in COVID-19
pneumonia? In ensi e Ca e Med. 2020,46, 1908–1910. [C ossRe ] [PubMed] [PubMed Cen al]
7.
Demi, L.; Wol am, F.; Kle sy, C.; De Sil es i, A.; Fe e i, V.V.; Mulle , M.; Mille , D.; Fele i, F.; Wełnicki, M.; Buda, N.; e al. New
In e na ional Guidelines and Consensus on he Use o Lung Ul asound. J. Ul asound Med. 2023,42, 309–344. [C ossRe ]
8.
Gil-Rod íguez, J.; Pé ez de Rojas, J.; A anda-Lase na, P.; Bena en e-Fe nández, A.; Ma os-Ruiz, M.; Pe eg ina-Ri as, J.A.;
Gui ao-A abal, E. Ul asound indings o lung ul asonog aphy in COVID-19: A sys ema ic e iew. Eu . J. Radiol. 2022,148,
110156. [C ossRe ] [PubMed] [PubMed Cen al]
9.
Amisha; Malik, P.; Pa hania, M.; Ra hau , V.K. O e iew o a i icial in elligence in medicine. J. Fam. Med. P im. Ca e 2019,8,
2328–2331. [C ossRe ] [PubMed] [PubMed Cen al]
10.
Ci incione, G.; Canna a, S.; Cicce i, G.; P inzi, F.; Cu ie i, T.; Lo ino, M.; Mili ello, C.; Pase o, E.; Vi abile, S. T ans o me -Based
App oach o Melanoma De ec ion. Senso s 2023,23, 5677. [C ossRe ]
11.
Bek a¸s, B.; Em e, ˙
I.E.; Ka al, E.; Gulsecen, S. Classi ica ion o Mammog aphy Images by Machine Lea ning Techniques. In
P oceedings o he 2018 3 d In e na ional Con e ence on Compu e Science and Enginee ing (UBMK), Sa aje o, Bosnia and
He zego ina, 20–23 Sep embe 2018; pp. 580–585. [C ossRe ]
12.
Pacu a i, A.C.; Bha a ai, S.; Muhammad, A.; A am, C.; Mede le, A.O.; Rosca, O.; B a osin, F.; Bogdan, I.; Fe icean, R.M.; Bi is,
M.; e al. Diagnos ic Accu acy o Machine Lea ning AI A chi ec u es in De ec ion and Classi ica ion o Lung Cance : A Sys ema ic
Re iew. Diagnos ics 2023,13, 2145. [C ossRe ] [PubMed]
13.
Men o, F.; Pe one, T.; Fiengo, A.; Sma giassi, A.; Inchingolo, R.; Solda i, G.; Demi, L. Deep lea ning applied o lung ul asound-
ideos o sco ing COVID-19 pa ien s: A mul icen e s udy. J. Acous . Soc. Am. 2021,149, 3626–3634. [C ossRe ]
Appl. Sci. 2024,14, 11930 23 o 23
14.
Khan, U.; A akh eh, S.; Men o, F.; Me , G.; Sma giassi, A.; Inchingolo, R.; Tu si, F.; Macioce, V.; Pe one, T.; Iacca, G.; e al.
Low-complexi y lung ul asound ideo sco ing by means o in ensi y p ojec ion-based ideo comp ession. Compu . Biol. Med.
2023,169, 107885. [C ossRe ] [PubMed]
15.
Ba os, B.; Lace da, P.; Albuque que, C.; Conci, A. Pulmona y COVID-19: Lea ning Spa io empo al Fea u es Combining CNN
and LSTM Ne wo ks o Lung Ul asound Video Classi ica ion. Senso s 2021,21, 5486. [C ossRe ] [PubMed]
16.
Ke dega i, H.; Phung, N.T.H.; McB ide, A.; Pisani, L.; Nguyen, H.V.; Duong, T.B.; Raza i, R.; Thwai es, L.; Yacoub, S.; Gomez, A.; e al.
B-Line De ec ion and Localiza ion in Lung Ul asound Videos Using Spa io empo al A en ion. Appl. Sci. 2021,11, 11697. [C ossRe ]
17.
Lucassen, R.T.; Ja a i, M.H.; Duggan, N.M.; Jowka , N.; Meh ash, A.; Fische i, C.; Be nie , D.; P en ice, K.; Duhaime, E.P.; Jin,
M.; e al. Deep Lea ning o De ec ion and Localiza ion o B-Lines in Lung Ul asound. IEEE J. Biomed. Heal h In o m. 2023,27,
4352–4361. [C ossRe ] [PubMed]
18.
an Sloun, R.J.G.; Demi, L. Localizing B-Lines in Lung Ul asonog aphy by Weakly Supe ised Deep Lea ning, In-Vi o Resul s.
IEEE J. Biomed. Heal h In o m. 2020,24, 957–964. [C ossRe ]
19.
Howell, L.; Ing am, N.; Lapham, R.; Mo ell, A.; McLaughlan, J.R. Deep lea ning o eal- ime mul i-class segmen a ion o
a e ac s in lung ul asound. Ul asonics 2024,104, 107251. [C ossRe ] [PubMed]
20.
Ronnebe ge , O.; Fische , P.; B ox, T. U-ne : Con olu ional ne wo ks o biomedical image segmen a ion. In Medical Image
Compu ing and Compu e -Assis ed In e en ion–MICCAI 2015, P oceedings o he 18 h In e na ional Con e ence, Munich, Ge many, 5–9
Oc obe 2015; P oceedings, Pa III 18; Sp inge In e na ional Publishing: Be lin/Heidelbe g, Ge many, 2015; pp. 234–241.
21.
Muñoz, M.; Cosa insky, G.; C uza, J.F.; Camacho, J. Deep Lea ning-Based Lung Ul asound image segmen a ion o eal- ime
analysis. In P oceedings o he 2023 IEEE In e na ional Ul asonics Symposium (IUS), Mon eal, QC, Canada, 3–8 Sep embe
2023; pp. 1–4. [C ossRe ]
22. Ke as-Une . A ailable online: h ps://gi hub.com/ka olzak/ke as-une / ee/mas e (accessed on 1 Ma ch 2023).
23.
Ok ay, O.; Schlempe , J.; Folgoc, L.L.; Lee, M.; Hein ich, M.; Misawa, K.; Mo i, K.; McDonagh, S.; Hamme la, N.Y.; Kainz, B.; e al.
A en ion u-ne : Lea ning whe e o look o he panc eas. a Xi 2018, a Xi :1804.03999.
24.
Camacho, J.; Muñoz, M.; Geno és, V.; He aiz, J.L.; O ega, I.; Bela a, A.; González, R.; Sánchez, D.; Giacche a, R.C.; T ueba-
Vicen e, Á.; e al. A i icial In elligence and Democ a iza ion o he Use o Lung Ul asound in COVID-19: On he Feasibili y o
Au oma ic Calcula ion o Lung Ul asound Sco e. In . J. T ansl. Med. 2022,2, 17–25. [C ossRe ]
25.
Tung-Chen, Y.; Ossaba-Vélez, S.; Acos a Velásquez, K.S.; Pa a-Go do, M.L.; Díez-Tascón, A.; Villén-Villegas, T.; Mon e o-
He nández, E.; Gu ié ez-Villanue a, A.; T ueba-Vicen e, Á.; A enas-Be engue , I.; e al. The Impac o Di e en Lung Ul asound
P o ocols in he Assessmen o Lung Lesions in COVID-19 Pa ien s: Is The e an Ideal Lung Ul asound P o ocol? J. Ul asound
2022,25, 483–491. [C ossRe ] [PubMed] [PubMed Cen al]
26.
F i sch, C.; Camacho, J.; Ibañez, A.; B izuela, J.; Giacche a, R.; González, R. A Full Fea u ed Ul asound NDE Sys em in a S anda d
FPGA. In P oceedings o he Eu opean Cong ess on Non-Des uc i e Tes ing (ECNDT), Be lin, Ge many, 25–29 Sep embe 2006;
ISBN 3-931381-86-2.
27.
Lucas, B.D.; Kanade, T. An Image Regis a ion Technique wi h an Applica ion o S e eo Vision. In P oceedings o he Image
Unde s anding Wo kshop, Washing on, DC, USA, 23 Ap il 1981; pp. 121–130.
28. Pa el, D.; Upadhyay, S. Op ical low measu emen using Lucas-Kanade me hod. In . J. Compu . Appl. 2013,61, 6–10. [C ossRe ]
29.
Ped egosa, F.; Va oquaux, G.; G am o , A.; Michel, V.; Thi ion, B.; G isel, O.; Blondel, M.; P e enho e , P.; Weiss, R.; Dubou g, V.;
e al. Sciki -lea n: Machine Lea ning in Py hon. J. Mach. Lea n. Res. 2011,12, 2825–2830.
30.
Iko un, A.M.; Ezugwu, A.E.; Abualigah, L.; Abuhaija, B.; Heming, J. K-means clus e ing algo i hms: A comp ehensi e e iew,
a ian s analysis, and ad ances in he e a o big da a. In . Sci. 2023,622, 178–210. [C ossRe ]
31.
PyQ G aph. A ailable online: h ps://pyq g aph. ead hedocs.io/en/la es /ge ing_s a ed/in oduc ion.h ml (accessed on 1
Feb ua y 2022).
32.
Mul ip ocessing—P ocess-Based Pa allelism. A ailable online: h ps://docs.py hon.o g/3/lib a y/mul ip ocessing.h ml (ac-
cessed on 1 Oc obe 2022).
33.
Go don, G.A.; Canumalla, S.; Ti mann, B.R. Ul asonic C-scan imaging o ma e ial cha ac e iza ion. Ul asonics 1993,31, 373–380.
[C ossRe ]
34.
He aiz, J.L.; F eijo, C.; Camacho, J.; Muñoz, M.; González, R.; Alonso-Roca, R.; Ál a ez-T oncoso, J.; Bel án-Rome o, L.M.;
Be nabeu-Wi el, M.; Blancas, R.; e al. In e -Ra e Va iabili y in he E alua ion o Lung Ul asound in Videos Acqui ed om
COVID-19 Pa ien s. Appl. Sci. 2023,13, 1321. [C ossRe ]
35.
Reis, M.; Vés ias, M.; Ne o, H. Designing Deep Lea ning Models on FPGA wi h Mul iple He e ogeneous Engines. ACM T ans.
Recon igu able Technol. Sys . 2024,17, 1–30. [C ossRe ]
36.
Xu, Y.; Wang, Y.; Chen, Q.; Hu, H.; Huang, H.; Lin, L.; Chen, Y.-W.; Li, J.; Lin, H. FPGA O ien ed Ligh weigh Deep Lea ning
In e ence o Li e Cance Segmen a ion. In P oceedings o he 2024 IEEE In e na ional Symposium on Biomedical Imaging
(ISBI), A hens, G eece, 27–30 May 2024; pp. 1–5.
Disclaime /Publishe ’s No e: The s a emen s, opinions and da a con ained in all publica ions a e solely hose o he indi idual
au ho (s) and con ibu o (s) and no o MDPI and/o he edi o (s). MDPI and/o he edi o (s) disclaim esponsibili y o any inju y o
people o p ope y esul ing om any ideas, me hods, ins uc ions o p oduc s e e ed o in he con en .