SYSTEMATIC REVIEW
published: 28 Ap il 2022
doi: 10.3389/ comp.2022.869140
F on ie s in Compu e Science | www. on ie sin.o g 1Ap il 2022 | Volume 4 | A icle 869140
Edi ed by:
Pe e Kokol,
Uni e si y o Ma ibo , Slo enia
Re iewed by:
Ricca do Rosa i,
Ma che Poly echnic Uni e si y, I aly
Qian Du,
GNS Heal hca e, Uni ed S a es
*Co espondence:
Fabiano Papaiz
[email p o ec ed]
Special y sec ion:
This a icle was submi ed o
Digi al Public Heal h,
a sec ion o he jou nal
F on ie s in Compu e Science
Recei ed: 03 Feb ua y 2022
Accep ed: 24 Ma ch 2022
Published: 28 Ap il 2022
Ci a ion:
Papaiz F, Dou ado MET J , Valen im
RAM, Mo ais AHF and A ais JP
(2022) Machine Lea ning Solu ions
Applied o Amyo ophic La e al
Scle osis P ognosis: A Re iew.
F on . Compu . Sci. 4:869140.
doi: 10.3389/ comp.2022.869140
Machine Lea ning Solu ions Applied
o Amyo ophic La e al Scle osis
P ognosis: A Re iew
Fabiano Papaiz1,2,3*, Ma io Emílio Teixei a Dou ado J .1,4,
Rica do Alexsand o de Medei os Valen im1, An onio Higo F ei e de Mo ais 1,3 and
Joel Pe diz A ais2
1Labo a o y o Technological Inno a ion in Heal h (LAIS), Fede al Uni e si y o Rio G ande do No e, Na al, B azil, 2Cen e o
In o ma ics and Sys ems o he Uni e si y o Coimb a, Depa men o In o ma ics Enginee ing, Uni e si y o Coimb a,
Coimb a, Po ugal, 3Ad anced Nucleus o Technological Inno a ion, Fede al Ins i u e o Rio G ande do No e, Na al, B azil,
4Depa men o In e nal Medicine, Fede al Uni e si y o Rio G ande do No e, Na al, B azil
The p ognosis o Amyo ophic La e al Scle osis (ALS), a complex and a e disease,
ep esen s a challenging and essen ial ask o be e comp ehend i s p og ession
and imp o e pa ien s’ quali y o li e. The use o Machine Lea ning (ML) echniques
in heal hca e has p oduced aluable con ibu ions o he p ognosis ield. This a icle
p esen s a sys ema ic and c i ical e iew o p ima y s udies ha used ML applied o
he ALS p ognosis, sea ching o da abases, ele an p edic o bioma ke s, he ML
algo i hms and echniques, and hei ou comes. We ocused on s udies ha analyzed
bioma ke s commonly p esen in he ALS disease clinical p ac ice, such as demog aphic,
clinical, labo a o y, and imaging da a. Hence, we in es iga e s udies o p o ide an
o e iew o solu ions ha can be applied o de elop decision suppo sys ems and
be used by a highe numbe o ALS clinical se ings. The s udies we e e ie ed om
PubMed, Science Di ec , IEEEXplo e, and Web o Science da abases. A e comple ing
he sea ching and sc eening p ocess, 10 a icles we e selec ed o be analyzed and
summa ized. The s udies e alua ed and used di e en ML algo i hms, echniques,
da ase s, sample sizes, bioma ke s, and pe o mance me ics. Based on he esul s,
h ee dis inc ypes o p edic ion we e iden i ied: Disease P og ession, Su i al Time,
and Need o Suppo . The bioma ke s iden i ied as ele an in mo e han one s udy we e
he ALSFRS/ALSFRS-R, disease du a ion, Fo ced Vi al Capaci y, Body Mass Index, age
a onse , and C ea inine. In gene al, he s udies p esen ed p omisso y esul s ha can
be applied in de eloping decision suppo sys ems. Besides, we discussed he open
challenges, he limi a ions iden i ied, and u u e esea ch oppo uni ies.
Keywo ds: Amyo ophic La e al Scle osis, p ognosis, Machine Lea ning, heal h in o ma ics, li e a u e e iew
1. INTRODUCTION
Amyo ophic La e al Scle osis (ALS) is a a e, incu able, and p og essi e disease ha a ec s he
neu ons o he human mo o sys em. The communica ion be ween he b ain and muscles is
g adually in e up ed, leading pa ien s o pa alysis and dea h. I s causes a e unknown, ypically
commi s men and women be ween he ages o 40 and 70. The a e age li e expec ancy is 3–5 yea s
a e symp oms onse , and he wo ldwide incidence is abou 1.9 cases pe 100,000 indi iduals
Papaiz e al. Machine Lea ning Applied o ALS P ognosis
pe yea . ALS is clinically he e ogeneous, p esen ing di e en
si es o disease onse , ex a-mo o in ol emen s, p og ession
a es, and su i al imes among hei pa ien s (Ande sen
e al., 2012; Chiò e al., 2014; Swinnen and Robbe ech , 2014;
Ha diman e al., 2017). Being ALS a complex disease, p o iding
an accu a e p ognosis becomes a challenge o he physicians
(e.g., su i al ime, disease p og ession, momen o in oducing
speci ic ea men s). Thus, i is essen ial o iden i y ele an
biological ma ke s (bioma ke s) and unde s and how hey a e
ela ed o ALS disease p og ession. Bioma ke s a e pa ame e s
collec ed om he pa ien s ha can be used o con i m a
disease p esence (diagnosis), ollow up a disease p og ession
(p ognosis) o ea men esponse (moni o ing), and calcula e
he p obabili y o de eloping a disease ( isk) (G oup, 2001). They
can comp ise di e en da a ypes, such as clinical, biome ic,
imaging, bio luid, and gene ic. P e ious s udies iden i ied help ul
bioma ke s ha can assis in ALS p ognosis, such as age a
symp om onse , diagnosis delay, weigh loss, bulba si e o
onse , a e o unc ional and espi a o y impai men o e ime,
mic oRNAs, neu o ilamen s, and labo a o y es s (ALS, 1996;
Ceda baum e al., 1999; Kollewe e al., 2008; Chiò e al., 2009;
Va ghese e al., 2013; Ha diman e al., 2017; Walle e al., 2017).
Resea ches using A i icial In elligence echniques, like
Machine Lea ning (ML) algo i hms, ha e been success ully
applied o imp o e he diagnosis and p ognosis o diseases,
such as he ecen ad ances in he oncology ield (Kou ou
e al., 2015; O’Shea e al., 2016). The ML ield aims o
de elop compu e p og ams capable o lea ning using p e ious
expe ience ( aining da a) wi hou being explici ly p og ammed
o his. ML algo i hms could ex ac in o ma ion om he
aining da a, ans o m i in o knowledge, and use i o sol e
di e en ca ego ies o p oblems (e.g., classi ica ion, eg ession,
clus e ing, Samuel, 1988). In heo y, he g ea e he amoun
o aining da a a ailable, he g ea e he algo i hm’s lea ning
and pe o mance (Mi chell, 1997; Kuba , 2017). In his sense,
ha ing access o ALS pa ien da a is c ucial o pe o m ele an
s udies in he p ognos ic a ea and c ea e ML solu ions o help
physicians in hei daily wo k. The analysis o medical da a
usually in ol es dealing wi h high-dimensional da a, co e ing
a la ge numbe o bioma ke s. Thus, some ML echniques
(e.g., Fea u e Selec ion, Dimensionali y Reduc ion) can be
applied o ans o m a complex da ase in o a simple one by
iden i ying he mo e ele an bioma ke s, which imp o e he
lea ning pe o mance, da a collec ing e iciency, and algo i hm
unde s anding (Lee and Ve leysen, 2007; B ank e al., 2011). ML
algo i hms can be used o de elop Clinical Decision Suppo
Sys ems (CDSS). The CDSS a e compu e p og ams designed
o help physicians make mo e app op ia e and imely decisions
abou hei pa ien s (Be ne e al., 2007; Beele e al., 2014;
Gul epe e al., 2014; CDS, 2015; Rosa i e al., 2020; Romeo
and F on oni, 2022). These sys ems usually p o ide p ognos ic
p edic ions o imp o e he decision-making p ocess and, hus,
imp o e he pa ien ’s quali y o li e. Some bene i s include
imp o ing pa ien s’ quali y o ca e, ea men e iciency, esou ce
planning, and educing cos s. CDSS also ep esen s a aluable
ool o p omo e knowledge dissemina ion among all in e es ed
heal h wo ke s. ML-based CDSS can imp o e clinical decisions
TABLE 1 | Resea ch ques ions.
RQ Ques ion
01 Wha a e he ALS da abases used in he s udy?
02 How many pa ien s comp ise he coho o he s udy?
03 Wha a e he ypes o p edic ion add essed by he s udy?
04 Wha a e he ML algo i hms and echniques used in he s udy?
05 Wha a e he bioma ke s e alua ed and he mos ele an iden i ied by
he s udy?
06 Wha a e he pe o mances o he used ML algo i hms?
by helping physicians analyze and make in e ences on a la ge
amoun o pa ien da a. Howe e , some ML app oaches p esen
esul s ha can no be easily unde s ood, dec easing hei
in e p e abili y (e.g., A i icial Neu al Ne wo ks o Suppo
Vec o Machines). In e p e abili y e e s o how well a pe son
can unde s and he decisions made by he ML algo i hm (Mille ,
2019). This issue can di icul he p ocess o accep ance and
in eg a ion o a CDSS in he clinical en i onmen ou ine.
Consequen ly, he de elopmen o a CDSS mus ha e conce ned
abou in e p e abili y issues, being anspa en enough so
ha heal h wo ke s can unde s and how any suppo was
o e ed.
Many coun ies p esen inancial limi a ions on hei heal h
sys em. This ac makes i un easible o collec complex and cos ly
bioma ke s (e.g., gene ic) in p ima y ca e. In his manne , i is
essen ial o ca y ou s udies conside ing hese limi a ions o
de elop compu a ional solu ions (e.g., CDSS) ha can assis a
highe numbe o p ima y ca e uni s.
The main objec i e o his s udy is o in es iga e ML
app oaches on ALS p ognosis ha analyzed less complex
bioma ke s, which can be po en ially applied o de elop clinical
decision suppo sys ems o assis physicians in he eal-wo ld
ALS clinical se ing. We ocused on s udies ha analyzed
bioma ke s commonly p esen in he ALS disease clinical
p ac ice, such as demog aphic, clinical (including unc ional,
espi a o y, and nu i ional), labo a o y, and imaging da a.
Hence, we in es iga e s udies using bioma ke s ob ained h ough
a less complex p ocess, aiming o p o ide an o e iew o
solu ions ha can be applied o de elop decision suppo sys ems
and be used on a la ge scale in p ima y ca e, conside ing
inancial limi a ions. In his sense, we did no include s udies
using omics da a (i.e., genomic, ansc ip omic, p o eomic,
and me abolomic). We desc ibed he ecen ad ances in his
a ea, he cu en ly a ailable da ase s, he bioma ke s analyzed,
he ML algo i hms and echniques used, he mos ele an
bioma ke s iden i ied, and hei ou comes. Besides, we discussed
he open challenges, he limi a ions iden i ied, and u u e
esea ch oppo uni ies.
2. METHODS
This sys ema ic e iew aims o in es iga e ML solu ions applied
o ALS p ognosis. In his sense, we elabo a ed esea ch ques ions
(RQ) o guide he conduc o his a icle, which a e p esen ed
F on ie s in Compu e Science | www. on ie sin.o g 2Ap il 2022 | Volume 4 | A icle 869140
Papaiz e al. Machine Lea ning Applied o ALS P ognosis
TABLE 2 | Inclusion c i e ia.
IC Desc ip ion
01 A icles published in Jou nals
02 A icles w i en in English
03 A icles published be ween Janua y 2011 and Ap il 2021
04 A icles in he In o ma ion Technology, Compu e Enginee , o
Compu e Science ela ed a eas
TABLE 3 | Exclusion c i e ia.
EC Desc ip ion
01 Re iew a icles
02 Duplica e a icles
03 A icles no ela ed o Machine Lea ning applied o ALS p ognosis
04 A icles using omics da a (i.e., genomic, ansc ip omic, p o eomic, and
me abolomic)
in Table 1. Nex , we pe o med he ollowing s ages: (i) sea ch
a icles ela ed o ALS p ognosis using ML in scien i ic da abases,
(ii) apply he inclusion c i e ia, (iii) apply he exclusion c i e ia,
and (i ) analyze and summa ize he selec ed a icles.
In he i s s age, he ele an li e a u e was ob ained
om he PubMed,Science Di ec ,IEEEXplo e, and Web o
Science da abases. The sea ch was pe o med in Ap il 2021
using he ollowing sea ch que y: (“a i icial in elligence” OR
“machine lea ning” OR “deep lea ning”) AND (“amyo ophic
la e al scle osis” OR “mo o neu one disease”) AND (“p edic ”
OR “p ognosis” OR “p og ession”). We used he Rayyan Web
Applica ion (Ouzzani e al., 2016) o o ganize he esul ing
a icles and also o pe o m he emaining s ages.
In he second and hi d s ages, we applied he Inclusion (IC)
and Exclusion (EC) C i e ia o il e he a icles acco ding o he
scope o his a icle (see Tables 2,3). We conside ed only a icles
published in Jou nals, w i en in English, and published be ween
Janua y 2011 and Ap il 2021 (IC-01, IC-02, and IC-03). A icles
ha did no belong o he In o ma ion Technology, Compu e
Enginee , o Compu e Science ela ed a eas we e no included
(IC-04). Nex , we ca ied ou he emo al o he e iew a icles
(EC-01), he duplica e en ies (EC-02), and a icles no ela ed o
ML applied o ALS p ognosis (EC-03). Then, he a icles using
omics da a we e emo ed (EC-04).
Finally, in he ou h s age, he selec a icles we e ho oughly
ead, which allowed he inal analysis and accomplishmen o he
objec i es o his esea ch.
3. RESULTS
Figu e 1 illus a es he sea ch and sc eening p ocess o his
sys ema ic e iew. The sea ch que y and all inclusion c i e ia
we e used o pe o m he da abase sea ches. A o al o 52 a icles
we e e ie ed, whe e wo e iew a icles we e immedia ely
excluded. A e he emo al o 15 duplica es, 35 a icles we e
chosen o abs ac e iew. A o al o 25 s udies we e excluded
due o he use o omic da a (n= 6) and no being ela ed o ML
applied o ALS p ognosis (n= 19). A e comple ing he sea ching
and sc eening p ocess, 10 a icles we e selec ed o be analyzed
and summa ized. The ollowing sec ions p esen he esul s ha
add ess he esea ch ques ions de ined in his s udy (Table 1).
3.1. ALS Da ase s and Sample Sizes
Di e en da ase s we e analyzed and hei sample sizes anged
om 41 up o o e 10,000 samples. Table 4 desc ibes all he
da ase s analyzed. Mos o he s udies (60%) analyzed da a om
he PRO-ACT (A assi e al., 2014) da ase , p obably because i
was he only publicly a ailable. The o he da ase s used we e local
o p op ie a y. The da a o ma s analyzed included abula (all
s udies) and image ( an de Bu gh e al., 2017). Mo e de ail abou
he sample size used by each s udy a e desc ibed in Tables 6–8.
3.2. Types o P edic ion Add essed
Based on he included s udies, h ee dis inc ypes o p edic ion
we e iden i ied: Disease P og ession,Su i al Time, and Need o
Suppo (mo e de ail in Table 5). Kue ne e al. (2019) add essed
he Disease P og ession and Su i al Time ypes simul aneously.
The Disease P og ession p edic ion aimed o es ima e he
pa ien ’s s a e a a gi en momen in he u u e and was he ype
mos add essed by he s udies included (70%). The Su i al Time
p edic ion aimed o es ima e he occu ence o dea h om a
baseline da e o a poin - ime in he u u e, such as he p obabili y
o dea h a e 12 mon hs om symp oms onse . The Need o
Suppo p edic ion aimed o es ima e he momen when pa ien s
will need mo e specialized suppo .
3.3. P edic i e Machine Lea ning
App oaches
Fo Disease P og ession p edic ion, mos s udies aimed o
es ima e changes in he ALS Func ional Ra ing Scale (ALSFRS)
o he Re ised ALS Func ional Ra ing Scale (ALSFRS-R) o e
ime. Two o he s udies aimed o classi y pa ien s conce ning
hei disease p og ession a es (Slow/Fas Kue ne e al.,
2019, Low/High G eco e al., 2021). Table 6 de ails he a ge
p edic ions, bes ML algo i hm, pe o mance, da ase s, samples
size, echniques, alida ion s a egies, and bioma ke s e alua ed
o each s udy.
The s udies ha add essed he Su i al Time p edic ion aimed
o classi y he pa ien s in o su i al g oups and es ima e he
p obabili y o dea h a e a speci ic ime in e al. an de Bu gh
e al. (2017) aimed o classi y pa ien s in o Sho (<25 mon hs),
Medium (25−50 mon hs), o Long (>50 mon hs) su i al
g oups. Kue ne e al. (2019) aimed o es ima e he p obabili y
o su i al a e 12, 18, and 24 mon hs. G ollemund e al. (2020)
aimed o es ima e he p obabili y o pa ien s being ali e a e 12
mon hs. All h ee s udies used he da e o symp oms onse as
he baseline da e. The cha ac e is ics o each s udy a e de ailed
in Table 7.
Pi es e al. (2018) was he unique s udy ha add essed he
Need o Suppo p edic ion, aiming o es ima e he need o Non-
In asi e Ven ila ion (NIV) suppo a e 3, 6, and 12 mon hs. The
cha ac e is ics o his s udy a e de ailed in Table 8.
F on ie s in Compu e Science | www. on ie sin.o g 3Ap il 2022 | Volume 4 | A icle 869140
Papaiz e al. Machine Lea ning Applied o ALS P ognosis
FIGURE 1 | PRISMA low cha o his s udy.
3.4. Bioma ke s E alua ed and he Mos
Rele an Iden i ied
As p e iously men ioned, we ocused on he bioma ke s
commonly p esen in he ALS disease clinical p ac ice, being
ob ained in a less cos ly and complex way. The bioma ke s
e alua ed comp ise clinical, demog aphic, i al signs, espi a o y,
unc ional, labo a o y, imaging, neu ophysiological, and
medica ion da a. Fo mo e de ail, please see column Bioma ke s
E alua ed in Tables 6–8. All he selec ed s udies e alua ed he
ALS Func ional Ra ing Scale (ALSFRS) o he Re ised ALS
Func ional Ra ing Scale (ALSFRS-R) bioma ke s. This ac
highligh s he impo ance o hese bioma ke s in moni o ing
ALS pa ien s.
Table 9 depic s he mos ele an bioma ke s iden i ied in
he s udies, wi h he in o ma ion abou hei associa ed ypes
o p edic ion. They comp ised clinical, imaging, unc ional,
espi a o y, and labo a o y da a. The bioma ke s iden i ied as
ele an in mo e han one s udy we e he ALSFRS/ALSFRS-R (n
= 7), disease du a ion (n= 5), Fo ced i al capaci y (n= 4), Body
mass index (n= 2), age a onse (n= 2), and C ea inine (n= 2).
3.5. Desc ip ion o he S udies
an de Bu gh e al. (2017) demons a ed he posi i e impac
o using Magne ic Resonance Images (MRI) along wi h
clinical in o ma ion o classi y ALS pa ien s in o h ee su i al
g oups: Sho (<25 mon hs), Medium (25−50 mon hs),
and Long (>50 mon hs). The bioma ke s e alua ed we e
clinical in o ma ion (e.g., si e o onse , age a onse , ALSFRS
slope, FVC) and MRI images (S uc u al Connec i i y and
B ain Mo phology da a) om 135 ALS pa ien s. They
de eloped Deep Neu al Ne wo ks models and e alua e
hem in ou scena ios using di e en bioma ke s se s: (i)
only Clinical Da a, (ii) only S uc u al Connec i i y MRI
Da a, (iii) only B ain Mo phology MRI Da a, and (i )
combining Clinical and MRI Da a. The g ea e accu acy
was ob ained using he Clinical-MRI combined da a (84%)
compa ed o he o he h ee s a egies (Clinical: 69%; S uc u al
Connec i i y MRI: 63%; B ain Mo phology MRI: 63%). They
poin ed ou he powe o Deep Neu al Ne wo ks in making
p edic ions using complex da a. Howe e , he ela ionships
be ween inpu and ou pu a iables could no be easily
F on ie s in Compu e Science | www. on ie sin.o g 4Ap il 2022 | Volume 4 | A icle 869140
Papaiz e al. Machine Lea ning Applied o ALS P ognosis
TABLE 4 | Lis o da ase s used in he s udies, in e sely o de ed by he sample
size.
Da ase Samples Re e ences
PRO-ACT +10,000 Go don and Le ne , 2019; Halbe sbe g
and Le ne , 2019; Kue ne e al., 2019;
Tang e al., 2019; G ollemund e al., 2020;
Hadad and Le ne , 2020
I eland-I alia 1,479 Kue ne e al., 2019
Tel A i —Sou asky
Medical Cen e
1,328 Hadad and Le ne , 2020
Lisbon—Sain Ma y’s
Hospi al
1,214 Pi es e al., 2018; Leão e al., 2021
Pa is—Te ia y e e al
Cen e o ALS
646 G ollemund e al., 2020
T ophos Company 431 G ollemund e al., 2020
Exonhi Pha ma 172 G ollemund e al., 2020
U ech —Uni e si y
Medical Cen e
135 an de Bu gh e al., 2017
I alia 41 G eco e al., 2021
TABLE 5 | Types o p edic ion add essed by he s udies.
Type Numbe
o s udies
Re e ences
Disease
p og ession
7Go don and Le ne , 2019; Halbe sbe g and
Le ne , 2019; Kue ne e al., 2019; Tang e al.,
2019; Hadad and Le ne , 2020; G eco e al.,
2021; Leão e al., 2021
Su i al ime 3 an de Bu gh e al., 2017; Kue ne e al.,
2019; G ollemund e al., 2020
Need o suppo 1 Pi es e al., 2018
ecognized, needing mo e in es iga ion o unde s and ALS
p og ession be e .
Pi es e al. (2018) de eloped a model o p edic when a pa ien
will need NIV suppo acco ding o a gi en ime window (3,
6, and 12 mon hs). They used he Po uguese ALS Da ase (n
=1,070), combining he s a ic and empo al da a in o a da a
s uc u e called snapsho , which con ains all in o ma ion abou
a pa ien a a speci ic da e. The pa ien s we e di ided in o h ee
disease p og ession g oups (Slow, Neu al, and Fas ) and, o
each g oup, hei espec i e snapsho s we e used as lea ning
ins ances o e alua e se e al ML models. A Fea u e Selec ion
Ensemble app oach was used o selec he ele an bioma ke s
o each g oup. The Random Fo es model ob ained he bes
pe o mance o 3, 6, and 12 mon hs ime window alues. The
ele an bioma ke s p esen in all g oups we e BMI, FVC, and
VC. O he ele an bioma ke s (p esen in 75% o he ime) we e
age a onse , disease du a ion, and ALSFRS sco e. The au ho s
epo ed he ad an age o using specialized ML models o
di e en pa ien g oups (e.g., disease p og ession g oups) a he
han c ea e gene alized models ea ing all he pa ien s simila ly.
Halbe sbe g and Le ne (2019) demons a ed he bene i o
using empo al modeling, sequence clus e ing, and sequen ial
pa e n mining o p edic he las pa ien s a e eco ded
(ALSFRS sco e) based on his pas in o ma ion. To ind ele an
de e io a ion pa e ns in empo al pa ien s da a hey de eloped
a amewo k consis ing o h ee s ages: (i) g oup pa ien s
wi h simila p og ession using hie a chical clus e ing based
on Dynamic Time Wa ping, (ii) pe o m pa e n mining o
ound ou common unc ional de e io a ion pa e ns among
pa ien s based on he SPADE sequence mining algo i hm,
and (iii) de elop a Random Fo es model o classi y pa ien s
in o hei mos simila clus e o p edic hei nex disease
s a e. The pe o mance ob ained by he p oposed amewo k
(Accu acy: 73, F1 sco e: 0.68, Mean Absolu e E o : 0.3) was
supe io ela ed o wo o he benchma k models (Random
Fo es and Long Sho -Te m Memo y, bo h using no empo al
modeling). They used s a ic (e.g., age a onse , ime om
onse , gende ) and longi udinal (ALSFRS sco es and subsco es)
da a o 2,590 subjec s om he PRO-ACT da ase . The mos
impo an p edic o s epo ed we e he p e ious ALSFRS sco e,
he p e ious ALSFRS D essing subsco e, he p e ious Climbing
S ai s subsco e, he p e ious Tu ning in Bed subsco e, he ime
om disease onse , and he de e io a ion pa e n e med <E,G,I>
(i.e., a sequen ial declining in he W i ing,D essing, and Walking
ALSFRS subsco es).
Go don and Le ne (2019) e alua ed he capaci y o o dinal
classi ie s o p edic he unc ional decline o he pa ien s. They
used da a abou he i s and las pa ien isi s om he PRO-
ACT da ase (n=3,772), analyzing he ollowing bioma ke s:
clinical, demog aphic, ALSFRS, FVC, medica ion, i al signs,
and labo a o y es s. The a ge a iables we e all en ALSFRS
i ems (ques ions) sepa a ely. The pa ien s a es we e mapped
o he ALSFRS i ems, hus co ela ing pa ien s a e o disease
p og ession o each poin in ime. Add essing he o dinal na u e
o he ALSFRS, hey e alua ed he ollowing o dinal classi ie s:
Cumula i e Link Models (CLM), O dinal Decision T ees (ODT),
and Cumula i e P obabili y T ee (CPT). To e alua e hei
pe o mances, hey de ined a penalizing sys em ha accoun s
o a ious e o se e i ies di e en ly. Thus, a classi ie was less
penalized when i p edic ed he alue o 2 ins ead o 1 when he
eal alue was 3. These h ee classi ie s we e compa ed wi h he
Random Fo es (RF), a non-o dinal classi ie . The esul s showed
ha he CLM and ODT o dinal classi ie s p esen ed a simila
pe o mance and ou pe o med he RF classi ie ega ding he
Mean Absolu e E o measu ed in he bes expe imen scena io
(CLM: 0.62−1.06; ODT: 0.63−1.01; RF: 1.01−1.61). Fo ea u e
selec ion, he au ho s implemen ed an algo i hm based on he J3
sca e ing ma ix c i e ion o each ALSFRS i em indi idually.
The mos ele an p edic o s we e he FVC, he si e o onse ,
he ime om onse , and he labo a o y es s C ea inine, CK,
Chlo ide, Phospho us, and Alkaline Phospha ase.
A c owdsou cing s a egy was p esen ed in Kue ne e al.
(2019), whe e we e selec ed 30 eams a ound he wo ld o
pa icipa e in an ALS s a i ica ion challenge. They asked he
pa icipan s o c ea e ML models o pe o m p edic ion asks
using he PRO-ACT and he I ish-I alian Regis ies da ase s. The
eams used pa ien da a om he i s h ee mon hs and we e
limi ed o e alua e only six o all bioma ke s a ailable. The a ge
p edic ions we e he Disease P og ession a 12 mon hs (decline o
he Func ional Ra e Scale) and he P obabili y o Su i al a 12,
F on ie s in Compu e Science | www. on ie sin.o g 5Ap il 2022 | Volume 4 | A icle 869140

Papaiz e al. Machine Lea ning Applied o ALS P ognosis
TABLE 6 | O e iew o ML app oaches on disease p og ession.
Re e ences Ta ge p edic ion Bes algo i hm Pe o mance Bioma ke s e alua ed Da ase
(Samples)
Techniques Valida ion
Halbe sbe g and
Le ne (2019)
Las pa ien s a e
(ALSFRS sco e)
eco ded based on his
pas in o ma ion
SPADE + DTW +
Clus e ing Me hod
Accu acy: 73
F1 Sco e: 0.68
MAE: 0.30
Tabula : Clinical,
demog aphic,
labo a o y, ALSFRS
PRO-ACT
(2,590)
Hold-ou
Go don and Le ne
(2019)
Las pa ien s a e
(ALSFRS subsco es)
eco ded based on his
pas in o ma ion
CLM, and ODT MAE (min-max):
-CLM: 0.62−1.06
-ODT: 0.63−1.01
Tabula : Clinical,
demog aphic, i al
signs, labo a o y,
ALSFRS
PRO-ACT
(3,772)
FS 10-Fold CV
Kue ne e al. (2019) -ALSFRS sco e a 12
mon hs, using da a om
he i s 3 mon hs and
only 6 bioma ke s.
-Pa ien s classi ica ion
in o slow/ as
p og ession g oups.
GBM, and RF GBM (PRO-ACT):
-Z-sco e: ≈12
RF (I eland-I alia)
-Z-sco e: ≈6
Tabula : Clinical,
demog aphic, i al
signs, labo a o y, FVC,
SVC, ALSFRS
PRO-ACT
(10,723)
I eland-I alia
(1,479)
Hold-ou
Tang e al. (2019) ALSFRS sco e and FVC
a 12 mon hs, using 1s
isi and 3-mon h da a
BART (ALSFRS),
and RF (FVC)
BART:
-R2: 0.22
- RMSE: 0.55
- Co : 0.47
RF:
-R2: 0.68
- RMSE: 14.27
- Co : 0.83
Tabula : Clinical,
demog aphic, pulse,
BMI, FVC, labo a o y,
Riluzole medica ion,
ALSFRS
PRO-ACT
(2,424)
FS
MI
5-Fold CV
Hadad and Le ne
(2020)
ALSFRS sco e a se e al
ime in e als, a ying
om 6 up o 24 mon hs
XGBoos RMSE: 2.65−5.57
MAE: 1.98−4.42
Tabula : Clinical,
demog aphic, i al
signs, FVC, labo a o y,
ALSFRS
PRO-ACT
(3,171)
Tel A i (1,328)
FIA Hold-ou
G eco e al. (2021) Pa ien s classi ica ion
in o low/high
p og ession a es
g oups
SVM Accu acy: 87.25 Tabula : Clinical,
demog aphic,
labo a o y, ALSFRS-R
I alia
(41)
FS LOO CV
Leão e al. (2021) Changes in he
ALSFRS-R sco e and
subsco es (be o e and
a e NIV)
Ex ension o DBN Accu acy: 74−88
Sensi i i y: 57−95
AUC: 75−98
Tabula : Clinical,
demog aphic, El
Esco ial, BMI, C9o 72,
FVC, MIP, MEP, PNRA,
ALSFRS, ALSFRS-R
Lisbon (1,214) MI 5-Fold CV
ALSFRS, ALS unc ional a ing scale; ALSFRS-R, e ised ALS unc ional a ing scale; NIV, non-in asi e en ila ion; BMI, body mass index; FVC, o ced i al capaci y; SVC, slow i al
capaci y; MIP, maximum inspi a o y p essu e; MEP, maximum expi a o y p essu e; PNRA, ph enic ne e esponse ampli ude; SPADE, sequen ial pa e n disco e y using equi alence
class; DTW, dynamic ime wa ping; CLM, cumula i e link models; ODT, o dinal decision ees; GBM, gene alized boos ing model; RF, andom o es ; BART, Bayesian addi i e eg ession
ee; SVM, suppo ec o machine; DBN, dynamic Bayesian ne wo k; AUC, a ea unde he ROC cu e; MAE, mean absolu e e o ; MSPE, mean squa ed p edic ion e o ; R2,
coe icien o de e mina ion; RMSE, oo mean squa e e o ; Co , Pea son’s co ela ion coe icien ; FS, ea u e selec ion; FIA, ea u e impo ance analysis; MI, missing da a impu a ion;
CV, c oss- alida ion; LOO, lea e-one-ou .
18, and 24 mon hs. Rega ding he su i al p edic ion, one eam
ou pe o med he o he s signi ican ly using a Gaussian P ocess
Reg ession model, p esen ing a be e app oach in leading wi h
he igh -censo ed pa ien ou come (dead o ial d opou ). The
bes models ela ed o he disease p og ession p edic ion used he
Gene alized Boos ing Model and he Random Fo es algo i hms.
The mo e ele an bioma ke s we e disease du a ion, age a
onse , si e o onse , gende , weigh , BMI, espi a o y exams
(FVC and SVC), labo a o y es s (C ea inine and Segmen ed
Neu ophils), and ALSFRS sco es and subsco es. Based on he
ele an bioma ke s chosen by he eams, he au ho s ha e
iden i ied ou dis inc pa ien g oups: Slow P og essing, Fas
P og essing, Ea ly S age, and La e S age. The main bioma ke s
ela ed o each g oup we e also de ailed in his s udy, whe e
he au ho s highligh ed he impo ance o he ALSFRS Bulba
subsco e (ques ions 1−3) in disc imina ing be ween g oups.
Tang e al. (2019) add essed p edic ions in changing o
he ALSFRS sco e and in he FVC pe cen age. They used
s a ic and longi udinal bioma ke s om he PROC-ACT da ase
(n=2,424), including only hose pa ien s wi h in o ma ion
abou ALSFRS sco es o e ime. The longi udinal da a we e
ans o med in o signa u e ec o s agg ega ing s a is ics alues
(minimum, median, maximum, and slope). Using da a om
he i s isi and a he 3-mon h, he au ho s c ea e models
o p edic he changes in he ALSFRS slope a 12-mon h.
The e alua ed models (Random Fo es and Bayesian Addi i e
Reg ession T ee) achie ed modes esul s (Co ela ion: 0.47;
RMSE: 0.55; R2: 0.22), hus, indica ing he di icul y in p edic ing
F on ie s in Compu e Science | www. on ie sin.o g 6Ap il 2022 | Volume 4 | A icle 869140
Papaiz e al. Machine Lea ning Applied o ALS P ognosis
TABLE 7 | O e iew o ML app oaches on su i al ime.
Re e ences Ta ge p edic ion Bes algo i hm Pe o mance Bioma ke s e alua ed Da ase
(Samples)
Techniques Valida ion
an de Bu gh e al.
(2017)
Pa ien s classi ica ion
in o Sho (<25
mon hs), Medium
(25−50), and Long
(>50) su i al g oups
Deep neu al
ne wo ks
Accu acy: 84 Tabula : Clinical,
demog aphic, C9o 72,
FTD, El Esco ial,
ALSFRS.
Image: MRI.
U e ch (135) FS
MI
Hold-ou
Kue ne e al. (2019) P obabili y o dea h
wi hin 12, 18, and 24
mon hs
Gaussian
Reg ession
PRO-ACT:
-Z-sco e: ≈14.5
I eland-I alia:
-Z-sco e: ≈13
Tabula : Clinical,
demog aphic, i al
signs, labo a o y, FVC,
SVC, ALSFRS
PRO-ACT
(10,723)
I eland-I alia
(1,479)
Hold-ou
G ollemund e al.
(2020)
1-yea su i al
p edic ion, classi ying
pa ien s in o high,
in e media e, and low
su i al a es g oups
UMAP BAcc: 91%
F1 Sco e: 96%
Tabula : Clinical,
demog aphic, ALSFRS
PRO-ACT (3971)
T ophos (431)
Exonhi (172)
Pa is (646)
Hold-ou
ALSFRS, ALS unc ional a ing scale; ALSFRS-R, e ised ALS unc ional a ing scale; MRI, magne ic esonance image, FTD, on o empo al demen ia; FVC, o ced i al capaci y; SVC,
slow i al capaci y; UMAP, uni o m mani old app oxima ion and p ojec ion; BAcc, balanced accu acy; FS, ea u e selec ion; MI, missing da a impu a ion.
TABLE 8 | O e iew o ML app oach on need o suppo .
Re e ences Ta ge p edic ion Bes algo i hm Pe o mance Bioma ke s e alua ed Da ase
(Samples)
Techniques Valida ion
Pi es e al. (2018) Pa ien s need o NIV
suppo a 3, 6, and 12
mon hs o h ee
p og ession g oups
(slow, neu al, and as )
RF Slow: (3/6/12
mon hs)
- AUC: 81/87/91
- Sens: 70/72/78
- Spec: 76/83/86
Neu al: (3/6/12
mon hs)
- AUC: 76/82/86
- Sens: 58/62/79
- Spec: 78/83/77
Fas : (3/6/12
mon hs)
- AUC: 72/81/79
- Sens: 51/71/74
- Spec: 77/76/71
Tabula : Clinical,
demog aphic, El
Esco ial, BMI, C9o 72,
VC, FVC, P0.1, SNIP,
MIP, MEP, NIV, PNRA,
PNRL, CE, CF, ALSFRS,
ALSFRS-R
Lisbon (1070) FS
DB
10-Fold CV
ALSFRS, ALS unc ional a ing scale; ALSFRS-R, e ised ALS unc ional a ing scale; NIV, non-in asi e en ila ion; BMI, body mass index; FVC, o ced i al capaci y; SVC, slow i al
capaci y; VC, i al capaci y; P0.1, ai way occlusion p essu e; SNIP, sni nasal inspi a o y p essu e; MIP, maximum inspi a o y p essu e; MEP, maximum expi a o y p essu e; PNRA,
ph enic ne e esponse ampli ude; PNRL, ph enic ne e esponse la ency; CE, ce ical ex ension; CF, ce ical lexion; RF, andom o es ; AUC, a ea unde he ROC cu e; Sens,
sensi i i y; Spec, speci ici y; FS, ea u e selec ion; DB, da a balancing; CV, c oss- alida ion.
12-mon h ALSFRS slope using he only baseline and 3-mon hs
da a. Fea u e Selec ion was pe o med using he Random Fo es
and he Knocko Fil e me hods. A e combining he op-
anked bioma ke s e u ned by bo h me hods, he bes p edic i e
bioma ke s we e he ALSFRS sco e, he disease du a ion, he
FVC, and he Absolu e Monocy e Coun . To p edic he FVC
Pe cen age changes be ween 3 and 12 mon hs, Random Fo es
models we e es ed in wo scena ios (ei he including he baseline
FVC o no ). The bes esul s we e ob ained using he FVC a
baseline da a, demons a ing he powe o his bioma ke , which
inc eased he co ela ion om 0.67 o 0.83. The au ho s also
applied unsupe ised classi ica ion (K-Means) o ind dis inc
pheno ypes g oups, ounding ou balanced clus e s among
he pa ien s. Howe e , i was conside ed imp ac ical o clea ly
unde s and how he g oups di e due o he high numbe o
bioma ke s de ined o each g oup du ing he clus e ing p ocess.
Hadad and Le ne (2020) s udied p edic ion o he ALSFRS
sco e in se e al ime in e als, a ying om 6 o 24 mon hs.
Tempo al (Long Sho Te m Memo y—LSTM) and non-
empo al (Random Fo es , XGBoos , and Mul ilaye Pe cep on)
models we e e alua ed o e he PRO-ACT da ase (n=3,171).
To be used by he non- empo al models, he longi udinal
da a we e ans o med in o ec o s con aining agg ega ed
alues (mean, s anda d de ia ion, slope, minimum, maximum).
Each model was es ed using 60 di e en andomly gene a ed
con igu a ions, and hei a e aged pe o mances we e compa ed
(Roo Mean Squa e E o and Mean Absolu e E o ). The
XGBoos model ob ained supe io pe o mance o he mos
F on ie s in Compu e Science | www. on ie sin.o g 7Ap il 2022 | Volume 4 | A icle 869140
Papaiz e al. Machine Lea ning Applied o ALS P ognosis
TABLE 9 | Mos ele an bioma ke s iden i ied, associa ed p edic ions, and e e ences.
Type Bioma ke Associa ed p edic ions/Re e ences
Disease p og ession Su i al ime Need o suppo
Clinical Age a disease onse Halbe sbe g and Le ne , 2019 Kue ne e al., 2019 –
Body Mass Index (BMI) Kue ne e al., 2019; Leão e al., 2021 Kue ne e al., 2019 –
Disease du a ion Go don and Le ne , 2019; Halbe sbe g
and Le ne , 2019; Kue ne e al., 2019;
Tang e al., 2019; Leão e al., 2021
Kue ne e al., 2019 –
Si e o onse Go don and Le ne , 2019 – –
Imaging Magne ic Resonance Imaging – an de Bu gh e al., 2017 –
Func ional ALSFRS Halbe sbe g and Le ne , 2019; Kue ne
e al., 2019; Tang e al., 2019; Hadad and
Le ne , 2020
Kue ne e al., 2019; G ollemund e al.,
2020
Pi es e al., 2018
ALSFRS-R Leão e al., 2021 –Pi es e al., 2018
Respi a o y Fo ced Vi al Capaci y (FVC) Go don and Le ne , 2019; Kue ne e al.,
2019; Tang e al., 2019
Kue ne e al., 2019 Pi es e al., 2018
Maximal expi a o y p essu e (MEP) Leão e al., 2021 – –
Maximal inspi a o y p essu e (MIP) Leão e al., 2021 – –
Slow i al capaci y (SVC) Kue ne e al., 2019 Kue ne e al., 2019 –
Vi al capaci y (VC) – – Pi es e al., 2018
Labo a o y Absolu e monocy e coun Tang e al., 2019 – –
Alanine ansaminase (ALT) Tang e al., 2019 – –
Alkaline phospha ase Go don and Le ne , 2019 – –
Calcium Tang e al., 2019 – –
Chlo ide Go don and Le ne , 2019 – –
Choles e ol—To al G eco e al., 2021 – –
Choles e ol—high-densi y (HDL) G eco e al., 2021 – –
C ea ine kinase (CK) Go don and Le ne , 2019 – –
C ea inine Go don and Le ne , 2019; Kue ne e al.,
2019
– –
Hema oc i Tang e al., 2019 – –
Phospho us Go don and Le ne , 2019 – –
Po assium Tang e al., 2019 – –
Segmen ed neu ophils Kue ne e al., 2019 – –
U ine Ph Kue ne e al., 2019 – –
Vi amin B12 G eco e al., 2021 – –
ime in e als e alua ed (RMSE: 2.65−5.57, MAE: 1.98−4.42),
being mo e p ecise o sho e han longe in e als. The ele an
p edic i e bioma ke s we e he ALSFRS subsco es. In ano he
expe imen , hese models we e e alua ed in wo scena ios: (i)
ained wi h he PRO-ACT and es ed wi h he TASMC da ase (n
=1,328), and (ii) ained and es ed using only he TASMC da a.
The sho - e m p edic ions (up o 6 mon hs) we e mo e p ecise
using models ained wi h he PRO-ACT, and he XGBoos
ob ained he bes esul s again. The au ho s highligh ed ha
he PROC-ACT con ains da a om clinical ials ha may no
e lec he eali y p esen ed by he clinical en i onmen pa ien s
due o he inclusion/exclusion c i e ia used. Thus, hei pa ien s
end o be younge and o ha e a slowe disease p og ession,
in addi ion o ha ing mo e isi s egis e ed han he usual
clinical pa ien s. To add ess his p oblem, hey p oposed a
inal expe imen applying he Domain Adap a ion app oach
o de elop p edic i e models using he PRO-ACT da a and
imp o e hei pe o mances using pa ien clinical da a. Fi s ly,
LSTM and Mul ilaye Pe cep on models we e ained using
only da a om he PRO-ACT. Then, he aining phase was
complemen ed using he TASMC da a o ine- une he models
o he clinical da a. The esul s demons a ed ha he use o
domain adap a ion imp o ed he p edic i e pe o mance o
bo h models.
G ollemund e al. (2020) p esen ed a dimensionali y educ ion
model o p edic 1-yea su i al a es. The bioma ke s analyzed
we e gende , si e onse , age, weigh , disease du a ion, ALSFRS
sco es, ALSFRS slopes, and i died o no a e one yea . They
F on ie s in Compu e Science | www. on ie sin.o g 8Ap il 2022 | Volume 4 | A icle 869140
Papaiz e al. Machine Lea ning Applied o ALS P ognosis
combined da a om ou da ase s (PRO-ACT, T ophos, Exonhi ,
and Pa is Te ia y Re e al Cen e ), o aling 5,220 samples.
The ob ained da ase was u he di ided in o de elopmen
and alida ion se s. A e , he high-dimensional da a om he
de elopmen se we e educed and p ojec ed on o 2D space
h ough he Uni o m Mani old App oxima ion and P ojec ion
(UMAP) algo i hm. Thus, he au ho s we e able o p ojec
in o ma ion abou he pa ien s in o a 2D g aph. The 2D da a
we e di ided in o h ee 1-yea su i al p obabili y zones: High
(90%), In e media e (80%), and Low (58%). Then, he alida ion
se was used o e alua e he p oposed model, and he esul s we e
compa ed wi h he Random Fo es and he Logis ic Reg ession
models. The UMAP model ob ained be e classi ica ion esul s
(F1 sco e: 96%, Balanced Accu acy: 91%) when compa ed o he
a e age esul s o he o he models (F1 sco e: 50%, Balanced
Accu acy: 60%). The adop ed app oach also helped iden i y he
bioma ke s wi h highe o lowe co ela ion wi h he su i al
p edic ion. Fo example, he age and ALSFRS sco e p esen ed
a high co ela ion, while he gende and weigh showed a low
co ela ion. Howe e , he o al comp ehension o he ela ionship
be ween inpu and ou pu a iables canno be ob ained because
he adop ed model is conside ed a black-box app oach, which
deg ades i s in e p e abili y.
Despi e G eco e al. (2021) aimed o ind blood analy es o
dis inguish pa ien s who ha e ALS om hose wi h Lowe Mo o
Neu on Disease (LMND), hey also s udied he classi ica ion o
hese pa ien s wi h ela ion o hei disease p og ession a es
(High o Low). They analyzed clinic, demog aphic, and blood
(108 analy es) da a om 41 ALS pa ien s. An SVM model was
de eloped, and he Recu si e-Fea u e-Elimina ion algo i hm was
used as a ea u e selec ion me hod. This model ob ained an
accu acy o 87.25% in classi ying ALS pa ien s in o he High and
Low g oups using he i s 16 anked analy es, indica ing he
po en ial o using blood da a as p edic o bioma ke s. Ele a ed
le els o Vi amin-B12, To al Choles e ol, and HDL we e ela ed
o a highe disease p og ession a e.
Leão e al. (2021) p oposed a p edic i e model based on
Dynamic Bayesian Ne wo ks (DBN), including bo h s a ic and
longi udinal da a. They accessed da a om he Po uguese ALS
da ase (n=1,214), and he a ge p edic ion was he disease
p og ession (ALSFRS sco e and subsco es) ela ed o he need
o NIV suppo . To be p ocessed by he DBN model, he
longi udinal da a we e con e ed in o ime-se ies da a and hen
di ided in o Be o e NIV and A e NIV subse s. Thus, hey we e
able o de e mine he mos ele an bioma ke s ela ed o hese
wo essen ial disease s ages. The au ho s de eloped a p edic i e
model, e med s dDBN amewo k, which uses s a iona y DBNs
o p edic disease p og ession and non-s a iona y DBNs o
de e mine how he bioma ke s analyzed change o e ime in each
subse . The a e age esul s o p edic ing disease p og ession
we e abo e 80% o bo h subse s ega ding he Accu acy,
Sensi i i y, and AUC me ics, demons a ing he po en ial o
he p oposed me hodology. G aphs we e gene a ed o isualize
how he bioma ke s change o e ime, displaying hei alues
in di e en ime s eps o each s age (be o e and a e NIV).
This app oach allowed iden i ying some in e es ing ela ionships,
as ollowing men ioned. The Maximum Expi a o y P essu e
(MEP) was conside ed he mos impo an espi a o y exam o
p edic he pa ien en ila o y decline be o e he need o NIV
suppo . The ALSFRS Bulba subsco e had mo e in luence on
disease p og ession a e NIV han be o e NIV. The BMI and
Disease Du a ion had a s onge in luence han he o he s a ic
bioma ke s o bo h subse s.
4. DISCUSSION
This s udy sys ema ically e iewed he li e a u e o iden i y
ele an s udies ha used ML app oaches o assis ALS disease
p ognosis. As explained be o e in Sec ion 2, we ocused on hose
s udies comp ising bioma ke s commonly p esen in he daily
ALS clinical p ac ice. We iden i ied 10 s udies and de ailed hei
he a ge p edic ions, bes ML algo i hm, pe o mance, da ase s,
samples size, echniques, alida ion s a egies, bioma ke s
e alua ed, and he mos ele an bioma ke s iden i ied.
4.1. ALS Da ase s and Da a P ep ocessing
No ably, he s udies accessed da ase s ha concen a e ALS
pa ien s om Eu ope and he Uni ed S a es o Ame ica. Da a
om o he egions we e no analyzed (e.g., Sou h Ame ica,
A ica, o Asia). We conside his analysis essen ial o con i m
(o no ) i he p edic i e ML solu ions can be b oadly gene alized
and i di e en da ase s can be combined o compose an e en
mo e ele an ALS da ase . Mos o he s udies (60%) analyzed
da a om he PRO-ACT da ase . PRO-ACT is he la ges public
ALS da ase a ailable, con aining o e 10,000 samples, se ing
as a basis o se e al s udies on ALS disease, and sui able
o de eloping ML solu ions. Howe e , some s udies included
ad ised ha he PRO-ACT has limi a ions ha can inc ease he
isk o c ea ing biased models (Tang e al., 2019; G ollemund
e al., 2020; Hadad and Le ne , 2020). P e ious s udies also
epo ed hese PRO-ACT limi a ions, and he isk o i does no
ep esen he clinical pa ien popula ion due o he inclusion and
exclusion c i e ia used in he clinical ials (Chio e al., 2011;
A assi e al., 2014). Fo ins ance, hei pa ien s end o be younge
and p esen ewe unc ional impai men s. In his sense, using
a alida ion s a egy ha includes an ex e nal da ase ep esen s
an al e na i e o dec ease bias isk and achie e a mo e eliable
ML algo i hm e alua ion. This s a egy was u ilized by Hadad
and Le ne (2020) and G ollemund e al. (2020).Hadad and
Le ne (2020) c ea ed a aining da ase combining samples om
he PRO-ACT (100%) and Tel A i (90%) da ase . The samples
emaining (10%) o he Tel A i da ase we e used o es he
model. G ollemund e al. (2020) pe o med he alida ion using
he Pa is da ase , which was no used in he aining and es ing
s ages. P e e ably, he ex e nal da ase should con ain da a om
he clinical pa ien popula ion.
When designing ML solu ions, we need o be awa e o
issues ha can a ec he pe o mance and eliabili y o he
model, such as missing alues o da a imbalance. The PRO-
ACT da ase p esen ed a conside able amoun o missing alues
wha caused ha only 32% o i s samples could be used in
p ac ice. Thus, i is aluable o e alua e how he missing da a
impu a ion me hods can help o inc ease he sample size.
an de Bu gh e al. (2017) and Tang e al. (2019) used a mo e
F on ie s in Compu e Science | www. on ie sin.o g 9Ap il 2022 | Volume 4 | A icle 869140