ORIGINAL ARTICLE
CONELPABO: composi e ne wo ks lea ning ia pa allel Bayesian
op imiza ion o p edic emaining use ul li e in p edic i e main enance
Da id Solı
´s-Ma ı
´n
1,2
•Juan Gala
´n-Pa
´ez
1,2
•Joaquı
´n Bo ego-Dı
´az
1,2
Recei ed: 13 Ma ch 2024 / Accep ed: 3 Janua y 2025 / Published online: 29 Janua y 2025
The Au ho (s) 2025
Abs ac
Main aining equipmen and machine y in indus ies is impe a i e o maximizing ope a ional e iciency and p olonging
hei li espan. The adop ion o p edic i e main enance enhances esou ce alloca ion, p oduc i i y, and p oduc quali y by
p oac i ely iden i ying and add essing po en ial equipmen anomalies h ough igo ous da a analysis be o e hey escala e
in o c i ical issues. Consequen ly, hese measu es s eng hen ma ke compe i i eness and gene a e a o able economic
ou comes. In many applica ions, senso s ope a e a high equencies o cap u e da a o e ex ended pe iods. This wo k
in oduces CONELPABO (Composi e Ne wo ks Lea ning ia Pa allel Bayesian Op imiza ion), a amewo k o analyzing
long ime se ies da a, pa icula ly o p edic ing he emaining use ul li e o a sys em o componen . I uses a di ide-and-
conque s a egy o manage he exponen ial g ow h in he hype pa ame e sea ch space du ing Bayesian Op imiza ion and
o accele a e model aining by 50%. Addi ionally, his s a egy enables he aining o deepe ne wo ks wi h limi ed
esou ces. The use ulness o he amewo k is demons a ed h ough wo case s udies, in which i achie es s a e-o - he-a
esul s, showing ha CNN-CNN and RNN-RNN a chi ec u es a e highly e ec i e o long ime-se ies da a. These
a chi ec u es ou pe o m many exis ing app oaches and challenge he common academic ocus on CNN-RNN hyb ids.
Keywo ds P ognos ics and heal h managemen Remaining use ul li e Deep lea ning Deep lea ning Bayesian
op imiza ion
Lis o symbols
XRaw signals da ase
e
XNo malized signals da ase
Yk
RUL o a uni ka ime
e
Xk
No malized signals a ime o a uni k
LwLeng h o he sliding window
e
Xk
No malized signals o a uni kbe ween
Lwand
TULkTo al use ul li e o he uni k
Ck
Cycle numbe o he uni ka ime
CbsCon olu ion block size
NcbNumbe o con olu ion blocks
Neb;Neb Numbe o encoding and decoding
N b Numbe o ecu en blocks o a RNN
N u Numbe ecu en uni s o a RNN laye
d a e Dila ion a e o a con olu ion
cxNeu ons in he x- h ully connec ed laye
con ; c; ou Ac i a ion unc ion o a con olu ion laye ,
ully connec ed laye and he ou pu laye
o a ne wo k.
KsKe nel size o a con olu ion
l1;l2Weigh s o L1and L2 egula iza ion
l Lea ning a e
BsBa ch size
A
jj Numbe o da a a ibu es
s ide S ep be ween windows applied o he aw
inpu
sw RUL p edic ion smoo hing window size
Juan Gala
´n-Pa
´ez and Joaquı
´n Bo ego-Dı
´az ha e con ibu ed
equally o his wo k.
&Da id Solı
´s-Ma ı
´n
[email p o ec ed]
Juan Gala
´n-Pa
´ez
[email p o ec ed]
Joaquı
´n Bo ego-Dı
´az
[email p o ec ed]
1
Depa amen o de Ciencias de la Compu acio
´n e In eligencia
A i icial, Uni e sidad de Se illa, Se illa, Spain
2
Da ik Ingelligence, S.A, Se ille, Spain
123
Neu al Compu ing and Applica ions (2025) 37:7423–7441
h ps://doi.o g/10.1007/s00521-025-10995-z(0123456789().,- olV)(0123456789().,- olV)
1 In oduc ion
P ope main enance o indus ial equipmen and machine y
is c i ical o ensu e ope a ing e iciency and ex end hei
li espan, esul ing in lowe eplacemen and epai cos s
and a long- e m posi i e economic impac . Addi ionally,
he a ailabili y o he sys em is a key ac o o conside .
Es ima ing he p obabili y o ailu e can help p e en
unexpec ed mal unc ions by enabling imely main enance
in e en ions, he eby inc easing equipmen a ailabili y
and main aining he p oduc ion low [36]. The ield o
esea ch has gained conside able a en ion wi h he eme -
gence o he Indus y 4.0 pa adigm [58,65].
P edic i e main enance, based on con inuous da a
moni o ing and he u iliza ion o ad anced echnologies
such as ib a ion analysis o wea senso s, p oac i ely
iden i ies po en ial ailu es be o e hey occu . P e en ing
unplanned down ime and op imizing he a ailabili y o
p oduc i e asse s, esul s in signi ican cos sa ings and,
he e o e, inc eased p o i abili y. This app oach no only
a oids he subs an ial cos s associa ed wi h machine y
ailu e bu also mi iga es he in ica e p ocesses in ol ed in
handling sys em ailu es [58,64]. This also imp o es
esou ce planning, p oduc i i y and p oduc quali y, and
ma ke compe i i eness, esul ing in a posi i e economic
impac bo h in e nally and ex e nally [47].
Es ima ing he emaining use ul li e (RUL) o a com-
ponen o sys em is a c i ical ask wi hin P ognos ics and
Heal h Managemen (PHM), and in ol es analyzing he
sys em’s beha io o e ime o p edic i s u u e eliabili y
and deg ada ion [46,54]. The main objec i e o RUL
analysis is o de e mine when he sys em will ail o when a
ce ain le el o deg ada ion will be eached. Solu ions able
o success ully es ima e he RUL o a sys em can help
indus ies mi iga e losses caused by unplanned down ime
and epai ing expenses [20].
The main app oaches o es ima ing he RUL o a sys-
em, in he ield o PHM, could be classi ied as model-
based o da a-d i en. Model-based me hods exploi physi-
cal and s a is ical modeling o design a deg ada ion model
ha p edic s he sys em’s deg ada ion end. De eloping
deg ada ion models o highly complex sys ems is a chal-
lenging ask as i equi es an in-dep h unde s anding o he
physical cha ac e is ics o he componen s ha a e p one o
ailu e, and he ypes o ailu es hese may expe ience [73].
Fu he mo e, model-based me hods ely on ce ain
assump ions ha can in oduce biases, hus educing hei
p edic i e pe o mance.
Da a-d i en me hods ha e gained popula i y due o hei
abili y o le e age la ge olumes o da a gene a ed by
mode n senso s and sys ems (e.g., In e ne o Things). This
kind o me hod allows he p ocessing o his o ical da a o
ex ac pa e ns, wi h he goal o de ec ing di e en
deg ada ion ends. Many app oaches ha e been explo ed
in he li e a u e, including neu al ne wo ks (NN) [57], deep
lea ning (DL) [13], and ensemble me hods based on deci-
sion ees [1,37]. Suppo ec o machines (SVM) ha e
been applied o p edic he RUL o ai c a engines in
s udies such as [42] and [43]. To handle unce ain y,
Bayesian ne wo ks [41] and uzzy logic-based sys ems [6]
ha e been p oposed, leading o mo e obus RUL
p edic ions.
DL models a e among he mos popula and p omising
da a-d i en me hods. O e he pas decade, DL echniques
ha e been widely u ilized, pa icula ly o complex asks
in ol ing high-dimensional nonlinea da a. DL has
demons a ed ema kable success in di e se ields, such as
Image P ocessing, Na u al Language P ocessing and Signal
P ocessing, among o he s. Consequen ly, i is no su p is-
ing ha DL-based app oaches ha e gained widesp ead use
in PHM esea ch.
As p e iously no ed, RUL p edic ion in ol es dealing
wi h complex condi ion moni o ing da a om sys ems ha
can consis o mul iple subsys ems and ailu e ypes. This
means ha moni o ing da a will comp ise mul iple a i-
ables and dimensions, making i c ucial o da a-d i en
RUL p edic ion me hods handle high-dimensional da a.
While DL algo i hms a e known o hei abili y o handle
high-dimensional da a, dimension educ ion echniques a e
o en employed o educe compu a ional complexi y, and
se e as egula iza ion mechanisms. Such echniques can
help o simpli y and gene alize he models used o RUL
p edic ion, and hus, ha e become a common ool in he
ield.
1.1 Aim o he pape
Many wo ks p opose ne wo k a chi ec u es wi hou speci-
ying how hey de i ed hose a chi ec u es. We emphasize
he impo ance o a igo ous ne wo k a chi ec u e sea ch o
enhance he c edibili y o published esea ch and educe
he isk o o e i ing. Howe e , his igo comes wi h he
disad an age o equi ing conside able ime, pa icula ly
when alida ing he me hod ac oss nume ous a chi ec u es
and wi h a c oss- alida ion app oach.
Taking his in o accoun , he goal o his wo k is no o
ou pe o m exis ing me hods in p edic i e accu acy bu o
signi ican ly educe he ime and memo y esou ces
equi ed o aining deep lea ning models, especially when
a ne wo k a chi ec u e sea ch is in ol ed.
The p ima y goal o his pape is o demons a e he
e ec i eness o Pa allel Bayesian Op imiza ion (BO) in
aining composi e ne wo ks and well-es ablished
me hodologies, such as hyb id models (CNNs and RNNs),
7424 Neu al Compu ing and Applica ions (2025) 37:7423–7441
123
o add ess a speci ic challenge: p edic ing he RUL using
long da a sequences.
Addi ionally, we explo e he use o a ious hyb id DL
echniques in p edic ing he RUL o a sys em by add essing
wo dis inc case s udies. The concep o hyb id DL is
ex ended o include composi e s uc u es, whe e combi-
na ions such as RNN-RNN o CNN-CNN a e conside ed.
Thus, his s udy is no limi ed solely o CNN-RNN hyb id
models; ins ead, i examines all possible combina ions o
CNN and RNN a chi ec u es. To he bes o ou knowl-
edge, such a comple e s udy has no been p e iously
conduc ed.
To achie e his goal, a comp ehensi e amewo k was
designed o analyzing e y long da a sequences, pa icu-
la ly ime se ies da a, in he con ex o RUL p edic ion.
This amewo k in eg a es a me iculously designed c oss-
alida ion and hype pa ame e op imiza ion p ocess, lead-
ing o op imal models o analyzing such sequences. To
his aim, a neu al a chi ec u e sea ch (NAS) app oach
enhanced by BO has been used.
The inno a ion o his wo k is e lec ed in how we
combine hese elemen s o add ess challenges speci ic o
he applica ion domain. By using composi e ne wo ks and
s a egically sepa a ing ea u e ex ac ion and p edic ion
componen s, we op imize esou ce u iliza ion and make he
BO sea ch space mo e ac able. This app oach educes
aining ime and achie es s a e-o - he-a (SOTA) esul s.
The es o he pape is s uc u ed as ollows. The nex
sec ion (Sec . 2) p o ides a b ie desc ip ion o he RUL
p oblem and he main DL me hods and echniques con-
side ed in his wo k. Sec ion 3is de o ed o desc ibe he
p oposed amewo k CONELPABO, which aims o sim-
pli y he modeling p ocess o his ype o p oblem,
h ough p ep ocessing echniques, adjus ing he ime win-
dow o RUL models, and exploi ing o he speci ic cha -
ac e is ics o he RUL challenge. Sec ion 4in oduces he
case s udies and hei main ea u es, and p esen s he
esul s ob ained by applying he p oposed me hodology on
bo h. Sec ion 5p o ides an analysis o he achie emen s o
his wo k and highligh s i s s eng hs and weaknesses. The
pape concludes wi h Sec . 6, p o iding some conside a-
ions abou he conduc ed wo k.
1.2 Rela ed wo k
RNN and CNN ha e been widely used o p edic ing he
RUL o machine y. Fo example, in [69,72,74] he au ho s
success ully apply basic RNN models o p edic RUL. In
[45], a dual-channel LSTM a chi ec u e was u ilized,
inco po a ing momen um smoo hing in o he p edic ions.
Bidi ec ional RNNs ha e been s udied o RUL es ima ion
in wo ks like [21] and [32].
O he wo ks ha e explo ed he applica ion o CNNs o
p edic RUL. Fo example, [30] and [29] a e no ewo hy.
[29] p oposes a Mul i-Scale CNN (MS-CNN), which
consis s o h ee mul i-scale blocks (MS-BLOCKs) whe e
con olu ion ope a ions o h ee di e en sizes a e applied
in pa allel o ex ac ea u es a a ying scales.
Dimensionali y educ ion app oaches ha e also been
used o p edic RUL. In [35,56,70], encode -decode
ne wo ks we e ained in an unsupe ised manne . The
encode is hen used o c ea e a Heal h Index (HI) o o
eed o he models using a sliding window.
Recen ly, esea che s ha e been using combina ions o
DL a chi ec u es o le e age he unique ad an ages o
each. One o he mos common hyb id models consis s o a
CNN ollowed by a RNN. While he CNN ocuses on
ex ac ing spa ial ea u es, he RNN exploi s he empo al
dependencies be ween ime-se ies da a poin s. These
app oaches usually di ide ime-se ies da a in o subse-
quences using a sliding window app oach. Each subse-
quence is hen designa ed as inpu o a model o ea u e
ex ac ion. La e , he ex ac ed ea u es om he i s model
a e ed in o a second model o accomplish he inal ask.
Examples o hese wo ks include [5,8,27,48,
52,60,63,68].
In he con ex o BO, esea che s ha e de eloped a i-
ous me hods o enhance i s applica ion in NAS. Much o
he esea ch in his domain has concen a ed on designing
inno a i e su oga e models [33,53,55,67] and encoding
schemes o neu al a chi ec u es [12,62,66].
The cu se o dimensionali y p esen in he BO sea ch
space ha e been app oached oo in some wo ks.
LaNAS[61] uses a hie a chical pa i ioning s a egy o
di ide he sea ch space in o good o bad egions ha
con ain ne wo ks wi h simila pe o mance me ics and
lead he sea ch owa d good egions.
Ou side he con ex o NAS, [23] and [17] p opose
decomposing he a ge unc ion in o addi i e s uc u es.
While his app oach can be e ec i e, i is limi ed o
unc ions ha a e amenable o addi i e decomposi ion,
es ic ing i s applicabili y o a na ow subse o op i-
miza ion p oblems. A mo e gene al app oach is p esen ed
in [4], which add esses he op imiza ion o composi e
unc ions o he o m ðxÞ¼gðhðxÞÞ. In hei amewo k,
gis modeled as an expensi e- o-e alua e black-box unc-
ion, while h ep esen s a compu a ionally inexpensi e
unc ion ha can be e alua ed o app oxima ed e icien ly.
1.3 Con ibu ions
In compa ison wi h exis ing li e a u e, ou wo k di e ges
signi ican ly in i s ocus and me hodology. While many
s udies ha e explo ed hyb id ne wo k a chi ec u es, such as
CNN-RNN combina ions [5,8,27,48,52,60,63,68],
Neu al Compu ing and Applica ions (2025) 37:7423–7441 7425
123
he e has been limi ed in es iga ion in o simple a chi ec-
u es like CNN-CNN, RNN-RNN, o uncon en ional
combina ions such as RNN-CNN. Ou analysis highligh s
ha some speci ic asks akes ad an age o hese a chi-
ec u es, challenging he p e ailing emphasis on hyb id
models in NAS.
Rega ding he BO p ocess, ou wo k add esses a simila
p oblem o ha explo ed in [4], which ocuses on op i-
mizing composi e unc ions o he o m ðxÞ¼gðhðxÞÞ.
Howe e , unlike [4], whe e he inne unc ion his assumed
o be cheap o e alua e, in ou case, he unc ion is
compu a ionally expensi e o e alua e. To add ess his
challenge, we design a dual pa allel BO p ocess inspi ed by
he addi i e decomposi ion me hods p oposed in [23]. This
app oach enables e icien explo a ion despi e he compu-
a ional demands associa ed wi h e alua ing . Mo eo e ,
by le e aging p e-compu ed embeddings, we signi ican ly
educe he ime equi ed o e alua e he g unc ion du ing
he BO p ocess, hus enhancing he o e all e iciency o he
amewo k.
Mo eo e , unlike [4] and [23], which do no ocus on
NAS, ou amewo k speci ically applies BO echniques o
NAS p oblems. This adap a ion in eg a es a dual pa allel
BO p ocess in o he con ex o neu al a chi ec u e op i-
miza ion, enabling a no el con ibu ion in he in e sec ion
o BO and NAS ields. To he bes o ou knowledge, no
p io wo k has explo ed his o simila app oaches wi hin
he NAS domain o RUL p edic ion asks.
The con ibu ions o his wo k a e summa ized as
ollows:
•The design o wo pa allel BO p ocess, enabling
e icien explo a ion o hype pa ame e s o ne wo k
a chi ec u es. The me hodology in ol es wo lea ning
s ages: (i) lea ning an encoding om aw da a, and (ii)
using his encoding o ain a inal p edic i e model.
Expe imen al esul s demons a e SOTA pe o mance
on benchma k da ase s, wi h a educ ion in GPU
memo y consump ion and a signi ican 50% educ ion
in aining ime. This enables he aining o la ge
ne wo ks wi h limi ed esou ces and allows o a
b oade ange o expe imen s du ing model sea ch.
•Empi ical e idence showing ha simple a chi ec u es,
such as CNN-CNN, can ou pe o m mo e complex
designs o speci ic da ase s (e.g., N-CMAPSS), while
RNN-RNN a chi ec u es excel in o he s (e.g., PRO-
NOSTIA). These indings challenge con en ional p e -
e ences o encode -decode o hyb id CNN-RNN
app oaches.
•The publica ion o he sou ce code o he amewo k
and expe imen al se ings, enhancing he ep oducibil-
i y o he esul s and enabling u u e esea ch.
2 Ma e ial and me hods
This sec ion p o ides an o e iew o he me hodologies
and ools used in his wo k. Fi s , he RUL es ima ion
p oblem is o mally de ined, emphasizing he key me ics
conside ed in his ask. Nex , he deep lea ning a chi ec-
u es u ilized in he s udy a e desc ibed, wi h a ocus on
Con olu ional Neu al Ne wo ks (CNN) and Recu en
Neu al Ne wo ks (RNN).
2.1 RUL p oblem de ini ion
The main goal in RUL is o es ima e he amoun o ime a
piece o equipmen will unc ion e ec i ely be o e needing
o be eplaced o o e hauled. The p oblem a hand in ol es
de eloping a model F o p edic he emaining use ul li e
y. This p oblem can be exp essed ma hema ically as he
ollowing op imiza ion p oblem:
a g min
hX
M
i¼1
Lyi^
yi
ðÞ ð1Þ
whe e Mis he numbe o obse a ions, Lis he loss
unc ion, yi ep esen s he ac ual RUL o he i- h obse -
a ion, and ^
yi ep esen s he p edic ed RUL o he i- h
obse a ion, wi h ^
yi¼FhðÞ and hbeing he pa ame e s o
he model F o be op imized. In his wo k, he loss unc ion
Lis de ined as he mean-squa e e o (MSE). MSE is
sensi i e o la ge e o s, which helps he model o ocus on
minimizing subs an ial p edic ion e o s. This cha ac e is-
ic is pa icula ly use ul in applica ions whe e la ge de i-
a ions om he ue alue a e c i ical and po en ially mo e
cos ly, like in RUL asks. While he ini ial choice o MSE
is mo i a ed by i s sensi i i y o la ge e o s, ma hema ical
p ope ies, and s anda d use in eg ession asks, we
acknowledge he impo ance o explo ing al e na i e loss
unc ions.
Addi ionally, in his wo k, he mean-absolu e e o and
he NASA sco ing we e compu ed o each expe imen .
The NASA sco ing unc ion (Ns,[51]) is de ined as:
Ns¼1
MX
M
i¼1
expðajyi^
yijÞ 1ð2Þ
whe e aequals 1
13 when ^
yi yand 1
10 o he wise. The NASA
sco ing unc ion o RUL p edic ion is ano he ele an
me ic. I emphasizes he accu acy o he p edic ion nea
he end o he equipmen li e, which aligns closely wi h
p ac ical applica ion needs. Inco po a ing his sco ing
unc ion as a pa o he e alua ion me ic could o e a
mo e comp ehensi e assessmen o he model pe o mance
in eal-wo ld scena ios.
7426 Neu al Compu ing and Applica ions (2025) 37:7423–7441
123
Figu e 1p esen s he me ics used in his wo k o p o-
ide a be e unde s anding o hei s eng hs and
weaknesses.
2.2 Deep lea ning
In his wo k, supe ised DL me hods a e employed o
add ess he men ioned RUL p edic ion p oblem. The e o e,
he model Fis a deep neu al ne wo k, and he pa ame e s h
o be op imized include he se o weigh s o he ne wo k,
along wi h o he addi ional hype pa ame e s (such as
lea ning a e, ba ch size, window size, e c.). The ne wo k is
ed wi h da a uni s X2Rn;T, ep esen ing a mul i a ia e
ime se ies inpu composed o na ibu es wi h Tda a
poin s each.
Nex , a concise o e iew o he di e en deep lea ning
me hods and a chi ec u es conside ed in his s udy is
p o ided.
2.2.1 Con olu ional neu al ne wo ks (CNNs)
CNNs[25] a e a ype o neu al ne wo k known o hei
success in asks like image p ocessing. These ne wo ks use
con olu ion and pooling ope a ions o ex ac local ea u es
om he inpu , implemen ing pa ame e sha ing and
educing compu a ional equi emen s.
Usually, CNNs can be di ided in o wo pa s. Fi s ly, he
con olu ional pa ocuses on ex ac ing ea u es. This pa
is o med by s acking con olu ional laye s and pooling
laye s, among o he ypes o laye s, such as ba ch no -
maliza ion laye s. The ex ac ed ea u es om he las laye
o his i s pa a e ed in o he second pa o make he
inal p edic ion. The second pa ypically consis s o a ew
dense o ully connec ed laye s. This a chi ec u e is
depic ed in Fig. 2A.
In his wo k, he ex ension o he CNN, known as MS-
CNN [29], has been conside ed. This a chi ec u e includes,
a he beginning o he ne wo k, a ew blocks o h ee
pa allel con olu ion laye s. Each o he con olu ions has a
di e en ke nel size. The ea u e maps o he h ee con-
olu ion ope a ions a e collec ed and conca ena ed o be
ed in o he nex block. This enables he ex ac ion o
ea u es a mul iple scales. This a chi ec u e is shown in
Fig. 2B.
2.2.2 Recu en neu al ne wo ks (RNNs)
RNNs[50] a e designed o handle sequen ial da a by
inco po a ing memo y mechanisms o cap u e empo al
dependencies. LSTM[19] cells imp o e upon s anda d
RNNs by managing long sequences and add essing he
anishing g adien p oblem h ough a ious ga es ha
con ol in o ma ion low. GRU[10] cells o e a simple
al e na i e wi h ewe pa ame e s, making hem compu a-
ionally e icien . Bo h RNN and LSTM/GRU a chi ec u es
a e used o cap u e ime-dependen ea u es and make
p edic ions.
Simila o CNNs, RNNs a e ypically di ided in o wo
pa s. The i s pa consis s o one o mo e ecu en laye s
ha a e adep a cap u ing empo al dependencies wi hin
sequen ial da a. The nex sec ion o he ne wo k u ilizes he
ea u es ex ac ed by he ecu en laye s o es ima e he
desi ed ou pu . This a chi ec u e is illus a ed in Fig. 2C.
3 CONELPABO amewo k
This sec ion in oduces he CONELPABO amewo k and
jus i ies he design decisions made o de ine i . Addi ion-
ally, he expe imen al se ings and pa ame e s used o
add ess each o he case s udies a e p o ided.
3.1 Modeling s ages
A ypical machine lea ning ask (in gene al e ms, no
speci ically wi hin he deep lea ning ield) usually ollows
he ollowing ou phases:
1. Da a p ep ocessing: In he da a p ep ocessing phase,
aw da a is cleaned and ans o med be o e aining he
models. The goal o his phase is o ensu e ha he da a
is p epa ed in a way ha maximizes he pe o mance o
he models.
2. Fea u e enginee ing: This phase in ol es selec ing
ea u es ele an o modeling, as well as c ea ing new
ones. Fea u e enginee ing usually equi es a deep
Fig. 1 MSE is sui able when i ’s impo an o hea ily penalize la ge
de ia ions, bu i can be o e ly in luenced by ou lie s. MAE p o ides
a s aigh o wa d in e p e a ion and is obus o ou lie s, bu ea s all
e o s equally. NASA Sco ing Me ic o e s a nuanced app oach o
RUL p edic ion by emphasizing he impac o e o s, pa icula ly la e
p edic ions, which is c ucial in many enginee ing applica ions
Neu al Compu ing and Applica ions (2025) 37:7423–7441 7427
123
unde s anding o he da a and p oblem a hand, and
o en can be an i e a i e p ocess o es ing and e ining
ea u e se s o achie e be e esul s.
3. Model selec ion: In he model selec ion phase, he
pe o mance o di e en models and hype pa ame e
se s is e alua ed and compa ed o selec he bes model.
4. Model alida ion: In his phase, he inal model is
ained on all p e-p ocessed ain da a a ailable, and
e alua ed using a es se , which was no used du ing
any s age o he model de elopmen , o assess i s
pe o mance.
The me hodology p oposed in his wo k simpli ies he
p e ious lis o phases by emo ing s eps 2 and 4. S ep 2 is
no longe necessa y since he ne wo ks a e ed wi h he aw
signal, and ea u e enginee ing is ca ied ou by he ne -
wo k i sel . S ep 4 can also be excluded, as models gen-
e a ed h ough c oss- alida ion can be used o make
in e ences by a e aging hei ou pu s. Using mul iple
models has he ad an age ha along wi h he a e age
p edic ion, a con idence in e al can be compu ed using he
p edic ions o each model.
The es o he sec ion co e s a ious aspec s ela ed o
he analysis o ime se ies da a. I begins by desc ibing he
p ocess o da a no maliza ion, which in ol es ans o ming
he da a in o a uni ied scale. Addi ionally, he sec ion
explains how he da a uni is di ided in o mul iple samples,
each associa ed wi h a speci ic RUL a ge ep esen ing he
emaining use ul li e. Then, he use o wo models in a
s acked manne o e ec i ely p ocess long ime se ies is
discussed. Finally, he hype pa ame e op imiza ion s a -
egy will be ou lined.
3.2 Da a p ep ocessing
Be o e aining he model, he da a we e no malized and
spli as desc ibed below:
•Da a s anda diza ion. No maliza ion ensu es ha all
ea u es ha e a simila impac on he lea ning p ocess
and p e en s any pa icula ea u e om domina ing he
aining, which can help he model o con e ge as e .
In his wo k, da a is no malized using da a s anda d-
iza ion, by sub ac ing he mean om each ea u e and
di iding by he s anda d de ia ion. This esul s in da a
ha ing a mean o ze o and a s anda d de ia ion o one:
x0¼xl
ð3Þ
whe e x2Xis an a ibu e o he mul i a ia e ime
se ies X.
•Time window p ocessing. The ime se ies ha e o be
pa i ioned using a sliding window ac oss he s anda d-
ized da a (see Fig. 4). The wid h o he window Lw, will
be op imized du ing he model selec ion phase (Sec .
3.5). Each da a uni inpu Xk
(Eq. 4) is a sequence o he
Lwmos ecen da a poin s o he no malized ea u es
e
Xk, ending a ime . The co esponding g ound-RUL
label is deno ed as y . Fo each uni , TkLwinpu s and
labels a e gene a ed, whe e Tkis he o al un- ime o
he uni in cycles.
Xk
¼½e
Xk
end Lw; :::; e
Xk
end ð4Þ
The RUL alue associa ed wi h each window yk
is
compu ed as TULkEk
, whe e TULkis he o al
Fig. 2 Schema ic o CNN and
RNN: Adepic s he common
CNN di ided in o he
con olu ional pa and he
eg ession pa . BShows an
ex ension o he CNN wi h an
addi ional pa o ex ac mul i-
scale ea u es. CIllus a es an
RNN composed o he i s pa
ocused on ex ac ing ime-
dependen ea u es and he
second pa ca ying ou he
p edic ion, wi h symbols
indica ing hype pa ame e s
op imized du ing he model
sea ch p ocess
7428 Neu al Compu ing and Applica ions (2025) 37:7423–7441
123
numbe o li e uni s in each da a expe imen k, and Ek
is
he numbe o li e uni s ha ha e elapsed om he
beginning o he da a expe imen a ime .Some
s udies, such as [29], o en se a cons an RUL a he
beginning o each expe imen . This is due o he
assump ion ha he e is no enough in o ma ion ini-
ially a ailable o cap u e he ini ial deg ada ion p ocess
accu a ely. In his s udy, his p ocedu e was no applied
ini ially o a oid po en ial bias.
3.3 C oss- alida ion
The la ge numbe o adjus able pa ame e s exis ing in
neu al ne wo ks makes i challenging o ind he combi-
na ion o hype pa ame e s ha yields he bes pe o mance
wi hou o e i ing he aining se . In addi ion, selec ing
he size o he sliding window is ano he c i ical aspec o
he p oposed me hodology.
In he li e a u e, he e exis a ious alida ion ech-
niques ha in ol e spli ing da a o a oid o e i ing. The
basic ‘‘single hold-ou s a egy’’ is usually no p e e ed as
i can lead o o e i ing on he hold-ou se du ing model
selec ion o hype pa ame e op imiza ion, esul ing in poo
gene aliza ion. In con as , he k- old c oss- alida ion
s a egy has he disad an age o dec easing he size o he
alida ion se as kinc eases ( he numbe o olds), making
i less ep esen a i e o he en i e da ase .
In his con ex , a k- old epea ed andom subsampling
alida ion s a egy is p e e ed. This s a egy andomly
di ides he da ase in o aining and alida ion se s, and he
p ocess is epea ed k imes. Du ing each epe i ion, a di -
e en andom spli is gene a ed, ensu ing ha each sample
is included in a alida ion se a leas once. This app oach
p o ides a be e ep esen a ion o he en i e da ase , and
he a e age o he esul s ob ained du ing he k epe i ions
is used o assess he pe o mance o he model. Figu e 3
illus a es he p ocess o k- old epea ed andom subsam-
pling alida ion.
In his wo k, k¼5 has been selec ed, and he chosen
alida ion se size is app oxima ely 20% o he expe imen
uni s o each da ase .
The alida ion s a egy p esen ed in many wo ks (e.g.,
[26,49], and [34]) consis in andomly selec ing a ce ain
pe cen age o da a poin s as es ing da a, while he
emainde is used as aining da a. The la e app oach
implies ha es ing and aining da a poin s a e in e lea ed,
p o iding he model wi h in o ma ion abou he es ing da a
dis ibu ion, ha is, including de ails abou u u e da a
poin s, which may lead o o e i ing, and he e o e, lead o
an unsa e model. In con as , in he p esen wo k, o
add ess such conce ns, he di ision o aining and
alida ion se s is based on ull uni s (such as bea ings o
ligh s), which cons i u es a sa e alida ion s a egy.
3.4 Composi e ne wo ks
The a chi ec u e p oposed in his wo k consis s o wo
s acked models and a inal mo ing a e age smoo hing
module, as shown in Fig. 4. The i s Fea u e Ex ac o
Ne wo k (FEN) aims o encode he aw signals, while he
second one, Reg ession Ne wo k A e (RNA), uses hese
encodings o p edic he RUL. Bo h models, FEN and
RNA, a e ained in a supe ised manne , wi h he inpu –
ou pu pai s being he ime-windowed signals o encodings
and he co esponding RUL labels, espec i ely.
The goal o he encoding s ep is o ans o m he aw
inpu signals in o a lowe -dimensional ep esen a ion,
cap u ing he mos ele an in o ma ion o p edic ing he
RUL using a limi ed numbe o da a poin s. This is
achie ed by aining a model o encode he inpu signals
in o a se o ea u es ha p ese e he impo an pa e ns
while emo ing noise and o he i ele an in o ma ion. The
esul ing encoding is hen used as he inpu o he RUL
p edic ion wi h he RNA model.
The FEN model is ained o p edic RUL, bu a e
aining, he las ou pu laye is emo ed, and he p e ious
laye is used o gene a e he encodings, which a e low-
dimensional ep esen a ions o he inpu s. The e o e, all he
ne wo k a chi ec u es es ed in his s udy, bo h CNN o
RNN, ha e wo ully connec ed laye s on op o gene a e
he inal p edic ion o he RUL. This is illus a ed in Fig. 4.
In his wo k, RNNs and CNNs ha e been chosen based
on hei p o en e ec i eness o ime-se ies analysis and
sequence p edic ion asks, which a e cen al o ou p ob-
lem. RNNs a e pa icula ly well-sui ed o modeling em-
po al dependencies in sequen ial da a, making hem
e ec i e o cap u ing he dynamic na u e o ime-se ies
da a. CNNs, on he o he hand, a e highly e ec i e in
ex ac ing local ea u es and pa e ns, which can be bene-
icial in iden i ying impo an cha ac e is ics wi hin long
ime-se ies sequences.
We acknowledge ha o he a chi ec u es, such as
ans o me s, ha e shown p omising esul s in a ious
Fig. 3 k- old c oss- alida ion s a egy (le ), and k- old andom
sampling s a egy ( igh )
Neu al Compu ing and Applica ions (2025) 37:7423–7441 7429
123
sequence modeling and ime-se ies p edic ion asks.
T ans o me s, wi h hei sel -a en ion mechanisms, could
po en ially o e addi ional bene i s in cap u ing long- ange
dependencies and imp o ing model pe o mance. How-
e e , due o limi ed ime and esou ces o expe imen a-
ion, his s udy is ocused on RNNs and CNNs.
In gene al, he choice o he a chi ec u e and hype pa-
ame e s will signi ican ly impac he e ec i eness o he
RUL p edic ion models. The e o e, i is impo an o
ca e ully e alua e and op imize hese aspec s o he model
in o de o achie e he bes possible pe o mance [14,22].
The ollowing Sec . 3.5 desc ibes he me hod o selec ing
hype pa ame e s conside ed in his wo k.
The ou pu o he ne wo ks was con igu ed o use he
ReLU ac i a ion unc ion, since in RUL es ima ion nega-
i e alues ha e no sense. Thus, cons aining he ou pu o
be posi i e bene i he con e gence o he lea ning p ocess.
Addi ionally, we in oduce a mo ing a e age smoo hing
(MAS) module o gene a e he inal RUL p edic ion. MAS
imp o es RUL p edic ions by educing noise and luc ua-
ions in he p edic ed RUL alues. Wi h MAS, p edic ion
s abili y is enhanced, and p e ious ends a e e ained,
which p o ide aluable in o ma ion o cu en ime
p edic ions.
3.5 Model selec ion
Model selec ion e e s o he p ocess o choosing he mos
app op ia e o op imal model om a se o candida e
models o a gi en ask o p oblem. I in ol es e alua ing
and compa ing di e en models based on hei pe o -
mance, complexi y, and o he ele an c i e ia. In his
wo k, he c i e ion used is he pe o mance o he model
ob ained by c oss- alida ion.
Bayesian op imiza ion (BO) is a echnique ha is used
o op imize black-box unc ions ha a e expensi e o
e alua e. BO can be used as a model selec ion echnique. In
his case, he black-box unc ion ains and e alua es a
model [16,44]. The inpu is he se o hype pa ame e
alues o be assessed in he cu en i e a ion, whe eas he
ou pu o he unc ion ( he goal o he op imiza ion p ocess)
is he model pe o mance. BO wo ks by building a su o-
ga e model o he black-box unc ion, which is upda ed
a e each e alua ion (i e a ion). The su oga e model is
used o decide he nex se o hype pa ame e s o be
Fig. 4 FEN-RNA composi e
ne wo ks. Fi s ly, he FEN
model is ained o lea n o
p edic he a ge , ha is, he
RUL o a ce ain sys em. Then,
a ea u e se is ex ac ed om
he ained FEN model and used
o ain he RNA model using
also he RUL as a ge . Finally,
he p edic ion o he RNA
passes h ough a mo ing
a e age smoo hing module o
p o ide he inal RUL
p edic ion
7430 Neu al Compu ing and Applica ions (2025) 37:7423–7441
123
e alua ed, in a way ha maximizes he expec ed
imp o emen o he black-box unc ion.
In his wo k, wo BO p ocesses a e u ilized o op imize
he hype pa ame e s, including hose ha de ine he ne -
wo k a chi ec u e. One challenge wi h BO is ha he sea ch
space g ows exponen ially wi h he numbe o hype pa-
ame e s conside ed. This expansion in he sea ch space
equi es a high numbe o i e a ions o ind good local
minima, which has p o en o be ex emely challenging
[59]. To add ess his issue, a di ide-and-conque s a egy is
adop ed. Ins ead o using only one BO p ocess o op imize
all he hype pa ame e s, wo pa allel BO p ocesses a e
employed. The i s BO p ocess conduc s he op imiza ion
o he FEN model, and he second one sea ches he bes
hype pa ame e s o he RNA-MAS composi ion, u ilizing
he las FEN model es ed. Du ing he execu ion o each
model sea ch i e a ion o he RNA-MAS model, he
weigh s used in he las sea ch i e a ion o he FEN a e
ixed, o a oid hei modi ica ion du ing he aining p o-
cess o he RNA model. This p ocess is desc ibed in de ail
in he Algo i hm 1.
Algo i hm 1 Model sea ch algo i hm
Each BO p ocess is guided by di e en sco es, which
a e used o selec he nex se o hype pa ame e s o be
e alua ed du ing he op imiza ion p ocess. Fo he BO
p ocess o he FEN, he sco e NFEN om he independen ly
ained model is u ilized. The sco e NFEN measu es he
con ibu ion o he FEN o he inal model. To guide he
BO p ocess o he RNA-NAS, he sco e NRNAMAS,
ob ained by aining he RNA-NAS model by eusing he
bes - ained FEN, is used.
I is impo an o no e ha while he BO p ocess uses he
NASA sco e, he ne wo k i sel is ained using he MSE.
To op imize he model, each BO p ocess is execu ed o
Ni e a ions. In his wo k, Nwas se o 100. In he i s 20
i e a ions, he inpu hype pa ame e s a e selec ed andomly
and used by he BO p ocess o c ea e he ini ial es ima ion
o he hype pa ame e sea ch space. Subsequen ly, Baye-
sian ules a e applied o selec he mos p omising se o
hype pa ame e s o be es ed in he nex i e a ion. This
p ocess con inues un il he comple ion o he desi ed
numbe o i e a ions.
Addi ionally, he ea ly s opping echnique is used as
s opping c i e ion du ing he aining o each model. This is
a common echnique used in Machine Lea ning o p e en
o e i ing. I in ol es moni o ing he alida ion loss du -
ing aining and s opping he aining p ocess when he
alida ion loss s ops imp o ing, e en i he aining loss
con inues dec easing. The pa ience ac o is a pa ame e
ha de e mines he numbe o epochs he model can con-
inue aining wi hou any imp o emen in alida ion loss
be o e s opping. In his wo k, he pa ience ac o has been
se o 8 epochs ( ha is, i in he las 8 epochs he e is no
imp o emen in he alida ion loss, he aining p ocess
will be s opped).
In addi ion, an adap i e lea ning a e is used, which is
educed by a ac o o 0.1 i no imp o emen in he ali-
da ion loss is obse ed du ing h ee aining epochs. This
echnique accele a es he model con e gence and p e en s
i om o e shoo ing he minimum o he loss unc ion.
The a chi ec u es e alua ed o he FEN, including CNN
and RNN, a e also candida e a chi ec u es in he RNA
model selec ion phase. The ou possible combina ions,
which a e CNN-CNN, CNN-RNN, RNN-RNN and RNN-
CNN, ha e been analyzed. Table 1p o ides a summa y o
he pa ame e anges used in he Bayesian op imiza ion
p ocess o each o he a chi ec u es u ilized. The majo i y
o he hype pa ame e s co espond o he a chi ec u e o
he ne wo k, excep o he s ide and sw. The s ide
hype pa ame e de ines he gap be ween da a poin s when
he sliding window is applied (see Fig. 4). The sw deno es
he MAS window size used o apply smoo hing o p e-
dic ing he inal RUL.
3.6 Me hodology mo i a ion
Once he me hodology and he amewo k ha e been
in oduced, his sec ion will s a e he a ionale behind he
p oposed app oach.
•E icien esou ce u iliza ion: Spli ing he ne wo k in o
a ea u e ex ac o and a p edic o allows aining mo e
complex models wi h limi ed GPU esou ces. This
di ision educes he compu a ional load a each s age
by dec easing he numbe o pa ame e s in each
ne wo k and he size o he inpu , enabling us o
Neu al Compu ing and Applica ions (2025) 37:7423–7441 7431
123
esul s ob ained wi h o he s a e-o - he-a me hods, u he
alida ing he e ec i eness o his app oach.
Addi ionally, classical machine lea ning me hods, such
as SVM and ee-based algo i hms, we e included in he
compa ison. The esul s highligh he supe io pe o mance
o he neu al ne wo k-based app oach o e hese adi ional
me hods, pa icula ly due o he ad an age o he ne wo k
wi h au oma ic ea u e ex ac ion. In con as , he classical
me hods ely hea ily on manual ea u e selec ion and
ex ac ion, which makes hem less e ec i e when dealing
wi h la ge ime-se ies da ase s. The neu al ne wo k models
a e able o lea n ep esen a ions di ec ly om aw da a,
educing he need o handc a ed ea u e enginee ing and
he bias his could in oduce, and gi ing hem a signi ican
edge in pe o mance.
The main ad an age o he p oposed me hod is he
signi ican educ ion in aining ime. By spli ing he ne -
wo k in o wo componen s-one o gene a e embeddings and
ano he o es ima e he inal RUL- his app oach no only
enables he p ecompu a ion o embeddings bu also educes
he hype pa ame e sea ch space, making he BO p ocess
mo e ac able. These ac o s con ibu e o he supe io i y
o his app oach o e o he s.
Equa ions 5and 6p o ide a ma hema ical explana ion
o his issue. Fo example, when aining he RNN-RNN
ne wo k on he PRONOSTIA da ase , he end- o-end ne -
wo k akes app oxima ely 720 s pe epoch. In con as ,
using he p oposed app oach, he FEN ne wo k akes 49 s
pe epoch, and he RNA ne wo k akes 220 s pe epoch, o
a o al o oughly 269 s pe epoch. I is also impo an o
no e ha gene a ing embeddings o he en i e da ase wi h
he FEN ne wo k akes a ound 5 s, dis ibu ed ac oss he
epoch. Thus, he o al educ ion in aining ime exceeds
50%. This ime sa ing is pa icula ly c ucial in he con ex
o model sea ch, whe e nume ous models need o be
e alua ed and es ed.
Rega ding he ne wo k size, di iding he end- o-end
ne wo k allows aining la ge models. I is no ewo hy
ha , wi h an end- o-end ne wo k app oach, he inpu
dimension is signi ican ly highe . Fo example, in he
p e ious example, he end- o-end ne wo k has an inpu
shape o (360x2x360), which consis s o 360 subsequences,
each wi h 360 da a poin s and wo ea u es. Wi h a loa 32
da a ep esen a ion and a ba ch size o 128, his con igu-
a ion equi es 0.1GB o GPU memo y jus o he inpu
da a. In con as , he inpu shape o he FEN ne wo k is
(360x2), and o he RNA ne wo k, i is (360x100).
The e o e, he educed inpu shape is ano he impo an
achie emen o he p oposed app oach in dec easing
esou ce usage.
The implica ions o hese indings a e signi ican o
bo h he esea ch and indus ial communi ies, pa icula ly
o hose wi h limi ed compu a ional esou ces o ime
cons ain s. This app oach is ad an ageous o o ganiza-
ions ha canno a o d ex ensi e compu a ional esou ces
o need o ob ain esul s as e .
Wi h espec o he N-CMAPSS da ase used in his
s udy, his da ase p o ides aluable insigh s and allows o
igo ous es ing o p edic i e models. Howe e , i is
essen ial o acknowledge ha simula ions, by hei na u e,
a e limi ed in hei abili y o cap u e he ull complexi y o
eal-wo ld scena ios. Al hough ou DNN models demon-
s a e high pe o mance and ag eemen wi h he simula ed
da a, his pe o mance may no di ec ly ansla e o eal-
wo ld p opulsion sys ems due o disc epancies be ween
simula ed and ac ual condi ions. The e o e, while ou
indings p o ide a s ong ounda ion and demons a e he
po en ial o he p oposed app oach, u he alida ion wi h
eal-wo ld da a is necessa y o con i m he gene alizabili y
and obus ness o he models.
The main weakness o he p oposed app oach is ha he
acquisi ion unc ion used by he BO p ocess o he FEN
model is comple ely independen o he inal FEN-RNA
model. As a esul , he hype pa ame e s selec ed by he BO
p ocess a e likely o be subop imal. P o iding in o ma ion
abou he con ibu ion o he FEN model o he inal FEN-
RNA model could be an in e es ing imp o emen . This is a
poin ha should be conside ed as an a ea o enhancemen
in u u e wo k.
6 Conclusions
The amewo k p esen ed in his wo k o e s a obus
app oach o RUL es ima ion ha does no equi e expe
knowledge o he sys em unde s udy. I es ablishes a
obus model sea ch and c oss- alida ion s a egy, mi i-
ga ing he isk o o e i ing. Applying Bayesian Op i-
miza ion di ec ly o he hype pa ame e s o bo h ne wo ks
Table 6 Compa ison o ou bes esul s in PRONOSTIA wi h o he
simila app oaches in he li e a u e and o he machine lea ning
algo i hms. Bold alue iden i y he bes pe o mance
Model MAPE RMSE
LSVM 39.19 ±11.03 0.31 ±00.24
SVM 1.0e3 ±2.0e3 0.16 ±00.01
RF 6.6e3 ±1.2e3 0.19 ±00.02
XGB 51.18 ±15.28 0.22 ±00.01
CNN-GRU [40] 42.34 0.19
Bi-GRU [9] 44.49 –
CNN-LSTM [18] 46.32 –
CONELPABO (ou s) 35.27 ±21.03 0.13 ±00.02
7438 Neu al Compu ing and Applica ions (2025) 37:7423–7441
123
would be in ac able. The e o e, a pa allel BO p ocess has
been designed o acili a e he con e gence o he model
sea ch in a educed numbe o i e a ions.
A se ies o expe imen s we e pe o med o compa e
se e al combina ions o s a e-o - he-a ne wo k a chi ec-
u es, selec ing he bes -pe o ming a chi ec u es o build
he inal solu ion. The p ocess o ob aining he solu ion
mainly in ol ed wo lea ning s ages: i s , an encoding o
he aw da a is lea ned and used as inpu o he second
lea ning s age, which p oduces he inal model capable o
es ima ing RUL. The p oposed me hodology has achie ed
esul s ha su pass he s a e-o - he-a ones. Rema kably,
he esul s e ealed ha CNN-CNN, despi e being one o
he mos basic a chi ec u es analyzed, exhibi ed ou s and-
ing pe o mance wi h he N-CMAPSS da ase . Con e sely,
when dealing wi h he PRONOSTIA da ase , he RNN-
RNN a chi ec u e ou pe o med he es o he con igu a-
ions. We belie e ha hese esul s a e qui e in e es ing o
he esea ch communi y, as p e ious wo ks in he li e a u e
ha e mainly ocused on CNN-RNN o encode -decode
a chi ec u es.
The p oposed me hodology no only achie es s a e-o -
he-a pe o mance esul s bu also educes esou ce usage,
including GPU memo y and aining ime. This educ ion
allows aining la ge ne wo ks wi h ewe esou ces and
inc eases he numbe o expe imen s ha can be conduc ed
du ing model sea ch.
To conclude, i is wo h no ing ha his wo k has been
de eloped wi h a ocus on ep oducible esea ch. To his
aim, he pa ame e anges used o model selec ion ha e
been de ailed. In addi ion, he sou ce code o ain he
models and alida e esul s can be ound in a Gi Hub
eposi o y.
2
7 Technical issues
Py hon 3.9 was he p ima y p og amming language used in
his wo k, and he amewo k was implemen ed using i .
Fo ne wo k design and aining, one o he la es e sions
o Tenso Flow [2] and Ke as [11] (i.e., Tenso low 2.0 and
Ke as 2.3, espec i ely) we e used. To op imize he
hype pa ame e s o he deep lea ning model, we used Tune
1.6.0 [31], which is a scalable hype pa ame e uning
lib a y de eloped by Ray. The compu a ional esou ces
used o he expe imen s we e a se e wi h 32GB o RAM
memo y, and ou h eads. We also used wo GTX 1080Ti
GPUs o dis ibu e he di e en ne wo k aining asks and
speed up he aining p ocess.
Acknowledgemen s This wo k has been suppo ed by G an
PID2023-147198NB-I00 unded by MICIU/AEI/10.13039/
501100011033 (Agencia Es a al de In es igacio
´n) and by FEDER,
UE, and by he Minis y o Science and Educa ion o Spain h ough
he na ional p og am ‘‘Ayudas pa a con a os pa a la o macio
´nde
in es igado es en emp esas (DIN2019-010887/AEI/10.13039/
50110001103)’’, o S a e P og amme o Science Resea ch and Inno-
a ions 2017-2020.
Funding Funding o open access publishing: Uni e sidad de Se illa/
CBUA. Spanish Na ional Plan o Scien i ic and Technical Resea ch
and Inno a ion (MICIU/AEI/10.13039/501100011033, DIN2019-
010887/AEI/10.13039/50110001103).
Da a a ailabili y This wo k used he public benchma ks da ase s
N-CMAPSS [3] and PRONOSTIA [39].
Decla a ions
Con lic o in e es The au ho s decla e ha hey ha e no Con lic o
in e es ela ed o his wo k.
Open Access This a icle is licensed unde a C ea i e Commons
A ibu ion 4.0 In e na ional License, which pe mi s use, sha ing,
adap a ion, dis ibu ion and ep oduc ion in any medium o o ma , as
long as you gi e app op ia e c edi o he o iginal au ho (s) and he
sou ce, p o ide a link o he C ea i e Commons licence, and indica e
i changes we e made. The images o o he hi d pa y ma e ial in his
a icle a e included in he a icle’s C ea i e Commons licence, unless
indica ed o he wise in a c edi line o he ma e ial. I ma e ial is no
included in he a icle’s C ea i e Commons licence and you in ended
use is no pe mi ed by s a u o y egula ion o exceeds he pe mi ed
use, you will need o ob ain pe mission di ec ly om he copy igh
holde . To iew a copy o his licence, isi h p://c ea i ecommons.
o g/licenses/by/4.0/.
Re e ences
1. Remaining Use ul Li e (RUL) P edic ion o Rolling Elemen
Bea ing Using Random Fo es and G adien Boos ing Technique,
ASME In . Mechanical Enginee ing Cong ess and Exposi ion,
ol 13: Design, Reliabili y, Sa e y, and Risk (2018)
2. Abadi M, Aga wal A, Ba ham P, e al (2015) Tenso Flow: la ge-
scale machine lea ning on he e ogeneous sys ems. h ps://www.
enso low.o g/, so wa e a ailable om enso low.o g
3. A ias Chao M, Kulka ni C, Goebel K e al (2021) Ai c a engine
un- o- ailu e da ase unde eal ligh condi ions o p ognos ics
and diagnos ics. Da a 6(1):5
4. As udillo R, F azie P (2019) Bayesian op imiza ion o composi e
unc ions. In: In e na ional con e ence on machine lea ning,
PMLR, pp 354–363
5. Azyus AF, Wijaya SK, Na ed M (2023) P edic ion o emaining
use ul li e using he CNN-GRU ne wo k: a s udy on main enance
managemen . So w Impac s 17:100535
6. Ba aldi P, Di Maio F, Al-Dahidi S e al (2017) P edic ion o
indus ial equipmen emaining use ul li e by uzzy simila i y and
belie unc ion heo y. Expe Sys Appl 83:226–241
7. Be ghou T, Mouss MD, Mouss LH e al (2022) P ogne : a
ans e able deep ne wo k o ai c a engine damage p opaga ion
p ognosis unde eal ligh condi ions. Ae ospace 10(1):10
8. Bo e
´A, Seman LO, Camponoga a E e al (2023) Machine aul
de ec ion using a hyb id CNN-LSTM a en ion-based model.
Senso s 23(9):4512
2
u l anonymized.
Neu al Compu ing and Applica ions (2025) 37:7423–7441 7439
123
9. Chen Y, Peng G, Zhu Z e al (2020) A no el deep lea ning
me hod based on a en ion mechanism o bea ing emaining
use ul li e p edic ion. Appl So Compu 86:105919
10. Cho K, Van Me ie
¨nboe B, Bahdanau D, e al (2014) On he
p ope ies o neu al machine ansla ion: encode -decode
app oaches. a Xi p ep in a Xi :1409.1259
11. Cholle F e al (2015) Ke as. h ps://gi hub.com/ cholle /ke as
12. Deng B, Yan J, Lin D (2017) Peephole: p edic ing ne wo k
pe o mance be o e aining. a Xi p ep in a Xi :1712.03351
13. Deu sch J, He D (2017) Using deep lea ning-based app oach o
p edic emaining use ul li e o o a ing componen s. IEEE T ans
Sys Man Cybe n Sys 48(1):11–20
14. Ding J, Ta okh V, Yang Y (2018) Model selec ion echniques: an
o e iew. IEEE Signal P ocess Mag 35(6):16–34
15. Dong X, Huang J, Yang Y, e al (2017) Mo e is less: a mo e
complica ed ne wo k wi h less in e ence complexi y. In: P o-
ceedings o he IEEE con e ence on compu e ision and pa e n
ecogni ion, pp 5840–5848
16. F azie PI (2018) A u o ial on bayesian op imiza ion. a Xi
p ep in a Xi :1807.02811
17. Ga dne J, Guo C, Weinbe ge K, e al (2017) Disco e ing and
exploi ing addi i e s uc u e o bayesian op imiza ion. In: A i-
icial in elligence and s a is ics, PMLR, pp 1311–1319
18. Hinchi AZ, Tkioua M (2018) Rolling elemen bea ing emaining
use ul li e es ima ion based on a con olu ional long-sho - e m
memo y ne wo k. P ocedia Compu Sci 127:123–132
19. Hoch ei e S, Schmidhube J (1997) Long sho - e m memo y.
Neu al Compu 9(8):1735–1780
20. Holmbe g K, Komonen K, Oedewald P, e al (2004) Sa e y and
eliabili y— echnology e iew. No. BTUO43-031209 in VTT
Resea ch Repo , VTT Technical Resea ch Cen e o Finland,
Finland
21. Huang CG, Huang HZ, Li YF (2019) A bidi ec ional LSTM
p ognos ics me hod unde mul iple ope a ional condi ions. IEEE
T ans Ind Elec on 66(11):8792–8802
22. Kadane JB, Laza NA (2004) Me hods and c i e ia o model
selec ion. J Am S a Assoc 99(465):279–290
23. Kandasamy K, Schneide J, Po
´czos B (2015) High dimensional
Bayesian op imisa ion and bandi s ia addi i e models. In:
In e na ional con e ence on machine lea ning, PMLR,
pp 295–304
24. Leal AS, Be on L, de Cas o San os LC (2022) Lea ning me h-
ods o emaining use ul li e p edic ion in a u bo an engine. In:
Anais do XIX Encon o Nacional de In elige
ˆncia A i icial e
Compu acional, SBC, pp 556–566
25. LeCun Y, Ha ne P, Bo ou L, e al (1999) Objec ecogni ion
wi h g adien -based lea ning. In: Shape, con ou and g ouping in
compu e ision. Sp inge , pp 319–345
26. Lee JE, Jiang JR (2019) Time se ies mul i-channel con olu ional
neu al ne wo k o bea ing emaining use ul li e es ima ion. In:
2019 IEEE Eu asia Con e ence on IOT. Communica ion and
Enginee ing (ECICE), IEEE, pp 408–410
27. Lee K, Kim JK, Kim J, e al (2018) CNN and GRU combina ion
scheme o bea ing anomaly de ec ion in o a ing machine y
heal h moni o ing. In: 2018 1s IEEE in e na ional con e ence on
knowledge inno a ion and in en ion (ICKII), pp 102–105,
h ps://doi.o g/10.1109/ICKII.2018.8569155
28. Li C, Sa
´nchez RV, Zu i a G e al (2016) Faul diagnosis o
o a ing machine y using ib a ion measu emen deep s a is ical
ea u e lea ning. Senso s 16(6):895
29. Li H, Zhao W, Zhang Y e al (2020) Remaining use ul li e p e-
dic ion using mul i-scale deep con olu ional neu al ne wo k.
Appl So Compu 89:106113
30. Li X, Ding Q, Sun JQ (2018) Remaining use ul li e es ima ion in
p ognos ics using deep con olu ion neu al ne wo ks. Reliab Eng
Sys Sa 172:1–11
31. Liaw R, Liang E, Nishiha a R, e al (2018) Tune: A esea ch
pla o m o dis ibu ed model selec ion and aining. a Xi p e-
p in a Xi :1807.05118
32. Liu C, Zhang Y, Sun J e al (2022) S acked bidi ec ional LSTM
RNN o e alua e he emaining use ul li e o supe capaci o . In J
Ene gy Res 46(3):3034–3043
33. Ma L, Cui J, Yang B (2019) Deep neu al a chi ec u e sea ch wi h
deep g aph Bayesian op imiza ion. In: IEEE/WIC/ACM in e na-
ional con e ence on web in elligence, pp 500–507
34. Ma Q, Zhang M, Xu Y, e al (2021) Remaining use ul li e es i-
ma ion o u bo an engine wi h ans o me -based deep a chi-
ec u e. In: 2021 26 h in e na ional con e ence on au oma ion and
compu ing (ICAC), IEEE, pp 1–6
35. Malho a P, TV V, Ramak ishnan A, e al (2016) Mul i-senso
p ognos ics using an unsupe ised heal h index based on LSTM
encode -decode . a Xi p ep in a Xi :1608.06154
36. Ma eus BC, Mendes M, To es Fa inha J e al (2023) Imp o ed
GRU p edic ion o pape pulp p ess a iables using di e en p e-
p ocessing me hods. P od Manu Res 11(1):2155263
37. Ma hew V, Toby T, Singh V, e al (2017) P edic ion o emaining
use ul li e ime ( ul) o u bo an engine using machine lea ning.
In: 2017 IEEE In . Con . on ci cui s and sys ems (ICCS), IEEE,
pp 306–311
38. Mei Y, Sa di S, Hodassman S e al (2020) Powe -law scaling o
assis wi h key challenges in a i icial in elligence. Sci Rep
10(1):19628
39. Nec oux P, Gou i eau R, Medjahe K, e al (2012) P onos ia: an
expe imen al pla o m o bea ings accele a ed deg ada ion es s.
In: IEEE in e na ional con e ence on p ognos ics and heal h
managemen , PHM’12., IEEE Ca alog Numbe : CPF12PHM-
CDR, pp 1–8
40. Ne o D, Pe ella L, Hen iques J e al (2023) A hyb id deep
lea ning-based app oach o olling bea ing aul p ognos ics.
IFAC-Pape sOnLine 56(2):6588–6593
41. Ng SS, Xing Y, Tsui KL (2014) A nai e Bayes model o obus
emaining use ul li e p edic ion o li hium-ion ba e y. Appl
Ene gy 118:114–123
42. Nie o PG, Ga cı
´a-Gonzalo E, Lashe as FS e al (2015) Hyb id
PSO-SVM-based me hod o o ecas ing o he emaining use ul
li e o ai c a engines and e alua ion o i s eliabili y. Reliab
Eng Sys Sa 138:219–231
43. O do
´n
˜ez C, Lashe as FS, Roca-Pa dinas J e al (2019) A hyb id
ARIMA-SVM model o he s udy o he emaining use ul li e o
ai c a engines. J Compu Appl Ma h 346:184–191
44. Pelikan M, Goldbe g DE, Can u
´-Paz E, e al (1999) Boa: The
Bayesian op imiza ion algo i hm. In: P oceedings o he gene ic
and e olu iona y compu a ion con e ence GECCO-99, Ci esee ,
pp 525–532
45. Peng C, Wu J, Wang Q e al (2022) Remaining use ul li e p e-
dic ion using dual-channel LSTM wi h ime ea u e and i s di -
e ence. En opy. h ps://doi.o g/10.3390/e24121818
46. Peng Y, Wang Y, Zi Y (2018) Swi ching s a e-space deg ada ion
model wi h ecu si e il e /smoo he o p ognos ics o emaining
use ul li e. IEEE T ans Ind In 15(2):822–832
47. Psa omma is F, May G, Azam i ei V (2023) En isioning main-
enance 5.0: Insigh s om a sys ema ic li e a u e e iew o
indus y 4.0 and a p oposed amewo k. J Manu Sys 68:376–379
48. Qiao H, Wang T, Wang P e al (2018) A ime-dis ibu ed spa-
io empo al ea u e lea ning me hod o machine heal h moni-
o ing wi h mul i-senso ime se ies. Senso s 18(9):2932
49. Ren L, Cui J, Sun Y e al (2017) Mul i-bea ing emaining use ul
li e collabo a i e p edic ion: A deep lea ning app oach. J Manu
Sys 43:248–256
50. Rumelha DE, Hin on GE, Williams RJ, e al (1985) Lea ning
in e nal ep esen a ions by e o p opaga ion
7440 Neu al Compu ing and Applica ions (2025) 37:7423–7441
123
51. Saxena A, Goebel K, Simon D, e al (2008) Damage p opaga ion
modeling o ai c a engine un- o- ailu e simula ion. In: 2008
in e na ional con e ence on p ognos ics and heal h managemen ,
IEEE, pp 1–9
52. Shche bako M, Sai C (2022) A hyb id deep lea ning amewo k
o in elligen p edic i e main enance o cybe -physical sys ems.
ACM T ans Cybe -Phys Sys (TCPS) 6(2):1–22
53. Shen Y, Li Y, Zheng J, e al (2023) P oxybo: Accele a ing neu al
a chi ec u e sea ch ia bayesian op imiza ion wi h ze o-cos
p oxies. In: P oceedings o he AAAI con e ence on a i icial
in elligence, pp 9792–9801
54. Soualhi A, Lam aoui M, Elyous i B e al (2022) PHM SURVEY:
implemen a ion o p ognos ic me hods o moni o ing indus ial
sys ems. Ene gies. h ps://doi.o g/10.3390/en15196909
55. Sp ingenbe g JT, Klein A, Falkne S, e al (2016) Bayesian
op imiza ion wi h obus Bayesian neu al ne wo ks. Ad Neu al
In o m P ocess Sys 29
56. Su C, Li L, Wen Z (2020) Remaining use ul li e p edic ion ia a
a ia ional au oencode and a ime-window-based sequence
neu al ne wo k. Qual Reliab Eng In 36(5):1639–1656
57. Tian Z, Wong L, Sa aei N (2010) A neu al ne wo k app oach o
emaining use ul li e p edic ion u ilizing bo h ailu e and sus-
pension his o ies. Mech Sys Signal P ocess 24(5):1542–1555
58. U bani M, Pe i D, B unelli M e al (2020) Main enance-Man-
agemen in Ligh o Manu ac u ing 4.0. Sp inge In e na ional
Publishing, Cham, chap 4:97–11. h ps://doi.o g/10.1007/978-3-
030-46103-4_5
59. Ve ma E, Chak abo y S (2021) Unce ain y-awa e labelled
augmen a ions o high dimensional la en space bayesian op i-
miza ion. In: Neu IPS 2021 wo kshop on deep gene a i e models
and downs eam applica ions
60. Wahid A, B eslin JG, In iza MA (2022) P edic ion o machine
ailu e in indus y 4.0: a hyb id CNN-LSTM amewo k. Appl
Sci 12(9):4221
61. Wang L, Xie S, Li T, e al (2019) Sample-e icien neu al
a chi ec u e sea ch by lea ning ac ion space. a Xi p ep in
a Xi :1906.06832
62. Wang L, Zhao Y, Jinnai Y, e al (2019) Alphax: explo ing neu al
a chi ec u es wi h deep neu al ne wo ks and mon e ca lo ee
sea ch. a Xi p ep in a Xi :1903.11059
63. Wang T, Guo D, Sun XM (2022) Remaining use ul li e p edic-
ions o u bo an engine deg ada ion based on concu en semi-
supe ised model. Neu al Compu Appl 34(7):5151–5160
64. Webe H, Do
¨ß T, Kaupp L e al (2022) Faul handling in indus y
4.0: de ini ion, p ocess and applica ions. Senso s. h ps://doi.o g/
10.3390/s22062205
65. We bin
´ska-Wojciechowska S, Winia ska K (2023) Main enance
pe o mance in he age o indus y 4.0: a bibliome ic pe o -
mance analysis and a sys ema ic li e a u e e iew. Senso s.
h ps://doi.o g/10.3390/s23031409
66. Whi e C, Ai R, Neiswange W, e al (2019) Neu al a chi ec u e
sea ch ia Bayesian op imiza ion wi h a neu al ne wo k model.
Neu IPS
67. Whi e C, Neiswange W, Sa ani Y (2021) Bananas: Bayesian
op imiza ion wi h neu al a chi ec u es o neu al a chi ec u e
sea ch. In: P oceedings o he AAAI con e ence on a i icial
in elligence, pp 10293–10301
68. Xia T, Song Y, Zheng Y e al (2020) An ensemble amewo k
based on con olu ional bi-di ec ional LSTM wi h mul iple ime
windows o emaining use ul li e es ima ion. Compu Ind
115:103182
69. Yousu S, Khan SA, Khu sheed S (2022) Remaining use ul li e
(RUL) eg ession using long-sho e m memo y (LSTM) ne -
wo ks. Mic oelec on Reliab 139:11477
70. Yu W, Kim IY, Meche ske C (2020) An imp o ed simila i y-
based p ognos ic algo i hm o RUL es ima ion using an RNN
au oencode scheme. Reliab Eng Sys Sa 199:106926
71. Yu X, Zang H (2008) T ans ome aul diagnosis based on ough
se s heo y and a i icial neu al ne wo ks. In: 2008 in e na ional
con e ence on condi ion moni o ing and diagnosis, IEEE,
pp 1342–1345
72. Yuan M, Wu Y, Lin L (2016) Faul diagnosis and emaining
use ul li e es ima ion o ae o engine using ls m neu al ne wo k.
In: 2016 IEEE in e na ional con e ence on ai c a u ili y sys ems
(AUS), IEEE, pp 135–140
73. Zhao Z, Liang B, Wang X e al (2017) Remaining use ul li e
p edic ion o ai c a engine based on deg ada ion pa e n lea n-
ing. Reliab Eng Sys Sa 164:74–83
74. Zheng S, Ris o ski K, Fa aha A, e al (2017) Long sho - e m
memo y ne wo k o emaining use ul li e es ima ion. In: 2017
IEEE In . Con . p ognos ics and heal h managemen (ICPHM),
IEEE, pp 88–95
Publishe ’s No e Sp inge Na u e emains neu al wi h ega d o
ju isdic ional claims in published maps and ins i u ional a ilia ions.
Neu al Compu ing and Applica ions (2025) 37:7423–7441 7441
123
