

Ci a ion: Ribei o, J.; And ade, P.;
Ca alho, M.; Sil a, C.; Ribei o, B.;
Roque, L. Play ul P obes o Design
In e ac ion wi h Machine Lea ning:
A Tool o Ai c a Condi ion-Based
Main enance Planning and
Visualisa ion. Ma hema ics 2022,10,
1604. h ps://doi.o g/10.3390/
ma h10091604
Academic Edi o : Al edo Milani
Recei ed: 16 Ma ch 2022
Accep ed: 2 May 2022
Published: 9 May 2022
Publishe ’s No e: MDPI s ays neu al
wi h ega d o ju isdic ional claims in
published maps and ins i u ional a il-
ia ions.
Copy igh : © 2022 by he au ho s.
Licensee MDPI, Basel, Swi ze land.
This a icle is an open access a icle
dis ibu ed unde he e ms and
condi ions o he C ea i e Commons
A ibu ion (CC BY) license (h ps://
c ea i ecommons.o g/licenses/by/
4.0/).
ma hema ics
A icle
Play ul P obes o Design In e ac ion wi h Machine Lea ning:
A Tool o Ai c a Condi ion-Based Main enance Planning
and Visualisa ion
Jo ge Ribei o * , Ped o And ade , Manuel Ca alho , Ca a ina Sil a , Be na de e Ribei o
and Licínio Roque
CISUC—Cen e In o ma ics and Sys ems, In o ma ics Enginee ing Depa men , Uni e si y o Coimb a,
3004-531 Coimb a, Po ugal; [email p o ec ed] (P.A.); [email p o ec ed] (M.C.);
[email p o ec ed] (C.S.); [email p o ec ed] (B.R.); [email p o ec ed] (L.R.)
*Co espondence: [email p o ec ed]
Abs ac :
Ai c a main enance is a complex domain whe e designing new sys ems ha include
Machine Lea ning (ML) algo i hms can become a challenge. In he con ex o designing a ool o
Condi ion-Based Main enance (CBM) in ai c a main enance planning, his case s udy add esses
(1) he use o Play ul P obing app oach o ob ain insigh s ha allow unde s anding o how o design
o in e ac ion wi h ML algo i hms, (2) he in eg a ion o a Rein o cemen Lea ning (RL) agen
o Human–AI collabo a ion in main enance planning and (3) he isualisa ion o CBM indica o s.
Using a design science esea ch app oach, we designed a Play ul P obe p o ocol and ma e ials, and
e alua ed esul s by unning a pa icipa o y design wo kshop. Ou main con ibu ion is o show
how o elici ideas o in eg a ion o main enance planning p ac ices wi h ML es ima ion ools and
he RL agen . Th ough a pa icipa o y design wo kshop wi h pa icipan s’ obse a ion, in which
hey played wi h CBM a e ac s, Play ul P obes a ou he elici a ion o use in e ac ion equi emen s
wi h he RL planning agen o aid he planne o ob ain a eliable main enance plan and u n possible
o unde s and how o ep esen CBM indica o s and isualise hem h ough a ajec o y p edic ion.
Keywo ds:
design; emaining use ul li e; isualisa ion; machine lea ning; ein o cemen lea ning;
condi ion based main enance; ai c a main enance planning
MSC: 68U35; 91A12; 68T20
1. In oduc ion
The Ai c a Main enance (AM) domain poses new challenges o he design o deci-
sion suppo sys ems such as Condi ion-Based Main enance (CBM). Human and Machine
Lea ning (ML) con luence can gi e ise o new decision suppo sys ems ha allow he
inc ease in he ai c a ’s ligh ime and he cos educ ion p omised by CBM [
1
]. This
echnique exploi s ML-based componen s and sys ems ailu e o ecas s o pe o m main e-
nance only when necessa y ins ead o using a ixed in e al app oach, inc easing ai c a
a ailabili y and sa e y while educing cos s.
In oducing ML algo i hms in c i ical and highly egula ed ope a ional con ex s e-
sis s expe imen a ion and aises addi ional challenges o design app oaches o human
owne ship and con ol o e new ML algo i hms. I , on one hand, in e ac ing wi h ML
ools equi es an app oach ha ecognises and empowe s he use o design new p ac ises,
on he o he hand, i is necessa y o design he echnology o a se o p ac ises ha a e
s ill nonexis en .
The use o he main enance planning ool, he eina e e e ed o as planne , has he
g ea esponsibili y o cons an ly ha ing a eliable plan. As such, i is na u al ha he/she
may dis us he ope a ion o a new ML agen . No only is i impo an o p o ide he
planne wi h an in e ac ion wi h he ML planning algo i hm ha mee s hei expec a ions,
Ma hema ics 2022,10, 1604. h ps://doi.o g/10.3390/ma h10091604 h ps://www.mdpi.com/jou nal/ma hema ics
Ma hema ics 2022,10, 1604 2 o 20
bu also ensu e ha he agen i sel esponds e ec i ely o he planning challenge by
gene a ing a good plan. Ai c a main enance has a signi ican in luence on he o al
ope a ing cos o an ai line. The e o e, i is impe a i e o imp o e he cu en planning
p ac ices and look owa ds main enance op imisa ion. An e icien planning algo i hm
aims o minimise he lee g ound ime and, consequen ly, inc ease i s a ailabili y and
enable e enue g ow h o he ai line.
CBM is made possible by using ML componen s o p oduce Remaining Use ul Li e
(RUL) es ima es o ai c a sys em componen s. S ill, in addi ion o p oducing he es i-
ma es, i is necessa y o p esen hem o he main enance planne so ha his in o ma ion
empowe s he human ole in he decision-making p ocess.
To gain insigh and be e unde s and how CBM planning could be in oduced in
a c i ical indus y such as a ia ion, we designed a CBM Play ul P obe. As p oposed by
Bodke and Kyng [
2
], we should ocus on pa icipa ion ha ma e s. The e o e, such a
p obe aims o empowe pa icipan s o play wi h al e na i e scena ios, de elop and exp ess
hei own unde s andings on he si ua ion in ques ion, o iden i y he necessa y changes
o he isualisa ion o he RUL as he in eg a ion o a Rein o cemen Lea ning (RL) agen
o collabo a ion in main enance planning.
The main esea ch ques ion o his wo k is: can he use o Play ul P obes enable
insigh s ha allow he design esea che s o unde s and how o design o in e ac ion
wi h ML algo i hms? Th ough he use o Play ul P obes i was possible (1) o ob ain a lis
o design ou comes ha allow us o unde s and how o design o in e ac ion wi h ML
algo i hms in a CBM con ex . Th ough hese design ou comes, i was possible o design
(2) he in eg a ion o a Rein o cemen Lea ning (RL) agen o human–AI collabo a ion in
main enance planning and (3) he isualisa ion o CBM indica o s ha will be included in
a u u e unnable e sion o he CBM planning ool.
As a u u e wo k, we wan o c ea e a unnable Play ul P obe o build on he knowledge
o domain expe s and p ac i ione s, empowe ing hem o specula e on how CBM planning
can wo k o hem. Mo eo e , we wan ed o ame he ML objec [
3
] by pu ing au oma ion o
es ima es and plan gene a ion o hei se ice. Howe e , i s , we need o unde s and how
Play ul P obes can be e be designed, aking he o m o a CBM main enance simula ion
game. This pape epo s on a Design Resea ch p ocess ha uns a Pa icipa o y Design
Wo kshop (PDW) o e alua ing a p oposed Play ul P obe design in he o m o a “ i ual
pape p o o ype”.
The nex sec ion e e s o some backg ound concep s ela ed o in o ma ion isuali-
sa ion, explainabili y and RL. Nex a e p esen ed wo ks ela ed o Play ul P obes. In he
Design Case sec ion we p esen ou case o s udy. The Me hod sec ion p esen s he ma e-
ials and me hodology o he play ul used in he PDW. In he Resul s we p esen con en
coding and he PDW con e sa ion analysis. In he Design Ou comes sec ion we p esen
he main ou comes designs esul ing om he use o Pay ul P obes as well as he esul
o hese design ou comes in he In eg a ion o he RL agen o human–AI collabo a ion
in main enance planning and he isualisa ion o CBM indica o s. Finally we p esen he
Discussion and Resul s.
2. Backg ound
To enable planne s o unde s and and in e ac wi h he gene a ed main enance plans,
hey need o be able o us and unde s and he in o ma ion p esen ed, how i came o
be and how likely o accu a e he p ognos ics/es ima es a e. We aim o do his by elying
on In o ma ion Visualisa ion (In oVis). Based on he de ini ions and concep s explo ed
in Aigne e al.
[4]
and Munzne
[5]
In oVis is he a ea ha s udies he use o isual
me apho s and a e ac s o mo e e icien ly con ey in o ma ion, making i mo e accessible
and unde s andable. One o he main subjec s o esea ch in In oVis is p ecisely how o
deal wi h big quan i ies o da a, specially ime-based da a.
Keeping in mind ha he ai line indus y is no p one o adical changes, i is impo -
an o conside he classical ways o ep esen ing ime and ime-based da a, such as he
Ma hema ics 2022,10, 1604 3 o 20
imeline, he line and a ea plo s [
4
,
5
]. To connec he classical In oVis echniques wi h his
mode n p oblem we elied on he “Wha ? Why? How?” abs ac analysis, as p esen ed by
Munzne
[5]
, which aims o pick apa and ca ego ise e e y aspec o a isualisa ion p ob-
lem o acili a e he compa ison and bo owing o me hods and echniques used in di e se
ields and con ex s.
The inc easing de elopmen o AI algo i hms and he need o humans o in e ac wi h
hem inc ease he need o p o iding guidelines o human–AI in e ac ion as [
6
,
7
]. W igh
e al.
[8]
p esen a de ailed compa a i e analysis o indus y human–AI In e ac ion Guide-
lines. Guzdial e al.
[9]
no ed, howe e , ha i is no ye clea how bes o design AI
in e aces ha ocus on explainabili y o co-c ea i i y. Abdul e al.
[10]
and
Wang e al. [11]
ag ee ha explainable, accoun able and in elligible sys ems emain key challenges. In
his line o hough , much esea ch has been pe o med on explainabili y: Zhou e al.
[12]
p esen s a comp ehensi e o e iew o me hods o he e alua ion o ML explana ions
and
Lina da os e al. [13],
a e iew o ML in e p e abili y me hods. In a mo e use -cen ed
pe spec i e, Bha e al.
[14]
syn hesise he limi a ions o cu en explainabili y echniques
ha hampe hei use by end use s.
Rein o cemen Lea ning (RL) has been used o main enance planning op imisa ion in
mul iple domains. Knowles e al.
[15]
used a basic Q-Lea ning in a main enance scheduling
p oblem o decide, a each s ep, i a main enance job should be pe o med o no . An
RL solu ion o op imise main enance scheduling in a low line sys em was p oposed by
Wang e al. [16].
Ba de e al.
[17]
uses an on-policy i s isi Mon e Ca lo o ob ain he
op imal eplacemen policy ha minimises he down ime o mili a y ucks, composed
o di e en ypes o componen s wi h andom ime- o- ailu e. In he a ia ion domain,
Hu e al. [18]
p opose he Q-lea ning algo i hm o sol ing he p oblem o ai c a long- e m
main enance decision op imisa ion.
3. Rela ed Wo k
We desc ibe ways o isualising he indica o s needed o conduc CBM and we desc ibe
me hods ha can be used o ca y ou main enance planning using ML and in e ac ing
me hods wi h ML planning algo i hms in Sec ion 2. Howe e , how should he use
explo a ion o he CBM planning me hods be enabled, helping o de elop he pa icipan
pe spec i e and app op ia ion o a new ool?
Cul u al P obes we e p oposed by Ga e e al., as “An app oach o use -cen ed
design o unde s anding human phenomena and explo ing design oppo uni ies” [
19
]
and “P obes a e collec ions o e oca i e asks mean o elici inspi a ional esponses om
people—no comp ehensi e in o ma ion abou hem, bu agmen a y clues abou hei
li es and hough s” [
20
]. J. Wallace e al. a gue ha he p ocess o media ing bo h he
ela ionship be ween pa icipan and esea che and pa icipan and he own eelings abou
a ques ion can be achie ed wi h design p obes ha p o ide mo e han jus inspi a ion o
he design [
21
]. Fu he mo e, cul u al p obes can be a ool o designe s o unde s and
use s [
22
]. F. Lange-Nielsen shows some s udies in which p obes a e used as a scien i ic
me hod o a design ool and [
23
,
24
] show how echnology p obes can be a p omising new
design ool in he design o new echnologies. Using suppo a e ac s, cul u al p obes
allow pa icipan s o documen hei ac i i ies and expe iences, o be used as esea ch
ma e ial. While collec ing he pe spec i e o he pa icipan s in he p ocess, his me hod
allows hem o explo e new hings beyond he expec ed.
The ole o play ulness in cul u al de elopmen has been ecognised a leas since
Huizinga
[25]
. Since hen, he e has been ex ensi e wo k on his opic in he scien i ic
communi y. Play ul p obing app oach uses games designed speci ically o he s udy, and
hese games a e ailo ed o he esea ch a ea and pu pose [
26
]. Sjo oll and Gulden
[27]
sugges s ha a game designed o play ul P obing “opens up o a play ul and au onomous
en i onmen o da a-ga he ing which in ol es lea ning abou indi idual and sha ed social
p ac ices”. The Play ul P obes echnique uses simila p inciples o hose o Cul u al P obes
while exploi ing games as a esea ch ool o enable lea ning and da a collec ion. Play ul
Ma hema ics 2022,10, 1604 4 o 20
P obes could also po en ially enable he explo a ion o he CBM planning me hods, helping
o de elop he pa icipan pe spec i e and app op ia ion o new ools [
28
]. A p elimina y
s udy [
29
] showed ha Play ul P obing a e ac s can be used o design new ML algo i hms
in a c i ical and highly egula ed ope a ional con ex .
4. Design Case
In he con ex o he de elopmen o a p o o ype planning ool in a new ai c a main-
enance CBM pa adigm, we s udied how he decision suppo ool should be designed o
(1) gi e he use a eading and unde s anding o he indica o s o p ognos ic ai c a com-
ponen s/sys ems ob ained by ex e nal sys ems and s o ed in Da abase and (2) o in eg a e
he ein o cemen lea ning (RL) agen o human collabo a ion in planning main enance as
can be seen in Figu e 1.
Figu e 1.
Condi ion-Based Main enance (CBM) planning ai c a ool a chi ec u e. The use in e -
ac s wi h a G aphical Use in e ace (GUI) ha eques s plans o he Rein o cemen Lea ning (RL)
au oma ic planning agen and eques s Remaining Use ul Li e (RUL) isualisa ions o a ce ain com-
ponen /sys em. Bo h he Au oPlan agen and he RUL isualisa ion ob ain he his o ical in o ma ion
om he Da abase.
Figu e 1 ep esen s he h ee componen s we p opose o a CBM Main enance Planning
Ai c a Tool. We can see ha i is composed by he au oma ic planning agen , by he
g aphical use in e ace ha allows he in e ac ion and e inemen o he plan by he
planne , and by isualisa ion o he RUL indica o s ha gi es he planne he con idence o
a pa icula main enance plan.
We did no ha e any in o ma ion abou wha indica o s we could use o RUL and
how hey could be isualised. We s a ed o use he RUL o he componen /sys em in
a simple way, jus as a alue in Fligh Hou s (FH). Rega ding he au oma ic planning
algo i hm, we ha e al eady de eloped a i s e sion o a planning algo i hm o long- e m
ou ine main enance when we s a ed his s udy.
5. Me hod
Play ul P obing app oach p oposed by Be nhaup e al.
[26]
uses games designed
speci ically o he s udy ailo ed o he esea ch a ea and pu pose o he s udy. I is he
app oach we used o s udy how o design he planning ool o a new CBM pa adigm
since i was in ended o explo e he design o he new ool by p o oking inspi a ional
esponses om pa icipan s. F om a p ac ical poin o iew, planne s play wi h he a e ac s
in pape p o o ype o m o sol e a main enance challenge p e iously p epa ed o he
wo kshop (see Figu e 2). In he game scena io, he planne s a e con on ed wi h a new
RUL in a main enance in he main enance plan. To sol e his game planne s ha e o mo e
he main enance blocks in ime and mo e ligh plans o o he ai planes o espec he new
RUL. Including planne s in his p ocess, we b ing pa icipan s o he de elopmen o he
ool om an ea ly s age, le ing hem app op ia e he new ool. To esea ch he design o he
Play ul P obes we adop ed he Design Science Resea ch (DSR) app oach [
30
]. This wo k
epo s lessons om a DSR i e a ion, wi h use ul esul s o e ine u u e DSR cycles, bu
also o in o m simila e o s by esea che s and p ac i ione s on simila esea ch p ocesses.
Ma hema ics 2022,10, 1604 5 o 20
In he con ex o he ReMap p ojec , we ec ui ed wo domain expe s in ai c a main-
enance managemen o conduc he design expe imen . The pa icipan s, bo h male and
be ween 20 and 40 yea s old ha e a good backg ound in a ia ion and p ac ical knowledge
o planning ools, al hough hey we e no daily p ac i ione s. The wo kshop was acili a ed
by he esea che , who ensu ed he applica ion o he p o ocol and cla i ica ion o doub s
on play scena ios and ma e ials, e.g., ole playing he gamemas e ole, and he designe
esea che who assis ed in he discussion. Wi h his expe imen , las ing 74 min, we expec ed
o open ele an ques ions abou how ML-based Remaining Use ul Li e (RUL) es ima es
could be ins umen ed as pa o he Play ul P obe simula ion.
A ec ed by COVID-19 measu es, hese expe imen s we e conduc ed online wi h he
igma web-based collabo a i e design pla o m [
31
], simula ing he pape p o o ype [
32
]
exe cise. Fo his expe imen , we played wi h simpli ied main enance scena ios ea u ing
wo k packages and only one RUL indica o o each ai c a main enance.
Nex , we will desc ibe he s eps o his p obing me hodology, om wo kshop p epa a-
ion o email in e iews.
1. Ma e ials. The basic elemen s o a main enance schedule (Figu e 2):
Fligh : blue ibbons ep esen ligh s ha some ai c a ( ow) can do a some ime.
Block: A p ede ined ou ine main enance (A-checks).
Clus e : A non- ou ine main enance composed by o he asks no included in Blocks.
This could ha e Due Da e, RUL, bo h, o none.
Can as: The “backg ound” o all a e ac s, he scena io wi h in o ma ion abou he
lee , ime ep esen a ions and es ic ions.
2.
Sol ing he P oblem Pa h. The beginning o he esolu ion was linea , only possible
in one di ec ion. Pa icipan s would be aced wi h he simples concep s o he ligh
plan and main enance. Subsequen ly, he esolu ion would lead o a pa h whe e use s
would necessa ily be aced wi h mo e complex issues such as con lic ing condi ions
and 90% con idence RUL.
This p obe was ins umen ed by placing isual a e ac s RUL es ima es wi h 90%
con idence o con on pa icipan s wi h si ua ions ha could lead o deba e and he
gene a ion o insigh s. The ques ions ha we wan o be aised by he pa icipan s
a e: Does i make sense o ha e a la ge deg ee o unce ain y? How do we ep esen i
o enable decisions?
3.
Tes speci ica ion. To p epa e he wo kshop, all a e ac s we e designed digi ally bu
p in ed and p e- es ed manually as in a common pape p o o ype exe cise. A e es ing
mul iple app oaches o ins umen he p obes wi h isual a e ac s, and adjus men s in
size and complexi y, he exe cise was mig a ed o a digi al collabo a ion ool (Figu e 2).
4.
B ie ing. In an ini ial pa o he expe imen /wo kshop, an in oduc ion was made
explaining wha he basic main enance elemen s o he game we e and demons a ing
how o sol e a simple p oblem.
5.
Play ul p ocedu e. In his pa o he expe imen al session, a i ac s we e p esen ed o
pa icipan s wi h a non- i ial main enance scheduling p oblem o be sol ed, i.e., a
p oblem ha needs se e al plays bo h in he a i ac s o he main enance and a i ac s
o ligh s o espec he new RUL (Figu e 2). The pa icipan ’s oice and he collabo a-
i e can as we e eco ded p esen ing hei ideas and playing wi h he ep esen a ions
o sol e he main enance p oblem. The acili a o answe ed pa icipan s’ ques ions
abou whe he hey could ake a ce ain ac ion o no . Fu he mo e, he ale ed when
hey we e igno ing some impo an condi ions while ying o explo e he p oblem.
6.
Deb ie ing. A e pa icipan s sol ed he scheduling p oblem, a wide discussion
space opened, namely on he ole o RUL isualisa ion and he use o an ML planning
agen in he planning p ocess.
7.
Email In e iew. A e iewing he eco ding, some ques ions we e sen o he pa -
icipan s. The in en ion was o cla i y o deepen he e lec ions ha hey exp essed
du ing phases 5 and 6.

Ma hema ics 2022,10, 1604 6 o 20
Figu e 2.
Main enance scheduling p oblem p esen ed o he pa icipan s in he expe imen al session. Fligh s : blue ec angles; Blocks: ed ec angles; Clus e s:
o ange ec angles; S a : he yellow s a inside Clus e ep esen a new main enance p oblem p esen ing a es ic i e RUL.
Ma hema ics 2022,10, 1604 7 o 20
This PDW gene a ed audio and ideo eco dings. The con e sa ion be ween pa ic-
ipan s and he ideo wi h he manipula ion o game a e ac s ook place in phase 5 and
discussion in phase 6. Da a we e analysed by spli ing in o small ime segmen s, coded
in o g oups acco ding o he i s analysis ca ego ies in he con e sa ion.
6. Wo kshop Resul s
In his sec ion he esul s ob ained in he Pa icipa o y Design Wo kshop (PDW) will
be p esen ed, i s illus a ing he empo al coding o he opics co e ed, and in a second
pa an analysis o he con e sa ion ob ained.
6.1. Con en Coding
The wo kshop was s a ed by explaining he basic main enance elemen s in he game
and illus a ing how o sol e a linea p oblem. In his pa , las ing 10 min, he pa icipan s
clea ed some doub s abou he game mechanics bu did no in e ac wi h a e ac s. The
expe imen ollowed wi h a non- i ial main enance scheduling p oblem o be sol ed.
Pa icipan s’ dialogue and collabo a i e can as we e eco ded, while discussing ideas and
manipula ing plan a e ac s o sol e he main enance p oblem.
The acili a o in e ened o: (a) answe ques ions abou whe he o no some ac ions
we e possible; (b) ale pa icipan s when hey we e missing ele an in o ma ion; (c) y
o encou age hem o u he explo e aspec s o he p oblem, o assess in o ma ional o
ac ion needs. The session de eloped eely ollowing he p oblem o be sol ed, wi h no
cons ain s ega ding o de ing o pa icipan s’ ac ions o managing concu ency among
open explo a ions, a ou ing dialogue while suppo ing each o he ’s explo a ion. A e
pa icipan s sol ed he scheduling p oblem, a wide discussion ocused on he ole o ML
in he planning p ocess. This pa las ed 74 min.
When we look a he ocus o he con e sa ion du ing s age 5 in Figu e 3we can see
ha a he beginning he pa icipan s alk abou he ep esen a ion o planning a e ac s
and deal wi h echnical issues ela ed wi h echnology un amilia o pa icipan s be o e he
wo kshop. Immedia ely a e he p oblem has been placed, pa icipan s s a alking abou
main enance- ela ed me a-speech (p ac ices, p ocedu es o egula ions bu no di ec ly
ela ed o he esolu ion o he main enance challenge). Only a 5:30, hey change he ocus
o sol e he p oblem, and only a 8 min, hey s a o manipula e he a e ac s. F om his
momen onwa ds, he pa icipan s do no lose ocus on sol ing he p oblem un il he end
o he exe cise. This p oblem esolu ion is accompanied al e na ely by momen s o a e ac
manipula ion o main enance- ela ed me a-speech.
Figu e 3. Focus o con e sa ion du ing s age 5.
Ma hema ics 2022,10, 1604 8 o 20
Re lec ions (on ML- ools and p ac ises) mainly ake place du ing s age 6, a e minu e
23, immedia ely a e he p oblem has been sol ed, as can be seen in Figu e 4. Du ing his
phase, i is no ewo hy a qui e in ense discussion abou main enance planning p ac ises.
The discou se al e na es be ween cu en p ac ises and specula ion on wha u u e p ac ises
will look like. The e lec i e discou se in his phase is di ided in o h ee majo blocks.
Be ween 23 and 40 min, we ind a speech oscilla ing be ween cu en and u u e p ac ises;
hen, be ween 42 and 58 min, he con e sa ion is ocused on u u e p ac ises and be ween
58 and 70 min, only cu en p ac ises a e discussed.
Figu e 4. Re lec ion u e ances in pa icipan s’ con e sa ion along he expe imen .
Conce ning he in oduc ion o he Remaining Use ul Li e concep , we could e i y
ha whene e he e is a dialogue abou ime o he con idence in e al, i comes wi h
a discussion on RUL meaning and implica ions i may ha e. This ook place mainly in
he i s block o he mixed discou se be ween cu en and u u e p ac ices (23–44). The
ML deba es ook place du ing his second block. I is in e spe sed be ween he o m o
in e ac ion and e lec ion on he unc ioning o he algo i hms and appea s a se e al poin s
in ime simul aneously. The hi d block was exclusi ely a e lec ion on cu en p ac ices.
Be ween he i s and he second block, a momen o e lec ion on he game (Play ul P obe)
i sel akes place, bu only o 2 min.
6.2. Con e sa ion Analysis
The discussion is based on he analysis o he meanings exp essed in con e sa ion du ing
he expe imen and on he compa ison wi h he eedback om he pa icipan s’ in e iew
answe s. The con e sa ion was e y ex ensi e; we will only ocus on he discussion ela ed o
he isualisa ion o he RUL and he in e ac ion wi h a machine lea ning algo i hm.
These pa icipan s did no s a immedia ely sol ing he p oblem. They begin by ad-
d essing he p oblem using me a-speech, sugges ing ha hey we e “ eading” he p oblem
i s and ob ain he igh connec ion be ween a e ac s and he main enance language ha
hey a e amilia wi h. They ook abou 5 min be ween he momen ha he p oblem was
placed, un il hey s a ed mo ing he elemen s in a e y in ica e collabo a ion p ocess,
such as analysing and nego ia ing he mo emen s as i hey we e lea ning o play a game
o chess. A minu e 13:30, hey decided o each ake a di e en ole “you do he ligh and
I do he main enance”(P1) pe haps as a o m o collabo a ion, bu some hing ha made
sense la e in e ms o eassessing p ac ices.
The pa icipan s ound i easy and clea o unde s and wha needed o be done.
Howe e , hey ound he RUL was no easy o in e p e and conside ed i as a ixed due
da e. P2 said “was qui e icky es ima e wha isk you ook when you in e p e ed he
RUL”, while P1 said he ep esen a ion o RUL equi es some men al e o o isualise,
Ma hema ics 2022,10, 1604 9 o 20
“was a bi challenging o de e mine he due da es o he asks, i equi ed some men al
e o s” P1 adding du ing he exe cise ha “ he di e ence be ween 95 and 99 in my head
is no playing a ole”. Despi e he di icul y o seeing he impac o he con idence le el
du ing he exe cise, hey ha e made an e o o unde s and i , e.g., P1 said “I won’ o
isk, because 90% is qui e high”. Du ing he exe cise, P2 sugges ed a RUL o 60H wi h a
con idence le el o 90%, “i would be nice i we could see (...) 65
±
6 h , han you kind
ha e an idea o how close he edge you a e”, and when asked i a boxplo could i , P1
answe ed “Yeh, I’m hinking ou aloud now, bu pe haps ins ead a squa e box, i could be a
kind o dis ibu ion”.
P1 said ha he planne wouldn’ ma e oo much in a p e en i e ask o a componen
ha usually ails wi h no impac (main enance consequences), bu i i is a componen ha
mus no ail because o he isk o an ai c a on he g ound si ua ion o a ligh being
cancelled; hen i would make a big di e ence. They sugges ed au oma ically isualising
he RUL on he imeline, and P1 also sugges ed i would be good o “ isualise ope a ion
impac ” as cos s, a ailabili y and he main enance componen s, asking P2 “Bu i could
ac ually depend on wha ’s hese 65 h based on, igh ? Wha kind o componen s we a e
alking abou !?”.
A some poin , P1 conside ed scheduling wo hou s main enance o e he limi , and
wonde ed “wha is he consequences o no making he exac Due Da e? wha ’s he
consequences o ha ing he componen illed be o e he p e en i e emo al?” and “How
c i ical is i i we don’ espec a RUL?”, sugges ing ha due da es maybe could be mo e
lexible, i he e u n is la ge enough. A he end o he exe cise, P1 ook a co-cons uc i e
mo e and s a ed using he collabo a ion ool o make some design p oposals. They s a ed
o d aw how his kind o dis ibu ion can occu , as shown in Figu e 5, using as a e e ence
he ep esen a ion o “T ends, Rul, & Unce ain y” by [
33
]. This ep esen a ion can also
be seen as a isual analogy based on how he a i al ime is modeled, bu in his case as a
iew o he isk.
Figu e 5.
The pa icipan p oposed isualizing he RUL wi h a “p obabili y dis ibu ion cu e”,
showing how i could be ep esen ed by oughly d awing he hin black cu e.
P1 complemen ed how his model can wo k when he e is a si ua ion wi h wo
main enance needs pa ially o e lapping, “ he planne can o example choose e e when
he a i al ime is ha close o each o he espec ing his po en ial o e lap (.. . ) we can jus
wide o he le and he igh as much as we can” as shown in Figu e 6.
Figu e 6.
The pa icipan p oposed isualizing wo RUL, showing how i could be ep esen ed by
oughly d awing he wo o e lap hin black cu es.
P2 was cau ious, saying ha wi h cu en p ac ices “we don’ wan ou planne o
access he echnical s a e o he plane (...) I wouldn’ be e y com o able wi h le ing
him decide whe he i ’s an accep able isk o ake”. P1 complemen ed “ ha ’s how we
wo k now, so i he e is a p ognos ic ale , hen somebody makes he due da e, and he
schedule espec s ha due da e, and he guy ha makes he due da e, doesn’ know abou
he schedule, he jus looks a he ask, looks a he c i icali y and hen he says, ok, his needs
Ma hema ics 2022,10, 1604 16 o 20
ep esen he Expec ed End o Li e (EEoL) o he componen mapped in ela ion o he RUL
p ognos ic. Tha is o any poin in he line plo o he ajec o y on he pas side, we ma k
he place in ime ha we expec he ajec o y o end. I he RUL is a PDF we ma k he
minimum, mean and maximum, o he wise jus one numbe . This esul s in an a ea o a
line plo , espec i ely. The a ea esembles a iangle o an in e ed o nado, poin ing o he
end o he ajec o y.
Figu e 11 p esen s a ough 3D ske ch o his ela ion, whe e we can see how he PDF
cu es o m he ed a ea.
We can see an example o a comple e ajec o y in Figu e 12, showing he comple e
ed shape.
By analysing he e ical de ia ion o he slan o he esul ing a ea o line i is much
easie o see i he p ognos ic model p edic ions we e consis en o i he e we e lo s o
co ec ions o e ime. This is because he g ound u h o a ajec o y, i ep esen ed in his
mapping would be a e ical line, cen ed on he end o li e.
I he e a e pa s o he ed a ea ha o e lap wi h he g ey backg ound i means ha
he co esponding RUL p ognos ics expec ed he componen o ail be o e ou cu en
ime. I he o e all shape is leaning o he igh , i means ha he componen is las ing
longe han expec ed. I he o e all shape is leaning o he le i means ha he componen
is de e io a ing as e han expec ed. This a e ac may help o in e how planning can
ela e o u u e beha iou . Fo ins ance, a low RUL can lead o a ligh plan ha makes he
componen las longe han expec ed as i was used on less damaging ou es— his would
be ep esen ed by he ed line poin ing o he igh side.
Figu e 11.
Ske ch ep esen a ion o he RUL ajec o ies and he PDF cu es in 3D. The black line
shows he cu en ajec o y and he ed lines a e o he pas ajec o ies o he same componen , all
in Fligh Hou s. He e we can see ha he ed a ea, men ioned p e iously, is o med by successi e
PDF cu es. In his igu e, he PDF cu es ep esen ed esul om he ajec o y compa ison, and no
om he model, as hese we e single alue RUL p ognos ics. As such he e a e wo ed a eas, he
da ke a ea shows he obse ed in e al and he ligh e one ma ks he 95% con idence in e al, ha
is 2 s anda d de ia ions away om he mean.
We also implemen ed a simple echnique o compa e a cu en RUL ajec o y wi h
pas ajec o ies o componen s o he same ype, o help con ex ualise i s beha iou and
make de ia ions om a “no mal” beha iou much mo e explici . I he e is a ailable da a
wi h pas RUL ajec o ies o o he componen s o he same ype, we can also ep esen he
pas ajec o ies as line plo s. Each pas ajec o y is mapped ela i e o he ime when hei
RUL p ognos ic was closes o he e e ence RUL, as shown in Figu e 13. This esul s in a
dis ibu ion, showing how long o he componen s o he same ype las ed a e ha ing a
simila p ognos ic. The compa ison me hod can be applied o single alue RUL ajec o ies,
o by using he mean alue o RUL PDF ajec o ies.

Ma hema ics 2022,10, 1604 17 o 20
Figu e 12.
RUL isualisa ion a he end o li e, wi h he esul ing iangula shape showing ha all
p edic ions we e e y close o he eal end o li e, specially a he s a and end o he ajec o y. The
black line ep esen s he cu en ajec o y. The ed line ma ks all he Expec ed Ends o Li e acco ding
o he RUL PDF mean, and he wid h o he a ea ma ks one s anda d de ia ion away om he mean.
Wi h his ep esen a ion we y o con ey mo e in o ma ion abou he beha iou o
each componen and also abou he p ognos ics accu acy, so ha he use can ha e a mo e
objec i e no ion abou he in insic unce ain y o any p edic ion.
Figu e 13.
RUL ajec o y compa ison showing some pas ajec o ies aligned wi h a cu en ajec o y.
The cu en ajec o y is ep esen ed wi h he black line and he pas ajec o ies a e ep esen ed wi h
he hin ed lines.The pas ajec o ies a e aligned by he ime hey had a p ognos ic simila o he las
p ognos ic o he cu en ajec o y. Wi h his ep esen a ion we can see how he o he ajec o ies
beha ed a e his poin . When he e is a new p ognos ic, he ajec o ies a e e-aligned using he
new p ognos ic alue.
8. Discussion and Re lec ion
The Play ul P obe app oach, ma e ialised in a sha ed digi al pape p o o ype, enabled
an explo a o y en i onmen in which esea che s and domain expe s we e able o explo e
di e se aspec s o ML adop ion in ai c a main enance. By playing wi h p obe a e ac s
o sol e a conc e e scena io, pa icipan s can ocus and e lec on changes o hei domain
p ac ices and open p oduc i e dialogue on how CBM main enance could be designed, as
e idenced by ou con en and speech analysis ela i e o he exe cise and ma e ialised in he
main design ou comes. By using he design ou comes as a lis o design equi emen s o he
ML componen s o ou design case, i was possible o e ol e he RL agen o Human–AI
collabo a ion in main enance planning and p opose he isualisa ion o CBM indica o s.
Rega ding he RL componen , a planning solu ion capable o op imising main enance
decision-making when acing new unexpec ed e en s was de eloped. This solu ion e ol ed
o e ime o a mo e ealis ic ep esen a ion o a main enance planning scena io by educing
he ime ho izon and including ai c a ligh s in he p oblem. The in oduc ion o he slo
Ma hema ics 2022,10, 1604 18 o 20
hold ea u e also allows he use o ha e mo e con ol o e he esul ing main enance plan.
The au oma ic planning algo i hm can deal wi h aul s o di e en u gency classes and
p ognos ics in o ma ion by op imising he scheduling o he espec i e main enance asks.
The goal is o achie e a balance be ween lee g ound ime and ime slack. Resul s indica e
ha he RL agen is able o p oduce be e main enance plans du ing aining by imp o ing
bo h KPIs. Fu he mo e, by aking in o conside a ion he cu en plan as a baseline, we
a oid c ea ing a comple ely new plan whene e new in o ma ion becomes a ailable, which
would be un easible in eal scena ios.
The e lec ion gene a ed in his exe cise allowed he pa icipan s o specula i ely
imagine how he planne could use his ool in he u u e by inco po a ing he p edic i e
RUL indica o s o ai c a componen s. The RUL indica o igge ed an ex ensi e speech
ha equi ed a cons an e lec i e discou se on cu en and u u e p ac ices. The wo kshop
allowed he in e p e a ion o he RUL indica o and poin ed o a possible o m o ep e-
sen a ion h ough PDF cu es. Al hough box plo s and PDF cu es a e eliable ways o
ep esen ing p obabili y dis ibu ions, in o de o be e isualise he deg ada ion ends i
was necessa y o con ex ualise he ins ance p ognos ics in hei ime ame, i.e., h ough
hei ajec o y. The e o e, he p oposed RUL de ailed isualisa ion was p ima ily ocused
on con ex ualising he p ognos ic ins ances wi hin hei cu en and pas beha iou s. I
can be di ided in o h ee pa s: he RUL ajec o y (Pas ); he cu en RUL p ognos ic
(P esen ); and he Expec ed End o Li e (Fu u e). In o de o p o ide in-dep h knowledge o
he p ognos ics, we p esen he componen ’s cu en condi ion, con ex ualise i in i s pas
beha iou and show how i can ansla e in o he u u e. By ep esen ing his in o ma ion
isually, he use can quickly g asp i he p oposed plan is i o he componen condi ion
o ake ac ion i some modi ica ions a e equi ed. Addi ionally, he isualisa ion idiom
migh help o in e how planning can ela e o u u e beha iou .
All hings conside ed, i was possible o specula i ely bu explici ly gene a e equi e-
men s o a unnable e sion o he planning ool ha includes he au oma ic ML planning
agen and gene a es he de ailed RUL isualisa ion inc easing he planne ’s con idence
in he main enance plan. Howe e , o he o ms o isualisa ion and in e ac ion wi h he
ML agen should be explo ed in u u e uns. The cu en s udy does no ye in o m he
accep ance by planne s o a unnable Play ul P obe as ei he a aining o inal use de ice,
which emains on ou p ojec agenda.
Au ho Con ibu ions:
Concep ualisa ion, J.R. and L.R.; in es iga ion, J.R.; me hodology, J.R.; su-
pe ision, C.S., B.R. and L.R.; alida ion, L.R.; w i ing—o iginal d a , J.R., P.A. and M.C.; w i ing—
e iew and edi ing, C.S., B.R. and L.R. All au ho s ha e ead and ag eed o he published e sion o
he manusc ip .
Funding:
This esea ch was unded by he Eu opean Union’s Ho izon 2020 esea ch and inno a ion
p og am unde he REMAP p ojec , g an numbe 769288.
Ins i u ional Re iew Boa d S a emen : No applicable.
In o med Consen S a emen :
In o med consen was ob ained om all subjec s in ol ed in he s udy.
Da a A ailabili y S a emen : No applicable.
Con lic s o In e es : The au ho s decla e no con lic o in e es .
Re e ences
1.
And ade, P.; Sil a, C.; Ribei o, B.; San os, B.F. Ai c a main enance check scheduling using ein o cemen lea ning. Ae ospace
2021,8, 113. [C ossRe ]
2.
Bødke , S.; Kyng, M. Pa icipa o y design ha ma e s—Facing he big issues. ACM T ans. Compu . Hum. In e ac .
2018
,25, 1–31.
[C ossRe ]
3.
Bødke , S.; Roque, L.; La sen-Lede , I.; Thomas, V. Taming a Run-Away Objec : How o Main ain and Ex end Human Con ol in
Human-Compu e In e ac ion? 2018. A ailable online: h ps://pu e.au.dk/ws/ iles/135965176/bodke _ aming_a_ unaway_
objec .pd (accessed on 15 Ma ch 2022).
4. Aigne , W.; Miksch, S.; Schumann, H.; Al, E. Visualiza ion o Time-O ien ed Da a; Sp inge : London, UK, 2011.
Ma hema ics 2022,10, 1604 19 o 20
5. Munzne , T. Visualiza ion Analysis & Design; CRC P ess, Taylo & F ancis G oup: Boca Ra on, FL, USA, 2015.
6.
Ame shi, S.; Inkpen, K.; Tee an, J.; Kikin-Gil, R.; Ho i z, E.; Weld, D.; Vo o eanu, M.; Fou ney, A.; Nushi, B.;
Collisson, P.; e al.
Guidelines o Human-AI In e ac ion. In P oceedings o he 2019 CHI Con e ence on Human Fac o s in Compu ing Sys ems—
CHI ’19, Glasgow, UK, 4–9 May 2019; ACM P ess: New Yo k, NY, USA, 2019; pp. 1–13. [C ossRe ]
7.
Holb ook, J. Human-Cen e ed Machine Lea ning . 2020. A ailable online: h ps://medium.com/google-design/human-
cen e ed-machine-lea ning-a770d10562cd (accessed on 16 Ap il 2020).
8.
W igh , A.P.; Wang, Z.J.; Pa k, H.; Guo, G.; Spe le, F.; El-Assady, M.; Ende , A.; Keim, D.; Chau, D.H. A Compa a i e
Analysis o Indus y Human-AI In e ac ion Guidelines. 2020. A ailable online: h p://xxx.lanl.go /abs/2010.11761 (accessed on
22 Oc obe 2020).
9.
Guzdial, M.; Liao, N.; Chen, J.; Chen, S.Y.; Shah, S.; Shah, V.; Reno, J.; Smi h, G.; Riedl, M.O. F iend, collabo a o , s uden ,
manage : How design o an AI-d i en game le el edi o a ec s c ea o s. In P oceedings o he Con e ence on Human Fac o s in
Compu ing Sys ems—P oceedings, Glasgow, UK, 4–9 May 2019; pp. 1–13. A ailable online: h p://xxx.lanl.go /abs/1901.06417
(accessed on 24 Ma ch 2020). [C ossRe ]
10.
Abdul, A.; Ve meulen, J.; Wang, D.; Lim, B.Y.; Kankanhalli, M. T ends and ajec o ies o explainable, accoun able and in elligible
sys ems: An HCI esea ch agenda. In P oceedings o he Con e ence on Human Fac o s in Compu ing Sys ems, Mon eal, QC,
Canada, 21–26 Ap il 2018. [C ossRe ]
11.
Wang, D.; Yang, Q.; Abdul, A.; Lim, B.Y.; S a es, U. Designing Theo y-D i en Use -Cen ic Explainable AI. In P oceedings o he
2019 CHI Con e ence on Human Fac o s in Compu ing Sys ems, Glasgow, UK, 4–9 May 2019; pp. 1–15.
12.
Zhou, J.; Gandomi, A.H.; Chen, F.; Holzinge , A. E alua ing he quali y o machine lea ning explana ions: A su ey on me hods
and me ics. Elec onics 2021,10, 593. [C ossRe ]
13.
Lina da os, P.; Papas e anopoulos, V.; Ko sian is, S. Explainable ai: A e iew o machine lea ning in e p e abili y me hods.
En opy 2021,23, 18. [C ossRe ]
14.
Bha , U.; Xiang, A.; Sha ma, S.; Welle , A.; Taly, A.; Jia, Y.; Ghosh, J.; Pu i, R.; Mou a, J.M.; Ecke sley, P. Explainable machine
lea ning in deploymen . In P oceedings o he 2020 Con e ence on Fai ness, Accoun abili y, and T anspa ency, Ba celona, Spain,
27–30 Janua y 2020; pp. 648–657. A ailable online: h p://xxx.lanl.go /abs/1909.06342 (accessed on 1 Feb ua y 2022). [C ossRe ]
15.
Knowles, M.; Baglee, D.; We m e , S. Rein o cemen Lea ning o Scheduling o Main enance. In Resea ch and De elopmen in
In elligen Sys ems XXVII; B ame , M., Pe idis, M., Hopgood, A., Eds.; Sp inge : London, UK, 2011; pp. 409–422. [C ossRe ]
16.
Wang, X.; Wang, H.; Qi, C. Mul i-agen ein o cemen lea ning based main enance policy o a esou ce cons ained low line
sys em. J. In ell. Manu . 2016,27, 325–333. [C ossRe ]
17. Ba de, S.R.A.; Yacou , S.; Shin, H. Op imal p e en i e main enance policy based on ein o cemen lea ning o a lee o mili a y
ucks. J. In ell. Manu . 2019,30, 147–161. [C ossRe ]
18.
Hu, Y.; Miao, X.; Zhang, J.; Liu, J.; Pan, E. Rein o cemen lea ning-d i en main enance s a egy: A no el solu ion o long- e m
ai c a main enance decision op imiza ion. Compu . Ind. Eng. 2021,153, 107056. [C ossRe ]
19. Ma elmaki, T.; Ko keakoulu, T. Design P obes; Uni e si y o A and Design: Helsinki, Finland, 2008.
20.
Ga e , W.W.; Bouche , A.; Penning on, S.; Walke , B. Cul u al p obes and he alue o unce ain y. In e ac ions
2004
,11, 53.
[C ossRe ]
21.
Wallace, J.; McCa hy, J.; W igh , P.C.; Oli ie , P. Making design p obes wo k. In P oceedings o he Con e ence on Human
Fac o s in Compu ing Sys ems, Pa is, F ance, 27 Ap il–2 May 2013; pp. 3441–3450. [C ossRe ]
22.
Celikoglu, O.M.; Ogu , S.T.; K ippendo , K. How Do Use S o ies Inspi e Design? A S udy o Cul u al P obes. Des. Issues
2017
,
33, 84–98. [C ossRe ]
23.
Lange-Nielsen, F.; La on , X.V.; Cassa , B.; Khaled, R. In ol ing playe s ea lie in he game design p ocess using cul u al p obes.
In P oceedings o he 4 h In e na ional Con e ence on Fun and Games, Toulouse, F ance, 4–6 Sep embe 2012; ACM P ess:
New Yo k, NY, USA, 2012; pp. 45–54. [C ossRe ]
24.
Hu chinson, H.; Mackay, W.; Wes e lund, B.; Bede son, B.B.; D uin, A.; Plaisan , C.; Beaudouin-La on, M.; Con e sy, S.; E ans,
H.; Hansen, H.; e al. Technology p obes: Inspi ing design o and wi h amilies. In P oceedings o he Con e ence on Human
Fac o s in Compu ing Sys ems, F . Laude dale, FL, USA, 5–10 Ap il 2003; pp. 17–24.
25. Huizinga, J. Homo Ludens: A S udy o he Play-Elemen in Cul u e; Angelico P ess: B ooklyn, NY, USA 2016.
26.
Be nhaup , R.; Weiss, A.; Ob is , M.; Tscheligi, M. Play ul p obing: Making p obing mo e un. In Lec u e No es in Compu e Science
(Including Subse ies Lec u e No es in A i icial In elligence and Lec u e No es in Bioin o ma ics); 4662 LNCS; Sp inge : Be lin/Heidelbe g,
Ge many, 2007; pp. 606–619._60. [C ossRe ]
27.
Sjo oll, V.; Gulden, T. Play p obes—As a p oduc i e space and sou ce o in o ma ion. In P oceedings o he 18 h In e na ional
Con e ence on Enginee ing and P oduc Design Educa ion, Design Educa ion: Collabo a ion and C oss-Disciplina i y, Aalbo g,
Denma k, 8–9 Sep embe 2016; The Design Socie y: Copenhagen, Denma k; 2016; pp. 342–347.
28. Ga e , B.; Dunne, T.; Pacen i, E. Design: Cul u al p obes. In e ac ions 1999,6, 21–29. [C ossRe ]
29.
Ribei o, J.; Roque, L. Play ully p obing p ac ice-au oma ion dialec ics in designing new ML- ools. In P oceedings o he
VideoJogos 2020: 12 h In e na ional Con e ence on Videogame Sciences and A s, Mi andela, Po ugal, 26–28 No embe 2020;
pp. 1–9.
30.
Vaishna i, V.K.; Pu ao, S., Eds. In P oceedings o he 4 h In e na ional Con e ence on Design Science Resea ch in In o ma ion
Sys ems and Technology, Philadelphia, PA, USA, 7–8 May 2009.
Ma hema ics 2022,10, 1604 20 o 20
31. Figma. Whe e Teams Design Toge he . 2020. A ailable online: h ps://www. igma.com (accessed on 13 July 2020).
32. Nielsen, J. The Usabili y Enginee ing Li e Cycle. Compu e 1992,25, 12–22. [C ossRe ]
33.
Goebel, K.; Saxena, A.; Daigle, M.; Celaya, J.; Roychoudhu y, I.; Clemen s, S. In oduc ion o p ognos ics. In P oceedings o he
Eu opean PHM Con e ence, D esden, Ge many, 3–5 July 2012.
34.
EASA. Ce i ica ion Speci ica ions and Guidance Ma e ial o Mas e Minimum Equipmen Lis (CS-MMEL). 2021. A ailable
online: h ps://www.easa.eu opa.eu/documen -lib a y/ce i ica ion-speci ica ions/cs-mmel-issue-3 (accessed on 1 May 2022).