Ci a ion: Lopez-de-Ipina, K.; I adi, J.;
Fe nandez, E.; Cal o, P.M.; Salle, D.;
Poologaind an, A.; Villa e de, I.;
Daelman, P.; Sanchez, E.; Requejo, C.;
e al. HUMANISE: Human-Inspi ed
Sma Managemen , owa ds a
Heal hy and Sa e Indus ial
Collabo a i e Robo ics. Senso s 2023,
23, 1170. h ps://doi.o g/10.3390/
s23031170
Academic Edi o : Yi Zhang
Recei ed: 27 No embe 2022
Re ised: 10 Janua y 2023
Accep ed: 14 Janua y 2023
Published: 19 Janua y 2023
Copy igh : © 2023 by he au ho s.
Licensee MDPI, Basel, Swi ze land.
This a icle is an open access a icle
dis ibu ed unde he e ms and
condi ions o he C ea i e Commons
A ibu ion (CC BY) license (h ps://
c ea i ecommons.o g/licenses/by/
4.0/).
senso s
A icle
HUMANISE: Human-Inspi ed Sma Managemen , owa ds
a Heal hy and Sa e Indus ial Collabo a i e Robo ics
Ka mele Lopez-de-Ipina 1,2,* , Jon I adi 2, Elsa Fe nandez 2, Pila M. Cal o 2, Damien Salle 3,*,
Anujan Poologaind an 1,4, I an Villa e de 3, Paul Daelman 3, Emilio Sanchez 5,6 , Ca alina Requejo 7
and John Suckling 1,*
1Depa men o Psychia y, Uni e si y o Camb idge, Camb idge CB2 3PT, UK
2
EleKin Lab, Sys ems Enginee ing and Au oma ion, Compu e s’ A chi ec u e and Technology, and En e p ise
Managemen Depa men s, Uni e si y o he Basque Coun y UPV/EHU,
20018 Donos ia-San Sebas ian, Spain
3Tecnalia Resea ch Cen e, Tecnalia Indus y and T anspo Di ision, 20009 Donos ia-San Sebas ia, Spain
4The Alan Tu ing Ins i u e, B i ish Lib a y, London NW1 2DB, UK
5
Depa men o Mechanical Enginee ing and Ma e ials, Enginee ing School, Uni e si y o Na a a, TECNUN,
20018 Donos ia-San Sebas ian, Spain
6CEIT, Manu ac u ing Di ision, 20018 Donos ia-San Sebas ian, Spain
7Cajal Ins i u e, Consejo Supe io de In es igaciones Cien í icas (CSIC), 28002 Mad id, Spain
*Co espondence: [email p o ec ed] (K.L.-d.-I.); [email p o ec ed] (D.S.); [email p o ec ed] (J.S.)
Abs ac :
The wo kplace is e ol ing owa ds scena ios whe e humans a e acqui ing a mo e ac i e
and dynamic ole alongside inc easingly in elligen machines. Mo eo e , he ac i e popula ion is
ageing and consequen ly eme ging isks could appea due o heal h diso de s o wo ke s, which
equi es in elligen in e en ion bo h o p oduc ion managemen and wo ke s’ suppo . In his
sense, he inno a i e and sma sys ems o ien ed owa ds moni o ing and egula ing wo ke s’ well-
being will become essen ial. This wo k p esen s HUMANISE, a no el p oposal o an in elligen
sys em o isk managemen , o ien ed o wo ke s su e ing om disease condi ions. The de eloped
suppo sys em is based on Compu e Vision, Machine Lea ning and In elligen Agen s. Resul s: The
sys em was applied o a wo-a m Cobo scena io du ing a Lea ning om Demons a ion ask o
collabo a i e pa s anspo a ion, whe e isk managemen is c i ical. In his en i onmen wi h a
wo ke su e ing om a men al diso de , sa e y is success ully con olled by means o human/ obo
coo dina ion, and isk le els a e managed h ough he in eg a ion o human/ obo beha iou models
and wo ke ’s models based on he wo kplace model o he Wo ld Heal h O ganiza ion. The esul s
show a p omising eal- ime suppo ool o coo dina e and moni o ing hese scena ios by in eg a ing
wo ke s’ heal h in o ma ion owa ds a success ul isk managemen s a egy o sa e indus ial
Cobo en i onmen s.
Keywo ds:
Cobo ; Machine Lea ning; isk managemen ; human/ obo beha iou ; ageing popula ion;
wo ke s’ diseases; indus ial heal h and sa e y
1. In oduc ion
Humani y is acing a new labou en i onmen whe e he highly-quali ied ac i e
popula ion is ageing, and highly-quali ied young popula ion ha e uns able employmen .
Mo eo e , wo king en i onmen s a e becoming inc easingly s ess ul, equi ing highe cog-
ni i e demands om indi iduals due o indus ial and echnological wo kplace p essu es.
Addi ionally, while adi ional psychosocial isks may a ise om a a ie y o no el wo king
condi ions, he adi ional job con en /con ex model does no apply o he mode n and
‘sma e ’ wo kplace. The e o e, p e en i e wo kplace measu es some imes may no be
applicable o hese new wo king scena ios. Fu he mo e, new eme ging ac o s should
be ca e ully in eg a ed in o wo kplaces because wo k is a aluable elemen o heal hy
Senso s 2023,23, 1170. h ps://doi.o g/10.3390/s23031170 h ps://www.mdpi.com/jou nal/senso s
Senso s 2023,23, 1170 2 o 18
ac i e ageing. In his ega d, cogni i e well-being is c ucial o an e icien and sa is ac o y
wo king li e [1].
Ageing induces heal h impai men s in he h ee aspec s o heal h de ined by he
Wo ld Heal h O ganiza ion (WHO) [
2
]: physical, men al and social. Addi ionally, cogni i e
impai men s s a in la e li e, and in he ea ly s ages i can lead o diseases such as anxie y,
dep ession, and in mo e ad anced s ages Alzheime ’s and Pa kinson’s ha p oduce a clea
nega i e impac on wo ke s’ pe o mance. Con e sely, inc easing social di icul ies such as
ca ing o dependan s, wo k-li e imbalance, exclusion isks and immig a ion, inc ease he
complexi y o cu en wo king en i onmen s.
In hese complex wo kplaces, inno a i e ools a e capable o p omo ing cogni i e well-
being, which is c ucial o main aining heal hy ageing and o con ibu ing o he economy
and socie y as highligh ed in he epo s o he Wo ld Heal h O ganiza ion ha p o ides an
essen ial ool o desc ibe en i onmen s in e ms o heal hy wo kplace models (Figu e 1).
Wi hin hese scena ios, se e al labou isks a e iden i ied, among o he s: (a) inc eased
a igue due o he loss o cogni i e well-being because o mul iple wo k and psychosocial
ac o s, (b) ea ly cogni i e impai men (ea ly neu odegene a ion ha is becoming epidemic
MCI (Mild Cogni i e Impai men ), PD (Pa kinson’s Disease), (c) inc eased le els o s ess
(anxie y), and (d) inc eased agili y and ch onici y o he wo ke ’s heal h. Subsequen ly, he
ollowing issues need o be ackled, which esul s in a g ea inc ease in occupa ional isks
and lea es o medical easons: (a) de e io a ion o men al heal h (anxie y and dep ession
diso de s), (b) agili y, (c) ch onici y, (d) lowe pe o mance, and (e) loss o unc ional
capaci y, pe o mance and p oduc ion as well as lowe quali y ou pu [1–3].
Senso s 2023, 23, x FOR PEER REVIEW 2 o 18
wo king condi ions, he adi ional job con en /con ex model does no apply o he mod-
e n and ‘sma e ’ wo kplace. The e o e, p e en i e wo kplace measu es some imes may
no be applicable o hese new wo king scena ios. Fu he mo e, new eme ging ac o s
should be ca e ully in eg a ed in o wo kplaces because wo k is a aluable elemen o
heal hy ac i e ageing. In his ega d, cogni i e well-being is c ucial o an e icien and
sa is ac o y wo king li e [1].
Ageing induces heal h impai men s in he h ee aspec s o heal h de ined by he
Wo ld Heal h O ganiza ion (WHO) [2]: physical, men al and social. Addi ionally, cogni-
i e impai men s s a in la e li e, and in he ea ly s ages i can lead o diseases such as
anxie y, dep ession, and in mo e ad anced s ages Alzheime ’s and Pa kinson’s ha p o-
duce a clea nega i e impac on wo ke s’ pe o mance. Con e sely, inc easing social di -
icul ies such as ca ing o dependan s, wo k-li e imbalance, exclusion isks and immig a-
ion, inc ease he complexi y o cu en wo king en i onmen s.
In hese complex wo kplaces, inno a i e ools a e capable o p omo ing cogni i e
well-being, which is c ucial o main aining heal hy ageing and o con ibu ing o he econ-
omy and socie y as highligh ed in he epo s o he Wo ld Heal h O ganiza ion ha p o-
ides an essen ial ool o desc ibe en i onmen s in e ms o heal hy wo kplace models
(Figu e 1). Wi hin hese scena ios, se e al labou isks a e iden i ied, among o he s: (a)
inc eased a igue due o he loss o cogni i e well-being because o mul iple wo k and
psychosocial ac o s, (b) ea ly cogni i e impai men (ea ly neu odegene a ion ha is be-
coming epidemic MCI (Mild Cogni i e Impai men ), PD (Pa kinson’s Disease), (c) in-
c eased le els o s ess (anxie y), and (d) inc eased agili y and ch onici y o he wo ke ’s
heal h. Subsequen ly, he ollowing issues need o be ackled, which esul s in a g ea in-
c ease in occupa ional isks and lea es o medical easons: (a) de e io a ion o men al
heal h (anxie y and dep ession diso de s), (b) agili y, (c) ch onici y, (d) lowe pe o -
mance, and (e) loss o unc ional capaci y, pe o mance and p oduc ion as well as lowe
quali y ou pu [1–3].
Figu e 1. HUMANISE’s amewo k consis s in he managemen o sa e y in Indus ial Collabo a i e
Robo ics wi h wo ke s su e ing om disease condi ions: (a) Indus ial Collabo a i e Robo ics
(CoBo ) scena io. (b) Wo ke ’s heal h condi ions h ough Wo ld Heal h O ganiza ion (WHO) wo k-
place model [2].
These condi ions ha e a se ious nega i e impac on amily and wo king li e: p eca -
iousness; educed access o heal h suppo ; mo e compe i i e and isola ed wo king en i-
onmen s; inc ease in psychosocial s ess le els ( ele-s ess); and inc ease in eme ging
isks. Taking in o conside a ion he nega i e impac s, some wo kplaces ha e e ol ed o
inco po a e home-based elewo king ha helps o balance amily-wo k li e, which usually
a ec s women wi h ca e esponsibili ies [4]. The COVID19 cu en si ua ion has accele -
a ed all hese changes. The e o e, as s a ed by he Eu opean Agency o Sa e y and Heal h
a Wo k ema k [5], In o ma ion and Communica ion Technologies (ICTs) and in elligen
machines such as obo s ha p o ide au oma ic human skills will undoub edly b ing ma-
jo new social and economic oppo uni ies as well as eme ging isks, helping o dec ease
he digi al and wo k gaps (gende , sala y, [1,4]).
Figu e 1.
HUMANISE’s amewo k consis s in he managemen o sa e y in Indus ial Collabo a i e
Robo ics wi h wo ke s su e ing om disease condi ions: (
a
) Indus ial Collabo a i e Robo ics (CoBo )
scena io. (
b
) Wo ke ’s heal h condi ions h ough Wo ld Heal h O ganiza ion (WHO) wo kplace
model [2].
These condi ions ha e a se ious nega i e impac on amily and wo king li e: p e-
ca iousness; educed access o heal h suppo ; mo e compe i i e and isola ed wo king
en i onmen s; inc ease in psychosocial s ess le els ( ele-s ess); and inc ease in eme ging
isks. Taking in o conside a ion he nega i e impac s, some wo kplaces ha e e ol ed o
inco po a e home-based elewo king ha helps o balance amily-wo k li e, which usually
a ec s women wi h ca e esponsibili ies [
4
]. The COVID19 cu en si ua ion has accele a ed
all hese changes. The e o e, as s a ed by he Eu opean Agency o Sa e y and Heal h a
Wo k ema k [
5
], In o ma ion and Communica ion Technologies (ICTs) and in elligen
machines such as obo s ha p o ide au oma ic human skills will undoub edly b ing majo
new social and economic oppo uni ies as well as eme ging isks, helping o dec ease he
digi al and wo k gaps (gende , sala y, [1,4]).
I will be essen ial o make indi iduals awa e o he impo ance o hese changes o
he wo kplace, and o inc ease he adap a ion capaci y o wo ke s by means o inno a i e
suppo ing ools ha also gua an ee pe sonal p i acy and da a p o ec ion wi hou inc eased
echnology cos [
2
–
4
]. This amewo k b ings challenges in o wo king en i onmen s and
Senso s 2023,23, 1170 3 o 18
sma sys ems based on A i icial In elligence and Collabo a i e Robo s (Cobo s) ha will
become essen ial esou ces o wo kplaces and indus y.
Inno a i e and sa e wo kplaces in ol ing Cobo s will be o ien ed owa ds moni o ing
and egula ing he wo ke s’ ac i i y, sa e y and secu i y, while also ying o p omo e
a mo e p oac i e and heal hie li es yle. The new cogni i e pla o ms o sma Cobo
ope a ion will include: sma managemen , egula ion and suppo , sa e y and secu i y
con ol, quali y and p oduc i i y imp o emen , wo ke s’ suppo and/o in elligen lea n-
ing scena ios. These no el machines equi e complex and in elligen human skills such
as p ecision, planning, mo ion, complex pe cep ion, lexibili y, e sa ili y and in elligen
con ol. In ac , cu en sys ems y o imi a e human beha iou s by in eg a ing ision,
p edic ion o in elligen managemen , and in he nea - u u e will also likely impe sona e
complex b ain p ocesses [5].
In ecen yea s, se e al indus ies ha e b ough obo ic echnology in o hei ma -
ke s: household obo s, en e ainmen obo s, logis ics obo s, public en i onmen obo s,
de ence applica ions, inspec ion and main enance obo s, p o essional cleaning obo s, ag i-
cul u al obo s, mo o ized human exoskele ons, cons uc ion and demoli ion obo s [
6
]. In
he lou ishing ield o medical obo ics, signi ican ad ancemen s include: obo ic su ge y
which p omo es he ansi ion om open o lapa oscopic su ge y and o he minimally
in asi e su gical p ocedu es [7], bionic p os heses, and ca e ake obo s [8].
The oppo uni ies o obo ics and au onomous sys ems in he ag icul u al- ood p o-
duc ion include: he de elopmen o ield obo s ha can assis wo ke s by ca ying
payloads and conduc ag icul u al ope a ions such as c op and animal sensing, weeding
and d illing, o in eg a ion o au onomous sys ems echnologies in o exis ing a m op-
e a ional equipmen such as ac o s [
9
]. These examples oge he demons a e ha he
bounda ies be ween indus ial and se ice obo ics a e blu ing [
10
], and ha human- obo
in e ac ions a e assuming a b oade ole, and becoming applicable o a wide a ie y o
applica ions whe e a close in e ac ion be ween humans and obo s is an icipa ed.
Human-Robo Collabo a ion (HRC), is a ou ed by wo cogni i e abili ies: in en ional
eading and con idence. A obo possessing hese abili ies could in e he non- e bal
in en ions o o he s and assess he likelihood ha hey will achie e hei goals, join ly
unde s anding wha ype and deg ee o collabo a ion hey equi e [11].
This wo k is pa o a esea ch line ocus on he imp o emen o e iciency and sa e y
in indus ial Cobo en i onmen s [
12
–
14
] and p esen s he i s app oach and p elimina y
esul s o HUMANISE, a human inspi ed sma managemen sys em o heal hy and sa e
indus ial collabo a i e obo ics (Cobo s). The sys em in eg a es he WHO wo kplace
model wi h p inciples om Machine Lea ning (ML), in elligen agen s, and ac ional
con ol (Figu e 1). The pape is o ganized as ollows: Sec ion 1con ains he in oduc ion;
Sec ion 2desc ibes Indus ial Cobo s; Sec ion 3includes me hods and ma e ials; Sec ion 4
p esen s he esul s and discussion. Finally, he concluding ema ks a e p esen ed in
Sec ion 5. Addi ionally, his wo k will be amed agains he backd op o an E hical,
Risk Managemen , and Regula o y amewo k o he long- e m implica ions o Cobo s
echnology, and hus will include use -o ien ed and uni e sal design me hodologies.
2. Collabo a i e Robo (Cobo ) in Indus ial En i onmen s
Cobo s a e obo s designed o au oma e p oduc ion p ocesses by sha ing a wo k
en i onmen h ough human-machine in e ac ion (Figu e 2). Due o he inhe en isks
o p oduc ion p ocesses, sa e y is a c i ical ac o when designing bo h he collabo a-
i e wo k mode and he coope a i e wo kspace [
12
–
17
]. This in eg a ion is occu ing
in se e al au oma ed elemen s: Cobo s, in elligen sys ems, high p ecision au oma ion,
augmen ed eali y, bionic (wea able exoskele on o p os hesis) o heal h obo ic sys ems
(neu osu ge y, neu o ehabili a ion).
Senso s 2023,23, 1170 4 o 18
Senso s 2023, 23, x FOR PEER REVIEW 4 o 18
e al au oma ed elemen s: Cobo s, in elligen sys ems, high p ecision au oma ion, aug-
men ed eali y, bionic (wea able exoskele on o p os hesis) o heal h obo ic sys ems (neu-
osu ge y, neu o ehabili a ion).
Figu e 2. Robo Kawada Nex age Open o Tecnalia [13].
The no ion o HRC, based on he co ela i e wo k o obo s and humans, aises ques-
ions abou he adi ional model o physical ba ie s ha sepa a e machines and wo ke s,
and highligh s he dwindling gap be ween bo h beha iou s. The e a e wo echnological
ac o s ha ha e made his pa adigm shi possible: (1) he in eg a ion o sa e y- ela ed
ea u es in he a chi ec u es o obo s and con ol sys ems, and (2) he use o mul imodal
in e aces o a mo e in ui i e, conscious, and sa e Human-Robo In e ac ion (HRI) [18,19].
Cobo sys ems a e a new obo ics echnology ha allows obo s and human ope a o s o
wo k oge he in ways ha we e p e iously impossible. The e a e ou di e en ypes o
collabo a i e ope a ions de ined in he ANSI/RIA obo sa e y s anda d documen s [20,21]
(1) Sa e y- a ed Moni o ed S op (SMS); (2) Hand Guiding (HG); (3) Speed and Sepa a ion
Moni o ing (SSM), and (4) Powe and Fo ce Limi ing (PFL).
The e o e, i is e y impo an ha Cobo s include sa e y s anda ds ha educe isks.
Al hough Cobo s ha e buil -in secu i y measu es ha allow sa e applica ions, his s a e
o en changes as soon as hey a e in eg a ed in o a wo k en i onmen . The e o e, sa e y
sys ems mus be ins alled o a oid isks o humans ha he obo may gene a e, as well
as sa e y measu es ela ed o he design o he wo k cell [21]. Mo eo e , u u e Cobo sys-
ems will no only need o adap o wo kplace condi ions, bu also need o inco po a e
ea u es o he wo ke ’s mood, heal h, and associa ed ac o s in o de o design obus
con ol sys ems o he wo kplace, and ul ima ely mi iga e isks [22–25].
Acco ding o some s udies, co ec ly p edic ing human ac ions h ough he unde -
s anding o human in en ion can p oduce sa e human- obo in e ac ions [25] and mo e
e icien human- obo eams [26]. Fu he mo e, i was ound ha obo planning using a
human planning model can p oduce plans ha can be gene alized be e han plans
lea ned wi hou such a model [27]. O he wo ks ha e de eloped me hods o p edic hu-
man in en ion based on low-le el human mo emen [28–30] as well as highe -le el mod-
els o human easoning based on modelling o social o ces [31]. I we look a he sa e y
assessmen in he HRC, some app oaches use objec i e measu es o plan and e alua e he
pe o mance o applica ions ha p esen collabo a i e “speed and sepa a ion moni o ing”
scena ios [32,33]. Howe e , hey also demons a e he di icul ies in iden i ying he mo-
men in ime du ing a obo ’s ajec o y whe e a speci ic algo i hm is he leas secu e, e-
qui ing simula ion o es ing wi h he en i e sys em.
In his sense, he LBR iiwa is he wo ld’s i s se ies-p oduced sensi i e (HRC-com-
pa ible) obo . LBR s ands o Leich bau obo e (Ge man o ligh weigh obo ) and iiwa
o in elligen indus ial wo k assis an [34]. LBR iiwa p o ides he necessa y skills o
humans and obo s o wo k in close coope a ion wi h a high e ec i eness and e iciency.
The Kuka LBR iiwa obo s wi h a payload o 7 kg a e equipped wi h o que senso s in
each join . These obo ic a ms can p o ide o ce and o que in o ma ion es ima ed om
he o que senso s placed in each join . In [12] au ho s wo ked on iden i ying dynamic
Figu e 2. Robo Kawada Nex age Open o Tecnalia [13].
The no ion o HRC, based on he co ela i e wo k o obo s and humans, aises
ques ions abou he adi ional model o physical ba ie s ha sepa a e machines and
wo ke s, and highligh s he dwindling gap be ween bo h beha iou s. The e a e wo
echnological ac o s ha ha e made his pa adigm shi possible: (1) he in eg a ion
o sa e y- ela ed ea u es in he a chi ec u es o obo s and con ol sys ems, and (2) he
use o mul imodal in e aces o a mo e in ui i e, conscious, and sa e Human-Robo
In e ac ion (HRI) [
18
,
19
]. Cobo sys ems a e a new obo ics echnology ha allows obo s
and human ope a o s o wo k oge he in ways ha we e p e iously impossible. The e
a e ou di e en ypes o collabo a i e ope a ions de ined in he ANSI/RIA obo sa e y
s anda d documen s [
20
,
21
] (1) Sa e y- a ed Moni o ed S op (SMS); (2) Hand Guiding (HG);
(3) Speed and Sepa a ion Moni o ing (SSM), and (4) Powe and Fo ce Limi ing (PFL).
The e o e, i is e y impo an ha Cobo s include sa e y s anda ds ha educe isks.
Al hough Cobo s ha e buil -in secu i y measu es ha allow sa e applica ions, his s a e
o en changes as soon as hey a e in eg a ed in o a wo k en i onmen . The e o e, sa e y
sys ems mus be ins alled o a oid isks o humans ha he obo may gene a e, as well as
sa e y measu es ela ed o he design o he wo k cell [
21
]. Mo eo e , u u e Cobo sys ems
will no only need o adap o wo kplace condi ions, bu also need o inco po a e ea u es o
he wo ke ’s mood, heal h, and associa ed ac o s in o de o design obus con ol sys ems
o he wo kplace, and ul ima ely mi iga e isks [22–25].
Acco ding o some s udies, co ec ly p edic ing human ac ions h ough he unde -
s anding o human in en ion can p oduce sa e human- obo in e ac ions [
25
] and mo e
e icien human- obo eams [
26
]. Fu he mo e, i was ound ha obo planning using a
human planning model can p oduce plans ha can be gene alized be e han plans lea ned
wi hou such a model [
27
]. O he wo ks ha e de eloped me hods o p edic human in en-
ion based on low-le el human mo emen [
28
–
30
] as well as highe -le el models o human
easoning based on modelling o social o ces [
31
]. I we look a he sa e y assessmen in
he HRC, some app oaches use objec i e measu es o plan and e alua e he pe o mance o
applica ions ha p esen collabo a i e “speed and sepa a ion moni o ing” scena ios [
32
,
33
].
Howe e , hey also demons a e he di icul ies in iden i ying he momen in ime du ing
a obo ’s ajec o y whe e a speci ic algo i hm is he leas secu e, equi ing simula ion o
es ing wi h he en i e sys em.
In his sense, he LBR iiwa is he wo ld’s i s se ies-p oduced sensi i e (HRC-compa ible)
obo . LBR s ands o Leich bau obo e (Ge man o ligh weigh obo ) and iiwa o in elli-
gen indus ial wo k assis an [
34
]. LBR iiwa p o ides he necessa y skills o humans and
obo s o wo k in close coope a ion wi h a high e ec i eness and e iciency. The Kuka LBR
iiwa obo s wi h a payload o 7 kg a e equipped wi h o que senso s in each join . These
obo ic a ms can p o ide o ce and o que in o ma ion es ima ed om he o que senso s
placed in each join . In [
12
] au ho s wo ked on iden i ying dynamic pa ame e s ha can
p edic he obo join o ques by using global op imiza ion. Fu he mo e, he p e ious
Senso s 2023,23, 1170 5 o 18
wo ks wi h his Cobo we e ocused on new schemes o sa e y and eal- ime eedback o
he p ocess s a e in a amewo k o coope a i e applica ions o indus ial en i onmen s
(Figu e 3). Thus, se e al a chi ec u es we e de eloped o manage lexibili y, eusabili y
wi h mul isenso y in elligen obo s [
13
] and he execu ion o ajec o y d i en collabo-
a i e asks by combining con ol, ajec o y coo dina ion and e ec i e obo - o-human
eedback [
14
], and by a pa h-d i en mobile co-manipula ion a chi ec u e de ini ion o sa e
and collabo a i e a eas [
15
]. Mo eo e , hese asks a e also in SHERLOCK p ojec [
16
],
which o ien ed he la es sa e obo ic echnologies including high payload collabo a i e
a ms, exoskele ons and mobile manipula o s in di e se p oduc ion en i onmen s.
Senso s 2023, 23, x FOR PEER REVIEW 5 o 18
pa ame e s ha can p edic he obo join o ques by using global op imiza ion. Fu he -
mo e, he p e ious wo ks wi h his Cobo we e ocused on new schemes o sa e y and
eal- ime eedback o he p ocess s a e in a amewo k o coope a i e applica ions o in-
dus ial en i onmen s (Figu e 3). Thus, se e al a chi ec u es we e de eloped o manage
lexibili y, eusabili y wi h mul isenso y in elligen obo s [13] and he execu ion o ajec-
o y d i en collabo a i e asks by combining con ol, ajec o y coo dina ion and e ec i e
obo - o-human eedback [14], and by a pa h-d i en mobile co-manipula ion a chi ec u e
de ini ion o sa e and collabo a i e a eas [15]. Mo eo e , hese asks a e also in SHER-
LOCK p ojec [16], which o ien ed he la es sa e obo ic echnologies including high pay-
load collabo a i e a ms, exoskele ons and mobile manipula o s in di e se p oduc ion en-
i onmen s.
Figu e 3. Scena io o he Use case 1: Cobo Lea ning om Demons a ion (L D).
3. Ma e ials and Me hods
3.1. Ma e ials
The ma e ials consis o a high- esolu ion ideo (A ound 5000 ames, 24 ames/s,
image size 1080 × 1920 pixels) o a eal ask eco ded in Tecnalia’s lexible obo ics labo -
a o y. Speci ically, a demons a ion cell was de eloped, which was designed o each he
possibili ies o e ed by Cobo s. The deployed Cobo s we e Kuka LBR iiwa ( wo a ms) on
a mobile pla o m wi h and ex e nal IO module.
In his wo k, in he amewo k o isk managemen in Cobo en i onmen s o ageing
and wo ke s su e ing om diseases, he so-called “collabo a i e pa s anspo a ion”
was selec ed because many indus ial sec o s equi e mo ing la ge pa s among di e en
a eas o he wo kplace, using a la ge amoun o he wo k o ce. Fu he mo e, one o he
mos ele an HRC asks was selec ed, he “Cobo Lea ning om Demons a ion (L D)”
ask. L D e e s o he p ocess used o ans e new skills o a machine by elying on
demons a ions om a use . This ask is inspi ed by he imi a ion p ocess o humans and
animals o acqui e new skills. L D p o ides an easy and in ui i e way o gi e new skills
o he Cobo [35]. Speci ically in his wo k du ing he L D ask, he wo ke demons a ed
o he Cobo s how o manage and anspo la ge pa s. The wo ke showed o bo h
Cobo s how: (i) o pick he la ge pa (posi ions); (ii) o mo e i h ough a pa hway o he
nex wo ks a ion; (iii) deposi i ; (i ) he Cobo s should coo dina e in each s ep o he ask.
This is he Use case 1 and i was a coo dina ed Cobo lea ning ask be ween he wo-a ms
o Cobo and a wo ke (Figu e 3). The moni o ing ideo sequences acqui ed by he ex e -
nal high-quali y came a we e p ocessed by a cus om oolbox in MATLAB o c ea e ac i i y
ime-se ies [36,37].
Th ee Use Cases (UC) o ien ed o isks gene a ed by wo ke ’s s ess we e c ea ed and
e alua ed, he eco ded eal- ask and wo simula ed asks:
1. UC1-Coo dina ion: his is he p e iously desc ibed eal ask, Cobo L D (Figu e 3) o
“collabo a i e pa s anspo a ion”.
2. UC2-S ess: his was a simula ed case o a s essed wo ke based on he isual eed-
back ea u es o UC1. An algo i hm gene a ed his simula ed case.
Figu e 3. Scena io o he Use case 1: Cobo Lea ning om Demons a ion (L D).
3. Ma e ials and Me hods
3.1. Ma e ials
The ma e ials consis o a high- esolu ion ideo (A ound 5000 ames, 24 ames/s,
image size 1080
×
1920 pixels) o a eal ask eco ded in Tecnalia’s lexible obo ics labo a-
o y. Speci ically, a demons a ion cell was de eloped, which was designed o each he
possibili ies o e ed by Cobo s. The deployed Cobo s we e Kuka LBR iiwa ( wo a ms) on a
mobile pla o m wi h and ex e nal IO module.
In his wo k, in he amewo k o isk managemen in Cobo en i onmen s o ageing
and wo ke s su e ing om diseases, he so-called “collabo a i e pa s anspo a ion”
was selec ed because many indus ial sec o s equi e mo ing la ge pa s among di e en
a eas o he wo kplace, using a la ge amoun o he wo k o ce. Fu he mo e, one o he
mos ele an HRC asks was selec ed, he “Cobo Lea ning om Demons a ion (L D)”
ask. L D e e s o he p ocess used o ans e new skills o a machine by elying on
demons a ions om a use . This ask is inspi ed by he imi a ion p ocess o humans and
animals o acqui e new skills. L D p o ides an easy and in ui i e way o gi e new skills o
he Cobo [
35
]. Speci ically in his wo k du ing he L D ask, he wo ke demons a ed o
he Cobo s how o manage and anspo la ge pa s. The wo ke showed o bo h Cobo s
how: (i) o pick he la ge pa (posi ions); (ii) o mo e i h ough a pa hway o he nex
wo ks a ion;
(iii) deposi
i ; (i ) he Cobo s should coo dina e in each s ep o he ask. This
is he Use case 1 and i was a coo dina ed Cobo lea ning ask be ween he wo-a ms o
Cobo and a wo ke (Figu e 3). The moni o ing ideo sequences acqui ed by he ex e nal
high-quali y came a we e p ocessed by a cus om oolbox in MATLAB o c ea e ac i i y
ime-se ies [36,37].
Th ee Use Cases (UC) o ien ed o isks gene a ed by wo ke ’s s ess we e c ea ed and
e alua ed, he eco ded eal- ask and wo simula ed asks:
1.
UC1-Coo dina ion: his is he p e iously desc ibed eal ask, Cobo L D (Figu e 3) o
“collabo a i e pa s anspo a ion”.
2.
UC2-S ess: his was a simula ed case o a s essed wo ke based on he isual
eedback ea u es o UC1. An algo i hm gene a ed his simula ed case.
Senso s 2023,23, 1170 6 o 18
3.
UC3-High isk: his was a Cobo isk beha iou UC whe e he Cobo s did no pe o m
in a coo dina ed way, and he e was high speed and accele a ed misbeha iou . An
algo i hm gene a ed his simula ed case.
Summing up, his wo k p esen s he p elimina y esul s o he HUMANISE sys em
e alua ed o e hese h ee UCs and in o ma ion o a Men al Diso de (anxie y). Mo eo e ,
he h ee UCs we e e alua ed o e wo L D asks wi h di e en isk le els: (i) Ex a La ge
size pa s wi h low speed (L D-XL); (ii) La ge size pa s wi h medium speed (L D-L).
3.2. Me hods
HUMANISE is o ien ed o cu en and u u e indus ial scena ios whe e lexibili y and
e sa ili y o small se ies, co- eedback and open collabo a ion among Cobo s and humans,
human- obo in e ac ion, eal- ime applica ions and/o sa e y a e essen ial equi emen s.
In hese wo king en i onmen s, eme ging isks appea , and in elligen suppo is
equi ed o bo h managemen and con ol o machines and p oduc ion. In his wo k, we
de elop HUMANISE, an in elligen managemen sys em o human ope a ion and obo ic
beha iou o heal hy and sa e indus ial use. The de eloped sys ems and ools a e based
on compu e ision and a i icial in elligence and a e applied o a wo-a m Cobo scena io
du ing a lea ning p ocess o lexible and e sa ile manu ac u ing.
3.2.1. O e all Me hodology and F amewo k
The amewo k o HUMANISE is o ien ed o he managemen o sa e y in indus ial
collabo a i e obo ics wi h wo ke s su e ing om disease condi ions. Thus, he main objec-
i es we e ocused on he ollowing measu able objec i es by in eg a ed
phases (Figu e 4)
:
1.
Knowledge ex ac ion: (i) Knowledge om he WHO heal h wo kplace models,
(ii) Knowledge
om he en i onmen al ac i i y da a o he isual eedback p o ided
by he pe cep ion sys em, (iii) Knowledge p o ided by he ML models.
2.
An in elligen agen o p o ide sma in e ac i e suppo o a specialis who could
in e ac and egula e he en i onmen . The agen will be guided in a u u e design by
means o human-b ain pa e ns.
3.
In elligen con ol and managemen o he Cobo s by he egula ion o bo h con ol
algo i hm pa ame e s and ajec o y coo dina ion.
Senso s 2023, 23, x FOR PEER REVIEW 6 o 18
3. UC3-High isk: his was a Cobo isk beha iou UC whe e he Cobo s did no pe -
o m in a coo dina ed way, and he e was high speed and accele a ed misbeha iou .
An algo i hm gene a ed his simula ed case.
Summing up, his wo k p esen s he p elimina y esul s o he HUMANISE sys em
e alua ed o e hese h ee UCs and in o ma ion o a Men al Diso de (anxie y). Mo eo e ,
he h ee UCs we e e alua ed o e wo L D asks wi h di e en isk le els: (i) Ex a La ge
size pa s wi h low speed (L D-XL); (ii) La ge size pa s wi h medium speed (L D-L).
3.2. Me hods
HUMANISE is o ien ed o cu en and u u e indus ial scena ios whe e lexibili y
and e sa ili y o small se ies, co- eedback and open collabo a ion among Cobo s and
humans, human- obo in e ac ion, eal- ime applica ions and/o sa e y a e essen ial e-
qui emen s.
In hese wo king en i onmen s, eme ging isks appea , and in elligen suppo is e-
qui ed o bo h managemen and con ol o machines and p oduc ion. In his wo k, we
de elop HUMANISE, an in elligen managemen sys em o human ope a ion and obo ic
beha iou o heal hy and sa e indus ial use. The de eloped sys ems and ools a e based
on compu e ision and a i icial in elligence and a e applied o a wo-a m Cobo scena io
du ing a lea ning p ocess o lexible and e sa ile manu ac u ing.
3.2.1. O e all Me hodology and F amewo k
The amewo k o HUMANISE is o ien ed o he managemen o sa e y in indus ial
collabo a i e obo ics wi h wo ke s su e ing om disease condi ions. Thus, he main ob-
jec i es we e ocused on he ollowing measu able objec i es by in eg a ed phases (Figu e
4):
1. Knowledge ex ac ion: (i) Knowledge om he WHO heal h wo kplace models, (ii)
Knowledge om he en i onmen al ac i i y da a o he isual eedback p o ided by
he pe cep ion sys em, (iii) Knowledge p o ided by he ML models.
2. An in elligen agen o p o ide sma in e ac i e suppo o a specialis who could
in e ac and egula e he en i onmen . The agen will be guided in a u u e design
by means o human-b ain pa e ns.
3. In elligen con ol and managemen o he Cobo s by he egula ion o bo h con ol
algo i hm pa ame e s and ajec o y coo dina ion.
Figu e 4. Diag am o he o e all me hodology o HUMANISE.
Figu e 4. Diag am o he o e all me hodology o HUMANISE.
Senso s 2023,23, 1170 7 o 18
3.2.2. Sma Managemen Sys em
The o e all objec i e o HUMANISE is o de elop a sma sys em o manage Cobo
wo king en i onmen s whils suppo ing wo ke s o imp o e hei heal h and sa e y
condi ions (Figu e 5). Speci ically, he p ojec will es ablish a wo king syne gy be ween
se e al b oad and mul idisciplina y a eas: heal h and p e en ion o occupa ional isks,
neu ology, psychia y, ac i e ageing, heal h ca e, elecommunica ions, modelling and
con ol, and compu e science. These ML and in elligen me hodologies will p edic and
analyse wo kplace si ua ions. Figu e 4shows he block diag am o he Sma Managemen
Sys em o HUMANISE. The a chi ec u e o he sys em is based on a closed loop sys em
wi h isual eedback and in elligen managemen o he pe o mance condi ions.
Senso s 2023, 23, x FOR PEER REVIEW 7 o 18
3.2.2. Sma Managemen Sys em
The o e all objec i e o HUMANISE is o de elop a sma sys em o manage Cobo
wo king en i onmen s whils suppo ing wo ke s o imp o e hei heal h and sa e y con-
di ions (Figu e 5). Speci ically, he p ojec will es ablish a wo king syne gy be ween se -
e al b oad and mul idisciplina y a eas: heal h and p e en ion o occupa ional isks, neu-
ology, psychia y, ac i e ageing, heal h ca e, elecommunica ions, modelling and con ol,
and compu e science. These ML and in elligen me hodologies will p edic and analyse
wo kplace si ua ions. Figu e 4 shows he block diag am o he Sma Managemen Sys em
o HUMANISE. The a chi ec u e o he sys em is based on a closed loop sys em wi h is-
ual eedback and in elligen managemen o he pe o mance condi ions.
Figu e 5. Diag am o he Sma Managemen Sys em.
The main componen s o he sys em a e:
1. The in elligen managemen module: he co e o his module consis s o he WHO
and isual en i onmen ML models (Suppo Vec o Machines, SVM), and an in el-
ligen agen . This las componen will manage he condi ions o he wo kplace by
sending ins uc ions and in o ma ion o he con ol module.
2. The con ol module: This componen adap s he Cobo beha iou and gi es suppo
o he wo ke s o a success ul and coo dina ed e olu ion o he ask imp o ing he
quali y o he p ocess. This eal- ime con ol and suppo will be essen ial o heal h
and sa e y wo k condi ions.
3. The pe cep ion encode . This componen manages in eal- ime he isual in o ma ion
ha en iches he in elligen managemen module.
3.2.3. WHO Heal h Wo kplace Model
HUMANISE also egula es condi ions by a sma managemen sys em, which in e-
g a es ML modelling based on WHO’s heal h wo kplace model (Figu e 1) ha includes:
1. Heal h and sa e y in he physical wo k en i onmen .
2. Heal h, sa e y, and well-being in he psychosocial wo k en i onmen .
3. Pe sonal heal h esou ces in he wo kplace.
4. En e p ise communi y esou ces.
3.2.4. Moni o ing and Con ol o he Cobo En i onmen
Figu e 6 shows he Wo k low o he Cobo en i onmen , which co e s wo main
asks:
1. Moni o ing: The moni o ing o he ac i i y a eas is used as co-manipula ion eedback
and suppo o wo ke s in isk.
2. In elligen isk managemen : Robo /Human (R/H) beha iou managemen , obo
con ol and wo ke suppo .
Figu e 5. Diag am o he Sma Managemen Sys em.
The main componen s o he sys em a e:
1.
The in elligen managemen module: he co e o his module consis s o he WHO and
isual en i onmen ML models (Suppo Vec o Machines, SVM), and an in elligen
agen . This las componen will manage he condi ions o he wo kplace by sending
ins uc ions and in o ma ion o he con ol module.
2.
The con ol module: This componen adap s he Cobo beha iou and gi es suppo
o he wo ke s o a success ul and coo dina ed e olu ion o he ask imp o ing he
quali y o he p ocess. This eal- ime con ol and suppo will be essen ial o heal h
and sa e y wo k condi ions.
3.
The pe cep ion encode . This componen manages in eal- ime he isual in o ma ion
ha en iches he in elligen managemen module.
3.2.3. WHO Heal h Wo kplace Model
HUMANISE also egula es condi ions by a sma managemen sys em, which in e-
g a es ML modelling based on WHO’s heal h wo kplace model (Figu e 1) ha includes:
1. Heal h and sa e y in he physical wo k en i onmen .
2. Heal h, sa e y, and well-being in he psychosocial wo k en i onmen .
3. Pe sonal heal h esou ces in he wo kplace.
4. En e p ise communi y esou ces.
3.2.4. Moni o ing and Con ol o he Cobo En i onmen
Figu e 6shows he Wo k low o he Cobo en i onmen , which co e s wo main asks:
Senso s 2023,23, 1170 8 o 18
1.
Moni o ing: The moni o ing o he ac i i y a eas is used as co-manipula ion eedback
and suppo o wo ke s in isk.
2.
In elligen isk managemen : Robo /Human (R/H) beha iou managemen , obo
con ol and wo ke suppo .
Senso s 2023, 23, x FOR PEER REVIEW 8 o 18
Figu e 6. Wo k low o he Cobo en i onmen : moni o ing and con ol, showing he s eps o image
acquisi ion, p e-p ocessing, da a ea men , Human/Robo (HR) beha iou analysis and in elligen
managemen o he en i onmen h ough WHO models o suppo he wo ke s.
The main ac i i ies in he p ocess a e:
• Cobo en i onmen moni o ing: he main in o ma ion is p o ided as eedback.
• Da a/image acquisi ion and p e-p ocessing: acquisi ion, bina iza ion and il e ing o
each ame, de ec ion o ac i i y a eas and ex ac ion o ac i i y cen oids o each
a ea.
• T ajec o y gene a ion: (i) o moni o ing, ex ac ion o n egions o in e es (ROIs) o
he ac i i y. (ii) o isk managemen , ajec o y gene a ion and ajec o y ea u e ex-
ac ion.
• Human/Robo (HR) isk de ec ion: HR beha iou analysis and in eg a ion wi h he
WHO models.
• Cobo en i onmen in elligen managemen : in elligen managemen o he en i on-
men by he agen wi h in o ma ion o he WHO models. Robo con ol by he in o -
ma ion o coo dina ion.
• Wo ke suppo : he sys em will p o ide suppo and also eedback by he moni o -
ing sys em.
In he nex sec ions he main ac i i ies a e desc ibed in de ail.
Da a Acquisi ion and P e-P ocessing
The moni o ing ideo sequences we e acqui ed by a high-quali y came a, and p o-
cessed by a cus om oolbox in MATLAB in o de o c ea e ac i i y ime-se ies [36–38].
Then, du ing he p e-p ocessing he ac i i y a eas (mo ion de ec ion) we e calcula ed
based on a ame di e ence me hod (F ame absolu e di e ence) be ween wo consecu i e
ames. Ac i i y de ec ion and mo ion es ima ion we e pe o med in o de o de ec HR
mo ing and simul aneously elimina e backg ound, noise and a i ac s.
In he nex s ep, images we e di ided in o n egions o in e es (ROI) in o de o mon-
i o each wi h he mean ac i i y o he pixels wi hin he ROI o eal ime p ocessing, and
con e ed o bina y images, and hen he main ac i i y a eas we e de ined applying mo -
phological il e s o educe he noise and o smoo hing. Moni o ing o he Cobo en i on-
men ac i i y: (a) Image o ac i i y di e ence in wo consecu i e ames. (b) Ac i i y (A)
in a ame. (c) ∆A in a ame (d) ∆∆A in a ame.
Fea u e Ex ac ion: T ajec o y Analysis
The e olu ion and coo dina ion o he componen s in he sys em (Cobo s and human)
was modelled by he ajec o ies o N ac i i y cen oids along ime. In his sense as in [36–
38] k-means algo i hm was selec ed because he cen oids need o cap u e he in o ma ion
p o ided by he main ac i i y a eas in o de o analyse he en i onmen coo dina ion. In
addi ion, as in p e ious wo ks [36–40] he e was essen ial knowledge on he ask (R/H
Figu e 6.
Wo k low o he Cobo en i onmen : moni o ing and con ol, showing he s eps o image
acquisi ion, p e-p ocessing, da a ea men , Human/Robo (HR) beha iou analysis and in elligen
managemen o he en i onmen h ough WHO models o suppo he wo ke s.
The main ac i i ies in he p ocess a e:
•Cobo en i onmen moni o ing: he main in o ma ion is p o ided as eedback.
•
Da a/image acquisi ion and p e-p ocessing: acquisi ion, bina iza ion and il e ing o
each ame, de ec ion o ac i i y a eas and ex ac ion o ac i i y cen oids o
each a ea
.
•
T ajec o y gene a ion: (i) o moni o ing, ex ac ion o n egions o in e es (ROIs) o he
ac i i y. (ii) o isk managemen , ajec o y gene a ion and ajec o y
ea u e ex ac ion
.
•
Human/Robo (HR) isk de ec ion: HR beha iou analysis and in eg a ion wi h he
WHO models.
•
Cobo en i onmen in elligen managemen : in elligen managemen o he en i-
onmen by he agen wi h in o ma ion o he WHO models. Robo con ol by he
in o ma ion o coo dina ion.
•
Wo ke suppo : he sys em will p o ide suppo and also eedback by he
moni o ing sys em.
In he nex sec ions he main ac i i ies a e desc ibed in de ail.
Da a Acquisi ion and P e-P ocessing
The moni o ing ideo sequences we e acqui ed by a high-quali y came a, and p o-
cessed by a cus om oolbox in MATLAB in o de o c ea e ac i i y ime-se ies [
36
–
38
]. Then,
du ing he p e-p ocessing he ac i i y a eas (mo ion de ec ion) we e calcula ed based on
a ame di e ence me hod (F ame absolu e di e ence) be ween wo consecu i e ames.
Ac i i y de ec ion and mo ion es ima ion we e pe o med in o de o de ec HR mo ing
and simul aneously elimina e backg ound, noise and a i ac s.
In he nex s ep, images we e di ided in o n egions o in e es (ROI) in o de o
moni o each wi h he mean ac i i y o he pixels wi hin he ROI o eal ime p ocessing,
and con e ed o bina y images, and hen he main ac i i y a eas we e de ined applying
mo phological il e s o educe he noise and o smoo hing. Moni o ing o he Cobo
en i onmen ac i i y: (a) Image o ac i i y di e ence in wo consecu i e ames.
(b) Ac i i y
(A) in a ame. (c) ∆A in a ame (d) ∆∆A in a ame.
Fea u e Ex ac ion: T ajec o y Analysis
The e olu ion and coo dina ion o he componen s in he sys em (Cobo s and hu-
man) was modelled by he ajec o ies o N ac i i y cen oids along ime. In his sense
Senso s 2023,23, 1170 9 o 18
as
in [36–38]
k-means algo i hm was selec ed because he cen oids need o cap u e he
in o ma ion p o ided by he main ac i i y a eas in o de o analyse he en i onmen coo -
dina ion. In addi ion, as in p e ious wo ks [
36
–
40
] he e was essen ial knowledge on he
ask (R/H ac i i y dis ibu ed by a eas, beha iou ea u es,) and also he e we e c i ical
equi emen s o eal- ime. Thus, wi hin each ame in he bina ized image, he coo dina es
o he cen es o all he ac i i y a eas we e calcula ed and hen k-means was applied o ind
he cen oids o he ac i i y clus e s (cen es, wi h x and y coo dina es).
Finally, he ac i i y was analysed in eal- ime by he ollowing ea u es o each
cen oid and each coo dina e (x and y):
•Numbe o ac i i y a eas.
•Veloci y, and speed (∆).
•Accele a ion (∆∆).
A da ase was c ea ed wi h he ea u es o he ajec o ies o he cen oids gene a ed
o all he ames ( ime-se ies).
HR Beha iou Modelling and Risk Managemen
We will de elop a hypo hesis-based modelling. In his sense, he modelling is based
on he e olu ion and coo dina ion o he sys em ha is ep esen ed by he e olu ion
and coo dina ion o N cen oids. Speci ically, In he UCs o his con olled scena io,
he e could be wo cen oids mainly ocused on he wo ke and he Cobo . Thus, ML
classi ie algo i hms we e selec ed aking in o accoun he knowledge abou he ask and
he obus ness o hese classi ie s o eal- ime and noise en i onmen equi emen s o ien ed
o a bina y case [
41
–
46
]. The scena io’s classes o modelling a e he numbe o cen oids
(NC =2, C1 and C2).
Then in o de o model he sys em in eal- ime condi ions whe e he pe o mance ime
is c i ical, wo classi ie s we e used:
1. Suppo Vec o Machines (SVM).
2.
Mul ilaye Pe cep on (MLP) de ined by L laye s o N neu ons and he Numbe o
Neu ons in Hidden Laye s (NNHL): L = 1 and NNHL= (numbe o ea u es + classes
numbe )/2.
Finally, he in o ma ion p o ided by he ML models is in eg a ed wi h he WHO
models o he wo ke , and an in elligen agen manages he coo dina ion and isk le els.
In addi ion, his las componen will manage he condi ions o he wo kplace by sending
ins uc ions and in o ma ion o he con ol module and wo ke suppo o eedback
and coo dina ion.
In elligen Managemen o he Cobo En i onmen and Wo ke Suppo
Subsequen ly, hese ea u es we e managed in he con ol module, his module adap s
hese ea u es o con ol he obo beha iou . Then, in eg a ing he p e ious ea u es in he
models, he in elligen agen sends he in o ma ion o manage he ac ional and classical
P opo ional-In eg al-De i a i e (PID) con ol algo i hms ha we e used in he con ol
module o adap he Cobo beha io and o suppo he wo ke in o de o adap he wo k
o a success ul e olu ion o he ask.
Sa e y s anda ds a e included in he WHO model in he En e p ise communi y e-
sou ces. The sa e y s anda ds ISO 10218-1:2012 and ISO 10218-2:2012 desc ibe he basic
isks associa ed o obo s and p o ide equi emen s o elimina e o adequa ely educe he
haza ds associa ed wi h hese isks. Cobo s mus also comply wi h ISO/TS 15066:2016
ha speci ies sa e y equi emen s o collabo a i e indus ial obo sys ems and he wo k
en i onmen , and supplemen s he equi emen s and guidance on collabo a i e indus ial
obo ope a ion gi en in ISO 10218-1 and ISO 10218-2 [
47
,
48
]. In addi ion, he wo ke can
e iew he ac i i y moni o ing (Figu e 7) and ecei es suppo and in o ma ion o adap
he /his beha iou o imp o e he ask pe o mance.
Senso s 2023,23, 1170 16 o 18
Au ho Con ibu ions:
K.L.-d.-I. designed he esea ch, designed and de eloped he ea u es and
he ma e ials, so wa e and sys em, pe o med he expe imen s, analysed he esul s, p epa ed
he igu es, and w o e he manusc ip . K.L.-d.-I., J.I., P.M.C. and E.F. de eloped he so wa e and
da ase , analysed he esul s and p epa ed he igu es. P.D., I.V. and D.S. designed and de eloped
he ea u es, so wa e and ma e ials and analysed he esul s. C.R. and A.P. analysed he esul s and
w o e he manusc ip . D.S. de eloped he so wa e and da ase , pe o med he expe imen s and
analysed he esul s. J.S. designed he sys em, analysed he esul s, p epa ed he igu es, and w o e
he manusc ip . E.S. designed he sys em, and w o e he manusc ip . All au ho s ha e ead and
ag eed o he published e sion o he manusc ip .
Funding:
This wo k is also based upon wo k om COST Ac ions CA18106 suppo ed by COST
(Eu opean Coope a ion in Science and Technology) and he Basque Go e nmen g an s, IT1489-22,
ELKARTEK21/109 and EUSK22/17.
In o med Consen S a emen :
In o med consen was ob ained om all subjec s in ol ed in
he s udy.
Da a A ailabili y S a emen :
The da ase s gene a ed by and/o analysed du ing he cu en s udy
a e no publicly a ailable due o e hics and p i acy equi emen s, bu hey a e a ailable om he
co esponding au ho upon easonable eques .
Acknowledgmen s:
This wo k is suppo ed in pa by he Uni e sidad del País Vasco/Euskal He iko
Unibe si a ea COLAB22/15, he Uni e si y o Camb idge, he Basque Go e nmen , Enginee ing
and Socie y and Bioenginee ing Resea ch G oups, GIC18/136, and ELKARTEK 18/99, 20/81, 21/16),
“Minis e io de Ciencia e Inno ación” (SAF2016 77758 R), FEDER unds, DomusVi, Founda ion
(FP18/76), Go e nmen o Gipuzkoa (DG19/29, DG20/25 p ojec s) and SHERLOCK 820689.
Con lic s o In e es : The au ho s decla e no con lic o in e es .
Re e ences
1.
Suppo ing Ac i e Ageing be o e Re i emen : A Sys ema ic Re iew and Me a-Analysis o Wo kplace Physical Ac i i y In e en-
ions Ta ge ing Olde Employees|BMJ Open. A ailable online: h ps://bmjopen.bmj.com/con en /11/6/e045818 (accessed on
7 Janua y 2023).
2.
Heal hy Wo kplaces: A Model o Ac ion. A ailable online: h ps://www.who.in /publica ions-de ail- edi ec /heal hy-
wo kplaces-a-model- o -ac ion (accessed on 7 Janua y 2023).
3.
OSH Managemen in he Con ex o an Ageing Wo k o ce|Sa e y and Heal h a Wo k EU-OSHA. A ailable online: h ps:
//osha.eu opa.eu/en/ hemes/osh-managemen -con ex -ageing-wo k o ce (accessed on 7 Janua y 2023).
4.
Eu opean Agency o Sa e y & Heal h a Wo k—In o ma ion, S a is ics, Legisla ion and Risk Assessmen Tools. A ailable online:
h ps://osha.eu opa.eu/en (accessed on 7 Janua y 2023).
5. Chen, B.; Vond ick, C.; Lipson, H. Visual Beha io Modelling o Robo ic Theo y o Mind. Sci. Rep. 2021,11, 424. [C ossRe ]
6.
IFR P esen s Wo ld Robo ics Repo 2020—In e na ional Fede a ion o Robo ics. A ailable online: h ps://i .o g/i -p ess-
eleases/news/ eco d-2.7-million- obo s-wo k-in- ac o ies-a ound- he-globe (accessed on 7 Janua y 2023).
7.
Pe e s, B.S.; A mijo, P.R.; K ause, C.; Choudhu y, S.A.; Oleyniko , D. Re iew o Eme ging Su gical Robo ic Technology. Su g.
Endosc. 2018,32, 1636–1655. [C ossRe ] [PubMed]
8. Riek, L.D. Heal hca e Robo ics. Commun. ACM 2017,60, 68–78. [C ossRe ]
9.
Ducke , T.; Pea son, S.; Blackmo e, S.; G ie e, B.; Chen, W.-H.; Cielniak, G.; Clea e smi h, J.; Dai, J.; Da is, S.; Fox, C.; e al.
Ag icul u al Robo ics: The Fu u e o Robo ic Ag icul u e 2018. A ailable online: h ps://uwe- eposi o y.wo k ibe.com/ou pu /
866226 (accessed on 18 Janua y 2023).
10.
EUni ed Robo ics—Eu opean Robo ics Associa ion—Eu opean Robo ics Indus y. A ailable online: h ps://www.eu-ni ed.ne /
euni ed+aisbl/ obo ics/euni ed- obo ics-eu opean- obo ics-associa ion-.h ml (accessed on 7 Janua y 2023).
11.
Vinanzi, S.; Cangelosi, A.; Goe ick, C. The Collabo a i e Mind: In en ion Reading and T us in Human-Robo In e ac ion. iScience
2021,24, 102130. [C ossRe ] [PubMed]
12.
Xu, T.; Fan, J.; Chen, Y.; Ng, X.; Ang, M.H.; Fang, Q.; Zhu, Y.; Zhao, J. Dynamic Iden i ica ion o he KUKA LBR Iiwa Robo wi h
Re ie al o Physical Pa ame e s Using Global Op imiza ion. IEEE Access 2020,8, 108018–108031. [C ossRe ]
13.
He e o, H.; Ou ón, J.L.; Pue o, M.; Sallé, D.; López de Ipiña, K. Enhanced Flexibili y and Reusabili y h ough S a e Machine-
Based A chi ec u es o Mul isenso In elligen Robo ics. Senso s 2017,17, 1249. [C ossRe ]
14.
Iba gu en, A.; Eimon ai e, I.; Ou ón, J.L.; Fle che , S. Dual A m Co-Manipula ion A chi ec u e wi h Enhanced Human–Robo
Communica ion o La ge Pa Manipula ion. Senso s 2020,20, 6151. [C ossRe ]
15.
Iba gu en, A.; Daelman, P. Pa h D i en Dual A m Mobile Co-Manipula ion A chi ec u e o La ge Pa Manipula ion in Indus ial
En i onmen s. Senso s 2021,21, 6620. [C ossRe ] [PubMed]
16. Home. A ailable online: h ps://www.she lock-p ojec .eu/ (accessed on 7 Janua y 2023).
Senso s 2023,23, 1170 17 o 18
17.
Dju ic, A.M.; U banic, R.J.; Rickli, J.L. A F amewo k o Collabo a i e Robo (CoBo ) In eg a ion in Ad anced Manu ac u ing
Sys ems. SAE In . J. Ma e . Manu . 2016,9, 457–464. [C ossRe ]
18.
Ajoudani, A.; Zanche in, A.M.; I aldi, S.; Albu-Schä e , A.; Kosuge, K.; Kha ib, O. P og ess and P ospec s o he Human–Robo
Collabo a ion. Au on Robo . 2018,42, 957–975. [C ossRe ]
19.
Villani, V.; Pini, F.; Leali, F.; Secchi, C. Su ey on Human–Robo Collabo a ion in Indus ial Se ings: Sa e y, In ui i e In e aces
and Applica ions. Mecha onics 2018,55, 248–266. [C ossRe ]
20.
F anklin, C.S.; Dominguez, E.G.; F yman, J.D.; Lewandowski, M.L. Collabo a i e obo ics: New e a o human- obo coope a ion
in he wo kplace. J. Sa e y Res. 2020,74, 153–160. [C ossRe ] [PubMed]
21.
RIA TR R15.606-2016—Technical Repo —Indus ial Robo s and Robo Sys ems—Sa e y Requi emen s—Collabo a i e Robo s.
A ailable online: h ps://webs o e.ansi.o g/s anda ds/ ia/ ia 156062016 (accessed on 7 Janua y 2023).
22.
Gual ie i, M.; Pas, A. en; Pla , R. Pick and Place Wi hou Geome ic Objec Models 2018. A ailable online: h ps://a xi .o g/
abs/1707.05615 (accessed on 18 Janua y 2023).
23.
Song, D.; Goldbe g, K.Y. App oxima e Algo i hms o a Collabo a i ely Con olled Robo ic Came a. IEEE T ans. Robo .
2007
,23,
1061–1070. [C ossRe ]
24.
The In e disciplina y Handbook o Pe cep ual Con ol Theo y—1s Edi ion. A ailable online: h ps://www.else ie .com/books/
he-in e disciplina y-handbook-o -pe cep ual-con ol- heo y/mansell/978-0-12-818948-1 (accessed on 7 Janua y 2023).
25.
Laso a, P.A.; Fong, T.; Shah, J.A. A Su ey o Me hods o Sa e Human-Robo In e ac ion. FNT Robo .
2017
,5, 261–349. [C ossRe ]
26.
Kanno, T.; Naka a, K.; Fu u a, K. A Me hod o Team In en ion In e ence. In . J. Hum. Compu . S ud.
2003
,58, 393–413. [C ossRe ]
27.
Choudhu y, R.; Swamy, G.; Had ield-Menell, D.; D agan, A.D. On he U ili y o Model Lea ning in HRI. In P oceedings o he 2019
14 h ACM/IEEE In e na ional Con e ence on Human-Robo In e ac ion (HRI), Daegu, Ko ea, 11–14 Ma ch 2019; pp. 317–325.
28.
Kelley, R.; Wigand, L.; Hamil on, B.; B owne, K.; Nicolescu, M.; Nicolescu, M. Deep Ne wo ks o P edic ing Human In en
wi h Respec o Objec s. In P oceedings o he P oceedings o he Se en h Annual ACM/IEEE In e na ional Con e ence on
Human-Robo In e ac ion, Associa ion o Compu ing Machine y, New Yo k, NY, USA, 5 Ma ch 2012; pp. 171–172.
29.
Pé ez-D’A pino, C.; Shah, J.A. Fas Ta ge P edic ion o Human Reaching Mo ion o Coope a i e Human-Robo Manipula ion
Tasks Using Time Se ies Classi ica ion. In P oceedings o he 2015 IEEE In e na ional Con e ence on Robo ics and Au oma ion
(ICRA), Sea le, WA, USA, 26–30 May 2015; pp. 6175–6182.
30.
Thompson, S.; Ho iuchi, T.; Kagami, S. A P obabilis ic Model o Human Mo ion and Na iga ion In en o Mobile Robo Pa h
Planning. In P oceedings o he 2009 4 h In e na ional Con e ence on Au onomous Robo s and Agen s, Welling on, New Zealand,
10–12 Feb ua y 2009; pp. 663–668.
31.
Lube , M.; S o k, J.A.; Tipaldi, G.D.; A as, K.O. People T acking wi h Human Mo ion P edic ions om Social Fo ces. In
P oceedings o he 2010 IEEE In e na ional Con e ence on Robo ics and Au oma ion, Ancho age, AK, USA, 3–7 May 2010;
pp. 464–469.
32.
Kuma , S.; Sa u , C.; Sahin, F. Su ey o Human–Robo Collabo a ion in Indus ial Se ings: Awa eness, In elligence, and
Compliance. IEEE T ans. Sys . Man Cybe n. Sys . 2021,51, 280–297. [C ossRe ]
33.
Ma el, J.A. Pe o mance Me ics o Speed and Sepa a ion Moni o ing in Sha ed Wo kspaces. IEEE T ans. Au om. Sci. Eng.
2013
,
10, 405–414. [C ossRe ]
34.
LBR Liwa. A ailable online: h ps://www.kuka.com/en-de/p oduc s/ obo -sys ems/indus ial- obo s/lb -iiwa (accessed on
7 Janua y 2023).
35.
Calinon, S. Lea ning om Demons a ion (P og amming by Demons a ion). In Encyclopedia o Robo ics; Ang, M.H., Kha ib, O.,
Siciliano, B., Eds.; Sp inge : Be lin/Heidelbe g, Ge many, 2018; pp. 1–8. ISBN 978-3-642-41610-1.
36.
Lopez-de-Ipina, K.; Solé-Casals, J.; Sánchez-Méndez, J.I.; Rome o-Ga cia, R.; Fe nandez, E.; Requejo, C.; Poologaind an, A.;
Faúndez-Zanuy, M.; Ma í-Massó, J.F.; Be ga eche, A.; e al. Analysis o Fine Mo o Skills in Essen ial T emo : Combining
Neu oimaging and Handw i ing Bioma ke s o Ea ly Managemen . F on . Hum. Neu osci. 2021,15, 648573. [C ossRe ]
37.
López-de-Ipiña, K.; Cepeda, H.; Requejo, C.; Fe nandez, E.; Cal o, P.M.; La uen e, J.V. Machine Lea ning Me hods o
En i onmen al-En ichmen -Rela ed Va ia ions in Beha io al Responses o Labo a o y Ra s. In P oceedings o he Unde s anding he
B ain Func ion and Emo ions; Fe ández Vicen e, J.M., Ál a ez-Sánchez, J.R., de la Paz López, F., Toledo Mo eo, J., Adeli, H., Eds.;
Sp inge In e na ional Publishing: Cham, Swi ze land, 2019; pp. 420–427.
38.
Egui aun, H.; López-de-Ipiña, K.; Ma inez, I. Applica ion o En opy and F ac al Dimension Analyses o he Pa e n Recogni ion
o Con amina ed Fish Responses in Aquacul u e. En opy 2014,16, 6133–6151. [C ossRe ]
39.
Requejo, C.; López-de-Ipiña, K.; Ruiz-O ega, J.Á.; Fe nández, E.; Cal o, P.M.; Mo e a-He e as, T.; Miguelez, C.;
Ca dona-G i oll, L.
; Cepeda, H.; Ugedo, L.; e al. Changes in Day/Nigh Ac i i y in he 6-OHDA-Induced Expe imen-
al Model o Pa kinson’s Disease: Explo ing P od omal Bioma ke s. F on . Neu osci. 2020,14, 590029. [C ossRe ]
40. Tabianan, K.; Velu, S.; Ra i, V. K-Means Clus e ing App oach o In elligen Cus ome Segmen a ion Using Cus ome Pu chase
Beha io Da a. Sus ainabili y 2022,14, 7243. [C ossRe ]
41.
Bue kle, A.; Ma ha u, H.; Al-Yacoub, A.; Lohse, N.; Bambe , T.; Fe ei a, P. An Adap i e Human Senso F amewo k o
Human–Robo Collabo a ion. In . J. Ad . Manu . Technol. 2022,119, 1233–1248. [C ossRe ]
42.
Sazono , E.; Hegde, N.; B owning, R.C.; Melanson, E.L.; Sazono a, N.A. Pos u e and Ac i i y Recogni ion and Ene gy Expendi u e
Es ima ion in a Wea able Pla o m. IEEE J. Biomed. Heal h In o m. 2015,19, 1339–1346. [C ossRe ]
Senso s 2023,23, 1170 18 o 18
43.
Me y, K.J.; Macdonald, E.; MacPhe son, M.; Aziz, O.; Pa k, E.; Ryan, M.; Spa ey, C.J. Classi ying Si ing, S anding, and Walking
Using Plan a Fo ce Da a. Med. Biol. Eng. Compu . 2021,59, 257–270. [C ossRe ]
44.
Jeong, G.-M.; T uong, P.H.; Choi, S.-I. Classi ica ion o Th ee Types o Walking Ac i i ies Rega ding S ai s Using Plan a P essu e
Senso s. IEEE Sens. J. 2017,17, 2638–2639. [C ossRe ]
45.
Liu, B.; Fu, W.; Wang, W.; Li, R.; Gao, Z.; Peng, L.; Du, H. Cobo Mo ion Planning Algo i hm o Ensu ing Human Sa e y Based on
Beha io al Dynamics. Senso s 2022,22, 4376. [C ossRe ]
46. Asaly, S.; Go lieb, L.-A.; Inba , N.; Reu eni, Y. Using Suppo Vec o Machine (SVM) wi h GPS Ionosphe ic TEC Es ima ions o
Po en ially P edic Ea hquake E en s. Remo e Sens. 2022,14, 2822. [C ossRe ]
47. 14:00-17:00 ISO 10218-1:2011. A ailable online: h ps://www.iso.o g/s anda d/51330.h ml (accessed on 7 Janua y 2023).
48.
Ma ine i, A.; Chemweno, P.K.; Nizamis, K.; Fosch-Villa onga, E. Rede ining Sa e y in Ligh o Human-Robo In e ac ion:
A C i ical Re iew o Cu en S anda ds and Regula ions. F on . Chem. Eng. 2021,3. [C ossRe ]
49.
Aswad, F.E.; Djogdom, G.V.T.; O is, M.J.-D.; Ayena, J.C.; Meziane, R. Image Gene a ion o 2D-CNN Using Time-Se ies Signal
Fea u es om Foo Ges u e Applied o Selec Cobo Ope a ing Mode. Senso s 2021,21, 5743. [C ossRe ]
50.
Tuan, H.M.; San ilippo, F.; Hao, N.V. Modelling and Con ol o a 2-DOF Robo A m wi h Elas ic Join s o Sa e Human-Robo
In e ac ion. F on . Robo . AI 2021,8, 223. [C ossRe ]
51. Baue , A.; Wollhe , D.; Buss, M. Human–Robo Collabo a ion: A Su ey. In . J. Human. Robo . 2008,05, 47–66. [C ossRe ]
52.
Moka am, S.; Ai ken, J.M.; Ma inez-He nandez, U.; Eimon ai e, I.; Came on, D.; Rolph, J.; Gwil , I.; McA ee, O.; Law, J.
A ROS-In eg a ed
API o he KUKA LBR Iiwa Collabo a i e Robo **The Au ho s Acknowledge Suppo om he EPSRC Cen e
o Inno a i e Manu ac u ing in In elligen Au oma ion, in Unde aking This Resea ch Wo k unde G an Re e ence Numbe
EP/I033467/1, and he Uni e si y o She ield Impac , Inno a ion and Knowledge Exchange G an “Human Robo In e ac ion
De elopmen ”. Equipmen Has Been P o ided unde he EPSRC G ea Technologies Capi al Call: Robo ics and Au onomous
Sys ems. IFAC-Pape sOnLine 2017,50, 15859–15864. [C ossRe ]
53.
Nagy, T.D.; Haidegge , T. Pe o mance and Capabili y Assessmen in Su gical Sub ask Au oma ion. Senso s
2022
,22, 2501.
[C ossRe ] [PubMed]
54.
B i o, T.; Quei oz, J.; Pia di, L.; Fe nandes, L.A.; Lima, J.; Lei ão, P. A Machine Lea ning App oach o Collabo a i e Robo Sma
Manu ac u ing Inspec ion o Quali y Con ol Sys ems. P ocedia Manu . 2020,51, 11–18. [C ossRe ]
55. Czubenko, M.; Kowalczuk, Z. A Simple Neu al Ne wo k o Collision De ec ion o Collabo a i e Robo s. Senso s 2021,21, 4235.
[C ossRe ]
56.
Khawaja, F.I.; Kanazawa, A.; Kinugawa, J.; Kosuge, K. A Human-Following Mo ion Planning and Con ol Scheme o Collabo a-
i e Robo s Based on Human Mo ion P edic ion. Senso s 2021,21, 8229. [C ossRe ]
57.
Kanazawa, A.; Kinugawa, J.; Kosuge, K. Adap i e Mo ion Planning o a Collabo a i e Robo Based on P edic ion Unce ain y o
Enhance Human Sa e y and Wo k E iciency. IEEE T ans. Robo . 2019,35, 817–832. [C ossRe ]
58.
Sa eea, M.; Ne o, P.; Bea ee, R. On-Line Collision A oidance o Collabo a i e Robo Manipula o s by Adjus ing o -Line
Gene a ed Pa hs: An Indus ial Use Case. Robo . Au on. Sys . 2019,119, 278–288. [C ossRe ]
59.
Pauliko á, A.; Gyu ák Babel’o á, Z.; Ubá o á, M. Analysis o he Impac o Human–Cobo Collabo a i e Manu ac u ing
Implemen a ion on he Occupa ional Heal h and Sa e y and he Quali y Requi emen s. In . J. En i on. Res. Public Heal h
2021
,
18, 1927. [C ossRe ]
60.
Ge lo , C.; Kon ad, K.; Bzdok, D.; Büsing, C.; Reindl, V. In e ac ing B ains Re isi ed: A C oss-B ain Ne wo k Neu oscience
Pe spec i e. Hum. B ain Mapp. 2022,43, 4458–4474. [C ossRe ]
61.
Chen, P.-H.A.; Qu, Y. Taking a Compu a ional Cul u al Neu oscience App oach o S udy Pa en -Child Simila i ies in Di e se
Cul u al Con ex s. F on . Hum. Neu osci. 2021,15, 703999. [C ossRe ] [PubMed]
62.
McDu , D.; el Kaliouby, R.; Senechal, T.; Am , M.; Cohn, J.F.; Pica d, R. A ec i a-MIT Facial Exp ession Da ase (AM-FED):
Na u alis ic and Spon aneous Facial Exp essions Collec ed “In- he-Wild”. In P oceedings o he 2013 IEEE Con e ence on
Compu e Vision and Pa e n Recogni ion Wo kshops, Po land, OR, USA, 23–28 June 2013; pp. 881–888.
63.
Mollahosseini, A.; Hasani, B.; Mahoo , M.H. A ec Ne : A Da abase o Facial Exp ession, Valence, and A ousal Compu ing in he
Wild. IEEE T ans. A ec . Compu . 2019,10, 18–31. [C ossRe ]
64.
Jiang, Z.; Ha a i, S.; C owell, A.; Maybe g, H.S.; Nema i, S.; Cli o d, G.D. Classi ying Majo Dep essi e Diso de and Response
o Deep B ain S imula ion O e Time by Analyzing Facial Exp essions. IEEE T ans. Biomed. Eng.
2021
,68, 664–672. [C ossRe ]
[PubMed]
65.
Wha Is Uni e sal Design|Cen e o Excellence in Uni e sal Design. A ailable online: h ps://uni e saldesign.ie/Wha -is-
Uni e sal-Design/ (accessed on 7 Janua y 2023).
Disclaime /Publishe ’s No e:
The s a emen s, opinions and da a con ained in all publica ions a e solely hose o he indi idual
au ho (s) and con ibu o (s) and no o MDPI and/o he edi o (s). MDPI and/o he edi o (s) disclaim esponsibili y o any inju y o
people o p ope y esul ing om any ideas, me hods, ins uc ions o p oduc s e e ed o in he con en .
