Oc obe 2022
Bea iz Mi anda
A new app oach o s udy gai impai men s in
Pa kinson’s disease based on mixed eali y
Bea iz Ma ia Redondo Mi anda
A new app oach o s udy gai
impai men s in Pa kinson’s
disease based on mixed eali y
Oc obe 2022
i
Bea iz Ma ia Redondo Mi anda
A new app oach o s udy gai
impai men s in Pa kinson’s disease
based on mixed eali y
Mas e disse a ion
Mas e Deg ee in Biomedical Enginee ing
Medical Elec onics
Disse a ion supe ised by
P o esso Doc o C is ina Manuela Peixo o dos San os
Oc obe 2022
DIREITOS DE AUTOR E CONDIÇÕES DE UTILIZAÇÃO DO TRABALHO
POR TERCEIROS
Es e é um abalho académico que pode se u ilizado po e cei os desde que espei adas as eg as e
boas p á icas in e nacionalmen e acei es, no que conce ne aos di ei os de au o e di ei os conexos.
Assim, o p esen e abalho pode se u ilizado nos e mos p e is os na licença abaixo indicada.
Caso o u ilizado necessi e de pe missão pa a pode aze um uso do abalho em condições não p e is as
no licenciamen o indicado, de e á con ac a o au o , a a és do Reposi ó iUM da Uni e sidade do Minho.
A ibuição-NãoCome cial-SemDe i ações
CC BY-NC-ND
h ps://c ea i ecommons.o g/licenses/by-nc-nd/4.0/
ii
AGRADECIMENTOS
Es a disse ação, desen ol ida du an e o úl imo ano, oi o esul ado de mui o abalho e es o ço que
nunca pode ia e acon ecido sem a con ibuição di e a e indi e a de mui as pessoas que me são mui o
que idas, es ando-me ag adece -lhes.
Em p imei o luga , que o ag adece à minha o ien ado a, P o esso a C is ina, pela opo unidade que
me deu em abalha nes e p oje o, po oda a mo i ação e o ien ação que oi dando, e ainda po odas
as euniões de in a minu os que se p olonga am po duas ho as, po que nem udo é abalho. Admi o-
a como p o issional, mulhe e mãe. Es ou ex emamen e ag adecida po oda a sua dedicação e supo e.
De seguida, a Helena. Pessoa que mais me acompanhou du an e es e ano e cé eb o do p oje o
+sense. Que o ag adece -lhe po odas as ho as em chamada mesmo após e mos passado o dia no
labo a ó io, po me e ajudado na ecolha de dados no hospi al, po me ensina li e almen e udo, a é
ecei as de bolo da caneca. A paciência, ajuda, dedicação e mo i ação mos adas não são mensu á eis.
À minha pa cei a de mes ado Ma a, o 4º ano oi uma ba alha encida ao lado dela. Ag adeço pelas
incon á eis noi es a abalha e a can a e as e ças- ei as no labo a ó io. A odos os meus amigos da
uni e sidade, ag adeço não só a companhia nas aulas mais dolo osas como odas as b incadei as, saídas
à noi e e po me e em dado a conhece mais deles e das e as deles. Ag adeço ambém às minhas
colegas de casa po ou i em odos os meus d amas e à melho pessoa que a uni e sidade me deu, a
minha a ilhada, Ma ia Pimen a, po se ão disponí el e ão amiga.
A odos os meus “amigos de pon e”, B una, Guida, Jucas, Ca oli, Inês, Nelson, Gina, Luces, Jaime,
ob igada po me acompanha em desde o secundá io, po odos os e ões, po odos os ca és de sábado
à noi e e po se em semp e um po o segu o. Apesa de nos conhece mos há mui os anos, ensinam-me
semp e uma coisa no a odas as semanas, nem que seja os no os sabo es da Água das Ped as.
Não menos impo an e, enho de ag adece ao Spo i y, às suas playlis s e a is as, po e em sido os
meus melho es amigos calman es du an e es e pe íodo. A música semp e se á um pedaço de mim,
mesmo que enha seguido o mundo da engenha ia.
Po im ag adeço à minha amília, aos meus pais e ambém à Lua, po se em o meu maio supo e
inancei o e emocional, po me e em dado as melho es condições que podia pedi , pelo in e esse que
mos am pela minha á ea mesmo que não pe cebam semp e. E ambém ao meu i mão po me
emp es a equipamen os e po me ajuda quando a ecnologia não que se minha aliada.
Mui o ob igada a odos.
Bia

iii
STATEMENT OF INTEGRITY
I he eby decla e ha ing conduc ed his academic wo k wi h in eg i y. I con i m ha I ha e no used
plagia ism o any o m o undue use o in o ma ion o alsi ica ion o esul s along he p ocess leading o
i s elabo a ion.
I u he decla e ha I ha e ully acknowledged he Code o E hical Conduc o he Uni e si y o Minho.
i
ABSTRACT
Pa kinson’s disease (PD) is he second mos common neu odegene a i e diso de a e
Alzheime 's disease. PD onse is a 55 yea s-old on a e age, and i s incidence inc eases wi h age. This
disease esul s om dopamine-p oducing neu ons degene a ion in he basal ganglia and is cha ac e ized
by a ious mo o symp oms such as eezing o gai , b adykinesia, hypokinesia, akinesia, and igidi y,
which nega i ely impac pa ien s’ quali y o li e. To moni o and imp o e hese PD- ela ed gai disabili ies,
se e al echnology-based me hods ha e eme ged in he las decades. Howe e , hese solu ions s ill
equi e mo e cus omiza ion o pa ien s’ daily li ing asks in o de o p o ide mo e objec i e, eliable, and
long- e m da a abou pa ien s’ mo o condi ions in home- ela ed con ex s. P o iding his quan i a i e da a
o physicians will ensu e mo e pe sonalised and be e ea men s. Also, mo o ehabili a ion sessions
os e ed by assis ance de ices equi e he inclusion o quo idian asks o ain pa ien s o hei daily
mo o challenges. One o he mos p omising echnology-based me hods is i ual, augmen ed, and mixed
eali y (VR/AR/MR), which imme se pa ien s in i ual en i onmen s and p o ide senso y s imuli (cues)
o assis wi h hese disabili ies. Howe e , u he esea ch is needed o imp o e and concep ualize e icien
and pa ien -cen ed VR/AR/MR app oaches and inc ease hei clinical e idence.
Bea ing his in mind, he main goal o his disse a ion was o design, de elop, es , and alida e
i ual en i onmen s o assess and ain PD- ela ed gai impai men s using mixed eali y sma glasses,
in eg a ed wi h ano he high- echnological mo ion acking de ice. Using speci ic i ual en i onmen s ha
igge PD- ela ed gai impai men s ( u ning, doo ways, and na ow spaces), i is hypo hesized ha
pa ien s can be assessed and ained in hei daily challenges ela ed o walking. Also, his ool in eg a es
on-demand isual cues o p o ide isual bio eedback and os e mo o aining. This solu ion was alida ed
wi h end-use s o es he iden i ied hypo hesis. The esul s showed ha , in ac , mixed eali y has he
po en ial o ec ea e eal-li e en i onmen s ha o en p o oke PD- ela ed gai disabili ies, by placing i ual
objec s on op o he eal wo ld. On he con a y, bio eedback s a egies did no signi ican ly imp o e he
pa ien s’ mo o pe o mance. The use expe ience e alua ion showed ha pa icipan s enjoyed
pa icipa ing in he ac i i y and el ha his ool can help hei mo o pe o mance.
Keywo ds: Pa kinson’s disease; Vi ual eali y; Augmen ed eali y; Mixed Reali y; Rehabili a ion; Gai
disabili ies; Senso y cueing; Bio eedback.
RESUMO
A doença de Pa kinson (DP) é a segunda doença neu odegene a i a mais comum depois da doença
de Alzheime . O início da DP oco e, em média, aos 55 anos de idade, e a sua incidência aumen a com
a idade. Es a doença esul a da degene ação dos neu ónios p odu o es de dopamina nos gânglios basais
e é ca ac e izada po á ios sin omas mo o es como o congelamen o da ma cha, b adicinesia,
hipocinesia, acinesia, e igidez, que a e am nega i amen e a qualidade de ida dos pacien es. Nas úl imas
décadas su gi am mé odos ecnológicos pa a moni o iza e eina es as desabilidades da ma cha. No
en an o, es as soluções ainda eque em uma maio pe sonalização ela i amen e às a e as diá ias dos
pacien es, a im de o nece dados mais obje i os, iá eis e de longo p azo sob e o seu desempenho
mo o em con ex os do dia-a-dia. A a és do o necimen o des es dados quan i a i os aos médicos, se ão
assegu ados a amen os mais pe sonalizados. Além disso, as sessões de eabili ação mo o a,
p omo idas po disposi i os de assis ência, eque em a inclusão de a e as quo idianas pa a eina os
pacien es pa a os seus desa ios diá ios. Um dos mé odos ecnológicos mais p omisso es é a ealidade
i ual, aumen ada e mis a (RV/RA/RM), que ime gem os pacien es em ambien es i uais e o necem
es ímulos senso iais pa a ajuda nes as desabilidades. Con udo, é necessá ia mais in es igação pa a
melho a e concep ualiza abo dagens RV/RA/RM e icien es e cen adas no pacien e e ainda aumen a
as suas e idências clínicas.
Tendo is o em men e, o p incipal obje i o des a disse ação oi concebe , desen ol e , es a e alida
ambien es i uais pa a a alia e eina as incapacidades de ma cha elacionadas com a DP usando
óculos in eligen es de ealidade mis a, in eg ados com ou o disposi i o de as eio de mo imen o.
U ilizando ambien es i uais especí icos que desencadeiam desabilidades da ma cha ( oda , po as e
espaços es ei os), é possí el es a hipó eses de que os pacien es possam se a aliados e einados nos
seus desa ios diá ios. Além disso, es a e amen a in eg a pis as isuais pa a o nece bio eedback isual
e omen a a eabili ação mo o a. Es a solução oi alidada com u ilizado es inais de o ma a es a as
hipó eses iden i icadas. Os esul ados mos a am que, de ac o, a ealidade mis a em o po encial de
ec ia ambien es da ida eal que mui as ezes p o ocam de iciências de ma cha elacionadas à DP.
Pelo con á io, as es a égias de bio eedback não p o oca am melho ias signi ica i as no desempenho
mo o dos pacien es. A a aliação ei a pelos pacien es mos ou que es es gos a am de pa icipa nos
es es e sen i am que es a e amen a pode auxilia no seu desempenho mo o .
Pala as-Cha e: Doença de Pa kinson; Realidade i ual; Realidade aumen ada; Realidade mis a;
Reabili ação; Desabilidades mo o as; Pis as senso iais; Bio eedback.
i
Table o Con en s
1 In oduc ion .............................................................................................................. 1
1.1 Mo i a ion ................................................................................................................... 2
1.2 P oblem s a emen ...................................................................................................... 4
1.3 Goals ........................................................................................................................... 5
1.4 Resea ch Ques ions ...................................................................................................... 6
1.5 Con ibu ions o Knowledge ......................................................................................... 6
1.6 Disse a ion S uc u e .................................................................................................. 7
2 Li e a u e Re iew ...................................................................................................... 8
2.1 In oduc o y Insigh ...................................................................................................... 9
2.2 Me hods .................................................................................................................... 10
2.2.1 Da a sou ces, sea ch s a egy and s udies selec ion ....................................................... 10
2.3 Resul s ....................................................................................................................... 10
2.3.1 Gene al Resul s ................................................................................................................ 10
2.3.2 VR/AR/MR in PD .............................................................................................................. 11
2.3.3 Technology suppo ing VR/AR/MR-based app oaches in PD .......................................... 13
2.3.4 Valida ion me hodology highligh s: pa icipan s, c i e ia s udy, se ing, p o ocols,
Schedule, me ics.......................................................................................................................... 16
2.4 Discussion .................................................................................................................. 27
2.4.1 How ha e he VR/AR/MR-based app oaches been applied in PD o help pa ien s
mi iga e gai disabili ies? .............................................................................................................. 27
2.4.2 Which echnologies ha e been used o suppo VR/AR/MR-based app oaches in PD? . 27
2.4.3 How ha e he VR/AR/MR-based app oaches been clinically alida ed in PD? ............... 28
2.5 Conclusions and Fu u e di ec ions .............................................................................. 29
3 Solu ion O e iew ................................................................................................... 32
3.1 P oblem desc ip ion ................................................................................................... 33
3.2 +sense ....................................................................................................................... 33
3.3 +sImme si e ............................................................................................................... 34
3.3.1 Mixed eali y sma glasses: Mic oso HoloLens 2 ......................................................... 35
3.3.2 Mo ion acking sys em: Xsens MVN Awinda ................................................................. 37
3.4 Conclusions ................................................................................................................ 39
4 Solu ion desc ip ion ................................................................................................ 40
xiii
M
MMSE
Mini-Men al S a e Examina ion
MoCA
Mon eal Cogni i e Assessmen
Mini-
BESTes
Mini-Balance E alua ion Sys ems Tes
MR
Mixed Reali y
M1
Moni o ing es 1 - dice
M2
Moni o ing es 2 - doo
M3
Moni o ing es 3 – na ow spaces
P
PD
Pa kinson’s Disease
PIGD
Pos u al Ins abili y and Gai Diso de
PTF
Pe cen age o ime ozen
PC
Pe sonal compu e
PD-
Pa kinson’s Disease pa ien s wi h eezing o gai ( eeze s)
PDQL
Pa kinson’s Disease Quali y o Li e ques ionnai e
Q
QoL
Quali y o Li e
S
SI
Semi imme si e
SSQ
Simula o Sickness Ques ionnai e
SUS
Sys em Usabili y Scale
SAC
S ess A ousal Checklis
T
T
T aining
TUG
Timed Up and Go es
TC1
Con ol es 1
TC2
Con ol es 2
T1
T aining es 1 – dice wi h a ows
T2
T aining es 2 – doo wi h oo p in s
T3
T aining es 3 – na ow spaces wi h oo p in s
U

xi
UPDRS
Uni ied Pa kinson’s Disease Ra ing Scale
UPSRS-III
Uni ied Pa kinson’s Disease Ra ing Scale pa III
V
VR
Vi ual Reali y
VGoT
Videogame-o ien ed aining
1 INTRODUCTION
2
This disse a ion p esen s he wo k ca ied ou o e he pas yea , in eg a ed in he scope o he
Mas e Deg ee in Biomedical Enginee ing a he Biomedical Robo ic De ices Lab (BiRDLAB) included in
he Cen e o Mic oElec oMechanical Sys ems (CMEMS), a esea ch cen e o he Depa men o
Indus ial Elec onics (DEI) o Uni e si y o Minho.
The p ojec main goal was o de elop and alida e a mixed eali y (MR) ool o he assessmen and
aining o gai disabili ies in Pa kinson’s disease. This solu ion was de eloped o b ing a new pa adigm
shi . Thus, by igge ing PD- ela ed gai impai men s, pa ien s can be assessed and ained in e e yday
si ua ions. The po en ial o mixed eali y, in eg a ed wi h a mo ion acking sys em, o mimic e e yday
en i onmen s, was es ed, aiming a mo e objec i e medical assessmen , and enhanced and mo i a ional
ehabili a ion exe cises. All he s eps pe o med o achie e his solu ion a e de ailed in his documen .
1.1 MOTIVATION
Pa kinson’s disease (PD) is he second mos common neu odegene a i e diso de a e
Alzheime 's disease [1]. I is belie ed ha PD physiopa hology lies in he loss o dopamine-gene a ing
neu ons in he basal ganglia, which is ela ed o human mo emen con ol. The i s dyskinesias appea
when he e is a de iciency in dopamine elease by hese cells [1], [2]. I s a e age onse is a 55 yea s-old
and i s incidence inc eases wi h age [1].
This disease is cha ac e ized by se e al symp oms wi h gai impai men s being he mos common
and disabling ones. Mo o symp oms include eezing o gai (FoG), b adykinesia (mo emen
slowness), hypokinesia ( educed mo emen ampli ude), akinesia (p oblems ini ia ing
mo emen ), es ina ion ( endency o speed up when pe o ming epe i i e mo emen s),
igidi y, pos u al ins abili y, and educed mo emen au oma ici y, all o hem diminishing
pa ien s’ quali y o li e [1]–[7]. Usually, pa ien s s a by educing hei walking eloci y, aking sho e
s eps, and p esen ing some gai asymme ies and e en ually su e ing mo o eezing e en s. FoG is
de ined as he “
sudden inabili y o con inue walking despi e he in en o main ain locomo ion” and “i is
episodic and a iable by na u e
”, being one o he mos debili a ing and di icul impai men s o assess
[3], [4], [7]. FoG e en s a e ypically igge ed by speci ic si ua ions, such as ini ia ion o walking,
u ning du ing s eady-s a e walking, acing objec s, s ess, dis ac ion, and nea ing doo ways. E en i
hese en i onmen s did no igge a comple e momen o gai blockage, hey con ibu ed o a dec ease
in he s ep leng h and eloci y, occu ing es ina ion and akinesias. Mo eo e , hese gai disabili ies can
be agg a a ed by dual- ask a en ional equi emen s [2], [3], [6]–[8].
3
Mo ion acking sys ems make i possible o moni o pa ien s’ mo o unc ion using wea able
senso s [9], while elec omyog aphy sys ems acqui e elec ical muscle ac i i y, e lec ing mo o
luc ua ions [10]. Bo h sys ems a e a he o e on o mo o unc ion’s moni o ing, enabling o ga he
da a wi h low-cos , po able, and minia u ized senso s. I has been possible o moni o kinema ic
and elec omyog aphy-d i en in o ma ion abou pa ien s’ mo o condi ions, such as hei gai
spa io empo al pa ame e s [2], [11], [12], FoG e en s du a ion and occu ence [2], [13]–[15],
muscles ac i i y [10], o pos u al changes [11], [12]. Indeed, his in o ma ion ep esen s i ial
da a o physicians o moni o , o e ime, he mo o s a e o hei pa ien s, con olling he
p og ession o he disease, especially i hey can access hese da a collec ed on pa ien s’ home
en i onmen s, du ing hei daily asks. This would esul in g ea e supe ision o he de elopmen
o he disease and would allow ea men s o be pe sonalised o he pa ien . Despi e he
con inuous p og ess o hese echnological solu ions in he con inuous moni o ing o PD-associa ed gai
disabili ies, i is s ill di icul o easibly assess and ain pa ien s o hei common daily asks.
Se e al esea che s ha e dedica ed hemsel es o he s udy o he neu ological o igin o hese
impai men s in o de o cus omize ea men me hodologies o PD- ela ed gai disabili ies [16]. Despi e
he scien i ic ad ances, di e en hypo heses a e s ill poin ed ou . I is only known ha he e may be a
ailu e in he ac i i y o ne e messages be ween he cen al ne ous sys em and he
e e en muscles, esponsible o mo emen , ha can cause mo o symp oms in PD [17]–
[19]. To o e come hese mo o symp oms, esea che s poin ed ou cueing-based in e en ions
[5]. These s a egies in ol e he use o ex e nal empo al o spa ial s imuli o acili a e mo emen , in he
o m o isual, audi o y and ib o ac ile cues. These cues con ibu ion consis s in bypassing aul s
in ne ous messages ha may be a he o igin o gai impai men s [17]–[19]. Indeed, hese
senso y cueing s a egies a e in eg a ed in o bio eedback de ices, which ha e al eady been explo ed in
he PD ield. These sys ems make use o wea able echnology o p o ide senso y acquisi ion and igge
a cue in o ma ion (bio eedback). They can de ec a dec ease in cadence o a change o he lowe leg
muscle ac i i y, and h ough he de ec ion o such mo o beha io s deli e senso y cues [8], [17]. Thus,
hese in e en ions could lead o a change in pos u al con ol, s ep pa e n, and un eeze gai eezing
e en s, p e en alls, and consequen ly could p omo e less a iabili y in gai and a mo e goal-o ien ed
gai . Fu he , wea able sys ems allow hei in eg a ion in o pa ien s’ e e yday asks, ensu ing g ea e
eedom o mo emen and com o [9]. Howe e , o he bes o knowledge, he e ec i eness o hese
echnologies o ehabili a ion has seldom been in es iga ed and alida ed in eal-li e
si ua ions. Thus, he use o i ual en i onmen s o imme se pa ien s in hose si ua ions could po en ia e
4
he bio eedback in e en ions. Fu he , hese s udies did no ollow a pa ien -cen ed app oach,
some did no use ully wea able sys ems and did no include modula sys ems, meaning hey
a e no easy o in eg a e wi h o he echnologies [2], [17]. Las ly, pa ien s and physicians we e
a ely included in he de elopmen phase and he e we e e y ew accu a e and objec i e e alua ion
me ics, plus i was unusual o use unc ional mo emen aining.
Vi ual, augmen ed, and mixed eali y eme ged in he las decades as a p omising
s a egy o allow pa ien s’ imme sion in cus omized i ual en i onmen s. In ac , hese
pe sonalised i ual and in e ac i e en i onmen s allow pa ien s o be placed in si ua ions whe e hey can
pe o m daily asks, ob aining mo e easible and na u al mo ion da a. When a modula de elopmen
a chi ec u e is used, his VR/AR/MR equipmen may be in eg a ed wi h o he moni o ing and ac ua ion
de ices, os e ing pa ien s’ mo o assessmen and aining. In his sense, in o de o o e come he
limi a ions encoun e ed, his disse a ion aims o explo e he use o mixed eali y (MR) in he assessmen
and aining o PD pa ien s du ing hei mo o asks. By de eloping a obus sys em o MR
in eg a ed wi h a high echnological mo ion acking de ice, capable o assessing PD- ela ed
gai disabili ies, i is expec ed o show how imme si e, in e ac i e MR echnology can o e a new
me hodological amewo k o moni o ing and aining gai - ela ed beha iou s in PD.
1.2 PROBLEM STATEMENT
Mo e eliable assessmen and consis en aining gea ed owa ds daily asks a e needed, and his
can be achie ed by imme sing PD pa ien s in i ual en i onmen s o imp o e hei mo o unc ion
assessmen and aining.
I is expec ed o
(i)
explo e he use o MR o de elop and design i ual en i onmen s close o pa ien s’
daily eali y;
(ii)
de elop a modula a chi ec u e capable o in eg a ing he MR app oach wi h a mo o
assessmen de ice;
(iii)
in es iga e he po en ial o senso y cues in imp o ing gai impai men s h ough
augmen a i e cues.
To add ess hese p oblem s a emen s, i is c ucial o ollow a use -o ien ed app oach capable o
de eloping a mo o assessmen and aining s a egy close o pa ien s’ daily needs. This disse a ion will
adop a sys ema ic app oach o answe hese key cons ain s.

5
1.3 GOALS
The ul ima e goal o his disse a ion was o design, de elop, es , and alida e h ee di e en i ual
en i onmen s, which ec ea e e e yday si ua ions, o assess and ain PD- ela ed gai disabili ies using
MR sma glasses and a mo ion acking sys em. One o he mos dis inguishable ea u es o his
disse a ion is i s mul i-disciplina y na u e spanning om neu osciences o algo i hms o MR, modula
a chi ec u es, wea able senso s, and bio eedback s a egies. Thus, his wo k equi ed dealing wi h
exis ing and on -end ha dwa e, designing i ual en i onmen s, gai analysis and segmen a ion,
alida ing p o ocols wi h end use s, and da a analysis.
To each his main goal, he ollowing s ep-goals and Key Pe o mance Indica o s (KPI) needed o be
de ined and achie ed:
Goal 1: Ga he knowledge abou VR/AR/MR s a egies used in PD o mo o aining
and assessmen , h ough li e a u e e iews, o answe he ollowing ques ions:
(i)
“How ha e he
VR/AR/MR-based app oaches been applied in PD o help pa ien s mi iga e gai disabili ies?”;
(ii)
“Which
echnologies ha e been used o suppo VR/AR/MR-based app oaches in PD?”; and
(iii)
“How ha e he
VR/AR/MR-based app oaches been clinically alida ed in PD?”. This goal ela es o KPI 1: summa ising
he li e a u e h ough a leas wel e a icles; wha a e he mos common gai disabili ies in PD; wha a e
he eal-wo ld si ua ions ha mos cause PD- ela ed gai disabili ies. Chap e 2 p esen s hese su eys.
Goal 2: Implemen a ion o a modula , use -cus omised, echnological solu ion based on
mixed eali y o imme se pa ien s in scena ios ha can igge PD- ela ed gai disabili ies.
Based on he li e a u e e iew, i ual en i onmen s ha e oke PD- ela ed gai disabili ies and ha a e
cus omisable acco ding o he use s' heigh will be de eloped. This will add ess he limi a ions iden i ied
in VR/AR/MR-based app oaches in PD. This goal ela es o KPI 2: de elopmen o h ee i ual
en i onmen s ha ep esen si ua ions ha ypically cause PD- ela ed gai disabili ies. Chap e 3 and
Chap e 4 desc ibe he ma e ials and p ocedu es o his solu ion.
Goal 3: Implemen a ion o a modula , use -cus omised, echnological solu ion based on
mixed eali y in eg a ed wi h ano he high- echnological mo ion acking de ice o help
pa ien s o e come PD- ela ed gai disabili ies. Ou comes co e he in eg a ion o a mo ion acking
sys em wi h MR echnologies based on combined isual senso y cues, mo ion analysis and augmen ed
eali y. This goal ela es o KPI 3: de elopmen o on-demand isual bio eedback s a egies in eg a ed in
HoloLens 2; de elopmen o a eal- ime ini ial and inal con ac de ec ion algo i hm wi h a pe o mance
highe han 96% o accu acy. Chap e 3 and Chap e 4 de ail he ma e ials, me hods and algo i hms
used and de eloped o achie e his solu ion.
6
Goal 4: Valida ion o he p oposed MR s a egy wi h end-use s. I is in ended o collec and
analyse da a o assess he usabili y, e iciency, and accep abili y o implemen ed s a egies (use -cen ed
app oach). This goal ela es o KPI 4: alida ion o he solu ions wi h a leas en end-use s; s a is ically
signi ican di e ences in spa io empo al pa ame e s be ween con ol and moni o ing es s, and la e
be ween moni o ing and aining es s; SSQ sco e lowe han six een poin s; IMI sco e g ea e han i e
poin s. Chap e 5 will e eal he ob ained esul s.
1.4 RESEARCH QUESTIONS
Conside ing he ul ima e goal o his disse a ion and he s ep-goals p esen ed, ele an esea ch
ques ions (RQs) we e iden i ied, as ollows:
RQ 1: How ha e he VR/AR/MR-based app oaches and echnologies been applied o suppo PD
pa ien s and how ha e hey been clinically alida ed? This ques ion ela es o Goal 1 and is answe ed in
Chap e 2.
RQ2: How o implemen a modula , use -cus omised, mixed eali y-based echnology solu ion ha
imme ses pa ien s in en i onmen s ha (1) cause PD- ela ed gai impai men s; and ha (2) help
o e come hese impai men s wi h he aid o a mo ion acking sys em and bio eedback s a egies? This
issue conside s Goal 2 and Goal 3. The answe is de eloped h oughou Chap e 3 and Chap e 4.
RQ3: How does he implemen ed modula echnological solu ion, based on mixed eali y in eg a ed
wi h a mo ion acking sys em and wi h bio eedback s a egies, a ec he mo o pe o mance o PD
pa ien s du ing assessmen and aining? This ques ion is linked o Goal 4 and is answe ed in Chap e
5.
The p esen ed RQs a e summa ized and answe ed in Chap e 6.
1.5 CONTRIBUTIONS TO KNOWLEDGE
The main con ibu ions o his disse a ion o knowledge a e:
• Re iew on VR/AR/MR-based app oaches cu en ly deployed in PD o ain and assess gai
disabili ies.
• De elopmen o h ee i ual en i onmen s and i ual asks ha cause gai impai men s,
cus omisable o each pa icipan .
7
• Implemen a ion and alida ion o an algo i hm o de ec ing ini ial and inal con ac s, in eal
ime. Also, in his scope, an algo i hm o es ima e spa io empo al me ics was implemen ed,
based on gai segmen a ion.
• De elopmen o wo isual bio eedback s a egies, cus omisable o each pa icipan , o help
o e come PD- ela ed gai impai men s.
I is expec ed ha he de eloped wo k will lead o he elabo a ion o a jou nal a icle.
Du ing his pe iod, I had he p i ilege o guiding wo s uden s o he In eg a ed Mas e s in Elec onic
Enginee ing, in a p ojec o he cu icula uni "P oje o In eg ado ". In addi ion, I applied o a g an om
he "Ve ão com Ciência" p og amme o he Fundação pa a a Ciência e Tecnologia (FCT).
1.6 DISSERTATION STRUCTURE
This manusc ip is o ganized in o six chap e s, as ollows.
Chap e 1 p esen s he mo i a ion, p oblem s a emen and he ul ima e goals o his wo k.
Chap e 2 ou lines a comp ehensi e e iew o cu en li e a u e abou VR/AR/MR-based app oaches
o s udy PD- ela ed gai disabili ies. The VR/AR/MR-based app oaches a e p esen ed and discussed,
ega ding he VR/AR/MR echnology, embedded senso s, i ual asks, and clinical ou comes. The
chap e inishes wi h a summa y o he indings.
Chap e 3 add esses he o e iew o he solu ion. I s a s by desc ibing he p oblem. Then, p ojec
+sense and i s modules a e p esen ed, as well as a desc ip ion o he ha dwa e included in he s a egy,
men ioning i s need and echnical cha ac e is ics.
Chap e 4 ou lines he solu ion desc ip ion. Fi s ly, an in oduc o y insigh is p esen ed, desc ibing
he se up o be used. Secondly, he use -cen ed design o he solu ion is p esen ed, iden i ying he i ual
asks and en i onmen s designed. Thi dly, he in eg a ion o he senso y sys em is desc ibed, s a ing by
explaining he eal- ime ini ial and inal con ac s de ec ion algo i hm, up o he es ima ion o
spa io empo al me ics, pe o med o line. Finally, he in eg a ion o he bio eedback s a egies is
explained.
Chap e 5 p esen s he alida ion p o ocol, esul s, and a c i ical discussion o he solu ion.
Fu he mo e, i p esen s hei limi a ions and possible explana ions, along wi h esea ch sugges ions and
imp o emen s.
Chap e 6 concludes he disse a ion, while p o iding a b ie analysis o he p ojec and i s esul s,
along wi h u u e esea ch insigh s.
2 LITERATURE REVIEW
15
I was obse ed ha he VR/AR/MR equipmen used by he selec ed s udies we e he Oculus Ri
DK2 [2], [17], [28], HTC Vi e [11], [12], [20], [25], Google Glass [13], [23], HoloLens [15], [18], and
HTC Vi e P o [22], [26]. Fu he mo e, in [21] a mic o display was a ached o he eyeglasses ame and
in [14] a p o o ype o cus om-made sma glasses was designed. Finally, in [24] and [27] sma glasses
we e no wo n, on he con a y, a compu e assis ed i ual eali y en i onmen (CAREN) and a C-Mill VR+
eadmill was used, espec i ely.
The i ual en i onmen imme sion anged om ully imme si e [2], [11]–[15], [17], [18], [20]–[26],
[28] o semi-imme si e [27], being equen ly used head-moun ed displays.
Vi ual asks included mo o ac i i ies, like climbing s ai s [14], u ning [14], [15], and walking s aigh
[11], [13], [14], [18], [21], [24], [27], [28], (
i.e.,
on a hallway as in [12], [17], [20]), o speci ic con ex s
ha could igge PD- ela ed gai disabili ies, such as c ossing i ual [2], [12], [20] o eal doo s [13].
Addi ionally, when VR/AR/MR was applied o mo o aining s a egies, a cue-o ien ed game was used
in [22], while in [23]–[26] pa ien s ollowed he asks indica ed on he i ual game, namely, dance [23],
na iga e a i ual boa [24], d i e a ball o he inish line [24], smash lying objec s [24], comple e i ual
wo ds [25] o play a box game [26]. An o e iew o some i ual asks and en i onmen s de eloped a e
shown in Figu e 2-3.
Rega ding he acquisi ion module, h ee sys ems we e ound: In e Sense IS900 [2], [17], Qualysis
[20], IMU [13]–[15], accele ome e (G-Senso ) [24] and Vi e acke s [22], [25], wi h none o hem being
buil in. IMUs we e placed in bo h ull body [14], [15] and lowe body [11], [13] con igu a ions.
Wi h espec o he ac ua ion module, i was iden i ied he ype and which de ice we e used, and
whe he i was buil in. All sys ems which had an ac ua ion module used buil -in ac ua o s, such as
augmen a i e isual cues o ea phones [13]–[15], [17], [18], [21], [22], [24]. Howe e , [15] also had a
non-buil -in ac ua o , namely, a speake o audi o y cueing. Fu he mo e, [24] used ou ypes o
ac ua o s ( isual, audio, es ibula and ac ile), [13] used h ee ypes o ac ua o s (audio, lashing ligh
and op ic low), [15] used wo ypes o ac ua o s ( isual and audio) whe eas [14], [17], [18], [21], [22]
only used one ype o ac ua o , isual. Besides, isual cues we e used mo e han audi o y cues.

16
2.3.4 VALIDATION METHODOLOGY HIGHLIGHTS: PARTICIPANTS, CRITERIA STUDY, SETTING,
PROTOCOLS, SCHEDULE, METRICS
Table 2-3 summa izes he alida ion me hodology o he selec ed s udies. I highligh s he
pa icipa ion and e alua ion o PD pa ien s, inclusion and exclusion c i e ia o pa icipan s selec ion,
se ing, expe imen al p o ocols, schedule, and he esea ch e alua ion me ics.
Uni ied Pa kinson’s Disease Ra ing Scale pa III (UPDRS-III) [2], [12]–[15], [17], [18], [20], [23]–
[25], [27], [28] and Hoehn and Yah scale (H&Y) [11], [12], [14], [15], [18], [20], [21], [23]–[25], [27],
[28] we e he mos commonly used a ing scales o symp oms o PD. Along wi h UPDRS-III, Pos u al
Ins abili y and Gai Diso de sub-sco e (PIGD) [14], Ac i i ies-speci ic Balance Con idence (ABC) scale
[11], [12], [22], [23], [28], Mini-Balance E alua ion Sys ems Tes (Mini-BESTes ) [11], [12], [20], [25],
[28] and UPDRS-II [24] we e used o e lec he e olu ion o mo o unc ion. To indica e he pa ien s’
cogni i e and men al s a e, he Mini-Men al S a e Examina ion (MMSE) scale was used in [11], [14], [15],
[27], Mon eal Cogni i e Assessmen (MoCA) in [12], [18], [20], [23], [28] and F on al Assessmen
Ba e y (FAB) scale in [13]–[15]. FOG-ques ionnai e (FOG-Q) was used in [2], [12]–[15], [17], [18], [20].
To assess simula o sickness symp oms, he Simula o Sickness Ques ionnai e (SSQ) was used [25].
Figu e 2-3 - O e iew o some i ual asks and en i onmen s aken om [2], [3], [12], [13], [19], [20], [22]–[26].
17
Table 2-3 – Clinical Highligh s o he de eloped VR/AR/MR echnologies o PD pa ien s, o e he las en yea s
Goal
Pape
Pa icipan s
C i e ia S udy
Se ing
P o ocol
Schedule
Me ics
N
Scales
Inclusion
Exclusion
A
[2]
10
- UPDRS-III;
- FOG-Q;
-
-
Labo a o y
Walk unde 3 di e en i ual
condi ions:
(1) no doo ;
(2) na ow doo way;
(3) s anda d doo way.
Single isi :
(1) amilia isa ion
phase;
(2) 18 ials (6x
each condi ion).
To al: 20min
- s ep cadence (mean and CV);
- s ep eloci y (mean and CV);
- s ep leng h (mean and CV);
- du a ion o FoG episodes
(mean and SD);
- % ials wi h a FoG episode
[11]
10
- UPDRS;
- Mini-
BESTes ;
- ABC scale;
- H&Y;
- MMSE;
- Diagnosis o PD; -
- H&Y≤3;
- MMSE > 24/30;
- No o he
pa hology
in e ac ing wi h gai
o causing
dizziness;
- No unco ec ed
isual de iciency;
- Abili y o walk 512
consecu i e s ides
(±10–
15 min);
-
Labo a o y
Walk in a andomized o de in
3 condi ions:
(1) O e g ound Walking;
(2) T eadmill Walking
(3) imme si e Vi ual Reali y on
T eadmill Walking
Single isi
- speed;
- s ep leng h;
- cadence;
- SSQ
[12]
10
- MDS-
UPDRS-III;
- NFoGQ;
- MoCA;
- Mini-BEST;
- ABC Scale;
- H&Y;
- Diagnosis o PD;
- Sel - epo ed FoG;
- Sel - epo ed
abili y o walk
400m wi hou
assis ance om a
de ice o ano he
pe son;
- No diagnosis o
demen ia;
- No unco ec ed
ision o hea ing
p oblems;
-
Labo a o y
Walk in 5 en i onmen s:
(1) Physical labo a o y wi hou
VR;
(2) i ual labo a o y wi hou
obs acles;
(3) i ual doo way;
(4) i ual hallway;
(5) i ual s ee scene wi h
c owds
Single isi
- gai speed;
- s ep leng h (mean and CV);
- s ep wid h;
- s ep ime;
- s ep ime asymme y;
- es ina ion;
- SSQ
[20]
12
- MoCA;
- NFOGQ;
- Mini-BEST;
- Diagnosis o PD
wi hou demen ia;
-
Labo a o y
Walk unde 4 condi ions:
(1) physical labo a o y;
(2) i ual labo a o y;
(3) i ual doo way;
Single isi
- kinema ic a iables;
- gai speed;
- s ep leng h (mean and CV);
- s ep ime;
18
- MDS-
UPDRS-III;
- H&Y;
- Sel - epo ed o
clinician-obse ed
FoG;
- Abili y o walk 400
m wi hou
assis ance om a
de ice o ano he
pe son;
- No unco ec ed
ision o hea ing
de ici s;
(4) i ual hallway.
- s ep ime asymme y;
- s ep wid h;
- DLS;
- es ina ion;
- SSQ
CoA
[13]
12
“end-o -
dose”
- UPDRS-III;
- NFOGQ;
- FAB;
- P esence o FoG
mo e han wice pe
day;
- Able o walk 20m
o e a la su ace
wi hou walking
aids;
- Signi ican cogni i e
impai men s;
- Como bidi ies ha
impai ed gai , o isual
impai men s;
Labo a o y
Walk on 4 di e en walking
cou ses in combina ion wi h 4
cueing condi ions:
(1) me onome;
(2) lashing ligh ;
(3) op ic low;
(4) no cue.
Single isi :
(1) amilia iza ion
phase;
(2) 16 di e en
cue-cou se
combina ions (2
ials each).
To al: 2.5h
- no. o FoG episodes;
- du a ion o FoG episodes;
- s ide leng h (mean and SD);
- speed;
- cadence (mean and SD);
- in e iew (use expe ience)
[18]
24
“on s a e”
- UPDRS-III;
- H&Y;
- MoCA;
- NFOGQ;
- Olde han 18
yea s;
- Diagnosis o PD;
- Expe ience FOG in
he dopamine gic
“ON” s a e;
- Addi ional
neu ological diseases
and/o o hopedic
p oblems;
- Inabili y o walk
independen ly;
Home/Labo a o y
(1) HOME: walk a eezing
p o oking ou e mul iple imes
wi h and wi hou wea ing he
HoloLens (wi hou Holocue);
(2) LABORATORY: amilia ize
pa icipan s o walking wi h he
(on-demand) holog aphic cues;
(3) HOME: equal o session 1
bu wea ing he HoloLens wi h
and wi hou he Holocue
3 sessions o
1.5h, one week
apa
- no. o FoG episodes;
- a e age du a ion o FoG
episodes;
- o al du a ion o FoG episodes;
- % ime ozen (PTF)
CoT
[21]
20
- H&Y
-
- conside able isual
de ici no
compensa ed by
co ec ion;
- ocula mo emen
dys unc ion;
- gai dis u bances due
o neu omuscula
diseases;
Labo a o y
Walk a s aigh ack o 10 m:
(1) baseline;
(2) online display o ;
(3) online display on;
(4) esidual e ec s;
(5) examina ion;
Single isi
- speed;
- s ide leng h
[14]
25
“end-o -
dose”
- UPDRS-III;
- UPDRS-
PIGD;
- Diagnosis o PD;
- Olde han 18
yea s old;
- His o y o s oke;
- Psychia ic disease;
Labo a o y
Walk on 3 di e en walking
cou ses in combina ion wi h 5
cue condi ions:
Single isi :
- no. o FOG episodes;
- % eezing ime;
- s ide leng h (mean and SD);
19
- H&Y;
- NFOGQ;
- MMSE;
- FAB;
- P esence o FoG
mo e han once pe
day;
- Se e e unco ec ed
isual o hea ing
impai men s;
- Como bidi y limi ing
ambula ion;
- Inabili y o walk
unaided;
- Deep b ain
s imula o o
apomo phine pump;
- Jejunal le odopa gel
in usion;
- MMSE sco e < 24;
(1) augmen ed isual cue ba s;
(2) augmen ed isual cue
s ai cases;
(3) con en ional 3D ans e se
ba s on he loo ;
(4) me onome;
(5) no cueing.
2 sessions
sepa a ed by
30min b eak
To al: 2.5h-3h
- cycle ime (mean and SD);
- cadence;
- speed;
- in e iew (use expe ience)
[17]
12
- UPDRS-III;
- FOG-Q;
-
-
Labo a o y
Walk using isual cues:
(1) 2 spa ial condi ions: 115%
and 130% o an indi idual’s
baseline s ep leng h and;
(2) 3 di e en empo al
condi ions: spa ial only
condi ion, 100 and 125%
baseline s ep cadence.
Single isi :
(1) amilia isa ion
phase;
(2) 6 di e en
cueing condi ions
(8x each
condi ion).
To al: 40min
- S ep leng h (mean and CV);
- S ep cadence (mean and CV);
- S ep eloci y (mean and CV);
(a baseline and pos
in e en ion)
[15]
16
“end-o -
dose”
- UPDRS-III;
- H&Y;
- MMSE;
- NFOGQ;
- FAB;
- Diagnosis o PD;
- P esence o FoG
mo e han wice pe
day;
- MMSE sco e < 24;
- FAB sco e < 13;
- Como bidi y causing
se e e gai
impai men s;
- Se e e bila e al
isual o audi o y
impai men s;
- Inabili y o pe o m a
180◦ u n unaided;
Labo a o y
Pe o m a se ies o 180º u ns
unde :
(1) an expe imen al condi ion
wi h AR isual cues and;
(2) wo con ol condi ions:
audi o y cues and no cues.
Single isi :
(1) 1 aining
session: 3 blocks
(15 ials each);
(2) 2 expe imen al
sessions: 3 blocks
(15 ials each).
- PTF;
- no. o FoG episodes;
- du a ion o FOG episodes;
- cadence;
- peak eloci y;
- s ide ime (mean and CV);
- s ep heigh (mean and CV);
- max head-pel is sepa a ion;
- ime o max head-pel is
sepa a ion;
- max medial CoM de ia ion;
- u n ime;
- in e iew (use expe ience)
[22]
5
-
- Diagnosis o PD;
- H&Y I-III;
- Able o walk
independen ly;
- Condi ions ha could
ha e a ec ed exe cise
unc ion;
Labo a o y
Play he game “T easu e Island
Ad en u e” wi h and wi hou
obs acles in combina ion wi h 3
le els: 35, 40, and 45cm
be ween he isual cues
1 session pe
week o 30min,
o e 3 weeks
- BBS;
- ABC;
- S ep dis ance;
- Leg aising;
(a baseline and pos
in e en ion)
20
VgoT
[23]
7
“ON”
- UPDRS-III;
- H&Y;
- MoCA;
- Diagnosis o PD;
- H&Y > III;
- MDS-UPDRS-III > 57;
- Unable o wea o
ope a e Google Glass;
- Demen ia;
Home
Comple e a leas 3 modules o
MTG pe day
E e y day o 3
weeks
- Mini-BESTes ;
- one-leg s ance;
- TUG;
- dual- ask;
- ABC scale;
- BDI;
- PDQL;
- in e iew
[24]
22
- UPDRS-II;
- UPDRS-III;
- H&Y;
- Mini-
BESTes ;
- MMSE;
- Diagnosis o PD;
- H&Y≤3;
- MMSE ≥ 24
- age > 85 yea s;
- p esence o se e e
medical and
psychia ic illness
po en ially in e e ing
wi h he VR aining
Labo a o y
Comple e ou scena ios:
(1) Na iga e a i ual boa
h ough a slalom cou se;
(2) walk ac oss he boa d;
(3) d i e a ed ball, mo ing he
oad up o he inish line;
(4) swa a lying objec s ha
eme ge along he pa h
20 con en ional
physio he apy
sessions + 3-
mon h es + 20
sessios o CAREN
aining
- BBS;
- TUG;
- UPDRS-II;
- UPDRS-III;
- FES-I;
- H&Y;
- 10MWT;
- s ide leng h;
- cycle ime;
- s ance phase/ ime;
- swing phase/ ime;
- pe cen age o single- and
double-limb suppo ;
- speed;
- cadence;
- s ep leng h;
- s ep wid h
[25]
9
“ON”
- UPDRS-III;
- H&Y;
- Mini-
BESTes ;
- SSQ;
- Diagnosis o PD;
- No mo o
luc ua ions;
- H&Y I-III;
- Olde han 18
yea s old;
- Walking
independen ly;
- S able medica ion;
- Uncon olled,
in olun a y
mo emen s
(dyskinesia);
- Musculoskele al
inju ies;
- Pain ha limi ed
mo emen ;
Labo a o y
Comple e a puzzle ha
consis ed o a wo d wi h
missing le e s loca ed a eye
le el in he i ual en i onmen
3 sessions o
30min each, o e
1 week
To al: 1h30
- SSQ;
- ITC-SOPI;
- IMI;
- SUS;
[26]
4
-
- H&Y II;
- Inabili y
o co ec ly espond o
he assessmen
p o ocol;
- P esence o
ca dio ascula ,
pulmona y, o
Labo a o y
Play he game BOX VR
2 sessions, 2
weeks apa ;
1s session:
(1) amilia iza ion
phase wi h S eam
VR Home (9min);
- SUS;
- SSQ;
- GEQ-pos game;
- in e iew (use expe ience)

21
musculoskele al
condi ion;
- P esence o se e e
isual loss;
- Ve igo, epilepsy, and
psychosis;
(2) aining: game
gym (3min);
2nd session:
(1) amilia iza ion
phase wi h
TheBlue;
(2) aining: game
gym.
[27]
29 PIGD +
23 non-
PIGD
2h a e
medica ion
- UPDRS-III;
- H&Y;
- MMSE;
- Diagnosis o
p ima y PD;
- <75 yea s old
- H&Y s age I–III
(“on” pe iod);
- MMSE>24 (>20
o hose wi h only
p ima y school
educa ion);
- Se ious
complica ions o
como bidi ies;
- Special ea men
equi ed o o he
como bidi ies;
- Deep b ain
s imula ion o in i o
implan s;
- A ypical o seconda y
PD;
- Como bidi ies a ec
walking;
- se e e cogni i e,
isual, and hea ing
impai men ;
- Using a psycho opic
subs ance;
Labo a o y
Comple e 5 modules o C-Mill
aining in each aining session
1 session o
30min pe day o
7 days:
(1) amilia iza ion
phase;
(2) modules o C-
Mill aining.
- 10-me e walking es ;
- TUG es ;
- BBS;
- Pos u e sway;
- Gai adap abili y;
- Bo g 6-20 Ques ionnai e;
- pe cei ed isk o alling;
- PDQL
T
[28]
11
“ON”
- UPDRS-III;
- H&Y;
- MoCA;
- ABC;
- Mini-
BESTes ;
- Able o walk o 30
min on a eadmill;
- 19<MoCA<30;
- No o he
neu ological
diso de s;
-
Labo a o y
Walk o 20 min on a eadmill
while iewing a i ual ci y
scene
Single isi
To al: 20min
- CoP excu sion;
- SSQ;
- SAC;
(a baseline and pos
in e en ion)
[Re .]: s udy e e ence; A: assessmen ; CoA: cue-o ien ed assis ance; CoT: cue-o ien ed aining; VGoT: ideogame-o ien ed aining; T: aining; UPDRS-III: Uni ied Pa kinson’s Disease Ra ing Scale pa III; FOG-Q:
F eezin o Gai ques ionnai e; FAB: F on al Assessmen Ba e y; H&Y: Hoehn and Yah scale; MMSE: Mini-Men al S a e Examina ion; UPDRS-PIGD: Uni ied Pa kinson's Disease Ra ing Scale - Pos u al Ins abili y and
Gai Diso de ; Mini-BesTes : Mini-Balance E alua ion Sys ems Tes ; SSQ: Simula o Sickness Ques ionnai e; MoCA: Mon eal Cogni i e Assessmen ; ABC: Ac i i ies-speci ic Balance Con idence scale; CV: coe icien
o a ia ion; SD: s anda d de ia ion; PTF: pe cen age ime eezing; DLS: double limb suppo ; CoM: Cen e o Mass; BBS: Be g Balance Scale; ITC-SOPI: Independen Tele ision Commission Sense o P esence
In en o y; IMI: In insic Mo i a ion In en o y; SUS: Sys em Usabili y Scale; GEQ-pos game: Game expe ience ques ionnai e-pos game; TUG: Timed Up and go Tes ; BDI: Beck Dep ession In en o y; PDQL: Pa kinson's
Disease Quali y o Li e ques ionnai e; CoP: Cen e o P essu e; SAC: S ess A ousal Checklis .
22
Inclusion c i e ia included he diagnosis o PD [11], [12], [14], [15], [18], [20], [22]–[25], [27], abili y
o walk independen ly [11]–[14], [18], [20], [22], [25], mo o luc ua ions absence [13]–[15], [25],
musculoskele al inju ies absence [21], [25], lack o o he neu ological diso de s [14], [18], [24], [28],
lack o se e e bila e al isual o audi o y impai men s [11]–[15], [20], [21], [26], [27], abili y o pe o m
a 180º u ns unaided [15], lack o cogni i e impai men s [13], [23], lack o deep b ain s imula ion [27]
o apomo phine pump and jejunal le odopa gel in usion [14], p esence o s able medica ion [25]. Fo he
s udies ha e alua ed FoG, he p esence o his symp om was also an inclusion c i e ia [12]–[15], [18],
[20]. Addi ionally, speci ic sco es o PD scales we e used o include pa icipan s: 19<MoCA>30 in [28];
MMSE>24 in [11], [14], [15], [24], [27]; FAB sco e>13 in [15]; H&Y s age I-III in [22], [25] , H&Y s age
I-III while on medica ion in [27]; H&Y s age II in [26]; H&Y s age<III in [11], [23], [24]. Mo eo e , some
s udies used clinical cha ac e is ics as inclusion c i e ia, namely, age: olde han 18 yea s old in [14],
[18], [25]; younge han 75 yea s old [27] and younge han 85 yea s old [24].
Rega ding he alida ion scena ios, all [2], [11]–[15], [17], [20]–[22], [24]–[28] a icles conduc ed
an in e en ion in a labo a o y se ing apa om [18], [23] which ollowed a home-based app oach.
Those a icles ha e alua ed FoG used he ollowing me ics: du a ion o FoG episodes [2], [13], [15],
[18], pe cen age o ials wi h a FoG episode [2], numbe o FoG episodes [13]–[15], [18] and pe cen age
o eezing ime [14], [15], [18]. Mo eo e , gai - ela ed me ics we e used, such as s ep cadence (mean
[2], [11], [13]–[15], [17], [24], coe icien o a ia ion (CV) [2], [17] and s anda d de ia ion (SD) [11],
[13]), s ep eloci y (mean [2], [11]–[14], [17], [20], [21], [24], CV [2], [17] and SD [11], [24]), s ep
leng h (mean and CV [2], [11], [12], [17], [20], [24]), s ide leng h (mean and SD [13], [14], [21], [24]),
peak eloci y [15], s ep ime (mean and asymme y [12], [20]), s ide ime (mean and CV [15]), s ep
wid h (mean [12], [20], [24] and SD [24]), es ina ion [12], [20], kinema ic a iables [20], double limb
suppo (DLS) [20], s ep heigh (mean and CV [15]), maximum head pel is sepa a ion [15], ime o
maximum head-pel is sepa a ion [15], maximum medial cen e o mass (CoM) de ia ion [15], u n ime
[15], cycle ime (mean and SD [14], [24]), s ep dis ance [22], leg aising [22], s ance and swing phase
[24], pe cen age o single limb suppo [24], pos u e sway [27], gai adap abili y [27], 10-me e walking
es (10MWT) [24], [27] and cen e o p essu e (CoP) excu sion [28], in o de o analyse gai pe o mance.
In e ms o balance analysis, Be g Balance Scale (BBS) [22], [24], [27], ABC scale [22], [23], Timed
Up and Go Tes (TUG) [24], [27] and dual- ask [23], one-leg s ance [23], Mini-BESTes [23], Falls E icacy
Scale In e na ional (FES-I) [24] and pe cei ed isk o alling [27] we e assessed. Fu he mo e, Simula o
Sickness Ques ionnai e (SSQ) [11], [12], [20], [24]–[26], [28] was conduc ed o e alua e simula o
23
sickness symp oms, Independen Tele ision Commission Sense o P esence In en o y (ITC-SOPI) [25] o
check pe cei ed sense o p esence, In insic Mo i a ion In en o y (IMI) [25] o sco e le els o mo i a ion,
Sys em Usabili y Scale (SUS) [25], [26] o e alua e sys em o e all usabili y, Pa kinson's Disease Quali y
o Li e Ques ionnai e (PDQL) [23], [27] o e alua e quali y o li e (QoL), Beck Dep ession In en o y (BDI)
[23] o assess dep essi e diso de s a us, Bo g 6-20 Ques ionnai e [27] o check pa icipan s’ pe cei ed
exe ion and a igue and S ess A ousal Checklis (SAC) [28] o assess s ess. Finally, in [13]–[15], [23],
[26] an in e iew was conduc ed on use expe ience, and in [26] a game expe ience ques ionnai e-pos
game (GEQ-pos game) was also unde aken.
Gómez-Jo dana e al. [2] p oposed a s udy o assess i he p esence o i ual doo ways in a i ual
en i onmen could induce FoG he same way eal doo ways do. Fo expe imen al p o ocols, he e we e
h ee g oups, a g oup o heal hy pa icipan s as a con ol g oup, a g oup o PD pa ien s wi hou FoG and
a g oup o PD pa ien s wi h FoG, named as eeze s (PD- ). All g oups walked along a hallway unde h ee
di e en i ual condi ions (no doo , na ow doo way (100% o shoulde wid h) and s anda d doo way
(125% o shoulde wid h)). The p esence o i ual doo s esul ed in a educ ion on s ep leng h and eloci y
and an inc ease on gai a iabili y, wi h he wo s alues occu ing o PD- . The na ow doo was he one
ha p o oked he mos FoG.
Lheu eux e al. [11] aimed o assess he e ec s o adding an op ic low displayed h ough an
imme si e i ual eali y headse du ing eadmill walking on gai . PD pa ien s we e ins uc ed o walk in
a andomized o de in 3 condi ions: (i) o e g ound walking; (ii) eadmill walking; and (iii) imme si e i ual
eali y on eadmill walking. As a esul , a g ea e s ep leng h and lowe cadence we e ob ained. SSQ was
simila be ween he (ii) and (iii) condi ions.
Yamagami e al. [12] in ended o in es iga e whe he i ual en i onmen s ha eplica e FoG-
p o oking si ua ions would exace ba e gai impai men s associa ed wi h FoG compa ed o unobs uc ed
VR and physical labo a o y en i onmen s. Pa icipan s pe o med a se ies o walking asks on i e di e en
en i onmen s (physical labo a o y wi hou VR; i ual labo a o y wi hou obs acles; i ual doo way; i ual
hallway; i ual s ee scene wi h c owds). The esul s showed ha FoG-p o oking VR en i onmen s could
exace ba e gai impai men s ha a e ela ed o FoG.
Besha a e al. [20] aimed o examine he e ec s o i ual doo ways and hallways on gai kinema ics
among people wi h PD and FoG. Pa icipan s pe o med a se ies o walking asks on ou di e en
condi ions (physical labo a o y; i ual labo a o y; i ual doo way; i ual hallway). As a esul , kinema ic
changes commonly associa ed wi h FoG episodes we e ob ained.
24
Zhao e al. [13] in ended o e alua e hy hmic isual and audi o y cueing in a labo a o y se ing.
Pa icipan s pe o med a se ies o walking asks on ou di e en walking cou ses (wide u n, na ow u n,
ull u n, and doo way) in combina ion wi h h ee cues (me onome, lashing ligh and op ic low). A mo e
s able gai pa e n wi h he aid o hese cues was ob ained bu FoG did no diminish signi ican ly. The
me onome was mo e e ec i e han hy hmic isual cues and p e e ed by mo e pa icipan s.
Gee se e al. [18] explo ed un amilia i y and habi ua ion e ec s associa ed wi h wea ing he HoloLens
on FoG and e alua ed he po en ial immedia e e ec o Holocue on alle ia ing FoG in he home
en i onmen . Pa ien s pe o med h ee sessions o 1.5h, scheduled one week apa . In he i s session,
pa icipan s walked a eezing p o oking ou e mul iple imes wi h and wi hou wea ing he HoloLens
(wi hou Holocue), in hei homes. Session 2 ook place in a labo a o y and consis ed o indi idually
cus omise he cues o he Holocue applica ion in e ms o in e cue dis ance and p e e ed ype o cues
and amilia ize pa icipan s o walking wi h he holog aphic cues. Finally, he las session ook place again
a he pa ien s’ home. Pa icipan s walked he same ou e wi h he same condi ions as in session 1, while
wea ing he HoloLens wi h and wi hou he Holocue applica ion. Wea ing he HoloLens (wi hou Holocue)
did signi ican ly inc ease he numbe and du a ion o FOG episodes, bu his un amilia i y e ec
disappea ed wi h habi ua ion o e sessions. Holocue had o e all no immedia e e ec on FOG, al hough
objec i e and subjec i e bene i s we e obse ed o some indi iduals, mos no ably hose wi h long and/o
many FOG episodes.
Bada ny e al. [21] s udied he e ec s o isual eedback cues on gai . The i ual en i onmen
consis ed o a i ual iled loo in a checke boa d a angemen . The expe imen al p o ocol was di ided
in o 5 phases: (i) walking wi hou he de ice; (ii) walking wi h he de ice placed on bu wi h he display
u ned o ; (iii) walking wi h he display u ned on; (i ) walking wi hou he de ice a e a 15-minu e b eak;
and ( ) e-e alua ion o baseline pe o mance wi hou he de ice one week a e he i s examina ion. The
esul s sugges ed ha wea ing he de ice u ned o esul ed in a negligible e ec o abou 2%. Wi h he
display u ned on, 56% o he pa ien s imp o ed hei gai speed o s ide leng h o bo h. A e emo ing
he de ice, 68% o he pa ien s showed o e 20% imp o emen in ei he gai speed o s ide leng h o
bo h. One week la e , 36% o he pa ien s showed o e 20% imp o emen in baseline pe o mance wi h
espec o he p e ious es .
Janssen e al. [14] in es iga ed he usabili y o 3D augmen ed eali y cues compa ed o con en ional
3D ans e se ba s on he loo and audi o y cueing, in educing FoG and imp o ing gai pa ame e s.
Pa ien s we e p esen ed o h ee walking cou ses (walking s aigh , s op and s a and u ning) wi h i e
cue condi ions ( wo expe imen al condi ions: AR isual cues ba s, AR isual cues s ai case; and h ee
31
The e o e, a sys ema ic app oach was ollowed o iden i y he equi emen s o he sys em, om he
poin o iew o he use and he echnologies, conside ing he limi a ions iden i ied in he li e a u e e iew,
allowing o mo e on o he nex asks o he disse a ion.
Table 2-4 - Iden i ied limi a ions o cu en VR/AR/MR-based app oaches and guidelines o hei mi iga ion
Limi a ions
Guidelines o be ollowed
Technological
Sma glasses cha ac e is ics:
hea y, uncom o able, monocula
and wi h a na ow ield o iew
Sma glasses should be mo e ligh weigh ,
com o able, wi h a use - iendly design,
binocula and wi h an adequa e eye calib a ion
and ield o iew
Explo e he use o mixed eali y
Unknown moni o ing sys ems’
con ibu ion
Explo e he use and po en ial o o he in eg a ed
moni o ing sys ems speci ied o di e en gai
impai men s
Unclea co ela ion be ween i ual
en i onmen s and asks and be e
assessmen and aining
S udy which a e he bes i ual en i onmen s and
i ual asks o mo o assessmen and aining
Valida ion
Failu e o ca y ou usabili y, sa e y,
and easibili y ques ionnai es
Assessmen o he VR/AR/MR-app oach should include
usabili y, sa e y and easibili y ques ionnai es
and mo e objec i e es s
Implemen a ion o a sui able amilia iza ion phase
wi h VR/AR/MR equipmen
Sho - e m in e en ions and ollow-
up absence
VR/AR/MR should inco po a e ea men p o ocols o
se e al sessions pe week, o se e al weeks wi h
longe ollow-up in e als
Con ol g oup absence
VR/AR/MR should in eg a e a con ol g oup o s udy
he e ec s o i
Con ol condi ion absence
VR/AR/MR should in eg a e a con ol condi ion o
dis inguish dis ac ion by he sma glasses

3 SOLUTION OVERVIEW
33
The ollowing chap e speci ies he ma e ials and me hods used o de elop he p oposed s a egy and
o acqui e and p ocess all he da a equi ed. This includes i) an o e iew o he solu ion ound, s a ing
by summa ising he p oblem aised; ii) a p esen a ion o he p ojec in which his disse a ion is
in eg a ed; and iii) he espec i e p ojec module o which his disse a ion has con ibu ed; and i ) an
in oduc ion o he de ices and sys ems used.
3.1 PROBLEM DESCRIPTION
F om he li e a u e e iew i was concluded ha VR/AR/MR s a egies ha e he po en ial o no only
imme se pa ien s in en i onmen s ha ec ea e daily si ua ions which may igge PD- ela ed gai
disabili ies bu also o in eg a e bio eedback s a egies o help pa ien s eme ge om hese
disabili ies. Howe e , some limi a ions we e iden i ied, such as he ac ha he sma glasses we e hea y,
no usabili y, sa e y o easibili y ques ionnai es we e used, and he lack o con ol condi ions in he
p o ocols, p e en ing a clea discussion o he esul s ob ained.
Thus, a new s a egy mus include pa ien s wi h Pa kinson’s disease as a ge audience
and will be implemen ed based on MR echnology, i.e., he combina ion o eal-wo ld imme sion and
i ual objec s in e ac ion, in eg a ed wi h a mo ion acking sys em. Pa ien s’ mo o pe o mance
will be eco ded and assessed by he mo ion acking sys em, which should p esen a eal- ime
synch oniza ion wi h he MR echnology. Fu he , he analysis o use s’ mo ion will make i possible
o p o ide on-demand isual cues, ollowing a isual bio eedback s a egy.
In his sense, his disse a ion expec s o (i) de elop i ual en i onmen s ha lead o gai
impai men s and (ii) in eg a e a bio eedback s a egy ha enables pa ien s o o e come hese episodes.
The e o e, h ee di e en i ual en i onmen s we e de eloped, in which pa ien s we e imme sed and
encou aged o pe o m mo o asks, ha co esponded o h ee si ua ions ha ypically cause PD gai
impai men s ( u ning, c ossing doo s, na ow spaces). In addi ion, a bio eedback s a egy based on isual
cues was p oposed o imp o e pa ien s’ mo o pe o mance in he same i ual en i onmen s.
This solu ion is in eg a ed in he +sImme si e module o he +sense p ojec , which is p esen ed in
he nex sec ion.
3.2 +SENSE
This disse a ion is in eg a ed and in ended o con ibu e o he +sense p ojec . The p ojec aims o
imp o e pa ien s’ quali y o li e, p omo ing less dependence on hi d pa ies by imp o ing hei mobili y
34
and mo o au onomy. In his sense, +sense o e s on -end high- ech solu ions based on wea able
bio eedback de ices which ely on acquisi ion, in e p e a ion, and eedback o pa ien s’ senso imo o
in o ma ion. Cu en ly, +sense is di ided in o ou modules, as shown in Figu e 3-1: (1) +sBio eedback;
(2) +sMo ion; (3) +sC-suppo and (4) +sImme si e. The de elopmen o his disse a ion con ibu ed
o he ou h module.
3.3 +SIMMERSIVE
This module b ings a new pa adigm shi by using mixed eali y app oaches, in eg a ed wi h a mo ion
acking de ice and bio eedback s a egies, as a complemen a y ool o mo o moni o ing and aining o
PD- ela ed gai disabili ies. The h ee i ual en i onmen s allow he imme sion o he use in e e yday
si ua ions, ha ing o pe o m daily mo o asks, in o de o achie e a mo e eliable mo o assessmen and
ehabili a ion. Thus, his disse a ion b ings a s ep o wa d in he knowledge o how mixed eali y-based
mo o assessmen and aining can be applied in Pa kinson's disease. The +sImme si e conside s he
mul i ac o ial na u e o PD and inno a es by con ibu ing wi h a pa ien -cen ed app oach. Bea ing his in
mind, wo de ices we e used: (1) mixed eali y sma glasses; and (2) mo ion acking sys em, explained
in de ail in he ollowing sec ions.
Figu e 3-1 - +sense modules.
35
3.3.1 MIXED REALITY SMART GLASSES: MICROSOFT HOLOLENS 2
In o de o implemen he mixed eali y s a egy, he Mic oso HoloLens 2 was used, a ully
imme si e, po able, and wea able comme cial se up de ice o augmen ed/mixed eali y, Figu e 3-2. I
consis s o an AR/MR headse and a USB Type-C cable, which allows o cha ge he sma glasses and
connec hem o o he de ices such as compu e s.
Some o he HoloLens 2 sys em speci ica ions a e men ioned in Table 3-1 [30]. HoloLens 2 p esen s
see- h ough holog aphic lenses, enabling o see he eal wo ld, ne e losing he sense o eali y. Mo eo e ,
hey ha e se e al senso s ha allow head and eye acking, making i possible o he glasses o always
know whe e he use is in space. These sma glasses ha e buil -in speake s and mic ophone. In his
sense, beyond he isual eedback, hey can also p o ide audi o y eedback o use s. One o he bigges
s eng hs is ha hey can unde s and he human and he en i onmen h ough hand acking, eye
acking, oice, 6DoF acking and spa ial mapping, making hese glasses use - iendly. In addi ion, hey
only need a USB Type-C cable o connec o a compu e , hey a e ligh weigh (556g), one can wea
glasses unde hem, and hei ba e y las s up o 3 hou s o ac i e use.
This as ange o ea u es o HoloLens 2 mo i a ed i s selec ion, as i was in ended o use mixed
eali y sma glasses ha allow holog ams o be placed in eal space, while s ill seeing he eal wo ld.
Figu e 3-2 - Mic oso HoloLens 2.
36
Table 3-1 - Speci ica ions o HoloLens 2 sma glasses [30]
HoloLens 2 Technical Speci ica ions
Display
Op ics
See- h ough holog aphic lenses (wa eguides)
Resolu ion
2k 3:2 ligh engines
Holog aphic densi y
2.5k adian s (ligh poin s pe adian )
Eye-based ende ing
Display op imiza ion o 3D eye posi ion
Senso s
Head acking
4 isible ligh came as
Eye acking
2 IR came as
Dep h
1-MP ime-o - ligh (ToF) dep h senso
IMU
Accele ome e , gy oscope, magne ome e
Came a
8-MP s ills, 10800p30 ideo
Audio and speech
Mic ophone a ay
5 channels
Speake s
Buil -in spa ial sound
Human
unde s anding
Hand acking
Two-handed ully a icula ed model, di ec
manipula ion
Eye acking
Real- ime acking
Voice
Command and con ol on-de ice; na u al language
wi h in e ne connec i i y
Windows Hello
En e p ise-g ade secu i y wi h i is ecogni ion
En i onmen
unde s anding
6DoF acking
Wo ld-scale posi ional acking
Spa ial Mapping
Real- ime en i onmen mesh
Mixed Reali y Cap u e
Mixed holog am and physical en i onmen pho os
and ideo
Compu e and
connec i i y
SoC
Qualcomm Snapd agon 850 Compu e Pla o m
HPU
Second-gene a ion cus om-buil holog aphic
p ocessing uni
Memo y
4-GB LPDDR4x sys em DRAM
S o age
64-GB UFS 2.1
Wi-Fi
Wi-Fi: Wi-Fi 5 (802.11ac 2x2)
Blue oo h
5
USB
USB Type-C
Fi
Single size
Yes
Fi s o e glasses
Yes
Weigh
566g
So wa e
Windows Holog aphic Ope a ing Sys em
Mic oso Edge
Dynamics 365 Remo e Assis
Dynamics 365 Guides
3D Viewe
Powe
Ba e y li e
2–3 hou s o ac i e use
Cha ging
USB-PD o as cha ging
Cooling
Passi e (no ans)

37
HoloLens 2 has some ecommended sys em equi emen s (Table 3-2)[31] ha he hos compu e
mus mee o p ope ly enjoy he expe ience. A compu e TUF Gaming wi h a NVIDIA GeFo ce GTX 1060
GPU was used o un and connec he so wa e needed o build he mixed eali y ool. Acco ding o Table
3-2, he compu e TUF Gaming comp ises all he minimum and ecommended equi emen s o use
HoloLens 2 sys em.
Table 3-2 - Compa ison o ecommended sys em equi emen s o using HoloLens 2 [31] and he speci ica ions o he used
compu e (TUF Gaming FX505GM_FX505GM)
Componen
Recommended sys em equi emen s
TUF Gaming
FX505GM_FX505GM
CPU
64-bi wi h 4 co es o equi alen
In el® Co eTM i7-8750H CPU 2.20GHz
GPU
Di ec X 11.0 o la e
WDDM 1.2 d i e o la e
NVIDIA GeFo ce GTX 1060
RAM
8 GB o mo e
32 GB
Ope a ing
sys em
64-bi Windows 10 P o, En e p ise, o Educa ion
(Hype -V suppo )
Windows 11 Home 22H2
3.3.2 MOTION TRACKING SYSTEM: XSENS MVN AWINDA
The IMU-based mo ion cap u e sys em elies on MVN Awinda (Xsens, Enschede, The Ne he lands)
[32], [33] gi en i s eliabili y o body mo ion analysis in ee-li ing condi ions. The lowe body
con igu a ion (Figu e 3-3a) comp ises a o al o 7 wea able Wi eless Mo ion T acke s (MTw) senso s
(Figu e 3-3b) which a e placed on he body h ough adjus able s aps (Figu e 3-3c). This sys em
collec s he lowe -body kinema ic da a ha will be used o s udy he pa icipan s’ mo o pe o mance and
ac acco dingly. Fu he mo e, his sys em was used o communica e wi h Uni y so wa e, p o iding
in o ma ion abou he occu ence o gai ini ial o inal con ac (IC/FC) so ha he bio eedback could ac
in HoloLens 2.
Figu e 3-3 – Xsens MVN Awinda componen s. (a) lowe body con igu a ion; (b) MVN Mo ion
T acke (MTw); (c) MVN Awinda s aps; (d) MVN Awinda s a ion.
38
The MTw senso s ha e embedded accele ome e s, gy oscopes and magne ome e s ha p o ide 3D
accele a ion, 3D angula eloci y and 3D magne ic ield, espec i ely [32], [33]. These measu emen s
become pa icula ly in e es ing o posi ion and o ien a ion es ima ion o human body segmen s. Thus, i
was possible o de elop an algo i hm (Sec ion 4.3) o de ec ing ini ial and inal con ac s, acco ding o
he da a coming om Xsens, namely he angula eloci y in y and he linea eloci y in z o he oo
senso s. Da a om he MTw senso s a e wi elessly ansmi ed and synch onised by he Awinda S a ion
(Figu e 3-3d).
Du ing he da a acquisi ion sessions, i was used he MVN Analyze P o 2021.2, an easy- o-use
so wa e o eal- ime iewing and eco ding, which allows he expo o mo ion cap u e da a o hi d pa y
applica ions [32]. Fu he mo e, his so wa e has a s eaming ea u e which enables compu e s o
s eam he cap u ed da a o e a ne wo k o o he clien compu e , in eal- ime, Figu e 3-4.
This eal- ime ne wo k s eaming p o ocol is based on Use Da ag am P o ocol (UDP). The UDP P o ocol
is unidi ec ional, is s a eless and does no equi e he ecei e o answe incoming packe s, which allows
g ea e speed. Upon his, Xsens has de eloped plug-ins, a ailable o Uni y3D, o ee a asse s o e, o
usage wi h hi d pa y ools as a clien applica ion, allowing o ecei e mo ion cap u e da a in eal- ime.
The da a con en in he da ag am is de ined by he speci ic p o ocol se . Each da ag am s a s wi h a 24-
by e heade ollowed by a a iable numbe o by es o each body segmen , depending on he selec ed
da a p o ocol. All da a is sen in ‘ne wo k by e o de ’, which co esponds o big-endian no a ion. The
heade con ains he ype o he da a and some iden i ica ion in o ma ion, so he ecei ing end can apply
i o he igh a ge [34].
Thus, a new session was c ea ed o each “equipped” olun ee and an h opome ic da a was
measu ed and egis e ed o build he pe son’s biomechanical model. A e , a calib a ion me hod is
pe o med o align he MTw senso s wi h he use ’s body segmen s by he “Npose + Walk” ask. When a
success ul calib a ion is achie ed and he “s eam” ea u e is on, as well as he "Linea Segmen
Kinema ics", “Angula Segmen Kinema ics” and “Time code” da ag ams selec ed, i is inally possible
o s a eco ding a eal- ime session acco ding o he de ined p o ocol.
39
3.4 CONCLUSIONS
A e an ex ensi e li e a u e e iew abou he cu en ly VR/AR/MR-based app oaches used in PD, i
was no iced ha mixed eali y may be he bes echnology o be used wi h indi iduals wi h Pa kinson's
disease, as i allows i ual and in e ac i e objec s o be added o he eal wo ld, wi hou e e losing he
sense o eali y.
Thus, his disse a ion aims o explo e his echnology no only o he assessmen bu also o he
aining o PD- ela ed disabili ies. To his end, his disse a ion is inse ed in he +sense p ojec ,
con ibu ing o he +sImme si e p ojec module and makes use o wo high- ech equipmen , namely he
HoloLens 2 mixed eali y sma glasses and he Xsens mo ion acking sys em.
Figu e 3-4 – Real- ime s eaming ea u e in MVN Analyze P o.
4 SOLUTION DESCRIPTION
47
The da a acquisi ion p o ocol consis ed o pe o ming wo di e en ials, h ee imes each, which
consis ed o walking in a s aigh line along 10 me e s: (1) wi h he sma glasses OFF; and (2) wi h
he sma glasses ON, showing a i ual scena io (scena io 3, na ow spaces). In o al each
pa ien pe o med 6 ials.
A e he expe imen al p o ocol was comple ed, he acqui ed da a was analysed in o line. To do his,
he ials we e expo ed in MVN Analyze P o and hen he expo ed ials (m nx iles) we e loaded in o
MATLAB. A e wa ds, he eal- ime IC/FC de ec ions we e compa ed wi h he Xsens oo con ac signals
(g ound u h), as depic ed in Figu e 4-2. Thus,
(1) The angula eloci y signal in he y-di ec ion om bo h ee was compa ed wi h he espec i e
oo con ac signal o assess he pe o mance o eal- ime iden i ica ion o IC;
(2) The eloci y signal in he z-di ec ion om bo h ee was compa ed wi h he espec i e oo con ac
signal o assess he pe o mance o eal- ime iden i ica ion o FC.
De ec ed gai e en s we e e alua ed conside ing hei accu acy (Equa ion 4-1), p ecision (Equa ion
4-2), sensi i i y (Equa ion 4-3), and speci ici y (Equa ion 4-4). These me ics po ay he pe o mance
o he de eloped algo i hm. T ue posi i es (TP) co esponded o he gai e en s co ec ly iden i ied,
ue nega i es (TN) ep esen ed gai e en s ha he algo i hm co ec ly de ec ed as a non-e en , alse
posi i es (FP) co esponded o gai e en s no co ec ly iden i ied and alse nega i es (FN) he e en s
ha should had been de ec ed. Fu he mo e, ad ance and delayed de ec ions we e also assessed
based on hei pe cen age o occu ence and du a ion. Ad ance and delayed de ec ions we e conside ed
om he TP de ec ions.
Xsens
HoloLens 2
Figu e 4-4 –Rep esen a ion o he de ices used in he e i ica ion es s.

48
Equa ion 4-1
𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦 (%)=𝑇𝑃+𝑇𝑁
𝑇𝑃+𝑇𝑁+𝐹𝑃+𝐹𝑁
Equa ion 4-2
𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 (%)= 𝑇𝑃
𝑇𝑃+𝐹𝑃
Equa ion 4-3
𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 (%)= 𝑇𝑃
𝑇𝑃+𝐹𝑁
Equa ion 4-4
𝑆𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦 (%)= 𝑇𝑁
𝑇𝑁+𝐹𝑃
4.3.2.2 RESULTS AND DISCUSSION
Table 4-4 p esen s he pe o mance o he eal- ime IC and FC de ec ion algo i hm. I shows he
accu acy, p ecision, sensi i i y, speci ici y and, ad ance and delayed de ec ions (by means o hei
pe cen age o occu ence and du a ion).
Table 4-4 – Resul s o he e i ica ion es s
Me ic
Mean (±SD)
Accu acy (%)
98.93 (± 1.38)
P ecision (%)
100.00 (± 0.00)
Sensi i i y (%)
97.87 (± 2.75)
Speci ici y (%)
100.00 (± 0.00)
Delays ( eq %))
0.27 (± 0.52)
Delays ( ime (s))
0.01(± 0.02)
Ad ances ( eq %))
0.19 (± 0.37)
Ad ances ( ime (s))
0.02 (± 0.03)
The p oposed algo i hm showed o be signi ican ly accu a e (mean o 98.93%), sensi i e (mean o
97.87%), p ecise (100%), and speci ic (100%) o he es s pe o med, meaning ha he de eloped
algo i hm is able o de ec , wi hou much e o , he ini ial and inal con ac s, p esen ing su icien capaci y
o in eg a e he bio eedback s a egy, ha ing eached he KPI3, which de ined 96% as he pe cen age o
accu acy o be me , ha is, in a space o 10 me e s whe e 20 s eps a e aken, he algo i hm de ec s 19
IC/FCs. Howe e , an adjus men o he h esholds was subsequen ly made using exis ing da a om he
49
p ojec da abase o 9 PD pa ien s. Ne e heless, u he modi ica ions may ha e o be made in he u u e
conside ing he he e ogenei y o PD and he in a- and in e -subjec a iabili y.
4.3.2.3 CONCLUSIONS
The p oposed eal- ime IC and FC de ec ion algo i hm has shown o be accu a e, sensi i e, p ecise,
and speci ic. The adap abili y in oduced in he IC and FC de ec ion ensu es g ea e obus ness o he
sys em in he e en ual occu ence o pe u ba ions. These aspec s make his algo i hm sui able o be
in eg a ed wi h an ac ua ion sys em, i.e., wi h a bio eedback s a egy. Howe e , he e a e some u u e
challenges such as he need o alida e his algo i hm wi h (1) da a collec ed om PD pa ien s; (2) da a
collec ed om PD pa ien s a a ious s ages o he disease; and (3) da a collec ed o e ime.
4.3.3 SPATIOTEMPORAL METRICS ESTIMATION
In o de o assess whe he he imme si e i ual en i onmen s we e able o igge PD- ela ed gai
disabili ies and whe he he isual bio eedback was able o help pa ien s o e come hese impai men s,
spa io empo al me ics we e es ima ed using he mo ion da a cap u ed by Xsens. Thus, a
code was de eloped in MATLAB o his es ima ion.
Table 4-5 p esen s he calcula ed spa io empo al pa ame e s, as well as he de ini ion and o mula
o each and he uni s o measu emen . Fu he mo e, he a iabili y (SD) and asymme y (AS) o
hese me ics we e also calcula ed.
50
Table 4-5 - Spa io empo al pa ame e s: desc ip ion, o mula and uni s [39]
Spa io empo al
pa ame e
De ini ion
Fo mula
Measu ed
uni s
S ep du a ion
Time be ween he con ac o wo
consecu i e limbs in g ound
𝐼𝐶𝑖+1 − 𝐼𝐶𝑖
Seconds
S ide du a ion
Du a ion o one gai cycle, i.e., he
in e al be ween wo sequen ial ini ial
con ac s on he g ound by he same
limb
𝐼𝐶𝑖+2 − 𝐼𝐶𝑖
Seconds
S ance phase
du a ion
Du a ion o s ance phase o a io o
s ance phase du a ion wi h s ide
du a ion
(𝐹𝐶𝑖+1 − 𝐼𝐶𝑖)×100
𝑆𝑡𝑟𝑖𝑑𝑒 𝑑𝑢𝑟𝑎𝑡𝑖𝑜𝑛
Seconds o
pe cen age
Swing phase
du a ion
Du a ion o swing phase o a io o
swing phase ime wi h s ide du a ion
(𝑆𝑡𝑟𝑖𝑑𝑒𝑇𝑖𝑚𝑒𝑖+1 −𝑆𝑡𝑎𝑛𝑐𝑒𝑇𝑖𝑚𝑒𝑖)×100
𝑆𝑡𝑟𝑖𝑑𝑒 𝑑𝑢𝑟𝑎𝑡𝑖𝑜𝑛
Seconds o
pe cen age
Double suppo
phase du a ion
In e al o ime o he double suppo
phase o a io o double suppo phase
du a ion wi h s ide du a ion
(𝐼𝐶𝑖+1 −𝐹𝐶𝑖)×100
𝑆𝑡𝑟𝑖𝑑𝑒 𝑑𝑢𝑟𝑎𝑡𝑖𝑜𝑛
Seconds o
pe cen age
S ep leng h
Dis ance ha one pa o he oo
mo es in on o he same pa o he
o he oo du ing each s ep
2√2𝐿ℎ−ℎ2 ,ℎ= ∬ 𝑎𝑐𝑐𝑒𝑙𝑒𝑟𝑎𝑡𝑖𝑜𝑛𝑣𝑒𝑟𝑡𝑖𝑐𝑎𝑙
𝐼𝐶𝑖+1
𝐼𝐶𝑖
Me e s
S ide leng h
Dis ance be ween wo consecu i e
ini ial con ac s on he g ound by he
same limb
𝑠𝑡𝑒𝑝 𝑙𝑒𝑛𝑔𝑡ℎ𝑖+ 𝑠𝑡𝑒𝑝 𝑙𝑒𝑛𝑔𝑡ℎ𝑖+1
Me e s
Veloci y
Dis ance co e ed by he whole body in
a gi en ime
𝑠𝑡𝑒𝑝 𝑙𝑒𝑛𝑔𝑡ℎ𝑖
𝑠𝑡𝑒𝑝 𝑑𝑢𝑟𝑎𝑡𝑖𝑜𝑛𝑖
Me e s pe
second
Cadence
Numbe o s eps aken in a speci ic
ime
𝑣𝑒𝑙𝑜𝑐𝑖𝑡𝑦𝑖×60
𝑠𝑡𝑒𝑝 𝑙𝑒𝑛𝑔𝑡ℎ𝑖
S eps pe
minu e
ROM
Range o he signals
𝑚𝑎𝑥 (𝑎𝑐𝑐𝑒𝑙𝑒𝑟𝑎𝑡𝑖𝑜𝑛𝑥,𝑦,𝑧)
−min(𝑎𝑐𝑐𝑒𝑙𝑒𝑟𝑎𝑡𝑖𝑜𝑛𝑥,𝑦,𝑧)
Me e s pe
second
squa ed
RMS
Rela es o he ib a ion le els o a
signal
𝑟𝑚𝑠 (𝑎𝑐𝑐𝑒𝑙𝑒𝑟𝑎𝑡𝑖𝑜𝑛𝑥,𝑦,𝑧)
Me e s pe
second
squa ed
JERK
Fi s ime de i a i e o accele a ion
𝑑𝑖𝑓𝑓 (𝑎𝑐𝑐𝑒𝑙𝑒𝑟𝑎𝑡𝑖𝑜𝑛𝑥,𝑦,𝑧)
Me e s pe
second cubed
ROM: ange o mo ion; RMS: Roo mean squa e.
51
Figu e 4-5 is a ep esen a ion o he gai cycle o a heal hy subjec , o easie in e p e a ion o he
concep s.
When he pa icipan is exposed o i ual en i onmen s in ended o assess mo o pe o mance,
empo al me ics, such as s ep du a ion, a e expec ed o inc ease and spa ial me ics, including s ep
leng h and eloci y, o dec ease, as hese pa ien s end o p esen a mo e cau ious beha iou in
pe o ming hese asks, leading o slowe and smalle s eps [2], [12], [20]
Con e sely, when bio eedback is used, empo al me ics a e expec ed o dec ease, and spa ial
me ics a e expec ed o inc ease, i.e., mo e s able gai pa e n, as e , and bigge s eps [1], [13]–[15],
[17].
4.4 BIOFEEDBACK INTEGRATION
Rega ding bio eedback s a egies, wo modali ies we e de eloped, one in open loop and he o he in
closed loop, which a e explained below.
4.4.1 OPEN LOOP STRATEGY
The bio eedback s a egy o i ual scena io 1, co ido wi h dices, consis ed o using isual cues ha
indica ed he pa h o be aken by he hand wi h he dice, om he ini ial loca ion o he dice o he
co esponding colou ed box. To his end, a ows we e d awn in he ai , in he colou o he co esponding
Figu e 4-5 - Gai cycle o a heal hy subjec .
52
dice, o help he use o u n. A ows we e chosen because hey ep esen di ec ion and mo emen , ying
o acili a e he u ning o he body. Each se o a ows displayed he colou ela i e o he playing dice so
ha he use would no be con used abou which dice is ca ying. This s a egy is in ended o make
u ning a mo e luid and easie ask, by p o iding isuospa ial cueing in o ma ion. Figu e 4-6 p esen s
he s a egy men ioned.
4.4.2 CLOSED LOOP STRATEGY
The closed-loop bio eedback s a egy was achie ed by he communica ion p o ocol be ween
MVN Analyze P o and Uni y, using he eal- ime IC/FC de ec ion algo i hm (Sec ion 4.3.1).
This way, he s a egy o i ual scena ios 2 ( i ual doo ) and 3 (na ow spaces) consis ed o
p esen ing isual cues on he loo in on o he use in a closed loop. In his case, i was chosen o use
oo p in s since hey show ele ance in he gai . The g een was chosen since he eye is mos sensi i e o
a yellowish-g een colou unde no mal ligh ing condi ions [35], [36] and, beyond ha , g een is ela ed o
“being igh ”, “p oceeding”, unlike de colou ed, as in a ic ligh s. Thus, when he use places he igh
oo on he loo , a igh IC o heel s ike is de ec ed, and he sys em will place a le g een oo p in o
se e as a spa ial guideline, indica ing whe e o place he le oo . On he con a y, once he use places
he le oo on he loo , a le IC o heel s ike is de ec ed, and he igh g een oo p in is displayed.
The e o e, a mo e luid and con inuous gai is mo i a ed. Figu e 4-7 shows a g een oo p in o he igh
oo a e de ec ion o a heel s ike om he le oo .
Figu e 4-6 - Bio eedback s a egy o scena io 1.

53
4.5 CONCLUSIONS
This chap e has p esen ed he me hods used o implemen a modula , use -cus omised, mixed
eali y-based echnology solu ion ha (1) imme ses pa ien s in en i onmen s ha cause PD- ela ed gai
impai men s; and (2) imme ses pa ien s in en i onmen s ha help o e come hese impai men s wi h he
aid o HoloLens 2, Xsens and bio eedback s a egies (RQ2).
The i ual en i onmen s and asks we e de ined, ha ing de eloped h ee di e en en i onmen s ha
aimed o ep esen he eal-li e si ua ions ha mos cause PD-gai disabili ies in PD pa ien s, namely (1)
u ning (scena io 1); (2) walking h ough doo s (scena io 2); and (3) walking in na ow spaces (scena io
3).
Rega ding bio eedback s a egies, a ows and oo p in s we e added o he i ual en i onmen s, and
i would be expec ed ha he use would ollow hese isual cues. The oo p in s we e p o ided in closed
loop, i.e. as he use walks, mo e oo p in s will appea which a e ac i a ed by he occu ence o ICs.
Thus, when he use pu s one oo on he g ound, he HoloLens p ojec s he oo p in ela i e o he
opposi e oo in o de o p omo e a mo e con inuous and luid gai . To make his possible, an algo i hm
was de eloped o de ec ICs and FCs in eal ime, based on adap i e h esholds.
Fu he mo e, a MATLAB code was de eloped o he es ima ion o spa io empo al me ics in o line,
so ha i was possible o e alua e he mo o pe o mance a e exposu e o he i ual en i onmen s and
a e exposu e o he bio eedback s a egies in hese pa ien s.
Figu e 4-7 – Display o he igh g een oo p in .
5 SOLUTION VALIDATION
55
This chap e desc ibes he me hodologies o alida ing he solu ion. Fi s ly, he p o ocol used in he
alida ion o he mixed eali y s a egies wi h indi iduals wi h PD is p esen ed. This alida ion p o ocol
ollowed a p e-pos expe imen al s udy design aiming o e alua e he subjec s’ mo o pe o mance. The
pa icipan s and hei cha ac e is ics a e speci ied, as well as he inclusion and exclusion c i e ia. Nex ,
he ma e ials used in he in e en ion a e desc ibed, ollowed by he da a acquisi ion me hods and he
s udy a iables. The da a p ocessing conduc ed o achie e he in ended ou comes measu es is exposed,
as well as he s a is ical analysis pe o med. Finally, he esul s ob ained a e p esen ed, as well as a
de ailed discussion o hem, answe ing RQ3.
5.1 INTRODUCTION
5.1.1 HYPOTHESIS, RESEARCH QUESTION AND STUDY DESIGN
PD is cu en ly incu able, so i s ea men consis s o applying in e en ions ha slow down he apid
p og ession o he disease. The mixed eali y s a egies de eloped in his disse a ion aim o b ing he
day- o-day eali y o he pa ien close o he medical appoin men . In ac , i becomes c i ical ha hese
pa ien s a e co ec ly and objec i ely assessed, since disease p og ession is e y as . In addi ion, i is
in ended o e i y whe he his s a egy could complemen ehabili a ion sessions, h ough a mo e un
and disease- ocused aining, using isual cues, wi h he aim o imp o ing hei mobili y and au onomy.
In ha sense, he esea ch ques ion o be answe ed is RQ3: “How does he implemen ed modula
echnological solu ion, based on mixed eali y in eg a ed wi h a mo ion acking sys em and wi h
bio eedback s a egies, a ec he mo o pe o mance o PD pa ien s du ing assessmen and aining?”
The s udy in ques ion consis ed o a c oss-sec ional s udy as an obse a ion o a de ined
popula ion was conduc ed a a single poin in ime. Exposu e o he in e en ion and ou come we e
de e mined simul aneously [40].
5.2 METHODOLOGY
The alida ion p o ocol wi h pa hological end-use s was conduc ed in Hospi al o B aga, wi h he
collabo a ion o he physicians om 2CA-B aga, ollowing he p inciples o he Decla a ion o Helsinki and
he O iedo Con en ion, in acco dance wi h he e hical guidelines o he E hics Commi ee in Li e and
Heal h Sciences (CEICVS 147/2021). All pa icipan s illed ou an in o med consen o pa icipa e in he
cu en esea ch.
56
5.2.1 PARTICIPANTS
Ele en subjec s (six emales and i e males) we e ec ui ed and accep ed o pa icipa e in his da a
collec ion. A lis o inclusion and exclusion c i e ia was ou lined in o de o selec he pa icipan s.
Pa icipan s we e ec ui ed i hey had: I) diagnosis o PD acco ding o he UK Pa kinson’s Disease Socie y
B ain bank c i e ia; II) p esence o eezing o gai ; III) Hoehn and Yah s age be ween 1 and 4; IV) age
be ween 45 and 85 yea s old; and V) able o walk wi hou assis ance. Exclusion c i e ia we e: I) p esence
o como bid diso de s likely o a ec gai , including s oke, o hopaedic disease, heuma ologic disease,
o he neu ological and musculoskele al diso de s, ca dio ascula and pulmona y diseases; II)
signi ica i e cogni i e impai men (MMSE<24); III) ob ious mo o impai men s; IV) isual acui y de ici s;
V) audiome ic de ici s; VI) pain ha may a ec walking; and VII) inabili y o pe o m a 180º u n wi hou
assis ance. Table 5-1 p esen s he pa icipan s’ de ailed clinical cha ac e is ics and an h opome ics.
Table 5-1 - Demog aphic in o ma ion abou he PD pa icipan s
Pa icipan
ID
Gende
(M/F)
Age
(yea s)
Body
heigh
(cm)
Body
mass
(kg)
Clinical
S a e
NFoG-Q
UPDRS,
Pa III
H&Y
PD- 01
F
74
160
62
ON
27
66
4
PD- 02
F
48
166
65
ON
24
4
PD- 03
F
67
164
73
ON
24
71
3
PD 04
F
59
155
71
ON
0
18
2
PD 05
M
75
168
80
ON
0
12
2
PD- 06
M
71
163
85
ON
24
19
2
PD- 07
M
65
172
60
ON
25
15
1
PD- 08
F
70
163
77
ON
13
22
2
PD- 09
F
47
169
64
ON
12
14
1
PD- 10
M
83
160
75
ON
26
57
4
PD- 11
M
84
162
62
ON
26
42
3
Mean (±STD)
-
67.55 (±
11.71)
163.82 (±
4.55)
70.36 (±
7.96)
-
18.28 (±
9.88)
31.86
(±22.03)
2.29
(±1.08)
ID: iden i ica ion; PD: indi idual wi h Pa kinson's disease; PD- : indi idual wi h Pa kinson's disease and eezing o gai ; M:
male; F: emale; NFoG-Q: New eezing o gai ques ionnai e; UPDRS-III: Uni ied Pa kinson’s disease a ing scale – pa III;
H&Y: Hoehn and Yah scale; STD: s anda d de ia ion.
5.2.2 MATERIALS
The ma e ials used in he alida ion phase we e he HoloLens 2 mixed eali y sma glasses and he
Xsens mo ion acking sys em. In addi ion, a documen eco ding he pa icipan s' demog aphic and
clinical in o ma ion was also used. HoloLens 2 was used o display he i ual en i onmen s and o p o ide
he isual cues (a ows and g een oo p in s). In u n, Xsens was used o s eam ine ial da a o Uni y a
63
5.2.6 STATISTICAL ANALYSIS
The s a is ical analysis was pe o med h ough IBM SPSS so wa e e sion 25.0 ( o Windows)
(IMP Co p, A monk, NY, USA). Fi s ly, desc ip i e s a is ics we e ob ained o summa ise he esul s
(means and s anda d de ia ions) o each g oup and he da a no mali y was assessed using Shapi o-
Wilk es . The popula ion was conside ed no mally dis ibu ed i he signi icance alue was highe han
0.05. This s udy p esen s pai ed samples as he samples a e om he same pa icipan s, Table 5-8.
In his sense, o compa e wo pai ed g oups, pai ed es was pe o med o he pa ame ic me ics and
he Wilcoxon signed- ank es was applied o a iables whe e he assump ion o no mali y was no e i ied.
When mo e han wo pai ed g oups we e o be compa ed, epea ed-measu es ANOVA was used o
no mal popula ions, on he con a y, he F iedman es was used o non-no mal a iables.
All s a is ical es s we e execu ed conside ing a con idence le el o 95% (α = 0.05). The s a is ical
es s we e conduc ed o e alua e he ollowing null hypo hesis (H0): “ he e a e s a is ically
signi ican di e ences be ween in e en ions”. I
p
- alue<0.05, he H0 is accep ed.
Table 5-8 - Pa ame ic and non-pa ame ic es s
Type o da a
Goal
Measu emen
(o no mal popula ions)
O de , esul o measu e
(o non-no mal popula ions)
Compa e wo pa ed samples
Pai ed es
Wilcoxon es
Compa e mo e han wo pa ed
samples
Repea ed-measu es ANOVA
F iedman es
5.3 RESULTS
This subchap e aims o p esen he esul s om he alida ion p o ocol wi h PD pa ien s. The esul s
a e di ided in o h ee sec ions. Fi s ly, he esul s conce ning he mo o assessmen a e p esen ed,
ollowed by he esul s ela ed o he mo o aining. In addi ion, he esul s o eezing gai episodes
(numbe and du a ion) pe es a e shown. Finally, he answe s o he use ’s expe ience e alua ion es s
a e depic ed.
Ele en pa ien s unde wen he es s, wi h en pa ien s comple ing all es s. Pa ien 10 d opped ou
due o a igue/no in e es . Mos pa icipan s success ully comple ed all he es s p o ided, in an a e age
du a ion o 40 minu es.

64
5.3.1 IMMERSIVE VIRTUAL FRAMEWORK FOR MOTOR ASSESSMENT
Fi s ly, he aim was o check whe he wea ing he HoloLens 2 OFF would ha e any in luence on he
mo o unc ion o hese pa ien s. To his end, he TC1 and TC2 es s we e compa ed. Rega dless o he
ou come o he compa ison o hese es s, he TC1 was selec ed o be compa ed o he o he es s o
he pu pose o esul s analysis. Nex , i was aimed o s udy he po en ial o mixed eali y o cause PD-gai
disabili ies by compa ing he TC1 wi h he M2 and, la e , he TC1 wi h he M3.
Thus, he i s s ep o he s a is ical analysis was o pe o m a desc ip i e analysis by es . The esul s
o his s ep a e shown in Table 5-10, o con ol es s 1 and 2 (TC1, TC2) and moni o ing es s 2 and 3
(M2 and M3).
A e wa ds, he no mali y o he ea u es was e i ied using he Shapi o-Wilk es . As mos o he
ea u es did no show a no mal dis ibu ion, he Wilcoxon es was chosen o compa ison o wo pai ed
samples. The esul s o his es a e also shown in Table 5-10, in which he di e ence o a ce ain
ea u e be ween he se e al es s is conside ed signi ican i he signi icance alue is smalle han 0.05
and hese a e in bold.
Rega ding con ol es s TC1 and TC2, only a ew me ics, namely, s ep and s ide leng h, eloci y
and AS swing ime, p esen ed s a is ically signi ican di e ences, since he p- alue < 0.05, co obo a ing
he null hypo hesis. A e wa ds, he compa ison be ween TC1 and M2 showed s a is ically signi ican
di e ences in he me ics s ep and s ide du a ion, s ance, swing and double suppo phase, eloci y, SD
s ep and s ide du a ion, SD s ance and swing phase, SD s ep leng h, SD cadence and AS s ance and
swing ime, co obo a ing he null hypo hesis. Finally, he compa ison be ween TC1 and M3 showed
s a is ically signi ican di e ences in almos all me ics, so he null hypo hesis is co obo a ed. All hese
me ics a e in bold.
Th oughou da a acquisi ion, he esea che s isually assessed he exis ence o gai eezing episodes.
La e , du ing da a p ocessing, hese episodes we e excluded and hei numbe , a e age and o al du a ion
we e coun ed pe es . The esul s a e p esen ed in Table 5-9.
Figu e 5-5 p esen s wo QR codes showing ideos o pa ien 7 pe o ming es s M2 and M3.
65
Table 5-9 - Numbe , a e age and o al du a ion os eezing o gai episodes in con ol es s and moni o ing es s 2 and 3
Tes
Numbe o episodes
A e age du a ion (s)
To al du a ion (s)
TC1
0
0
0
TC2
0
0
0
M2
4
6.73
74
M3
0
0
0
Figu e 5-5 – Videos o pa icipan 7 pe o ming he M2 and M3 es s.
66
Table 5-10 - Spa io empo al me ics and hei desc ip i e s a is ics, Shapi o-Wilk es and Wilcoxon es o con ol es s and scena ios 2 and 3
TC1
TC2
Wilcoxon
Tes
(sig)
(TC1-TC2)
M2
Wilcoxon
Tes
(sig)
(TC1-M2)
M3
Wilcoxon
Tes
(sig)
(TC1-M3)
Me ic
Mean
S d
de ia ion
Shapi o-
Wilk
(sig)
Mean
S d
de ia ion
Shapi o-
Wilk
(sig)
Mean
S d
de ia ion
Shapi o-
Wilk
(sig)
Mean
S d
de ia ion
Shapi o-
Wilk
(sig)
S ep du a ion
0.618
0.087
0.298
0.618
0.082
0.144
0.811
0.740
0.233
0.254
0.022
0.812
0.360
0.001
0.003
S ide du a ion
1.236
0.176
0.345
1.235
0.165
0.122
0.868
1.478
0.472
0.218
0.025
1.615
0.722
0.001
0.005
S ance phase
62.057
3.047
0.168
62.688
3.295
0.081
0.053
67.056
5.887
0.383
0.000
67.360
7.848
0.002
0.000
Swing phase
37.943
3.047
0.168
37.312
3.295
0.081
0.053
32.914
5.937
0.361
0.000
32.640
7.848
0.002
0.000
Double suppo
phase
24.196
6.104
0.170
25.508
6.550
0.102
0.058
34.562
12.255
0.247
0.000
34.442
16.290
0.001
0.000
S ep leng h
0.568
0.096
0.378
0.519
0.135
0.022
0.004
0.509
0.165
0.075
0.053
0.500
0.127
0.613
0.003
S ide leng h
1.143
0.199
0.265
1.046
0.257
0.033
0.005
1.019
0.344
0.108
0.053
0.994
0.258
0.596
0.004
Veloci y
0.934
0.198
0.226
0.853
0.218
0.047
0.012
0.750
0.275
0.128
0.002
0.702
0.215
0.273
0.000
Cadence
99.395
13.881
0.149
99.988
13.399
0.035
0.744
95.866
27.003
0.399
0.396
89.104
21.928
0.042
0.006
SD s ep ime
0.040
0.022
0.000
0.055
0.059
0.000
0.616
0.222
0.274
0.000
0.001
0.240
0.370
0.000
0.002
SD s ide ime
0.055
0.037
0.000
0.076
0.090
0.000
0.828
0.307
0.370
0.000
0.002
0.317
0.447
0.000
0.003
SD s ance ime
0.045
0.026
0.000
0.068
0.092
0.000
0.616
0.280
0.344
0.000
0.001
0.259
0.396
0.000
0.002
SD swing ime
0.031
0.022
0.000
0.037
0.033
0.000
0.557
0.101
0.101
0.000
0.005
0.147
0.202
0.000
0.003
SD s ep leng h
0.139
0.067
0.049
0.119
0.052
0.043
0.184
0.180
0.102
0.043
0.039
0.159
0.105
0.001
0.420
SD s ide leng h
0.182
0.121
0.002
0.134
0.089
0.000
0.145
0.252
0.166
0.072
0.133
0.211
0.171
0.002
0.528
SD eloci y
0.229
0.119
0.008
0.191
0.084
0.087
0.170
0.247
0.126
0.259
0.396
0.204
0.125
0.004
0.231
SD cadence
6.170
2.530
0.002
7.453
5.207
0.001
0.500
19.854
19.948
0.000
0.004
13.895
15.022
0.000
0.039
AS s ep ime
0.034
0.023
0.092
0.032
0.022
0.299
0.695
0.114
0.179
0.000
0.231
0.136
0.173
0.000
0.017
AS s ide ime
0.009
0.011
0.000
0.005
0.006
0.001
0.316
0.026
0.051
0.000
0.446
0.033
0.068
0.000
0.758
AS s ance ime
0.018
0.016
0.038
0.027
0.030
0.002
0.085
0.048
0.057
0.000
0.031
0.069
0.119
0.000
0.078
AS swing ime
0.011
0.016
0.000
0.031
0.033
0.000
0.004
0.047
0.062
0.000
0.011
0.094
0.140
0.000
0.008
AS s ep leng h
0.112
0.079
0.053
0.121
0.092
0.044
0.777
0.102
0.071
0.080
0.845
0.109
0.077
0.058
0.983
AS s ide leng h
0.043
0.043
0.002
0.026
0.031
0.001
0.085
0.038
0.040
0.003
0.586
0.039
0.043
0.002
0.472
AS eloci y
0.176
0.130
0.023
0.184
0.132
0.111
0.349
0.159
0.090
0.103
0.913
0.173
0.104
0.158
0.616
AS cadence
5.546
3.910
0.241
5.768
3.701
0.176
0.557
7.072
8.293
0.000
0.845
9.726
7.091
0.006
0.071
67
5.3.2 IMMERSIVE VIRTUAL FRAMEWORK FOR MOTOR TRAINING
This subchap e p esen s he esul s o he bio eedback con ibu ion o mo o pe o mance, i.e., es s
M1 wi h T1 (dice), M2 wi h T2 (doo ) and M3 wi h T3 (na ow spaces) we e compa ed. Thus,
a desc ip i e analysis pe es was i s pe o med and hen he no mali y o he ea u es was s udied,
using Shapi o-Wilk es . As mos o he me ics did no p esen a no mal dis ibu ion, he Wilcoxon es
was used o compa ison o wo pai ed samples. The esul s o hese es s a e shown in Table 5-12,
Table 5-13 and Table 5-14, in which he di e ence o a ce ain ea u e be ween he se e al es s is
conside ed signi ican i he signi icance alue is smalle han 0.05.
Rega ding scena io 1, no me ics showed s a is ically signi ican di e ences, ejec ing he null
hypo hesis. In u n, scena io 2 showed ha only he me ics s ep du a ion, s ide du a ion, eloci y,
cadence, and SD eloci y, p esen ed s a is ically signi ican di e ences, since he p- alue < 0.05,
co obo a ing he null hypo hesis. Finally, scena io 3 had wo spa io empo al me ics ha showed
s a is ically signi ican di e ences, namely s ep leng h and cadence. Thus, he null hypo hesis is
co obo a ed. All hese me ics a e in bold.
Th oughou da a acquisi ion, he esea che s isually assessed he exis ence o gai eezing episodes.
La e , du ing da a p ocessing, hese episodes we e excluded and hei numbe , a e age and o al du a ion
we e coun ed pe es . The esul s a e p esen ed in Table 5-11.
Figu e 5-6 p esen s six QR codes showing ideos o pa ien 7 pe o ming es s M1, T1, M2, T2, M3
and T3.
Table 5-11 - Numbe , a e age and o al du a ion os eezing o gai episodes in moni o ing es s 1, 2, and 3 and aining
es s 1, 2 and 3
Tes
Numbe o episodes
A e age du a ion (s)
To al du a ion (s)
M1
0
0
0
T1
0
0
0
M2
4
6.73
74
T2
6
9.78
107.6
M3
0
0
0
T3
4
3.13
34.4
68
Table 5-12 - Spa io empo al me ics and hei desc ip i e s a is ics, Shapi o-Wilk es and Wilcoxon es o scena io 1
M1
T1
Wilcoxon Tes
(sig)
Me ic
Mean
S d
de ia ion
Shapi o-Wilk
(sig)
Mean
S d
de ia ion
Shapi o-Wilk
(sig)
ROM X (+)
3.630
1.427
0.049
3.472
1.269
0.002
0.390
ROM Y (+)
3.616
1.279
0.050
3.241
0.851
0.266
0.372
ROM Z (+)
4.720
3.046
0.000
3.985
3.422
0.000
0.168
RMS X (+)
0.478
0.217
0.003
0.508
0.214
0.018
0.178
RMS Y (+)
0.480
0.180
0.195
0.491
0.149
0.279
0.615
RMS Z (+)
0.380
0.157
0.092
0.383
0.222
0.001
0.833
JERK X (-)
0.002
0.006
0.007
0.001
0.005
0.005
0.158
JERK Y (-)
0.002
0.004
0.013
0.000
0.003
0.502
0.123
JERK Z (-)
0.000
0.004
0.054
0.001
.003
0.041
0.661
Table 5-13 - Spa io empo al me ics and hei desc ip i e s a is ics, Shapi o-Wilk es and Wilcoxon es o scena io 2
M2
T2
Wilcoxon Tes
(sig)
Me ic
Mean
S d
de ia ion
Shapi o-Wilk
(sig)
Mean
S d
de ia ion
Shapi o-Wilk
(sig)
S ep du a ion
0.758
0.228
0.274
0.848
0.298
0.000
0.031
S ide du a ion
1.513
0.463
0.230
1.696
0.606
0.000
0.028
S ance phase
67.517
5.724
0.536
69.522
5.243
0.393
0.064
Swing phase
32.451
5.775
0.505
30.478
5.243
0.393
0.071
Double suppo phase
35.519
11.918
0.252
38.728
10.415
0.524
0.170
S ep leng h
0.518
0.165
0.093
0.499
0.189
0.135
0.948
S ide leng h
1.038
0.345
0.135
0.991
0.408
0.045
0.777
Veloci y
0.747
0.283
0.118
0.630
0.268
0.008
0.016
Cadence
93.068
25.000
0.319
79.108
15.118
0.282
0.002
SD s ep ime
0.227
0.282
0.000
0.209
0.257
0.000
0.446
SD s ide ime
0.315
0.380
0.000
0.285
0.278
0.000
0.327
SD s ance ime
0.292
0.351
0.000
0.261
0.255
0.000
0.332
SD swing ime
0.098
0.103
0.000
0.118
0.146
0.000
0.231
SD s ep leng h
0.182
0.104
0.070
0.168
0.092
0.006
0.647
SD s ide leng h
0.254
0.171
0.087
0.210
0.135
0.004
0.586
SD eloci y
0.246
0.130
0.181
0.197
0.127
0.000
0.028
SD cadence
18.870
20.107
0.000
13.568
7.317
0.003
0.557
AS s ep ime
0.118
0.183
0.000
0.090
0.075
0.044
0.616
AS s ide ime
0.027
0.053
0.000
0.025
0.029
0.002
0.845
Figu e 5-6 - Videos o pa icipan 7 pe o ming es s M1, T1, M2, T2, M3 and T3.

69
AS s ance ime
0.048
0.058
0.000
0.064
0.102
0.000
0.983
AS swing ime
0.047
0.063
0.000
0.060
0.119
0.000
0.286
AS s ep leng h
0.104
0.072
0.145
0.134
0.100
0.133
0.184
AS s ide leng h
0.040
0.040
0.005
0.039
0.044
0.000
0.647
AS eloci y
0.158
0.092
0.077
0.159
0.113
0.241
0.586
AS cadence
7.044
8.547
0.000
7.023
4.985
0.032
0.913
Table 5-14 - Spa io empo al me ics and hei desc ip i e s a is ics, Shapi o-Wilk es and Wilcoxon es o scena io 3
M3
T3
Wilcoxon es
(sig)
Me ic
Mean
S d
de ia ion
Shapi o-Wilk
(sig)
Mean
S d
de ia ion
Shapi o-Wilk
(sig)
S ep du a ion (-)
0.762
0.229
0.022
0.821
0.398
0.000
0.147
S ide du a ion (-)
1.512
0.449
0.023
1.627
0.746
0.000
0.147
S ance phase
67.644
7.198
0.001
66.531
3.850
0.223
0.520
Swing phase
32.356
7.198
0.001
33.469
3.850
0.223
0.520
Double suppo phase
35.295
15.152
0.001
33.065
8.002
0.232
0.314
S ep leng h (+)
0.483
0.107
0.418
0.544
0.128
0.229
0.044
S ide leng h (+)
0.960
0.217
0.539
1.071
0.263
0.146
0.070
Veloci y (+)
0.693
0.191
0.652
0.744
0.263
0.686
0.841
Cadence
89.330
16.968
0.027
83.140
19.546
0.290
0.024
SD s ep ime
0.192
0.303
0.000
0.161
0.236
0.000
0.811
SD s ide ime
0.277
0.393
0.000
0.199
0.239
0.000
0.809
SD s ance ime
0.220
0.343
0.000
0.203
0.324
0.000
0.936
SD swing ime
0.122
0.164
0.000
0.123
0.215
0.000
0.841
SD s ep leng h
0.144
0.098
0.000
0.162
0.106
0.001
0.421
SD s ide leng h
0.184
0.163
0.000
0.212
0.178
0.000
0.314
SD eloci y
0.190
0.108
0.003
0.209
0.135
0.001
0.904
SD cadence
13.360
15.084
0.000
9.254
4.855
0.087
0.445
AS s ep ime
0.124
0.174
0.000
0.136
0.294
0.000
0.421
AS s ide ime
0.030
0.070
0.000
0.037
0.103
0.000
0.557
AS s ance ime
0.047
0.078
0.000
0.142
0.403
0.000
0.445
AS swing ime
0.069
0.097
0.000
0.115
0.300
0.000
0.825
AS s ep leng h
0.118
0.073
0.111
0.111
0.104
0.013
0.398
AS s ide leng h
0.027
0.028
0.005
0.049
0.077
0.000
0.227
AS eloci y
0.173
0.103
0.273
0.140
0.107
0.008
0.122
AS cadence
8.772
6.882
0.000
7.226
6.754
0.008
0.277
5.3.3 SSQ AND IMI QUESTIONNAIRES
A he end o da a acquisi ion, pa icipan s comple ed he a o emen ioned accep abili y
ques ionnai es, SSQ and IMI (Appendix B – Subjec i e ques ionnai es). Table 5-15 p esen s he
esul s o hese ques ionnai es.
70
Table 5-15 - SSQ and IMI ques ionnai es esul s
Pa icipan ID
SSQ
IMI
In e es /Enjoymen
subscale
Value/Use ulness
subscale
PD- 01
4
5.67
5.33
PD- 02
3
5.67
5.33
PD- 03
4
7
6.33
PD 04
0
7
5.67
PD 05
0
7
7
PD- 06
4
5.33
7
PD- 07
0
7
7
PD- 08
3
6.67
6.67
PD- 09
1
7
7
PD- 10
1
7
6.67
PD- 11
1
7
6.33
Mean (±STD)
1.91 (± 1.62)
6.58 (± 0.64)
6.39 (± 0.63)
6.49 (± 0.64)
5.4 DISCUSSION
This subsec ion discusses he esul s ob ained in he p e ious subsec ion. The analysis is made o
bo h “mo o assessmen ” and “mo o aining” sepa a ely. In addi ion, a b ie discussion is elabo a ed
ega ding he occu ence o FoG episodes and also he esul s o he accep abili y ques ionnai es.
5.4.1 IMMERSIVE VIRTUAL FRAMEWORK FOR MOTOR ASSESSMENT
Looking a he mean alues o he me ics o he wo con ol es s (TC1 and TC2), i can be seen
ha hey ha dly a ied, wi h he excep ion o s ep and s ide leng h and eloci y. Ac ually, he e e ed
me ics plus AS swing ime p esen ed s a is ically signi ican di e ences. Thus, i is concluded ha he
use o HoloLens in luences mo o pe o mance in he men ioned me ics, e en i hey a e swi ched o .
This may be due o he p esence o he HoloLens lenses ha a e no comple ely anspa en , seeing some
e lec ions.
Vi ual en i onmen 2 (doo s) showed an inc ease in he mean alues o s ep and s ide du a ion,
and a dec ease in s ep and s ide leng h, eloci y, and cadence. These me ics beha ed as expec ed (jus
like [2]) wi h he excep ion o cadence, which should ha e inc eased.
In ac , his i ual en i onmen showed ou een me ics ou o wen y- i e (s ep and s ide du a ion,
s ance, swing and double suppo phase, eloci y, SD s ep and s ide du a ion, SD s ance and swing
phase, SD s ep leng h, SD cadence and AS s ance and swing ime) wi h s a is ically signi ican di e ences
71
be ween he con ol and moni o ing es s. This means ha he MR echnology eally dis u bed he mo o
pe o mance o he pa ien s in he way ha was expec ed, due o he p esence o i ual objec s and
especially he i ual doo ha was able o ec ea e a eal doo .
In u n, i ual en i onmen 3 (na ow spaces) showed an inc ease in he mean alues o s ep
and s ide du a ion and a dec ease in s ep and s ide leng h, eloci y, and cadence. Ac ually, hese me ics
beha ed as expec ed wi h he excep ion o cadence, which should ha e inc eased.
Obse ing he esul s o he Wilcoxon es , six een ou o wen y- i e me ics showed s a is ically
signi ican di e ences, demons a ing ha he MR echnology may in ac ha e igge ed PD-gai ela ed
disabili ies. This may be due o he p esence o he a ious i ual objec s ha c ea ed a na owe co ido
han he eal co ido , ac ing as obs acles o he pa icipan , making him ake smalle and slowe s eps.
Wi h ega d o he occu ence o FoG episodes, he moni o ing es s should ha e inc eased
hei numbe and du a ion. Howe e , a e analysing he esul s, i was ound ha moni o ing es 2
was he only one ha caused hese episodes. This may be due o he ac ha : (1) disease may be
“masked” by medica ion, causing pa icipan s no o su e om FoG, since da a acquisi ion was
pe o med 1h a e medica ion in ake on a e age, i.e., in “ON” phase; (2) he e ogenei y in he o igin o
FoG, since some pa icipan s epo ed ha hey su e om hese episodes in s ess ul si ua ions, o he
pa ien s su e igh a e waking up, as well as, o he pa ien s su e when hey a e in c owded places.
5.4.2 IMMERSIVE VIRTUAL FRAMEWORK FOR MOTOR TRAINING
Rega ding i ual en i onmen 1 (dices), one would expec he a e age ROM and RMS alues o
inc ease and he a e age JERK alues o dec ease wi h he use o he bio eedback s a egy. Howe e , by
analysing he esul s one no ices ha he a e age ROM alues o he h ee axes dec eased, in u n he
a e age RMS alues o all axes inc eased and he same happened o he a e age JERK alues. Thus,
only he RMS beha ed as expec ed.
F om he esul s o he Wilcoxon es , no s a is ically signi ican di e ences we e ound, making i
possible o men ion ha he isual bio eedback s a egy, namely he colou ed a ows, had no e iden
impac . This may ha e occu ed because he sma glasses do no ha e a su icien ield o iew (FOV),
o cing pa ien s o look in he di ec ion o he loo . Ano he eason could be ha he e was li le ime in
con ac wi h he bio eedback s a egy.
Vi ual en i onmen 2 (doo s) showed an inc ease in he mean alues o s ep and s ide du a ion,
a dec ease in he mean alues o s ep and s ide leng h as well as a dec ease in he mean alues o
eloci y and cadence. Ne e heless, s ep and s ide du a ion and cadence we e expec ed o dec ease,
72
s ep and s ide leng h we e expec ed o inc ease, jus as eloci y, since his bio eedback s a egy aims o
imp o e mo o pe o mance, ob aining la ge and as e s eps.
In ac , his i ual en i onmen showed some me ics (s ep and s ide du a ion, eloci y, SD eloci y
and cadence) wi h s a is ically signi ican di e ences be ween he moni o ing and aining es s. Thus,
bio eedback may ha e nega i ely a ec ed hese me ics once he mean alues beha ed con a y o wha
was expec ed. This may be due o (1) he pa icipan was le wai ing o he oo p in s, and (2) he eal-
ime IC and FC de ec ion algo i hm was no sui able o hese pa ien s' gai causing he oo p in s no o
appea igh away, inc easing hei eac ion ime. On he o he hand, he cadence alues beha ed as
expec ed. This may be due o he ac ha he oo p in s indica e spa ial in o ma ion o he pa icipan ,
i.e., whe e o place he nex oo .
In u n, i ual en i onmen 3 (na ow spaces) showed an inc ease in he mean alues o s ep
and s ide du a ion as well as s ep, s ide leng h and eloci y. On he o he hand, he cadence dec eased
i s mean alue wi h he use o he oo p in s. Ac ually, s ep and s ide leng h, eloci y and cadence
beha ed as i was expec ed. On he con a y, s ep and s ide du a ion should ha e dec eased.
The e a e wo spa io empo al me ics ha showed s a is ically signi ican di e ences, namely s ep
leng h and cadence. This may be due o he in en ion o he oo p in s o " o ce" he pa icipan o be
awa e o hem and o help planning whe e o place his ee , guiding him o he inish line. In addi ion, he
oo p in s had a p e-de ined dis ance be ween hem (dependen on he heigh o he pa icipan ),
in luencing he pa icipan o ollow and imi a e he isual cues.
In wha conce ns he occu ence o eezing o gai episodes, he numbe and du a ion o hese
episodes would be expec ed o educe o disappea in he bio eedback aining ials.
Howe e , his did no occu and may be due o (1) un amilia i y wi h isual cues, as pa icipan s epo ed
ha hey had ne e in e ac ed wi h hese; (2) educed ield o iew o he HoloLens 2 causing pa icipan s
o some imes ail o see isual cues.
5.4.3 SSQ AND IMI QUESTIONNAIRES
A e analysing he esul s ob ained o he SSQ i was concluded ha hey did no e lec any
symp oms (nausea, diso ien a ion, and oculomo o ) a e exposu e o he i ual en i onmen s.
Fu he mo e, no pa icipan s e bally indica ed ha hey had symp oms o simula o sickness.
In u n, he esul s o he IMI ques ionnai e showed ha he sys em ecei ed high a ings on he
in e es and alue subscales. The pa icipan s we e always happy and willing o pa icipa e in he es s,
79
[1] Y. W. Wang, C. H. Chen, and Y. C. Lin, “Balance Rehabili a ion Sys em o Pa kinson’s Disease
Pa ien s based on Augmen ed Reali y,”
2nd IEEE Eu asia Con e ence on IOT, Communica ion and
Enginee ing 2020, ECICE 2020
, pp. 191–194, Oc . 2020, doi:
10.1109/ECICE50847.2020.9302018.
[2] L. I. Gómez-Jo dana, J. S a o d, C. E. Pepe , and C. M. C aig, “C ossing Vi ual Doo s: A New
Me hod o S udy Gai Impai men s and F eezing o Gai in Pa kinson’s Disease,”
Pa kinsons Dis
,
ol. 2018, 2018, doi: 10.1155/2018/2957427.
[3] A. Mi elman
e al.
, “Gai impai men s in Pa kinson’s disease,”
Lance Neu ol
, ol. 18, no. 7, pp.
697–708, Jul. 2019, doi: 10.1016/S1474-4422(19)30044-4.
[4] J. Nonnekes, A. H. Snijde s, J. G. Nu , G. Deuschl, N. Giladi, and B. R. Bloem, “F eezing o gai :
a p ac ical app oach o managemen ,”
Lance Neu ol
, ol. 14, no. 7, pp. 768–778, Jul. 2015,
doi: 10.1016/S1474-4422(15)00041-1.
[5] M. Mancini, K. Smulde s, G. Ha ke , S. S ua , and J. G. Nu , “Assessmen o he abili y o open-
and closed-loop cueing o imp o e u ning and eezing in people wi h Pa kinson’s disease,”
Scien i ic Repo s 2018 8:1
, ol. 8, no. 1, pp. 1–9, Aug. 2018, doi: 10.1038/s41598-018-31156-
4.
[6] P. H. Chen, R. L. Wang, D. J. Liou, and J. S. Shaw, “Gai Diso de s in Pa kinson’s Disease:
Assessmen and Managemen ,”
In J Ge on ol
, ol. 7, no. 4, pp. 189–193, Dec. 2013, doi:
10.1016/J.IJGE.2013.03.005.
[7] R. Iansek, F. Huxham, and J. McGinley, “The sequence e ec and gai es ina ion in Pa kinson
disease: con ibu o s o eezing o gai ?,”
Mo Diso d
, ol. 21, no. 9, pp. 1419–1424, Sep. 2006,
doi: 10.1002/MDS.20998.
[8] R. Chong
e al.
, “Closed-Loop VR-Based In e ac ion o Imp o e Walking in Pa kinson’s Disease,”
Jou nal o No el Physio he apies 2011 1:1
, ol. 1, no. 1, pp. 1–7, Dec. 2011, doi: 10.4172/2165-
7025.1000101.
[9] A. L. Sil a de Lima
e al.
, “F eezing o gai and all de ec ion in Pa kinson’s disease using wea able
senso s: a sys ema ic e iew,”
J Neu ol
, ol. 264, no. 8, pp. 1642–1654, Aug. 2017, doi:
10.1007/S00415-017-8424-0.
[10] S. M. Rissanen, M. Koi u, P. Ha ikainen, and E. Pekkonen, “Ambula o y su ace
elec omyog aphy wi h accele ome y o e alua ing daily mo o luc ua ions in Pa kinson’s
disease,”
Clin Neu ophysiol
, ol. 132, no. 2, pp. 469–479, Feb. 2021, doi:
10.1016/J.CLINPH.2020.11.039.

80
[11] A. Lheu eux
e al.
, “Imme si e Vi ual Reali y o Res o e Na u al Long-Range Au oco ela ions in
Pa kinson’s Disease Pa ien s’ Gai Du ing T eadmill Walking,”
F on Physiol
, ol. 11, p. 1207,
Sep. 2020, doi: 10.3389/FPHYS.2020.572063/BIBTEX.
[12] M. Yamagami
e al.
, “E ec s o i ual eali y en i onmen s on o e g ound walking in people wi h
Pa kinson disease and eezing o gai ,”
Disabil Rehabil Assis Technol
, 2020, doi:
10.1080/17483107.2020.1842920.
[13] Y. Zhao
e al.
, “Feasibili y o ex e nal hy hmic cueing wi h he Google Glass o imp o ing gai in
people wi h Pa kinson’s disease,”
J Neu ol
, ol. 263, no. 6, p. 1156, Jun. 2016, doi:
10.1007/S00415-016-8115-2.
[14] S. Janssen
e al.
, “Usabili y o h ee-dimensional augmen ed isual cues deli e ed by sma
glasses on (F eezing o ) gai in pa kinson’s disease,”
F on Neu ol
, ol. 8, no. JUN, p. 279, Jun.
2017, doi: 10.3389/FNEUR.2017.00279/BIBTEX.
[15] S. Janssen
e al.
, “The E ec s o Augmen ed Reali y Visual Cues on Tu ning in Place in Pa kinson’s
Disease Pa ien s Wi h F eezing o Gai ,”
F on Neu ol
, ol. 11, p. 185, Ma . 2020, doi:
10.3389/FNEUR.2020.00185/FULL.
[16] D. J. Moo e, A. B. Wes , V. L. Dawson, and T. M. Dawson, “Molecula pa hophysiology o
Pa kinson’s disease,”
Annual Re iew o Neu oscience
, ol. 28. pp. 57–87, 2005. doi:
10.1146/annu e .neu o.28.061604.135718.
[17] L. I. Gómez-Jo dana, J. S a o d, C. E. Pepe , and C. M. C aig, “Vi ual Foo p in s Can Imp o e
Walking Pe o mance in People Wi h Pa kinson’s Disease,”
F on Neu ol
, ol. 9, no. AUG, Aug.
2018, doi: 10.3389/FNEUR.2018.00681.
[18] D. J. Gee se, B. Coolen, J. J. an Hil en, and M. Roe dink, “Holocue: A Wea able Holog aphic
Cueing Applica ion o Alle ia ing F eezing o Gai in Pa kinson’s Disease,”
F on Neu ol
, ol. 12,
Jan. 2022, doi: 10.3389/ neu .2021.628388.
[19] J. Janko ic, “Pa kinson’s disease: Clinical ea u es and diagnosis,”
Jou nal o Neu ology,
Neu osu ge y and Psychia y
, ol. 79, no. 4. BMJ Publishing G oup, pp. 368–376, 2008. doi:
10.1136/jnnp.2007.131045.
[20] A. Besha a
e al.
, “Vi ual eali y doo way and hallway en i onmen s al e gai kinema ics in people
wi h Pa kinson disease and eezing,”
Gai Pos u e
, ol. 92, pp. 442–448, Feb. 2022, doi:
10.1016/j.gai pos .2021.12.013.
81
[21] S. Bada ny, J. Aha on-Pe e z, Z. Susel, G. Habib, and Y. Ba am, “Vi ual eali y eedback cues o
imp o emen o gai in pa ien s wi h Pa kinson’s disease,”
T emo O he Hype kine Mo (N Y)
,
ol. 4, no. 0, p. 225, Ap . 2014, doi: 10.7916/D8V69GM4.
[22] Y. W. Wang, C. H. Chen, and Y. C. Lin, “Balance Rehabili a ion Sys em o Pa kinson’s Disease
Pa ien s based on Augmen ed Reali y,”
2nd IEEE Eu asia Con e ence on IOT, Communica ion and
Enginee ing 2020, ECICE 2020
, pp. 191–194, Oc . 2020, doi:
10.1109/ECICE50847.2020.9302018.
[23] T. Tunu , A. DeBlois, E. Ya es-Ho on, K. Rick o d, and L. A. Columna, “Augmen ed eali y-based
dance in e en ion o indi iduals wi h Pa kinson’s disease: A pilo s udy,”
Disabil Heal h J
, ol.
13, no. 2, Ap . 2020, doi: 10.1016/J.DHJO.2019.100848.
[24] R. S. Calab ò
e al.
, “Imp o ing mo o pe o mance in Pa kinson’s disease: a p elimina y s udy
on he p omising use o he compu e assis ed i ual eali y en i onmen (CAREN),”
Neu ological
Sciences
, ol. 41, no. 4, pp. 933–941, Ap . 2020, doi: 10.1007/S10072-019-04194-7.
[25] J. M. Finley, M. Go sis, V. Lympou idis, S. Jain, A. Kim, and B. E. Fishe , “Design and De elopmen
o a Vi ual Reali y-Based Mobili y T aining Game o People Wi h Pa kinson’s Disease,”
F on
Neu ol
, ol. 11, Jan. 2021, doi: 10.3389/FNEUR.2020.577713.
[26] P. Campo-P ie o
e al.
, “Can Imme si e Vi ual Reali y Videogames Help Pa kinson’s Disease
Pa ien s? A Case S udy,”
Senso s 2021, Vol. 21, Page 4825
, ol. 21, no. 14, p. 4825, Jul. 2021,
doi: 10.3390/S21144825.
[27] Y. Wang
e al.
, “E icacy o C-Mill gai aining o imp o ing walking adap abili y in ea ly and middle
s ages o Pa kinson’s disease,”
Gai Pos u e
, ol. 91, pp. 79–85, Jan. 2022, doi:
10.1016/J.GAITPOST.2021.10.010.
[28] A. Kim, N. Da akjian, and J. M. Finley, “Walking in ully imme si e i ual en i onmen s: an
e alua ion o po en ial ad e se e ec s in olde adul s and indi iduals wi h Pa kinson’s disease,”
J
Neu oeng Rehabil
, ol. 14, no. 1, Feb. 2017, doi: 10.1186/S12984-017-0225-2.
[29] H. R. Gonçal es, A. M. Rod igues, and C. P. San os, “Vib o ac ile bio eedback de ices in
Pa kinson’s disease: a na a i e e iew,”
Med Biol Eng Compu
, ol. 59, no. 6, pp. 1185–1199,
Jun. 2021, doi: 10.1007/S11517-021-02365-3.
[30] “HoloLens 2—O e iew, Fea u es, and Specs | Mic oso HoloLens.”
h ps://www.mic oso .com/en-us/hololens/ha dwa e# (accessed Oc . 05, 2022).
[31] “Hololens Sys em Requi emen s – Vangos P e neas.”
h ps://p e neas.com/2016/03/31/hololens-sys em- equi emen s/ (accessed Oc . 05, 2022).
82
[32] “MVN Use Manual,” 2005.
[33] “MTw_Awinda_Use _Manual”.
[34] “JMU Upda ed o elease MVN Analyze/Anima e 4.0 (new HW + suppo o TCP p o ocol),”
2013, Accessed: Oc . 07, 2022. [Online]. A ailable:
h ps://www.asse s o e.uni y3d.com/en/#!/con en /11338
[35] “Nondes uc i e E alua ion Techniques : Pene an .” h ps://www.nde-
ed.o g/NDETechniques/Pene an Tes /In oduc ion/ isualacui y.xh ml (accessed Oc . 14,
2022).
[36] R. Mo ei a, H. Goncal es, A. Rod igues, and C. San os, “The Po en ial o Visual Cues o O e come
F eezing o Gai in Pa kinson’s Disease*; The Po en ial o Visual Cues o O e come
F eezing o Gai in Pa kinson’s Disease*,” 2019. [Online]. A ailable:
h ps://www.ninds.nih.go /Diso de s/All-Diso de s/Pa kinsons-
[37] J. Figuei edo, P. Félix, L. Cos a, J. C. Mo eno, and C. P. San os, “Gai E en De ec ion in Con olled
and Real-Li e Si ua ions: Repea ed Measu es om Heal hy Subjec s,”
IEEE T ansac ions on Neu al
Sys ems and Rehabili a ion Enginee ing
, ol. 26, no. 10, pp. 1945–1956, Oc . 2018, doi:
10.1109/TNSRE.2018.2868094.
[38] H. R. Gonçal es, A. Rod igues, and C. P. San os, “Gai moni o ing sys em o pa ien s wi h
Pa kinson’s disease,”
Expe Sys Appl
, ol. 185, Dec. 2021, doi: 10.1016/j.eswa.2021.115653.
[39] J. Figuei edo, J. C. Mo eno, and C. P. San os, “A u o ial on human locomo ion o de eloping
sma gai ehabili a ion de ices.”
[40] J. Glogowski, “GSU Lib a y Resea ch Guides: Li e a u e Re iews: Types o Clinical S udy Designs”,
Accessed: Oc . 15, 2022. [Online]. A ailable:
h ps:// esea ch.lib a y.gsu.edu/c.php?g=115595&p=755213
[41] R. S. Kennedy, N. E. Lane, K. S. Be baum, and M. G. Lilien hal, “Simula o Sickness
Ques ionnai e: An Enhanced Me hod o Quan i ying Simula o Sickness,”
In J A ia Psychol
, ol.
3, no. 3, pp. 203–220, 1993, doi: 10.1207/s15327108ijap0303_3.
[42] “In insic Mo i a ion In en o y (IMI) Scale Desc ip ion.”
APPENDICES
84
APPENDIX A – EXPERIMENTAL PROTOCOL

85
86
87
88