Ad ances in Elec ical and Elec onic Enginee ing
218
PRELIMINARY RESULTS FOR A MONOCULAR MARKER-FREE GAIT
MEASUREMENT SYSTEM
J. Cou ney, A. de Pao
School o Elec ical, Elec onic and Mechanical Enginee ing
Na ional Uni e si y o I eland, Dublin, Bel ield, Dublin 4, Rep. o I eland.
E-mail: Jane.Cou ney@di .ie
Summa y This pape p esen s esul s om a no el monocula ma ke - ee gai measu emen sys em. The sys em was designed o
physical and occupa ional he apis s o moni o he p og ess o pa ien s h ough he apy. I is based on a no el human mo ion cap u e
me hod de i ed om model-based acking. Tes ing is pe o med on wo monocula , sagi al- iew, sample gai ideos – one wi h
bo h he en i onmen and he subjec ’s appea ance and mo emen es ic ed and one in a na u al en i onmen wi h un es ic ed
clo hing and mo ion. Resul s o he modelling, acking and analysis s ages a e p esen ed along wi h s anda d gai g aphs and
pa ame e s.
1 INTRODUCTION
1.1 Mo i a ion
This pape is p esen ed as pa o a gai
measu emen design p ojec . The goal o his p ojec is
o design a sys em, o he use o occupa ional and
physical he apis s, which would cap u e and analyse
human gai . Cu en me hods o gai measu emen
in ol e complex ma ke sys ems and a mul iple came a
se -up, he eby equi ing a dedica ed gai labo a o y
and ained he apis s, making he sys ems cumbe some
and di icul o use. He e we ha e designed a simple
single-came a sys em which is no only accu a e bu
also has a low p ocessing ime and can be used emo e
om he ilming loca ion, he eby elimina ing he need
o pa ien s o a el o a gai labo a o y. I is hoped
ha he simplici y o he sys em will encou age bo h
he apis s and pa ien s o pa icipa e in gai s udies and
make he mos o he echnology a ailable.
1.2 Ma ke -based Sys ems
Ma ke -based sys ems a e s ill he mos eadily
used me hod o gai analysis. Howe e , hey a e
ex emely di icul o use and p oblema ic, equi ing
speci ic equipmen and expe ise. This makes he
sys ems less po able and less accessible ou side o a
gai labo a o y which can be a signi ican issue when
pa ien s a e oo unwell o a el. In addi ion, a gai
labo a o y can be a e y in imida ing en i onmen ,
which can make pa ien s eel uncom o able – a majo
issue pa icula ly when dealing wi h young child en
and elde ly pa ien s.
In o de o a oid ma ke mo emen , ma ke s
canno be placed on clo hing as his will mo e ela i e
o he join s and bones being ma ked. The pa ien mus
be s ipped and he ma ke s a ached di ec ly o hei
skin. This again adds o he discom o ha pa ien s can
eel in a gai labo a o y en i onmen . Many gai
pa ien s a e elde ly s oke ic ims who do no eel a all
com o able walking in hei unde wea . One o he
majo mo i a ions o ou design was o elimina e he
need o s ip he pa ien .
E en wi h expe ience and knowledge, ma ke
placemen is s ill a di icul ask. The posi ion o he
ma ke s will ha e a signi ican e ec on he ou pu o
he sys em. E en sligh inaccu acies, pa icula ly
a ound he join s, can cause he sys em o ail. Once he
ma ke s ha e been placed accu a ely o begin wi h,
hey mus be kep in posi ion as he subjec walks.
They mus no in e e e wi h he eedom o mo emen
o he pa ien and he sys em mus be in ulne able o
ma ke occlusion. The equipmen equi ed o
measu emen can be uncom o able o wea and can
ha e a signi ican e ec on he subjec ’s eedom o
mo emen . Ac i e ma ke sys ems equi e a
ansmi e , powe supply and wi ing o be wo n by he
pa ien as hey walk. Passi e ma ke s a e o en
moun ed on special suppo s o p o uding s icks o
make hem mo e isible in he image. Again, his can
es ic he subjec ’s mo emen and limi s he posi ions
in which ma ke s can be placed ( o ins ance,
p o uding ma ke s canno be placed on he inside o
he subjec ’s legs as hey will easily be knocked while
walking).
This cumbe some equipmen is no only
di icul o a ach bu could ha e a signi ican e ec on
he mo emen o he subjec . While passi e ma ke
sys ems a e less in usi e, hey equi e mo e ma ke s o
compensa e o hei ulne abili y o occlusion. Since
passi e ma ke s a e e lec i e, hey e ec i ely
disappea when hey a e blocked om he in a- ed
ligh . This happens qui e egula ly du ing walking as
he subjec ’s a m na u ally swings back and o h,
he eby occluding any ma ke s a ound he pel ic
egion. All o hese issues make a ma ke -based sys em
di icul o design and ope a e. Ins ead, we en isage a
sys em ha will be so simple o use ha pa ien s could
be moni o ed in any local clinic, hospi al, su ge y o
e en in hei own homes.
In ac , occlusion is an issue e en wi h ma ke -
ee sys ems. Many a emp s a designing ma ke - ee
sys ems ha e been based on ea u e de ec ion and
acking [1] o on appa en mo ion [2]. A he poin o
c osso e o he legs, du ing he swing phase o gai ,
he image ea u es become less well de ined and i is
di icul o iden i y any appa en mo ion. Al hough his
is no echnically occlusion, he esul is he same: he
P elimina y esul s o a monocula ma ke - ee gai measu emen sys em
219
acking cues a e los . In de eloping a new ma ke - ee
sys em, his is a majo conside a ion.
1.3 Ma ke - ee Gai Analysis
The e a e cu en ly many esea ch g oups
s i ing o de elop he i s ully au oma ed ma ke -
ee sys em. The e a e, in ac , al eady some
comme cially a ailable ma ke - ee sys ems, e.g. [3].
Howe e , so a , none o he a ailable sys ems is
comple ely au oma ed and hey s ill equi e a gai
labo a o y en i onmen , se e al measu emen s o he
subjec and/o manual in e en ion a a ious s ages in
o de o ope a e eliably. While hese sys ems may
su ice in o he applica ions, hey ha e no been eadily
emb aced by he apis s as a be e al e na i e o
ma ke -based sys ems in moni o ing pa hological gai .
Al hough i may s ill be necessa y o use
speci ic equipmen o acqui e ce ain da a, e.g. o ce-
pla e measu emen s, we eel ha he e is no eason ha
gai kinema ics, he in o ma ion gleaned om ideo
da a, canno be measu ed emo e om a gai
labo a o y. The g ea es challenge om a mo ion
analysis poin o iew lies in analysis o he lowe
limbs in he sagi al plane. In ac , his is also whe e he
mos use ul in o ma ion is ga he ed o diagnosis and
in e p e a ion o gai da a. In some basic ma ke - ee
sys ems, he subjec is equi ed o wea di e en
colou ed s ockings on each leg o dis inguish he wo
legs om each o he , e.g. [4]. This s ays om he goal
o a non-in usi e and comple ely ma ke - ee sys em.
The di icul ies in he sagi al plane s em om he
simila i y in appea ance and p oximi y o he wo legs
and om he speed change du ing he swing phase o
gai . As he legs c oss du ing he gai cycle, he
image o he mo ing leg becomes blu ed and he
mo ing leg becomes indis inguishable om he
s a iona y leg. Wi h s anda d mo ion acking
echniques, his can lead o mo ion ec o s ha ing
e oneous ze o alues. This is why sagi al gai analysis
is such a challenge when a emp ing au oma ed mo ion
acking. Many cu en ma ke - ee echniques a e s ill
being imp o ed upon in his a ea. Re iew pape s [5]
and [6], co e ing he en i e a ea o human mo ion
cap u e including ma ke -based and ma ke - ee
analysis, can be consul ed o a mo e ho ough su ey
o he cu en s a e o human mo ion esea ch.
1.4 Design Goals
Wi h he in e es s o bo h pa ien s and he apis s
in mind, we ou lined he ollowing goals o he
p ojec :
• The sys em mus be comple ely au oma ed
equi ing no excessi e measu emen s o he pa ien
and no manual in e en ion.
• The sys em mus be simple o use.
• The ou pu will be a comple e se o sagi al plane
gai g aphs and pa ame e s.
• The inpu will be a single AVI ile con aining a
ilm o he pa ien walking.
• The gai ideo can be ilmed in any easonable
en i onmen wi hou signi ican es ic ions.
• The subjec can be ully clo hed in app op ia e
clo hing.
• The subjec can walk eely.
In addi ion, because his mo ion measu emen
sys em has a speci ic applica ion, we can apply ce ain
es ic ions o ou expec a ions:
• I is easonable o expec adequa e ligh ing and
con as in he ilming en i onmen .
• The da a will be ilmed om a s a iona y came a.
• The subjec will walk om one side o he came a
iew plane o he o he .
• The subjec will be ully isible in all ames om
head o oe.
• Clo hing will no hide he subjec ’s leg ou line, o
example, ski s may no be wo n.
• The heigh o he subjec is known.
In designing ou sys em, we ied o mee as many
o ou goals as possible whils minimising he
es ic ions on he sys em. An ini ial design a emp
was made p e iously bu he di icul y a he c osso e
o he legs du ing he gai cycle could no be o e come
and he un acked leg had o be manually emo ed
om each ame in o de o p o ide esul s [7]. Since
hen, howe e , a me hod has been disco e ed which
ou lines he acked leg in each ame, he eby
dis inguishing i om he un acked leg. This me hod
has been in eg a ed in o he o e all sys em and adap ed
o make i comple ely au oma ed and obus . The esul
is a ully au oma ed acking sys em ha succeeds in
eaching ou ou lined goals.
2 IMPLEMENTATION
2.1 Segmen a ion
The human mo ion cap u e me hod used he e is
based on a me hod in oduced by Nyogi and Adelson
[8]. In hei ‘XYT’ me hod, ideo ames om a
s a iona y came a a e s acked o c ea e a 3D block wi h
wo o i s dimensions ep esen ing ho izon al and
e ical di ec ions and i s hi d dimension being ime
(See Figu e 1).
Fig. 1: The XYT block o he gai labo a o y sequence.
Ad ances in Elec ical and Elec onic Enginee ing
220
A pic u e o he mo emen in he ideo is ob ained by
slicing he block in he XT di ec ion. In he XT slice,
he s a iona y backg ound appea s as e ical lines and
objec s mo ing ho izon ally ac oss he came a plane
appea as diagonal lines.
This image is pa icula ly use ul o ecognizing
and analysing walking because o an in e es ing
cha ac e is ic o he leg mo ion. In he case o a sagi al
iew o a human walking in on o a s a iona y
came a, a dis inc ly ecognisable b aided pa e n is
obse ed in slices a ound he leg heigh (see Figu e 2).
The b aids a e o med by he pe iodic mo ion o he
legs as hey pass h ough he swing and s ance phases
o he gai cycle. While he legs appea e y close o
each o he in he XY plane, causing occlusion and
in e e ence, in he XT plane hey a e e y clea ly
dis inguishable. I hese wo pa e ns can be ou lined
sepa a ely in he slice, he wo legs would be sepa a ed
om one ano he h oughou he ideo sequence.
Fig. 2: The pe iodic pa e n o he legs in mo ion in a sample
XT slice.
This ou lining was achie ed using an
au oma ically ini ialised snaking algo i hm. A he
ankle heigh o he subjec , an XT slice was ob ained
and an ini ial app oxima e o he snake was i o he
b aided image. This ini ial empla e was hen wa ped o
ollow he pa e n’s edges. Because o he simila i y o
he b aided pa e ns a locally connec ed slices, his
ini ial snake i ing was pe o med only once. A e
ha , he snake i ing p ocess was epea ed a each slice
om he ankle o he head o he subjec and he esul
o he snake i ing om he p e ious heigh was used
as he ini ialisa ion a he nex . This esul ed in a
comple e ou line o he acked egion, which is held
h oughou he ideo sequence (see Figu e 3).
Fig. 3: The ou line o he a ea o in e es in a sample ame.
2.2 Ellipse Fi ing
Once he ou line o he a ea being acked has
been ob ained and he body has been segmen ed, he
ou line segmen s can be used as inpu s o he ellipse-
i ing algo i hm. The ou line ob ained om he slice-
by-slice snake algo i hm is di ided using he body
segmen a ion in o he acked body pa s: he head,
o so, high and shank. Each o hese segmen ou lines
is passed o he ellipse- i ing algo i hm and an ellipse
is a ached o each one independen ly.
So a , he body pa s in each ame ha e been
posi ioned independen o each o he and independen
o hei loca ions in p e ious ames. Howe e , his
could cause anomalies in he esul s. In many human
mo ion acking algo i hms, he segmen s a e
posi ioned subjec o cons ain s and a e dependen on
he loca ions o hei p edecesso in he hie a chical
ee s uc u e o he body. This is a good way o
a oiding unlikely posi ioning bu i is p one o s aying.
Since each posi ioning is dependen on i s own
p e ious s a e and on he cu en s a e o i s connec ed
body pa s, one bad i would p opaga e h ough he
image and h ough he image sequence causing he
acking o ail.
While ou algo i hm a ely su e s om s aying
and eco e s quickly when i does, i can po en ially
esul in nonsensical conclusions. Using ou di ec
ellipse- i ing me hod, he esul will be he bes - i
ellipse wi h no limi a ions. This means ha , while we
can ob ain a good es ima e o he posi ion and
o ien a ion o he body pa , he size o he pa may
a y om ame o ame and he ela i e angles wi h
o he body pa s could be un easonable.
In o de o o e come his p oblem, we apply
cons ain s a e he ini ial app oxima e i has been
acqui ed h ough di ec ellipse i ing. Fi s ly, he
dimensions o each ellipse a e se o he a e age o e
he sequence. While he e may be some change in he
appa en dimensions as he pe son mo es sligh ly
owa d o away om he came a o as muscles lex, i
is easonable o assume ha he change will be
insigni ican . Nex , he angles o he ellipses a e
empo ally smoo hed using a 1D Gaussian il e . This
has an indis inguishable e ec on he isual esul s bu
i ensu es ha he body pa s a e no o a ing a
un easonable speeds om ame o ame and i
imp o es he gai g aphs. Las ly, he ela i e angles a e
checked o ensu e ha he join angles a e easonable.
The inal s age o he sys em design is he
ex ac ion o he gai da a om he acked body
model. Because he ellipses con ain in o ma ion abou
he dimensions and o ien a ions o he limb segmen s,
he g aph da a o he high and shin angles can be
ex ac ed di ec ly om he isual esul s. The lexion
angles a e he angles o he join s and so can be
calcula ed as he di e ence o he body segmen
angles. A ull se o sagi al iew gai g aphs along
wi h a ew signi ican gai pa ame e s a e p esen ed
he e. The me hod used in implemen ing his sys em is
desc ibed ully in [9].
P elimina y esul s o a monocula ma ke - ee gai measu emen sys em
221
3 RESULTS
3.1 Sys em
The sys em was implemen ed on a PC wi h a
2.66GHz Pen ium®4 p ocesso wi h 1GB o RAM.
The gai labo a o y ideo images we e ilmed wi h an
analogue ideo came a and we e cap u ed using an
ATI All-in-Wonde ®128 P o ideo cap u e ca d. The
na u al-en i onmen ideo images we e ilmed wi h a
USB2.0 webcam. Some success ul p elimina y es s
ha e been done using wo synch onised USB2.0
webcams wi h a iew o c ea ing a ully in eg a ed 3D
sys em in he u u e. The p og am was implemen ed in
Mic oso Visual C++ ® 6.0. The gai esul s we e
g aphed in MATLAB®.
3.2 Inpu Da a
The i s ideo was ilmed in a gai labo a o y
using a high quali y camco de and acquisi ion ca d.
The subjec is wea ing i ed spo swea wi h he legs
mos ly ba e and is walking wi hou a m swing. The
second ideo was ilmed wi h a s anda d webcam in a
no mal en i onmen , al hough he backg ound is kep
da k o ensu e ha he subjec is clea ly isible in
con as . The subjec is d essed in e e yday clo hes and
walking eely wi h a m swing.
In bo h ideo clips, he da a was cap u ed a 30
ames pe second as a 320 x 240, 24-bi uncomp essed
RGB AVI. Highe esolu ion would gi e be e esul s
bu would conside ably slow he sys em. The sample
ideo sequences used he e a e bo h app oxima ely ou
seconds long (126 ames) o alling app oxima ely
28MB. P ocessing ime is less han one minu e.
3.3 Visual Resul s
The isual da a is use ul o gauging he success
o he algo i hm and i could also be used o c ea e an
a a a o mimic he gai in a i ual 3D en i onmen .
This is a e y angible o m o ou pu bu is only ully
ealisable wi h comple e 3D gai da a i.e. including he
ans e se and co onal planes o mo emen and
including pel is and ankle da a. He e, we ha e
concen a ed on he sagi al plane and pa icula ly on
he main lowe limb a ea (i.e. he high and shank), as
his is he mos challenging egion in he acquisi ion o
gai in o ma ion.
Shown in Figu e 4 a e some sample ames
om he wo ideo sequences wi h he simple ellipse
body model a ached. No e he signi ican di e ence in
pic u e quali y and con as be ween he wo sequences
ye despi e his, he e is li le di e ence in he accu acy
o he model i ing. Howe e , because he leg is no
di ec ly isible in he second sequence, he e is an
una oidable ambigui y wi h ega d o he dimensions
o he limb segmen s. In he g aphical esul s, we plo
he o ien a ion o he segmen s h oughou he
sequence and so his ambigui y will no a ec he gai
measu emen s – ano he ad an age o his ellipse-based
me hod.
Fig. 4: Sample ames om he gai labo a o y ideo
sequence and he na u al-en i onmen ideo sequence,
showing he simple ellipse model a ached o he images o
he subjec s.
3.4 Gai Da a
As his sys em concen a es on he main lowe
limb sec ions, we ha e p esen ed g aphed da a o he
o a ion and ela i e angles o he high and shank.
These a e ypical gai g aphs used in gai kinema ics.
The da a in Figu e 5 is p esen ed om oo -s ike ( he
poin when he heel o he acked leg s ikes he
g ound) o oo -s ike.
(a)
(b)
Fig. 5: Gai da a acqui ed om (a) he gai labo a o y
sequence and (b) he na u al en i onmen sequence.
Ad ances in Elec ical and Elec onic Enginee ing
222
Along wi h he isual and g aphical esul s, a
ew signi ican gai pa ame e s a e no mally acqui ed
as pa o ull sagi al gai in o ma ion. These a e he
walking eloci y, he s ide leng h and he cadence.
The s ide leng h is simply he leng h o wo s eps ( he
s ep leng h and walking eloci y acquisi ion me hods
a e desc ibed p e iously) and he cadence is he
numbe o s eps pe minu e. The alues o he sample
ideo sequences a e p esen ed below.
Table 1: Gai pa ame e s o bo h ideo sequences
Gai
Labo a o y
Sequence
Na u al
En i onmen
Sequence
Uni s
Subjec ’s Heigh 175 180 cm
Walking Veloci y 111 117 cm/sec
S ide Leng h 175 170 cm
Cadence 87 96 s eps/min
4 DISCUSSION
Cu en ly, he e a e ew comme cial ma ke - ee
sys ems in use in gai analysis. While ma ke - ee
sys ems a e becoming mo e common in o he a eas
such as spo s science and anima ion, he accu acy and
de ail equi ed o gai analysis makes his sys em
design pa icula ly challenging. In esea ching he
cu en s a e o a ai s, we disco e ed ha he g ea es
obs acles in designing ei he ma ke -based o ma ke -
ee sys ems lie in sagi al-plane acquisi ion. This is
also whe e he mos in o ma ion can be gleaned in
diagnosing and analysing pa hological gai . Thus, i
was decided in his p ojec o concen a e on designing
a eliable ma ke - ee sys em o moni o ing sagi al-
plane mo emen in he gai cycle.
We ha e achie ed ou ou lined goals o building
a comple ely au oma ed, po able gai measu emen
sys em o use in sagi al-plane gai analysis. This
sys em will allow pa ien s’ gai o be eco ded in a
elaxed and con enien en i onmen wi hou he need
o a ained he apis o be p esen . Thus, he he apis s
can use hei aluable expe ise in diagnosing gai
a he han spending hei ime mas e ing and using
cumbe some ma ke sys ems. I is hoped ha his
sys em will be used by he apis s and pa ien s in he
Na ional Rehabili a ion Hospi al, Dublin whe e i was
de eloped, and will pe haps become pa o a mo e
comple e gai labo a o y design in he u u e.
REFERENCES
[1] Segen J., Pingali S., ‘A came a-based sys em o
acking people in eal ime’, P oceedings o he
In e na ional Con e ence on Pa e n Recogni ion,
pp. 63-67, Vienna, Aus ia, Augus 1996.
[2] Cai Q., Mi iche A., Agga wal, J.K., ‘T acking
human mo ion in an indoo en i onmen ’,
P oceedings o he 2nd In e na ional Con e ence on
Image P ocessing, ol. 1, pp. 215-218, Washing on
D.C., Oc obe 1995.
[3] Mik omak: www.mic omak.com
[4] Chaudha i A.M., B agg R.W., Alexande E.J.,
And iacchi T.P., ‘A ideo-based, ma ke less
mo ion acking sys em o biomechanical analysis
in an a bi a y en i onmen ’, BED Bioenginee ing
Con e ence, Vol. 50, pp. 777-778, 2001.
[5] Agga wal J.K., Cai Q., ‘Human mo ion analysis: a
e iew’, Compu e Vision and Image
Unde s anding, ol. 73, no. 3, pp. 428-440, 1999.
[6] Ga ila D.M., ‘The isual analysis o human
mo emen : a su ey’, Compu e Vision and Image
Unde s anding, ol. 73, no.1, pp. 82-98, 1999.
[7] Cou ney J., Applica ion o Digi al Image
P ocessing o Ma ke - ee Analysis o Human Gai .
Mas e s Thesis, Na ional Uni e si y o I eland,
Dublin, Janua y 2001.
[8] Nyogi S., Adelson E., ‘Analysing and ecognising
walking igu es in XYT’, P oceedings o he IEEE
Con e ence on Compu e Vision and Pa e n
Recogni ion, pp. 469–474, 1994.
[9] Cou ney J., Ma ke - ee Human Mo ion Cap u e
wi h Applica ions in Gai Analysis. PhD Thesis,
Na ional Uni e si y o I eland, Dublin, July 2005.