Ac i i y Recogni ion Sys em Using Non-in usi e
De ices h ough a Complemen a y Technique
Based on Disc e e Me hods
Miguel ´
Angel ´
Al a ez de la Concepci´on1, Luis Miguel So ia Mo illo1,
Luis Gonz´alez Ab il2, and Juan An onio O ega Ram´ı ez1
1Compu e Languages and Sys ems Dep ., Uni e si y o Se ille, 41012 Se ille, Spain
{maal a ez,lso ia,jo ega}@us.es
2Applied Economics I Dep ., Uni e si y o Se ille, 41018 Se ille, Spain
[email p o ec ed]
Abs ac . This pape aims o de elop a cheap, com o able and, spe-
cially, efficien sys em which con ols he physical ac i i y ca ied ou
by he use . Fo his pu pose an ex ended app oach o physical ac i -
i y ecogni ion is p esen ed, based on he use o disc e e a iables which
employ da a om accele ome e senso s. To his end, an inno a i e se-
lec ion, disc e iza ion and classifica ion echnique o make he ecog-
ni ion p ocess in an efficien way and a low ene gy cos , is p esen ed
in his wo k based on Ame a disc e iza ion. En i e p ocess is execu ed
on he sma phone and on a wi eless heal h moni o ing sys em is used
when he sma phone is no used aking in o accoun he sys em ene gy
consump ion.
Keywo ds: Con ex ual In o ma ion, Disc e iza ion Me hod, Mobile En-
i onmen , Quali a i e Sys ems, Sma -Ene gy Compu ing.
1 In oduc ion
In ecen yea s, hanks la gely o he inc eased in e es on moni o ing ce ain
sec o s o popula ion such as elde ly people wi h demen ia o people in ehabili-
a ion, ac i i y ecogni ion sys ems ha e expe ienced an inc ease in bo h numbe
and quali y esul s. Howe e , mos o hem a e in a high compu a ional cos and
hence, i canno be execu ed in o a gene al pu pose mobile de ice.
Calcula ion o he physical ac i i y o a use based on da a ob ained om
an accele ome e is a cu en esea ch opic. Fu he mo e, many wo ks is going
o be analyzed showing some iden ified limi a ions ha make hese sys ems
uncom o able o use s in gene al.
The fi s diffe ence obse ed be ween he sys ems de eloped is he ype o used
senso . The e a e sys ems using specific ha dwa e [1], while o he s use gene al
pu pose ha dwa e [2]. Ob iously, he use o gene ic ha dwa e is a benefi o
use s, since he cos o de ices and e sa ili y o hem a e poin s in hei a o .
No o men ion dec easing he loss and o ge ing isk due o hey ha e been
in eg a ed on an e e yday objec like use s’ sma phones.
Ano he diffe ence ound be ween he su eyed p oposals is he numbe and
posi ion o he senso s. In [3] can be seen ha he accele ome e senso is placed
in a glo e and a mul i ude o ac i i ies depending on he mo emen o he hand
a e ecognized. In con as , o he s udies use a ious senso s h oughou he body
[4], [5] o a wea able wi eless senso node wi h a s a ic wi eless non-in usi e
senso y in as uc u e [6] o ecognize hese ac i i ies. Acco ding o some com-
pa a i e s udies and p e ious wo ks based on mul iple senso s, hey a e mo e
accu a e.
Al hough, wo ks like [2], whe e a senso is a use s’ pocke o in he hipe, is
mo e com o able o hem. By his way, place hem in he moni o ed pe son is
easie , no o men ion ha he in as uc u e is much lowe .
Thus, he p esen ed wo k will is ocus on he ecogni ion o physical ac i i ies
ca ied ou by use s h oughou hei mobile de ices. So, i mus be paid special
a en ion o ene gy consump ion and compu a ional cos o used me hods. Also,
a wi eless heal h moni o ing sys em can be used o inc emen he use accep-
ance, i.e. he use does no ca y he mobile de ices all he ime in an indoo
en i onmen .
One s ep u he , some wo ks do no only use da a om accele ome e s, bu
use o he sou ces such as mic ophone, ligh senso o oice ecogni ion o de e -
mine he con ex o he use [7]. Howe e , hey p esen p oblems i.e. when he
en i onmen is noisy o he use is alone.
The e a e ela ed wo ks whe e da a o ac i i ies ecogni ion a e ob ained
h ough mobile de ices, bu hese da a a e sen o a se e o p ocess he in o -
ma ion [8]. Thus, compu a ional cos is no a handicap and because o his mo e
complex me hods a e used. In con as , he efficiency is a c ucial issue when
p ocessing is ca ied ou in he mobile de ice [9], [10].
To educe he cos associa ed o accele ome e signal analysis, his pape op s
o a no el app oach based on a disc e iza ion me hod. Thanks o disc e iza ion
p ocess, classifica ion cos is much lowe han wo king wi h con inuous a iables.
Because o his, i is possible o elimina e he co ela ion be ween a iables
du ing he ecogni ion p ocess and on he o he hand, o minimize he ene gy
consump ion om he p ocess.
Wo king in he domain o disc e e a iables o pe o m lea ning and ecogni-
ion o ac i i ies is a new app oach offe ed by his wo k. This decision was la gely
due o he high compu a ional cos equi ed o lea ning algo i hms based on
con inuous a iables used o his pu pose o e he yea s.
In [11], a labeling p ocess, like a disc e iza ion p ocess, is used o ob ain a
Quali a i e Simila i y Index (QSI), so i can be said ha a ans o ma ion o he
con inuous domain o he disc e e domain o alues o he a iables is beneficial
in ce ain aspec s.
Bu , be o e he sel - ecogni ion o lea ning, i is necessa y o ca y ou a
p ocess o Ame a disc e iza ion om i s algo i hm [12]. I has a numbe o ad-
an ages o e o he well-known disc e iza ion algo i hms like CAIM disc e iza-
ion algo i hm [13], i.e. i is unsupe ised and e y as . The mos no able o
hese is he small numbe o in e als gene a ed which acili a es and educes
he compu a ional cos o he ecogni ion p ocess.
I should be no ed ha many o hese s udies could be seen in ac ion du ing
he compe i ion E AAL 2012 [14] in Ac i i y ecogni ion ack. E AAL is an
annual in e na ional compe i ion ha add esses he challenge o e alua ion and
compa ison o Ambien Assis ed Li ing (AAL) sys ems and pla o ms, wi h he
final goal o assess he au onomy, independen li ing and quali y o li e ha
AAL sys ems may g an o hei end use s.
In his ack compe i ion, ou eams pa icipa ed in he challenge: CUJ ( om
he Uni e si y o Chiba, Japan) [15], CMU ( om Ca negie Mellon and U ah
Uni e si ies, USA) [16], DCU ( om Dublin Ci y Uni e si y, I eland) [17] and
USS ( om Uni e si y o Se ille, Spain) [12]. Finally, al hough CMU had he
bes accu acy in he esul s, USS won he compe i ion because i s simplici y
and in e ope abili y ga e good ma ks in all he e alua ed c i e ia.
In o de o imp o e he accu acy p oblems encoun e ed du ing he celeb a ion
o he E AAL 2012 compe i ion, some significan imp o emen s in Ame a dis-
c e iza ion algo i hm a e p oposed. Also, in addi ion o de ec specific ac i i ies,
he ba ome ic senso which is being included in he la es gene a ion o mobile
de ices is used.
Finally, in o de o answe he ques ion abou wha would happen i you
decide no o use you mobile de ice in an indoo en i onmen , as happens in
eal li e, a complemen a y wi eless de ice is also op ionally used.
The e a e o he simila E AAL compe i ions such as HARL [18], OPPOR-
TUNITY [19], HASC [20] o BSN con es [21].
The pape is o ganized as ollows: fi s , he ac i i y ecogni ion s ep is p e-
sen ed in Sec ion 2. Also, he da a collec ion and he se o ac i i ies a e p e-
sen ed. Sec ion 3 p esen s he me hodology o de e mine he ac i i y using he
Ame a disc e iza ion. Sec ion 4 epo s he ob ained esul s o applying he
me hodology. Finally, he pape conclusions wi h a summa y o he mos impo -
an poin s a e in Sec ion 5.
2 Ac i i y Recogni ion
The final eal sys em consis only o a sma phone and, op ionally, a wi eless
de ice, configu ed o de ec he compe i ion ac i i ies: lie, si , s and, walk, bend,
all and cycle.
2.1 Da a Collec ion
In con as o he needs o some s udies ha equi e a aining se o classi y a
ecognized ac i i y co ec ly, his pape educes he wai ing ime o ecogni ion,
p o iding alid in o ma ion o an ac i i y equen ly.
To his end, a aining se and a ecogni ion se a e ob ained using 5-second-
ime windows o fixed du a ion which has been de e mined empi ically as op i-
mum leng h om a pe o mance and an accu acy analysis o he sys em.
The ime leng h o fi e seconds o hese windows has been chosen because o
ou sys em is e y impo an o ensu e ha in each ime window he e is a leas
one cycle o ac i i y, whe e ac i i y cycle is defined as a comple e execu ion o
some ac i i y pa e ns. Fo example, wo s eps a e a walking ac i i y cycle and
one pedal s oke is he ac i i y cycle o cycling. I a leas one cycle o ac i i y
can no be gua an eed in each ime window, i is no possible o de e mine he
ac i i y om accele ome e pa e ns.
This analysis is pe o med based on he alues ob ained om he accele ome-
e , which significan ly imp o e he p ecision o he body- ela ed ac i i ies, and
a ba ome e o de ec en i onmen - ela ed ac i i ies, such as going ups ai s and
downs ai s. The la e senso has mos o en been in eg a ed in ecen mobile
de ices, allow o inc ease he o e all sys em accu acy de ec ion o ac i i ies.
So, based on hese ime windows ha con ain da a o each accele ome e
axis and educing he compu a ional cos o he new solu ion, signal module has
been chosen o wo k. This elimina es he p oblem caused by he de ice o a ion
[22]. Fu he mo e, i inc eases use com o wi h he sys em by emo ing he
es ic ion o keep he o ien a ion du ing he lea ning and ecogni ion p ocess.
Fo each da a in a ime window size N,ai=(ax
i,a
y
i,a
z
i), i=1,2,...,N whe e
x,yand z ep esen he h ee accele ome e axis, he accele ome e module is
defined as ollow:
|ai|=(ax
i)2+(ay
i)2+(az
i)2
Hence, he a i hme ic mean, he minimum, he maximum, he median, he s an-
da d and he mean de ia ion, and he signal magni ude a ea s a is ics a e ob-
ained o each ime window.
In addi ion o he abo e a iables, he ea e called empo a y a iables, a new
se o s a is ics called equency-domain ea u es om he equency domain
o he p oblem a e gene a ed. Thus, in o de o ob ain he equency-domain
ea u es, Fas Fou ie T ans o m (FFT) is applied o each ime window.
Fo he ba ome e senso , wo measu es a e ob ained o each ime window: a
he beginning and a he end, aking in o accoun he diffe ence be ween hem.
b=bN−b1
I is impo an o no e ha in his case, he absolu e alue is no akenin o accoun ,
con a y o wha was done wi h he alues ob ained om he accele ome e .
2.2 Se o Ac i i ies
Fa om being a s a ic sys em, he numbe and ype o ac i i ies ecognized
by he sys em depends on he use . Thanks o his p oposal when use s is
ca ying ou ac i i ies ha ha e no been lea ned be o e can be de e mined.
This is achie ed basing on he analysis o p obabili y associa ed o each pa e n
while use is pe o ming he ac i i ies. Ob iously, he numbe o ac i i ies o be
de ec ed will impac on he accu acy o he sys em. Especially i accele a ion
pa e ns be ween ac i i ies a e e y simila .
Fo a la ge numbe s o use s could be in e es ing ecognize a ew ac i i ies,
such as walking, si ing and alling. Bu o ano he use s, ac i i ies like d i ing
o biking would be impo an . Howe e , o ca y ou a compa a i e analysis o
he accu acy and pe o mance o he disc e e ecogni ion me hod p oposed be-
low, 8 ac i i ies we e aken in o accoun . These ac i i ies a e immobile, walking,
unning, jumping, cycling, d i e, walking-ups ai s and walking-downs ai s.
The e o e, he lea ning sys em allows he use o decide wha ac i i ies he/she
wan s he sys em o ecognize. This is highly use ul when he de e mina ion o
ce ain e y specific ac i i ies on moni o ed use s is equi ed.
3 Me hodology
3.1 Ame a Algo i hm
Le X={x1,x
2,...,x
n}be a da a se o an a ibu e Xo mixed-mode da a
such ha each example xibelongs o only one o he classes o class a iable
deno ed by
C={C1,C
2,...,C
},≥2
A con inuous a ibu e disc e iza ion is a unc ion D:X→Cwhich assigns
aclassCi∈C o each alue x∈Xin he domain o p ope y ha is being
disc e ized. Le us conside a disc e iza ion Dwhich disc e izes Xin o kdisc e e
in e als:
L(k;X;C)={L1,L
2,...,L
k}
whe e L1is he in e al [d0,d
1]andLjis he in e al (dj−1,d
j], j=2,3,...,k.
Thus, a disc e iza ion a iable is defined as L(k)=L(k;X;C) which e ifies
ha , o all xi∈X, a unique Ljexis s such xi∈Lj ha o i=1,2,...,n and
j=1,2,...,k. The disc e iza ion a iable L(k)o Xand he class a iable Ca e
ea ed om a desc ip i e poin o iew.
The main aim o he Ame a me hod [12] is o maximize he dependency
ela ionship be ween he class labels Cand he con inuous- alues a ibu e L(k),
and a he same ime o minimize he numbe o disc e e in e als k. Fo his,
he ollowing s a is ic is used:
Ame a(k)= χ2(k)
k(−1) whe e χ2(k)=N⎛
⎝−1+


i=1
k

j=1
n2
ij
n·inj·⎞
⎠
and nij deno es he o al numbe o con inuous alues belonging o he Ciclass
ha a e wi hin he in e al Lj,ni·is he o al numbe o ins ances belonging o
he class Ciand n·jis he o al numbe o ins ances ha belong o he in e al
Lj, o i=1,2,..., and j=1,2,...,k, ulfilling he ollowing:
ni·=
k

j=1
nij ,n
·j=


i=1
nij ,N=


i=1
k

j=1
nij

The o iginal de eloped algo i hm o ob ain he bes in e als wi h he Ame a
disc e iza ion is based on finding he cu off poin s ha p o ide he bes coeffi-
cien . To do his, he alues o he a iables a e so ed o find he fi s cu (local
maximum). Then, i e u ns he nex cu , and so on, un il he Ame a coeffi-
cien does no imp o e. This beha io causes he complexi y o he algo i hm
is quad a ic o de , O(n2). A g aphic wi h h ee local maximums can be seen in
Figu e 1.
Fig. 1. An example o Ame a coefficien alues wi h h ee local maximums
The p esen ed imp o emen in his wo k allows o find all cu s, allowing he
complexi y o he algo i hm would be o linea o de , O(n). Al hough he e is a
loss o p ecision, i is negligible o he field o s udy o his wo k, since i allows
o ob ain good esul s.
Finally, o each s a is ical Sp∈{S1,S
2,...,S
m}, he disc e iza ion p ocess
is pe o med, ob aining a ma ix o o de kp×2, whe e kpis he numbe o class
in e als and 2 deno es he in (Lp
i)andsup(Lp
i) in e al limi s io ps a is ical.
Hence, a h ee-dimensional ma ix con aining he s a is ics and he se o in e al
limi s o each s a is ic is called Disc e iza ion Ma ix and i is deno ed by
W=(wpij )
whe e p=1,2,...,m,i=1,2,...,k
pand j=1,2.
The e o e, Disc e iza ion Ma ix de e mines he in e al a which each da a
belongs o he diffe en s a is ical associa ed alues, ca ying ou a simple and
as disc e iza ion p ocess.
Class In eg a ion. The aim in he nex s ep o he algo i hm is o p o ide a
p obabili y associa ed wi h he s a is ical da a o each o he ac i i ies based
on p e iously gene a ed in e als. Fo his pu pose, he elemen s o he aining
se x∈Xa e p ocessed o associa e he label o he conc e e ac i i y in he
aining se . In addi ion, he alue o each s a is ic is calcula ed based on he
ime window.
Fo ca ying ou he p e ious p ocess, a Class Ma ix, V, is defined as a
h ee-dimensional ma ix ha con ains he numbe o da a om he aining se
associa ed wi h a Lin e al in a Cac i i y o each s a is ical So he sys em.
This ma ix is defined as ollows:
V=( pij )
whe e pij =#{x∈X|in (Lp
i)<x≤sup(Lp
i)},andS=Sp,C=Cj,
p=1,2,...,m,i=1,2,...,k
pand j=1,2,...,.
So, each posi ion in he Class Ma ix is uniquely associa ed wi h a posi ion
in he Disc e iza ion Ma ix de e mined by i s ange.
A his poin , he e is no only possible o de e mine he disc e iza ion in-
e al, bu he Class Ma ix helps o ob ain he p obabili y associa ed wi h he
disc e iza ion p ocess pe o med wi h he Ame a algo i hm.
Ac i i y-in e al Ma ix. The nex s ep is de e mined a h ee-dimensional
ma ix, called Ac i i y-In e al Ma ix and deno ed by U, which de e mines he
likelihood ha a gi en alue xassocia ed o a Ss a is ical co esponds o C
ac i i y in a Lin e al. This a io is based on ob aining he goodness o he
Ame a disc e iza ion and he aim is o de e mine he mos p obable ac i i y
om he da a and he in e als gene a ed o he aining se .
Each alue o Uis defined as ollows:
upij = pij
p·j
q=1,q=j1− piq
p·q
−1
whe e p·jis he o al numbe o ime windows o he aining p ocess labeled
wi h he jac i i y o he ps a is ic, and p=1,2,...,m,i=1,2,...,k
pand
j=1,2,...,
Gi en hese alues, U o he ps a is ic is defined as
Up=
⎛
⎜
⎜
⎜
⎜
⎜
⎜
⎝
up00 ... u
p0j... u
p0
.
.
.....
.
.....
.
.
upi0... u
pij ... u
pi
.
.
.....
.
.....
.
.
upkp0...u
pkpj...u
pkp
⎞
⎟
⎟
⎟
⎟
⎟
⎟
⎠
As can be seen in he defini ion o U, he likelihood ha a da a xis associa ed
wi h he in e al Lico esponding o he ac i i y Cj, depends no only on da a,
bu all he elemen s associa ed wi h he in e al Li o he o he ac i i ies.
Thus, each upij ma ix posi ion can be seen as a g ade o belonging ha a
gi en xis iden ified o Cjac i i y, ha i is included in he Liin e al o he
Sps a is ic.
Simila ly, he elemen s o Uha e he ollowing p ope ies:
–upij =0 ⇐⇒ pij =0∨ piq = p·q,q=j
–upij =1 ⇐⇒ pij = p·j= pi·
Figu e 2 shows he o e all p ocess desc ibed on his sec ion o ca y on da a
analysis and in e al de e mina ion.
Reco e y da a om
accele ome e
senso
End o empo al
window?
No Build empo al
window da a se
Yes
Ge ime-domain
measu es
Apply Fas Fou ie
T ans o m
p ocessing
Ge equency-
domain measu es
Remo e noise
applying il e s
Execu e Ame a
algo i hm o e each
a iable
Ge in e als o
each measu e and
associa ed ac i i y
(Disc e iza ion-
Ma ix)
numbe o da a
om he aining
se included on
each in e al (Class-
Ma ix)
Ob ain ela i e
p obabili ies o each
ac i i y o belong o
each in e al o
Class-Ma ix
(Ac i i y-In e al
Ma ix)
Sa e Ac i i y-
In e al Ma ix in o
use p o ile
Fig. 2. O e all p ocess o da a analysis and in e al de e mina ion
3.2 Classifica ion P ocess
Ha ing ob ained he disc e iza ion in e als and he p obabili ies o belonging
o each in e al, he p ocess by which he classifica ion is pe o med can be
desc ibed. This classifica ion is based on da a om he analysis o ime windows.
The p ocess is di ided in o wo main s eps: he way in which o pe o m he
ecogni ion o physical ac i i y is fi s desc ibed; and he p ocess o de e mine
he equency a which some pa icula ac i i y is hen p esen ed.
Classi ying Da a. Fo he classifica ion p ocess, he mo e likely ac i i y is
decided by a majo i y o ing sys em. As said abo e, his p ocess pa s om he
Ac i i y-In e al Ma ix and a se o da a x∈X o he Sse .
The e o e, i consis s in finding an ac i i y Ci∈C ha maximizes he like-
lihood. The abo e c i e ion is collec ed in he ollowing exp ession, deno ed by
mpa (mos likely ac i i y):
mpa(x)=Ck
whe e k=a g(maxjm
p=1 upij |x∈(in (Lp
i),sup(Lp
i)]). The exp ession shows
ha he weigh con ibu ed by each s a is ical o he likely calcula ion unc ion
is he same. This can be done unde he assump ion ha all s a is ical p o ide
he same in o ma ion o he sys em and he e is no co ela ion be ween hem.
Thus, he mpa ep esen s he ac i i y whose da a, ob ained h ough he p o-
cessing ime window, is mo e sui ed o he alue se om U. In his way, he
p oposed algo i hm no only de e mine he mpa, bu i s associa ed p obabili y.
F om his likelihood, ce ain ac i i ies ha do no adap well o se s o gene ic
classifica ion can be iden ified. I is an indica ion ha use is ca ying ou new
ac i i ies o which he sys em has no been ained p e iously.
Figu e 3 shows he o e all p ocess desc ibed on his sec ion o ecogni ion
p ocess om Ac i i y-In e al Ma ix calcula ed in he p e ious s age.
Reco e y da a om
accele ome e
senso
End o empo al
window?
No Build empo al
window da a se
Yes
Ge ime-domain
measu es
Apply Fas Fou ie
T ans o m
p ocessing
Ge equency-
domain measu es
Find an ac i i y such
ha he sum o
each in e al
associa ed
p obabili ies (in
Ac i i y-In e al
Ma iz) is
maximized
Sa e mos -likely
ac i i y in o use
ac i i y log
Fig. 3. O e all ecogni ion p ocess om da a senso s
4 Me hod Analysis
Once exposed he bases o he de eloped ac i i ies ecogni ion algo i hm, an
analysis o he new p oposal was pe o med. To do his, he new de elopmen
was compa ed wi h a ecogni ion sys em widely used based on neu al ne wo k. In
his case, bo h lea ning and ecogni ion was pe o med by con inuous me hods.
The es p ocess was conduc ed in a Google Nexus One o a g oup o 10 use s.
No ably, he ac i i y habi s o hese use s we e adically diffe en , since 5 o hem
we e unde 30 yea s while he es we e olde han his age. Fo his pu pose, a
documen was deli e ed o each use o desc ibing he ac i i y pe o med, s a
ime and end ime.
Finally, he lea ning p ocess consis ed on he pe o ming o each ac i i y ec-
ognized by he sys em o a ime o 6 minu es. As o he ecogni ion p ocess,
use s we e ollowed o e a pe iod o 72 hou s.
Mo eo e , he ene gy consump ion and he p ocessing cos o he sys em when
i is wo king on a mobile de ice a e conside ed. In his case, he conclusion
eached is ha he me hod based on Ame a educes he compu a ional cos
o he sys em by abou 50% (see Figu e 4. The ime needed o p ocess a ime
window by using nue al ne wo ks me hods is 1.2 seconds, while, o he Ame a-
based me hod is 0.6 seconds.