Assessmen o Biosignals o Managing
a Vi ual Keyboa d
Manuel Me ino1, Isabel Gómez1, Albe o J. Molina1, and Ke in Guzman2
1 Elec onic Technology Depa men , Uni e si y o Se ille
A d. Reina Me cedes s/n, 41012, Se ille, Spain
[email p o ec ed]
{igomez,almolina}@us.es
h p://ma ix.d e.us.es/g upo ais/
2 Guadal el S.A. Pas o y Lande o, 19, 41001, Se ille, Spain
[email p o ec ed]
Abs ac . In his pape we p opose an assessmen o biosignals o handling an
applica ion based on i ual keyboa d and au oma ic scanning. The aim o his
wo k is o measu e he e ec o using such applica ion, h ough di e en in e -
aces based on elec omyog aphy and elec ooculog aphy, on ca diac and elec-
ode mal ac i i ies. Fi e people wi hou disabili ies ha e been es ed. Each
subjec w o e wice he same ex using an elec omyog aphy in e ace in i s
es and elec ooculog aphy in he second one. Each es was di ided in o ou
pa s: ins uc ion, ini ial elax, w i ing and inal elax. The esul s o he es s
show impo an di e ences in he elec oca diog am and elec ode mal ac i i y
among he pa s o es s.
Keywo ds: a ec i e in e aces, biosignals, con ol sys em, disabili y.
1 In oduc ion
Acco ding o he Eu os a [1], he o al popula ion in Eu ope was 362 million in 1996
which 14.8% o he popula ion be ween 6 and 64 yea s old ha e physical, psychologi-
cal o senso y disabili ies. The e o e, assis i e and adap ed sys ems ha e o be de el-
oped. The aim o hese ones is o help use s in hei dia y asks, le ing hem o li e in
a mo e com o able way.
Focus on so wa e, in gene al, i can be claimed ha hese applica ions ha e a high
deg ee o lexibili y and cus omizable, bu hei h oughpu s ongly depends on he
adap ed de ice o in e ace he use needs o in e ac wi h i . The a o emen ioned
h oughpu migh also be d ama ically educed o people who su e om se e e
disabili y. Ano he lack o hese so wa e p og ams is ha hey canno be au oma ical-
ly adap ed when hey a e unning and sel -adjus ing o use equi emen s.
The goal o de eloping a me hod o ma ching pa ame e s ha could be modi ied
in an adap a i e ow-column scanning o each indi idual use is discussed in [2], bu ,
in ha pape con ex ual in o ma ion o he applica ion is conside ed ins ead o physio-
logical signals. Nowadays he e a e se e al esea ches ocus on in inco po a e
he
ea u e o sel -adap a ion in di e en sys ems o applica ions. So, based on his con-
cep he e a e pape s whose aim is modi y he en i onmen o adap o he use
needs/p e e ences (Ambien In elligence) o use biosignals o adjus an applica ion
au oma ically (Physiological Compu a ion [3]).
This pape desc ibes how he use o i ual keyboa d (VK) a ec s di e en physio-
logical signals and how hese ones could be used o inco po a e au oma ic adap a ion
o unc ioning pa ame e s in his VK applica ion.
2 Rela ed Wo k
Disable people need in e aces o be adap ed o hei physical and cogni i e skills.
Wha e e he use ’s ype o disabili y, he in e ace needs o de ec use ’s will in o de
o selec he highligh ed ow o column on he i ual keyboa d. In mos cases, use is
able o mo e some muscles and p ess a special swi ch, o con ol a mouse poin e on
compu e sc een h ough an accele ome e [4]. Bu some imes use ’s abili y is con-
s ained o mo e he eyes; he e o e an eye- acke is equi ed. Elec ic biosignals can
be used ins ead o hose sys ems. Fo ins ance, an elec omyog aphy (EMG) sys em
can eco d muscle ac i i y and subs i u e he swi ch, and also elec odes placed
a ound he eyes can be used o de ec gaze [5, 6]. Mo eo e , in e aces based on
B ain Compu e In e ace (BCI) don’ equi e use mo emen s [7] because his one
is based on elec oencephalog am (EEG).
Biosignals can be also used as a sou ce o in o ma ion when use emo ional s a e
mus be de e mined. Elec ode mal ac i i y (EDA) [8] and elec oca diog am (ECG)
a e wo o he biosignals mos used in his a ea. EDA has been used in se e al e-
sea ches abou s ess and cogni i e load [8-10]. I has been shown EDA is a good
me hod o de ec di e en emo ional s a es. A ypical expe imen o measu e he s ess
[9] is o make a subjec calcula e se e al a i hme ic ope a ions o di e en le el o
di icul in 4 minu es and 3 phases. F om phase o phase he le el o s ess goes up
by inc easing he ope a ion di icul y. Signi icance changes in EDA among phases
ha e been epo ed, such ha he cogni i e load was e alua ed co ec ly in 82.8% o
s udied cases.
I has also epo ed in se e al pape s ha he ca diac ac i i y is also a ec ed by he
kind o ask. In [10] he au ho s ha e used biosignals as objec i e indica o s o s a e
o use s, whe e EDA, ECG, EMG and b ea h equency we e eco ded while he sub-
jec s played a game.
3 Me hodology
3.1 Applica ion
The es s we e de eloped using a cus omizable VK [6] shown in igu e 1. Among
o he hings he VK allows: keys pe sonaliza ion, using di e en con ol in e aces
based on e en s and gene a ing a lis o p edic ed wo ds. An ex ended VK dis ibu ed
by ows o six cha ac e s each is selec ed. High equency cha ac e s in Spanish lan-
guage a e placed on i s ows. A ow/column scanning me hod was used o selec a
le e o a wo d on a p edic ion lis . The minimum numbe o selec ed cha ac e s used
o u n on p edic ion engine was se o 3. The scan ime o dwell ime was es ablished
in 1.6s o EMG con ol in e ace and 1.8s o EOG con ol in e ace based on 0.65
ule [11].
Fig. 1. Ex ended Le e VK
3.2 Signal Acquisi ion
Th ee di e en biosignals we e simul aneously eco ded du ing he es s. One o hem
was used o manage a VK applica ion (EMG o EOG), and he o he s we e egis e ed
o de e mine he s ess/ a igue o he use induced by he con ol in e ace (ECG,
EDA) [8-10].
The assembly used o eco d he di e en biosignal is shown in he igu e 2. A mo-
nopola assembly was employed o eco d he ECG signal ( igu e 2.a). The EDA as-
sembly was placed on he w is ( igu e 2.b) [12]. Th ee elec odes on he a m we e
used o he EMG signal ( igu e 2.c). Finally, o pick EOG signal up wo elec odes
by di ec ion ( igu e 2.d) whe e placed a ound he eye [13] and he g ound and e e -
ence senso s we e placed espec i ely in he igh and le ea s.
Fig. 2. Elec ode posi ions. a) ECG. b) EDA. c) EMG. d) EOG.
Ag/AgCl elec odes wi h sel -adhesi e/conduc i e gel we e used o eco d he bio-
signals. Two bioampli ie s we e employed: g.USBamp and g.MOBILab o gTec. The
i s o hem egis e ed he con ol biosignals wi h a sample equency o 512Hz o
EMG and 128Hz o EOG. A no ch il e in (48, 52)Hz was implemen ed o elimina e
he powe elec ic noisy, and a bandpass il e in (5, 200)Hz o EMG and (0.1, 30)Hz
o EOG. Also, he e sion 2.0 o he BCI2000 so wa e was employed [7] o send he
e en o he VK h ough a socke . The o he bioampli ie eco ded he ECG signal
wi h a equency o 256Hz, a no ch il e in (48, 52)Hz and a bandpass il e in (0.5,
100)Hz.
The EDA signal employed a di e en sys em: QSenso [12]. This one is a wea able
sys em ha wea s in he w is and uses Ag/AgCl elec odes. Di e en sample e-
quency can be employed and he da a a e s o ed in he in e nal memo y. 32Hz o
sample equency was se ing.
The e sion 7.6.0.324 o Ma lab applica ion was u ilized o o line analysis o he
eco ded da a.
3.3 P ocedu e
T ials a e ocused on ex inpu unc ion o he VK. EMG and EOG signal a e consi-
de ed like con ol in e aces. Fi e use s wi hou disabili ies be ween 26 and 45 yea s
old pa icipa ed in hese ials (mean 31.2, sd. 7.79); only one o hem had used he
applica ion be o e.
The empo al line o he expe imen is spli ed in o ou pa s:
1. Task explana ion and elec ode se ing. Use s a e ins uc ed abou he ask hey a e
going o do. Then elec odes o eco d EDA, ECG, EOG and EMG signal a e
placed.
2. Ten minu es o elax. Du ing his ime use is eading a magazine.
3. Main ac i i y. Use s inpu a p ede ined ex using i ual keyboa d. Command o
con ol he applica ion is based on biosignals. Du a ion o he ial is use depen-
den , his is, he e is no ime o inish i .
4. Res ime. 10 minu es o eco e , eading a magazine.
The main ask consis s in w i ing a ex o 43 wo ds (209 cha ac e s). The applica ion
is ope a ed by a single e en which, in u n, is gene a ed in wo di e en ways: Mus-
cle ac i i y by EMG signal p ocessing o Ho izon al Eye Mo emen by EOG analysis.
A w ong cha ac e had o be co ec ed. Howe e , he mis akes made by selec ing an
inco ec wo d om he p edic ion lis had o be igno ed, so he use had o w i e he
nex wo d.
The ECG and EDA signals a e eco ded in all phases, excluding phase 1 whe e he
use s a e ins uc ed. By measu ing hese signals, i is possible o acqui e objec i e
in o ma ion abou he use emo ional s a e. Two S a e-T ai Anxie y In en o y (STAI)
ques ionnai es we e also p o ided o subjec s. They illed hem in a he beginning o
he elax phases (phases 2 and 4). So, a subjec i e measu e o s ess is also achie ed.
4 Wo k and Resul s
Di e en pa ame e s o a ec i e signals ha e been esea ched. The hea a e
a iabili y (HRV) has been he main ex ac ed ea u e om ECG. Each HRV ec o
elemen shows he ime be ween wo neighbo hea bea s. The ex ac ed da a a e clas-
si ied in wo g oups: in o ma ion based on empo al analysis o based on equency
analysis. The s anda d de ia ion o bea - o-bea o NN in e als (SDNN), he squa e
oo o he mean squa ed di e ence o successi e NN in e als (RMSSD), he p opo -
ion o NN in e als ha di e by mo e han 50 ms (pNN50), he wid h o he
minimum squa e di e ence iangula in e pola ion o he highes peak o he his o-
g am o all NN in e als (TINN), and he en opy o HRV a e he pa ame e s o he
i s g oup. The o he g oup is based on calcula e he powe spec um densi y (PSD)
o he HRV. The PSD is di ided in 4 pa s: equency om DC o 0.003Hz (Ul a
Low F equency - ULF), om 0.003 o 0.04Hz (Ve y Low F equency - VLF), om
0.04 o 0.15Hz (Low F equency) and om 0.15 o 4Hz (High F equency - HF). One
addi ional pa ame e is he a io be ween HF and LF (HF/LF). The in luence o each
band abou he PSD was calcula ed as he sum o PSD o he band di ided by o e all
PSD sum. In all pa ame e s, a slide window o 5 minu es o wid h wi h a shi o 1
hea bea was applied.
Fi e pa ame e s we e ex ac ed om EDA signal: endency, a e age ampli ude,
mean o de i a i e, a e age inc ease in pe iods o ising, and ime a io inc ease. A
slide window o 5 minu es o wid h wi h a shi o 1 sample was applied.
The ex ac ed in o ma ion was analyzed h ough an analysis o a iance (ANOVA)
o each single subjec wi h a h eshold o 1% (alpha pa ame e - p < 0.01). Fou ana-
lyses o each subjec we e ealized: elax phase 1 s. main ask, main ask s. elax
phase 2, elax phase 1 s. elax phase 2, and elax phase 1 s. main ask s. elax
phase 2. Di e ence be ween using EMG o EOG in e aces was he goal o he nex
ANOVA analysis: elax phase 1 o EMG con ol s. elax phase 1 o EOG, main ask
o EMG s. main ask o EOG, and elax phase 2 o EMG s. elax phase 2 o EOG.
The expec ed esul s we e he elax phases we en’ a ec ed (p > 0.01) and he main
ac i i ies showed signi ica i e di e ence be ween EMG and EOG (p < 0.01). Howe -
e , he ANOVA esul s o majo i ies o subjec s in elax phases showed an impo an
in luence o unknown a iable (p < 0.01). So, a ia ions in ECG and EDA signals due
o doing main ask wi h EMG o EOG in e aces a e no conclude, in despi e o he
expec ed esul s o main asks we e ob ained.
Table 1. STAI es esul s o con ol in e aces
Phase Use 1 Use 2 Use 3 Use 4 Use 5
EMG EOG EMG EOG EMG EOG EMG EOG EMG EOG
Relax 1 40 46 43 38 43 47 52 50 43 44
Relax 2 44 45 44 35 41 36 46 45 38 37
The EMG and EOG con ol in e aces showed he kind o phase a ec s di ec ly all
EDA pa ame e s (p < 0.00045). Also, SDNN, RMSSD, pNN50 (p < 0.0007) and he
bands ULF and VLF (p < 0.0085) o ECG pa ame e s showed signi ica i e changes
when using he EMG in e ace, while SDNN, pNN50 (p < 0.0004) and all equency
bands (p = 0) we e a ec ed o EOG con ol . On he o he hand, he able 1 wi h he
STAI esul showed ha he di e en phases a ec ed o s ess s a e o he subjec s,
such ha he subjec i e s ess le el o wo o hem was highe in he end o he main
ask han in he begging o es o EMG. The o he s we e a s ess le el highe in he
begging o he es han in he end o main ask. Also, he a iance ( a .) o he mod-
ulus o he di e ence be ween phases in STAI es was bigge in EOG ( a . 8) han
EMG ( a . 3.44).

5 Conclusion and Planned Lines o Resea ch
This pape p esen s how he biosignals can be used o manage an applica ion om
wo poin s o iew: he olun a y con ol ac ions and sel -adap a ion based on he
measu e o emo ion. EMG and EOG a e used o con ol and EDA and ECG o meas-
u e emo ion.
To sum up, i ha e done EMG and EOG es and i has been de e mined by
ANOVA analysis wha a e he pa ame e s o EDA and ECG signals ha a e a ec ed
in di e en phases o ial. Signi ican changes we e obse ed in hese pa ame e s. So,
impo an di e ences we e ound be ween elax phases and main ask. Howe e , i
was no possible de e mina e ECG and EDA changes be ween EMG o EOG in e -
ace, because an unknown a iable a ec ed he elaxed pe iods.
In he u u e, he aim will be de elop an in elligen agen o inco po a e o he VK
applica ion. This one will adap au oma ically he so wa e depending on emo ional
s a e o he use o do easie he use o sys em. So, i is necessa y o de e mine exac ly
wha he causes a e which p oduce he changes in he biosignals, i ed o s essed, and
how hese s a es a ec o he measu ed pa ame e s.
Acknowledgemen s. This p ojec has been ca ied ou wi hin he amewo k o a
esea ch p og am: (p08-TIC-3631) – Mul imodal Wi eless in e ace unded by he
Regional Go e nmen o Andalusia.
Re e ences
1. Eu os a . Hea h S a is ics. Luxemb ou g: O ice o O icial Publica ions o he Eu opean
communu ies (2002), h p://epp.eu os a .ec.eu opa.eu ISBN: 92-894-3730-8
( e ie ed on Janua y 2012)
2. Simpson, R.C., Koes e , H.H.: Adap i e One-Swi ch Row-Column Scanning. IEEE T ans-
ac ions on Rehabili a ion Enginee ing 7(4), 464–473 (1999)
3. Fai clough, S.H.: Fundamen als o Physiological Compu ing. In e ac ing wi h compu -
e s 21(1-2), 133–145 (2009)
4. Gómez, I., Anaya, P., Cab e a, R., Molina, A.J., Ri e a, O., Me ino, M.: Augmen ed and
Al e na i e Communica ion Sys em Based on Dashe Applica ion and an Accele ome e .
In: Miesenbe ge , K., Klaus, J., Zagle , W., Ka shme , A. (eds.) ICCHP 2010. LNCS,
ol. 6180, pp. 98–103. Sp inge , Heidelbe g (2010)
5. Dhillon, H.S., Singla, R., Rekhi, N.S., Rameshwa , J.: EOG and EMG Based Vi ual Key-
boa d: A B ain-Compu e In e ace. In: 2nd IEEE In e na ional Con e ence on Compu e
Science and In o ma ion Technology (2009)
6. Me ino, M., Gómez, I., Ri e a, O., Molina, A.J.: Cus omizable So wa e In e ace o
Moni o ing Applica ions. In: Miesenbe ge , K., Klaus, J., Zagle , W., Ka shme , A. (eds.)
ICCHP 2010, Pa I. LNCS, ol. 6179, pp. 147–153. Sp inge , Heidelbe g (2010)
7. Schalk, G., McFa land, D.J., Hin e be ge , T., Bi baume , N., Wolpaw, J.R.: BCI 2000: A
Gene al-Pu pose B ain-Compu e In e ace (BCI) Sys em. IEEE T ansac ions on Biomedi-
cal Enginee ing 51(6), 1034–1043 (2004)
8. Boucsein, W.: Elec ode mal Ac i i y, 1s edn. Sp inge (1992)
9. Se z, C., A n ich, B., Schumm, J., La Ma ca, R.: Disc imina ing S ess om Cogni i e
Load Using a Wa eable EDA De ice. IEEE T ansac ions on In o ma ion Technology in
Biomedicine 14(2), 410–417 (2010)
10. Mand yk, R.L., Inkpen, K.M., Cal e , T.W.: Using Psychophysiological Technique o
Measu e Use Expe ience wi h En e ainmen Technologies. Beha io & In o ma ion
Technology 25(2), 141–158 (2006)
11. Simpson, R., Koes e , H.H., Lop es i, E.F.: Selec ing an app op ia e scan a e: he ule. 65.
Assis i e Technology 19(2), 51–58 (2007)
12. Poh, M.-Z., Swenson, N.C., Pica d, R.W.: A Wea able Senso o Unob usi e, Long-
Te m Assessmen o Elec ode mal Ac i i y. IEEE T ansac ions on Biomedical Enginee -
ing 57(5), 1243–1251
13. Me ino, M., Ri e a, O., Gómez, I., Molina, A., Do onzo o, E.: A Me hod o EOG Signal
P ocessing o De ec he Di ec ion o Eye Mo emen s. In: Senso De ices 2010, pp. 100–105
(2010)