Development of hands-free interaction techniques and alternative computer access solutions for people with motor-neuron impairments [original]

T itle Page
Develop m ent of Hands -fr ee Interaction T echniques and
Alternative Co mputer Access Solu tions for P eople w ith
Motor -neur on Impa irments

vorg elegt von
M. Sc.
Cagda s Esi yok
ORCID: 0000-0003-0229-0948

an der Fakultät IV – Elektrotechnik und Informatik
der T e chnischen Universität Berlin
zur Erlang ung des akade mischen Gra des

Doktor der I ngenieurwiss enschaften
– Dr .- Ing. –

ge nehmigte Dissertation

Promotionsausschuss:

V orsitzender:

Prof. Dr . Odej Kao

Gutachter:

Prof. Dr . Dr . h.c. Sahin Albayrak

Gutachter:

Prof. Dr . -Ing. Matthias Rötting

Gutachterin:

Prof. Dr . Ali ye T osun

T ag der wissenschaftlichen Aussprache: 18. August 2020

Berlin 2020

Intelligence is the a bility to adapt to ch an ge.
-Stephen Hawking-

i
Abstract
In our increasingl y di gitalized world, computers have become indispensable tools by
of fering several useful services in many aspects of life such as communication,
education, commerce, h ealth, social interaction, and entertainment. Un fortunately , most
people with motor-impairments have dif ficulties to acce ss such services because
conventional input devices are not concertedl y designed for them. Hands-fre e Comput er
Acce ssibilit y T ools (CA T s) help these people to a chieve aforesaid useful services for a
more inclusive and barrier -free life. However , h ands-free computer access is still a
challenging task for p eople with severe motor-impairments of the li mbs such a s
quadriplegics. Especiall y , when it comes to the peo ple who have onl y a sin gle voluntar y
ge sture above neck survived (such as blinks or toot h -clicks), hands- free computer
access with a single-ges ture becomes one of the most challeng ing tasks in human -
computer interaction (HCI ).
Throug h this thesis, we f ocus on the single -gesture based hands-fre e computer
access problem. The existing HC I solutions on this problem are mostl y based on
expensive dedicated devices be yond standard computer peripherals. Although the aim
of the unive rsal a ccess is enabling equal opportunit y b y r educing barriers, high -cost of
curre nt solutions creates a new barrier financially for the majorit y of tar get group.
Furthermore, most of the exist ing single-gesture b ased hands -free HCI techniques are
only compatible with a specific switch- accessible interface. T o overcome these
deficienc ies of th e existing HC I solutions, we propose our novel software switch
approac h. By following the principles of the software switch approach, w e a lso propose
four novel software s witches which ar e sin gle-gesture based HCI techniques named as
the Puf fCam, the Puf fMic, the H eadCam, and the H eadGyro. Unlike the exist ing
solutions, our proposed software switch es don' t require an y dedicated d evices, and they
are c ompatible with most swi tch-accessible interfaces.
Although the proposed s oftware switches can allow the users to interac t with a
computer b y a single puff/he ad gesture, the users require a CA T whi ch is capable of
converting the emulated switch presses b y s oftware switches into m eaningful

commands to operate a computer . In accordance with this requirement, we p r esent a
new single-switch accessible CA T called the G LOSTER 1.0 with a novel mouse
pointing technique w ithi n the scope of thesis.
In addition to the above- mentioned contributions, the inadequacies of the
existing evaluation tool s promote us to desig n a novel evaluation tool namely the
SITbe nch 1.0 . As a benchmark tool , it is able to serve not onl y for the p roposed softwa re
switches but also the other available switch-based interac tion techniques (S I T s). It
provides a quick er and more accurate switch e valuation proce ss b y coll ecting and
saving the objective da ta automatically .

iii

Zusammen fassung
In unserer zunehmend digitalisierten W elt sind Computer zu unverzichtbaren
W erkzeug en ge worden, da sie in vielen Bereichen des L ebens wie Kommunikation,
Bildung und H andel sowie Gesundh eit, soziale Interaktion und Unterhaltung
verschiedene nütz liche Dienste anbieten. Leider haben viele M enschen mi t motorischen
Bee inträchti gunge n Sch wierig keiten, auf solche Dienste zuz ugre ifen, da k onventionelle
Eingabegeräte nicht auf sie ab ge stimmt sind. F reihändige Computerzug angshilfen
(Hands-free Computer Accessibilit y T ools, CA T s) helfen diesen Menschen, die oben
ge nannten Dienste z u nutzen und somit ein inkl usiveres und barrierefreies L eben in der
Computerwelt zu erreichen. Der freihändige Com puterzugang ist jedoch imm er noch
eine He rausforderung für Menschen mit schweren motorischen Beeinträchtig ungen der
Gliedmaßen, wie z. B. Quadriplegiker . Besonders wenn es um M enschen geht, die mit
nur einer einzigen freiwilligen G este über de m Hals leben (wie Bli nzeln oder
Za hnklicken), wird der freihändige Computerzugang mit einer einz igen Geste z u einer
der anspruchsvollsten Aufgaben in der Mensch -Computer -Interaktion (HCI).
In dieser Arbeit konzentrieren wir uns auf das Problem des freihändigen
Computerzugangs mit einer einzigen Gest e. Die e xistierenden HC I- Lösungen für di eses
Problem basieren meist auf teuren dedizierten Geräten, die über die Standard -
Computerperipherie hina usgehen. Obwohl d as Ziel des universellen Zugangs darin
besteht, durch den Abbau von Barrieren Chancengleichheit zu ermög lichen, stellen die
hohen Kosten der derzeitige n Lösungen für die Mehrheit der Z ielgruppe finanziell eine
neue Barriere da r . Darüber hinaus sind die meisten der existierenden Single -Gesture-
basierten HCI-Freisprechverfa hren nur mit einer spezifischen, switch- kompatiblen
Schnittstelle kompatibel. Um diese Mänge l der bestehenden HC I-Lösungen zu
überwinden, s chlagen wir unser en neuartigen Software -Switch-Ansatz vo r . W ir folgen
den Prinzipi en des Software-Switch-Ansatzes und schlagen außerdem vier neue
Software-Switches vor , d ie auf Single -Gesture-basierten HC I-T echniken au fbauen und
als Puf fCam, PuffMic , HeadCam und HeadG y ro bezeichnet werden. I m G egensatz zu
den bestehenden Lösungen benöti gen uns ere vorge schlagenen Software-S witches keine

dedizierten Geräte und sind mit den m eisten Switch -z ugä nglichen Schnittstellen
kompatibel.
Obwohl die vorge schlagenen Software-Switch es den Benutz ern die I nterakti on
mit einem Comput er durch eine einzige Puf f -/Kopf-Geste ermöglichen können,
benötige n die Benutzer einen CA T , der in der Lage ist , die emulierten Swit ch-Pressen
durch Software-Switches in sinnvolle Befehle zur Bedienung eines C omputers
umzuwandeln. Entsprechend dieser Anf orderung stellen wir im R ahmen der
Doktorarbe it einen neuen mit einer neuartigen Mauszeigertechnik sowie einem einzigen
Switch zugängliche n CA T vor - namens GLOSTER 1.0.
Zusä tzlich z u den oben genannten Beiträgen haben uns die Unzulänglichkeiten
der bestehenden Ev aluierungswerkzeuge dazu bewegt, ein neuartiges
Evaluierungswerkzeug , n ämlich die SITbench 1.0, z u entwerfe n. A ls Bench mark-T ool
ist es in der Lage, ni cht nur für die vo r gesc hl agenen Software -Switches, s ondern auch
für die anderen verfü gbare n switch-basierten Interaktionstechniken (SIT s) zu dienen.
Es e rmöglicht ein en schnelleren und genaueren Bewertungsprozess der Switches,
indem es die objektiven Daten automatisch sa mm elt und speichert.

vii

Acknow ledgements
This work would not hav e been possible without the support, help and encouragement
of many people.
First of all, I would like to ex press m y gratitude t o m y thesis sup ervisor P rof. D r . -Ing.
Sahin Alba y rak for givin g me the chance and sup port to work on m y P hD topic at TU
Berlin. I also take thi s opportunit y to thank Prof. Dr . -Ing. M atthias Rötting and Prof.
Dr . Ali y e T osun, for rev ie w ing m y thesis.
In p articular I wish to thank Dr . Brijnesh.-J. Jain for giving me constant f eedback and
professional advice on my research, which ultim ately led to improved results.
Likew is e, I would like to thank Dr . Frank Hopf gartner and Dr . Fikr et Sivrikay a for their
precious comments, suggestions and support while forming this work.
In addition, I would li ke to express m y appreciation to the Ministry of Nationa l
Education Scholarship of the T urkish Republic for funding me during my PhD studies.
Special thanks go to great people who took part in the usability studies through this
thesis.
Further , I would like to thank m y colleagues from the Information Retrieval and
Machine Learning research group at DA I-Labor for their support and for providing a
friendly working e nvironment within this period.
I would also like to express my deepest gratitude to m y famil y . I alw a y s felt their support
through my e ducation.
La st but not least, I would like to deepl y thank m y wife. All of thi s would not have be en
possible without her unconditional support and sweet smile.

viii

List of Publicatio ns
A large part of thi s thesis is based on the following publications by the author:
Journal Articles
Chapters 2 and 3:
• Cagdas Esiyok , A y h an Askin, Aliy e T osun and Sahin Alba y rak, “ Sof twar e
Switches: Novel Hands-f r ee Interaction T echniques for Quadriplegics Bas ed on
Respiration–machine Interaction ”, Universal Access in the Infor mation So ciety ,
2019.
Chapt er s 2 and 4:
• Cagdas Esiyok , A y han Askin, Ali y e T osun and S ahin Albayrak, “ N ovel Hands-
Fr ee Interaction T echniques based on the Softwar e Switch Appr oach for
Computer Access with Head Movements ”, Univer sal Access in the Information
Society , 2020 .
Chapter 6:
• Cagdas Esiyok and Sahin Albayrak, “ SITbenc h 1.0: A Novel Swit ch-Based
Interaction T echnique Benchmark ”, Journal of Healthcare Engineering, 2019.
Other p ublications by the author outside the scope of this thesis:
Conference Paper
• Cagdas Esiyok , Benjamin Kille, Brijnesh.-J . Jain, Frank Hop fgartner and Sahin
Albay rak, “Users' Reading Habits in Online News Portals” , Procee din gs of the
5th Information Interac ti on in Context S y mposiu m, 201 4.
Book Chapter
• Cagdas Esiyok , and Sahin Alba y rak, “T witter Sentiment T racking for Pr edicting
Marketing T r ends” , S mart I nformation Systems, 2015.

List of Abbreviations

AAC

Augmentative and Alternative Communication

AL S

Amyotrophic la teral sclerosis

BCI

Bra in-Computer Interaction

CA T s

Computer Accessibilit y T ools

Control group

Co orP

Coordinate-based Pointing

Disability group

DS - NDG

Double switch Non-stop Driver Game

EEG

Electroencepha lograph y

EMG

Electromyogra ph y

EOG

Electrooculography

FDP

Forefinger distal pulp

False negatives

False positives

FPI J

Forefinger pr oxim al interphalang eal joint

GDP

Gross Domestic Product

Gyro

Gyroscope

HCI

Human-computer interaction

HFG

Hungr y Frog Game

IL O

Internationa l La bour Organisation

LED

Light-e mitting diode

MEMS

Microelec tromechanical system

Mic

Microphone

MMSE

Mini-Mental State Examination

NDG

Non-stop Driver Game

SIT s

Switch-based interaction tec hniques

SS - NDG

Single switch Non-stop Driver Game

SUS

System Usability Scale

T rue n egatives

T rue positives

TSMG

T ie-Smile y Matchin g Game

WLAN

W ireless local area ne twork

xii

xiii

T a ble of Contents
Title Page ....................................................................................................................... 0
Abstract .......................................................................................................................... i
Zusamme nfassu ng ................................................................ ....................................... iv
Acknowledgements .................................................................................................... vii
List of Publications ...................................................................................................... ix
List of Abbreviations ...................................................................................................xi
Table of Contents ...................................................................................................... xiii
List of Tables ............................................................................................................ xvii
List of Figures ............................................................................................................xix
1 Introduction .............................................................................................................. 1
1.1 Motivation ........................................................................................................... 2
1.2 Problem Statement .............................................................................................. 3
1.3 Researc h Questions ............................................................................................. 4
1.4 Thesis Contributions ........................................................................................... 6
1.5 Thesis Structure .................................................................................................. 8
2 The Software Switch Approach ............................................................................ 10
2.1 Related Works ................................................................................................... 11
2.2 Existing Problems ............................................................................................. 12
2.3 Principles of the Software Switch Approach .................................................... 13
2.4 Gesture Selection .............................................................................................. 14
2.5 Summary ........................................................................................................... 15
3 The PuffMic and the PuffCam : Novel Interaction Techniques Based on a
Single Respiration-Gesture ................................................................................... 17
3.1 Introduction ....................................................................................................... 18
3.2 Software Switches ............................................................................................. 20
3.2.1 The User Interface ................................ ................................................. 20
3.2.2 The PuffMic .......................................................................................... 21
3.2.3 The PuffCam ......................................................................................... 23
3.3 Evaluation ................................................................................................ ......... 26
3.3.1 Participants ............................................................................................ 26
3.3.2 Apparatus .............................................................................................. 29

xiv

3.3.3 Procedure .............................................................................................. 30
3.3.4 Objective Da ta based Results ................................................................ 31
3.3.5 Subjective Data ba sed Results .............................................................. 33
3.4 Conclusion and Discussion ............................................................................... 35
4 The HeadCam and the HeadGyro: Novel Interaction Tec hniq ues Based on a
Single Head-Gesture .............................................................................................. 40
4.1 Introduction ....................................................................................................... 41
4.2 Related Works ................................................................................................... 43
4.2.1 Head-operated Interaction Techniques without a Single Head-gesture
Acce ss Support ...................................................................................... 43
4.2.2 Head-operated Interaction Techniques with a Single H ead - gesture
Acce ss Support ...................................................................................... 44
4.3 Software Switche s ............................................................................................. 45
4.3.1 The User Interface ................................ ................................................. 45
4.3.2 Th e HeadCam ....................................................................................... 47
4.3.3 The Hea dG yro ....................................................................................... 49
4.4 Evaluation ................................................................................................ ......... 50
4.4.1 Participants ............................................................................................ 50
4.4.2 Apparatus .............................................................................................. 54
4.4.3 Tests ...................................................................................................... 54
4.4.4 Procedure .............................................................................................. 55
4.4.5 Objective Da ta based Results ................................................................ 56
4.4.6 Subjective Data ba sed Results .............................................................. 59
4.5 Conclusion and Discussion ............................................................................... 60
5 The GLOSTER 1.0: A new Single-sw itch Acce ssible CAT with a Novel Mouse
Pointing Technique ................................................................................................ 64
5.1 Introduction ....................................................................................................... 65
5.2 Related Works ................................................................................................... 66
5.3 Design of the GLOSTER 1.0 ............................................................................ 69
5.3.1 The User Interface ................................ ................................................. 69
5.3.2 Pointing: The CoorP Technique ............................................................ 71
5.3.3 Clicking ................................................................................................. 72
5.3.4 Typing ................................................................................................... 73
5.4 Evaluation ................................................................................................ ......... 73
5.4.1 Participants ............................................................................................ 73
5.4.2 Apparatus .............................................................................................. 74

5.4.3 Tests ...................................................................................................... 74
5.4.4 Procedure .............................................................................................. 77
5.4.5 Objective Da ta based Results ................................................................ 78
5.4.6 Subjective Data ba sed Results .............................................................. 78
5.5 Conclusion and Discussion ............................................................................... 79
6 The SITbench 1.0: A Novel Switch-Based Interaction Technique Benchm ark
.................................................................................................................................. 82
6.1 Introduction ....................................................................................................... 83
6.2 Design of the SITbe nch 1.0 .............................................................................. 86
6.2.1 Tie-Smiley Ma tching Game (TSMG) ................................................... 87
6.2.2 Non-stop Driver Game (NDG) ............................................................. 90
6.2.3 Hungr y Frog Game (HFG) .................................................................... 93
6.3 Evaluation ................................................................................................ ......... 94
6.3.1 Participants ............................................................................................ 95
6.3.2 Apparatus .............................................................................................. 95
6.3.3 Procedure .............................................................................................. 96
6.3.4 Objective Da ta based Results ................................................................ 97
6.3.5 Subjective Data ba sed Results .............................................................. 99
6.4 Conclusion and Discussion ............................................................................... 99
7 Conclusion and Outlook ...................................................................................... 103
7.1 Summary of the Thesis and C ontributions ..................................................... 104
7.2 Outlook ........................................................................................................... 109
7.3 Final Remarks ................................................................................................ . 111
8 Reference s ................................................................................................ ............. 114

xvi

xvii

List of T ables
T able 3.1: Age statistics of the participants according to the groups. .......................... 27
T able 3.2: Main characte ristics of the participants. ..................................................... 27
T able 4.1: Age statistics of the participants according to the groups. .......................... 52
T able 4.2: Main characte ristics of the participants. ..................................................... 52
T able 4.3: Statem ents of the SUS qu estionnaire with average scale values of all
participants. ∆ s ymbol is replaced with the Head Gyro and the HeadCam, respectivel y ,
during a ssessments. ...................................................................................................... 55
T able 4.4: M ean values o f the HeadG y ro and the HeadCam through evaluation metrics
of HFG (avera ge press t ime, the fastest p ress ti me, the slowest press time, avera ge
release time, the fastest re lease time, and the slowest release time) for all participants.
................................................................................................ ..................................... 59
T able 5.1: Summar y of the existing CA T s a llowing single-switch bas ed mouse pointing.
................................................................................................ ..................................... 68
T able 5.2: Age statistics of the participants according to the genders. ........................ 74
T able 5.3: The statements of the SUS questionnaire according to the GLOSTER 1.0
with average sca le values of all participants. ............................................................... 76
T able 6.1: Modified stat ements of a SUS questionnaire with avera ge scale values of all
participants. .................................................................................................................. 97

xviii

xix

List of Figures
Fig ure 3.1: The user inte rface of the Puf fMic and the Puf fCam. ................................ 21
Fig ure 3.2: Camera, lapt op and microphone positions during the experiments. ......... 22
Fig ure 3.3: Raw audio signal. ...................................................................................... 22
Fig ure 3.4: Absolute valu e of audio signal. ................................................................. 23
Fig ure 3.5: Smoothed au dio signal. ............................................................................. 23
Fig ure 3.6: A standard webcam modified with a post - it. ............................................. 24
Fig ure 3.7: Main steps of mot ion tracking algorithm. (a) take video frames via camera;
(b) appl y Eu clidean colour filter for each frame; (c) convert video frames to gray scale;
(d) detect the objec t; (e) track the position of the detected object. .............................. 25
Fig ure 3.8: T wo participants previous to t he experiments. .......................................... 27
Fig ure 3.9: The comfort assessment questionnaire with mean v alues (x -axis represents
the rating range). .......................................................................................................... 31
Fig ure 3.10: Mean values of the proposed software switches for all participants through
evaluation metrics (acc ur acy , p recision, recall, false positive rate) (**** p < 0.0001).33
Fig ure 3. 1 1: Mean values of interaction tech niques through evaluation metrics
(acc urac y , pr ecision, recall, false positive rate) according to the participant groups (Mix ,
DG, CG) (* p < 0.05; ** p < 0.01; *** p < 0.001). ........................................................ 34
Fig ure 4.1: (a) The initial state of the interfac e of the HeadCam and the HeadGyro
software switches. (b) Ro tational movements of a head. ............................................ 46
Fig ure 4.2: Six dif ferent intersection states of t he interf ace according t o rotational
movements of a head. .................................................................................................. 47

Fig ure 4.3: Steps of head tracking al gorithm: (a) ta ke video frames via camera; (b) appl y
Euclidean c olour filter for each frame; ( c) convert video fra mes to gray scale; (d) detect
all objects in eac h fra me; (e) choose the greatest object for each frame; (f) track the
position of the greatest object. ..................................................................................... 48
Fig ure 4.4: Th e placement of the smartphone on the use r's head for the HeadG y ro
software switch. ........................................................................................................... 49
Fig ure 4.5: T wo different stream data graphs based on the x -axis of the gy roscope sensor
of two dif ferent pa rticipants when participants nod their head. Blue an d red lines
repre sent unfiltered and Kalman filtere d g y roscope data, respectivel y . ...................... 51
Fig ure 4.6: Mean values of interaction te chniques acquired from all pa rticipants through
evaluation metrics of TSMG including accurac y , precision, recall, and false positive
rate (* p < 0.05). ........................................................................................................... 57
Fig ure 4.7: Mean v alues of the software switches t hrough evaluation metric s (acc urac y ,
precision, recall, false positive rate) according to the participant groups (Mix, DG, CG).
................................................................................................ ..................................... 58
Fig ure 4.8: Mean v alues of two software switch es for all pa rticipants through evaluation
metrics of HFG ( avera ge pr ess time, the fastest press time, the slow est press time,
average release time, the fastest release time, and the slowest release time) (* p < 0.05;
** p < 0.01). .................................................................................................................. 59
Fig ure 5.1: The use r interface o f the GLOSTER 1.0 including 4 modules. (a) The main
menu module to select the pointing, clickin g, an d t y ping functions. (b) T he numbers
module to t y pe the numbe rs and define the coordin ates of the target point. (c) The speller
module to t ype the letters. (d) The speech- generator module to convert the t y ped texts
to speech. ................................................................ ..................................................... 70
Fig ure 5.2: The main menu module to select the pointing, clickin g, and t y ping functions.
(a) Mouse left-click. (b) Mouse ri ght-click. (c ) Mouse point ing function. (d) Mouse
double-click. (e) Mouse middle-click. (f) T yping f unction. ........................................ 71

xxi

Fig ure 5.3: The user inter face of the GLOSTER 1. 0 which is divided int o 100 sub -parts
once the mouse c ursor po inting function is selected. .................................................. 72
Fig ure 5.4: The use r interface of the Point ingChallenge with three box es in diffe rent
sizes and colors. ........................................................................................................... 75
Fig ure 5.5: Mean values of the two mous e pointing techniques in terms of mispoint
count according to the two dif ferent scan-line sensitivi ty level where in (a) t he scan-line
sensitivit y is 10, (b) th e scan-line sensitivit y is 20. (*** p < 0.001; **** p < 0.0001). 78
Fig ure 6.1: W elcoming screen of the SITbench 1.0. ................................................... 87
Fig ure 6.2: Ex pected key assig nment module. ............................................................ 87
Fig ure 6.3: T ime-state m odel of an automa tic linear scanning sample. ...................... 88
Fig ure 6.4: Initial form of TSMG in t emplate 1. ......................................................... 88
Fig ure 6.5: Initial form of TSMG in ex pert mode. ...................................................... 89
Fig ure 6.6: A view of TSMG in t he end of a trial f ollowing a user performance without
any mistake. ................................................................................................................. 89
Fig ure 6.7: A general view from TSMG in the end of a trial following a us er
performa nce with several mistakes (i.e., with false negatives and false positives). .... 90
Fig ure 6.8: The initial form of SS -NDG where 1 shows left signal (i.e., or ange square
box); 2 is right signal; 3 represents green car , 4 is f inish line. .................................... 91
Fig ure 6.9: The initial form of DS-NDG in track 2. .................................................... 92
Fig ure 6.10: A vie w of DS -NDG in th e end of a t rial whe re th e use r r eached to the finish
line in track 3 without any crash. ................................................................ ................. 93

xxii

Fig ure 6.1 1: All three frames shown to the use r during a task: (a) the f rame shown until
a fly is appear ed; (b) the frame shown until the user activates the switch; ( c) the frame
shown once the user activates the switch. ................................................................ ... 93
Fig ure 6.12: A view of HFG in the end of a trial. ........................................................ 94
Fig ure 6.13: P ositions of forefinge r during experiments acc ording to two switch sit es
FDP (r epresented by x) and FPI J (represented b y z ): (a) switch press with FDP; (b)
switch relea se with FDP; (c) switch press with FPI J; (d ) switch release with FPI J. ... 95
Fig ure 6.14: Mean v alues of two switch sit es for all participants through e valuation
metrics of TSMG (accuracy , pr ecision, recall, false positi ve rate). (* p < 0.05; *** p <
0.001; **** p < 0.0001). ............................................................................................... 98
Fig ure 6.15: Mean values of two switch sites for all participants according to evaluation
metrics of SS -NGD as (a) the completion time and (b) the c rash count. (* p < 0.05; *** p
< 0.001). ....................................................................................................................... 98
Fig ure 6.16: Mean v alues of two switch sites for all participants through evaluation
metrics of HFG (average press ti me, average release time, the fa stest pr ess time, the
slowest press time, the fa stest release time, the slowe st release time) (**** p < 0.0001).
................................................................................................ ..................................... 99

xxiii

1
1 Intr oduction

1
Intro duction
W ithin the scope of this thesis, we provide comprehensive solutions to single-gesture
based hands- free computer access problem for people with severe -motor impairments.
In this int roductory chapter , we b egin with the mo tivation of the thesis which explains
why sin gle-gesture b ased hands-free computer access is important for people with
severe motor -neuron impairments in Section 1.1 . Then, the ex isting problems of the
curre nt solutions a re stat ed in S ection 1.2. Subse quently , addressed research questions
through this thesis are g iven in Section 1.3. After wards, we state the main contributions
of thi s thesis in accordance with the aim of pro viding compr ehensive solutions for
single-gesture based h ands -free computer access problem . L astly , Section 1.5 presents
the overall struc ture of th e thesis.

1.1 Motivation
Computers have become indispensable tools for t he genera l public, f acilitating man y
essential services in our increasingl y di gitalized world. Unfortunately , most people with
motor -impairments lack these services, since the conventional computer interaction
ways suc h as k eyboards are generall y inaccessible for them. The a bilit y of operating a
computer op ens the door for these p eople to achieve s everal useful services in man y
aspects o f life such as communication, commerce, health, entertainment, social
interaction, and edu cation for a more inclusive and barrier-free life, which leads to an
increased qualit y o f life by a ccessing intern et [ 1] . Hands-free C A T s pla y a vital role in
achieving these services [2, 3] . I n principle, the y can enable the users to operate a
computer with thei r uni mpaired ph y sical abilities such as head movements instead of
conventional way s. The famous theoretical ph y sicist Stephen Hawking —diag nosed
with amytrophic late ral s clerosis (A L S) — is a w ell-known example of a person who
utilized CA T s in s evera l aspects. Even after the loss of his mobilit y and speech, he was
still able to communicate and conduct scientific researc h b y means of a CA T with an
Augmentative and Altern ative Communication (AAC) technology throughout his life.
Unfortunately , Stephen Hawking was not an exceptional case. Th ere are millions
of people worldwide wh o require CA T s. I t is esti mated that there have been about one
billion people with several disabilities according to the W orld Report on Disabilit y in
201 1 [4]. Be sides, about 2% of the world population —between 1 10 and 190 mill ion
people— have s evere disabili ties in func tioning . Even onl y in United States, it is
predicted th at nearly 5.4 million people (between the ages of 18 and 64) live with
paralysis in 2013 [5] . People with motor-impa irments —as a result of ALS, carpal
tunnel s y ndrome, spinal cord injur y or de generative diseases — require assistive
technology solutions to have a more independent life.
On the other hand, according to the I nternational L abour Orga nisation (IL O)
statistics [ 6] published in 2007, an estimated 470 mi llion of the world ’s working a ge
people live with several disabilities. Althoug h there have been man y jobs that are
dependent on computer usage like software codi ng, exclusion of mi llions of workin g

age people with disabilities from labour fo rce l eads to an increase in the G ross Domestic
Product (GDP) lost world wide. Furthermore, as it is expected, the ones who can perform
a paid-job feel more c onf ident and independe nt bo th financially and ps y chologically .
1.2 Pr ob lem State me nt
Hands-free computer ac cess is a challenging task for people with se vere motor -
impairments of the limbs such as quadriplegics, si nce the y h ave serious diffic ulties to
control any bod y parts under neck. Esp ecially , when it comes to the people who have
only on e voluntary gesture above neck survived (such as blinks or tooth-clicks), hands-
free computer access with a sin gle-gesture b ecomes one o f the most challenging tasks
in human-computer interaction (H C I ).
In this section, we identi fy the exist ing problems under three domains in line
with our ultim ate goa l to provide comprehensive solutions for single-ge sture based
hands-free c omputer access problem.
Softwar e Switches: Single-gestur e based Hands-fr ee Interaction T echniques
Recent studies on sing le-gesture based hands-free computer access problem are
mostl y based on expensive dedicated devices bey ond st andard computer peripherals.
Considering that 80% of people with disabilities accommodate in poor and middle
income countri es [ 4], the majorit y o f these people have difficulties to afford most of
curre nt solutions [7 - 9]. Although the aim of th e universal access is enabling e qual
opportunity and a ccess to a service or a prod uct regardless of people's ph y sical
disabilities by reducing the barriers, high-cost of curre nt solutions cre ates a new ba rrier
financially for the majorit y of targ et group.
The other problem is that the majority of the current single-ge sture based ha nds-
free HCI techniques are only compatible with s pecific switch -accessible interfaces.
While some switch-acce ssibl e interfaces expect to receive a specific ke y board character
like enter , the others ex pec t to receive a mous e click. There is not any commonl y agreed
standard on this.

The GLOSTER 1.0: A Sin gle-gestur e Acc essible Hands -fr ee CA T
Although software switches proposed within this thesis can enable to interact
with a computer by a single-gesture, a CA T is re quired to control a compute r . As stated
in Section 1.1, the CA T s are capable of providin g several useful services for the ones
who c annot c ontrol a comput er with conventional wa y s. T yping a nd clicking are
ge nerall y achieved b y us ing switch es via a scanning method, while mouse pointing is
ge nerall y performed by head or e y e tracking methods . But, if the user has an only single-
ge sture unimpaired, all three fun ctions have to be performed with a single -
ge sture/single-switch. E specially , single-switch based mouse pointing is a very
challenging task in CA T s.
The SITbench 1.0: An Evaluation T ool
Evaluation process of a switch -based int eraction technique (SIT) —lik e the
proposed softwa re switches— requires an interdisciplinary team ef fort a nd takes a
considerable amount of time, since a SIT setup depends on man y va riables such as
switch t ype or switch site. Although collec ting subjective evaluation data fro m the users
is a very common approac h, we considered that the subjective evaluation d ata alone
might be m anipulated and unreliable for comparing S IT performances in many cases.
Because it is h ard to e valuate the measurable performance b y collecting su bjective data
instead of objective data, deter mining the optimum S IT setup (i.e., the most appropriate
combination of setup variables) c ould not be achieved at first attempts.
On the other hand, although collec tin g objective data is the most appropriate
method for performance evaluation, the existing objective evaluation methods in
literature are far from b eing a benchmark. Th e y a re mostl y desi gned to ev aluate just a
specific S IT , which make s them ineligible to be a b enchmar k wher e the other S I T s could
be evaluated via standardized test.
1.3 Resear ch Questions
In this section, we pre sen t the research questions addressed through this thesis .

Softwar e Switches: Single-gestur e based Hands-fr ee Interaction T echniques
T o deal with the related p roblems stated in Section 1.2, we first fo cus on the following
research que stion:
Q1: How to devise an efficient appr oach enabling single-gestur e based hand s-fr ee HCI?
Then, we addr ess the following res earch questions according to the most suitable hands-
free gestures identified by us.
Q2: How to devise a better technique that e nables a person to interact with a computer
by a single puff-gestur e?
Q3: How to devise a better technique that e nables a person to interact with a computer
by a single head-ge stur e?
The GLOSTER 1.0: A Sin gle-gestur e Acc essible Hands -fr ee CA T
In line with the aim of controlling a computer via the proposed int eraction techniques,
we address the following research que stion:
Q4: How to devise a better CA T that enables a person to contr ol a computer with a
single-gestur e?
The SITbench 1.0: An Evaluation T ool
In th e evaluation sta ge o f the proposed interaction techniques, the requirement of an
evaluation tool leads us to focus on the following researc h question:
Q5: How to devise a b etter tool that enables objective evaluation of swit ch-based
interaction techniques?

1.4 Thesis Contributions
Our contributions throu gh this thesis and rel ated publi cations are given in this section
under three domains.
Softwar e Switches: Single-gestur e based Hands-fr ee Interaction T echniques
In accordance with our ef forts to find a n effic ient solution to single-gesture
based hands -free compu ter access problem, we start with a li terature r eview of the
curre nt hands-free interaction techniques —in terms of the gestures used — to identify
the current problems . T o overcome the related exist ing problems stated in Section 1.2 ,
we propose our nov el software switch approach following our ef forts to answer the
research qu estion Q1. T o sum up, our softw are swit ch approach has two pri nciples: an
interaction technique ba sed on softwa re switch approach (1) should not require any
dedicated devi ces, and (2) shoul d be config urable to be compatible with the other
switch-accessible interfaces. W e also identif y the most suitable hands-free gestures as
puf f and head gestures in accordance with the principles of our so ftware switch
approac h. These gestures are then emplo y ed to interact with a computer via the
proposed software switches.
Then, the research questions Q2 and Q3 lead us to devise four nov el software
switches —called the PuffMic , the PuffCam, the HeadCam, and th e HeadG y ro — by
following the principles of our software switch approach. T wo usabilit y studies —
conducted with 82 participants in total— de monstrate that the proposed software
switches can allow interacting with a computer b y a single- ge sture in a w ay th at they
rece ive the user's gesture as an input signal to translate them into emulated switch
presses. W hile the Puf fMic and th e P uf fCam are based on pu f f- gesture , the HeadCam
and the He adGyro d epend on head-gestures (e.g., a head tilt ). Th e Puf fMic and the
Puf fCam software switches were presented in the following article:
• Cagdas Esiyok , A y h an Askin, Aliy e T osun and Sahin Alba y rak, “ Sof twar e
Switches: Novel Hands-f r ee Interaction T echniques for Quadriplegics Bas ed on

Respiration–machine Interaction ”, Universal Access in the Infor mation So ciety ,
2019.
The following article is partly based on the HeadCam and the HeadG yro software
switch es :
• Cagdas Esiyok , A y han Askin, Ali y e T osun and S ahin Albayrak, “ N ovel Hands-
Fr ee Interaction T echniques based on the Softwar e Switch Appr oach for
Computer Access with Head Movements ”, Univer sal Access in the Information
Society [under review ].
The GLOSTER 1.0: A S ingle-gestur e Acc essible Hands -fr ee CA T
As a result of focusing the researc h question Q4, we present a new singl e-switch
accessible C A T called the GLOSTER 1.0 with a novel mous e point ing te chnique called
the Coordinate-ba sed Pointing (CoorP ). I t allows the users to easil y c ontrol a computer
with a single-gesture b y employing the proposed software switches. B y m eans of the
proposed CA T , emulated switch presses b y so ftware s witches ca n b e converted into
meaningful commands to operate a computer . As an all- in -one solution, the GLOSTER
1.0 provides all thr ee functions —pointing, clicking, and t y ping— perf ormed b y a
mouse and a k e y board. Foll owing a usabilit y stud y with 20 participants, it is reve aled
that the CoorP performed better than the Cros sHair which is the most preferred
technique by the e xist ing CA T s.
The SITbench 1.0: An Evaluation T ool
W ithin the development process of the proposed software switches in
accordance with the r esearch questions Q2 and Q3, durin g evaluation step , we
encounter the related p roblems identified in Section 1.2 . Therefore, the research
question Q5 leads us to propose a novel benchm ark tool for performance evaluation
called the SITbench 1.0. It is demonstrated b y a usabilit y stud y with 8 participants that
the S I Tbench 1.0 provid es a quicker and mor e accurate switch evaluation process b y

collecting the objective data automatically . P art of the SITbe nch 1.0 was published in
the following article:
• Cagdas Esiyok and Sahin Albayrak, “ SITbenc h 1.0: A Novel Switch -Based
Interaction T echnique Benchmark ”, Journal of Healthcare Engineering, 2019 .
1.5 Thesis Structur e
Th e rest of this thesis is structured a s follows:
In Chapter 2, we propose the software switch approach en abling single -gesture based
hands-free HCI . In a dditi on, we also identif y the most suitable gestures to intera ct with
a computer in line with t he software switch approach. In C hapter s 3 and 4, we present
four novel interac tion techniques —called the Puf fMic, the Puf fCam, the HeadC am,
and the HeadGyro soft ware switches— b y following the principles of our software
switch approach. While C hapter 3 introduces the Puf fMic and the Puf fCam software
switches, which are ba sed on a single puf f- gesture, Chapter 4 pr esents single head-
ge sture based softw are s witches namel y th e H eadCam and the HeadG yro. In Chapter 5,
we introduce a new single-switch accessible CA T —namely the GL OSTER 1.0— with
a novel mouse pointing technique which allows to easil y control a computer by
employing the propos ed software switches. In Ch apter 6, a nov el benchmark tool for
performa nce ev aluation —called the SITbench 1.0— is proposed to provide a quicker
and more accurate swi tch evaluation process b y collecting the objective dat a
automatically . Lastly , Chapter 7 summ arizes and discusses the main contributions and
provides an outlook to future re search directions.

2 Th e Softwa r e S w itch Appr oach

2
The Softwar e Switc h Approach
In line with our ef forts to find an ef ficient solution to single-gesture based hands-free
computer access proble m, we b egin with a literature review of th e cu rrent hands -free
interaction techniques in terms of the gestures used. Following the literature review , w e
identify two major problems of current single- gesture based hands-free interaction
techniques: (1) the majorit y of current sin gle-gesture bas ed hands -free HCI t echniques
depend on dedica ted devices beyond standard c omputer peripherals; (2) current sing le-
ge sture based h ands-free solut ions in literature are onl y compatible with spe cific switch-
accessible interfaces. T o overcome these p roblems, we propose our novel software
switch approach. B riefly , the software switch approach h as two pri nciples: an
interaction technique ba sed on softwa re switch approach (1) should not require any
dedicated de vices, and (2) should be configurable to be compatible with switch-
accessible interfaces. I n this chapter , we also i dentify the most suitable hands-free
ge stures as pu f f and head ge stures in accordance wi th the principles our software s witch
approac h. These gestures are then emplo y ed to interact with a computer via the
proposed software switches in Chapters 3 and 4.

2.1 Related W orks
Many hands- free solutions which are mostl y based on dedicated devices (e. g., switches,
sensors) or just standard computer pe ripherals (e.g., a camera) ha ve been developed up
to now for computer access. People with mot or-impa irments are able to interact wit h
these devices b y using their unimpaired body gestures like head movements or ey e -
blinks. I n this section, to look from a broad er perspective, we review ha nds-free
solutions —instead of foc using just the sin gle-gesture bas ed hands-free solu tions— that
provide alternative means for computer access in terms of th e bod y gestures used. W e
separa t ed them into two main g roups according to the c ondition whether any dedicated
hardware is re quired exc ept for standard computer periphe rals.
• Hands-Fr e e HCI T echniq ues with Dedicated Devices: Th ese s y stems require
additional de dicated sensors, switches or devices beyond stand ard computer
peripherals to control a co mputer such as B rain-Computer Interaction (BC I) based
systems via Electroencephalo graphy (E EG) sensors [ 10, 1 1]; eye movement
operated s y stems based on Electrooculography (EOG) sensors [ 12, 13] or
dedicated cameras [ 14, 15]; head movement operated s y stems based on traditional
switches [16], inertial sensors [17, 18] or special cameras [ 19, 20]; sip -and-puf f
operated s y stems [21-23]; facial mimics opera ted s ystems based on
Electromyogra ph y (EMG) senso rs [24, 25]; tong ue operated s y stems [26, 27] ;
tooth-click operated s ystems [ 28]; mouth/lip jo y s tick operated s y st ems [29-31] ;
and chin operated systems [32-34] .
• Hands-Fr e e HCI T echniq ues with S tandard Computer Peripherals: These
systems c an be divided i nto two main groups as camera and microphone based
systems. In camera based s y st ems, a ca mera is set to focus on e y e-gaze [35] or
head movements [36-38] to transform them into mouse cursor movements on a
computer scr een. Facial gestures (such as e y e bli nk or ey ebrow -raising) c an also
be captured b y a camera to trigger a mouse click or a ke y stroke [39, 40] . On the
other hand, the majorit y of microphone based ha nds -free interaction techniques
for computer a ccess depend on speech ge sture s as commands where speech
recognition techniques are applied to control a mouse pointer [41, 42] or a

key board [43] . Similarly , non-sp eech gestures such as humming, can be
recognized b y sp eech recognition al gorithms to control a mouse point er [44, 45]
or a ke yboard emulator [ 46] . Additionally , puff as a respiration-gesture can also
be detected b y a mic rophone and emplo yed for clicking task in computer a ccess
[47] .
2.2 Existing Pr o blem s
Following a literature review in Section 2.1, h aving a broader point of view helped us
to rec ognize the existing problems. W e identified two major problems of the current
single-gesture ba sed han ds-free interaction techniques:
• Requir eme nt of Dedicated Devices : The majorit y o f current single - ge st ure
based hands-free HCI techniques depend on dedicated devices bey ond standard
computer peripherals. Besides, The W orld Report on Disabilit y reve als that 80%
of people with disabilit ies accommodate in poor and middle income countries
[4] , which means that most of these people might have serious dif ficulties to
af ford dedicated devices [7 -9]. Althoug h the aim of the universal access is
enabling equal opportun ity and access to a service or product regardless of
people's physical disabi lities by reducing b arriers, high -cost of dedi cated
devices crea tes a new barrier financiall y .
• Compatibility wit h Switch-accessible Interfaces : Current single -gesture
based hands-f ree solutions in literature are onl y compatibl e with specific switch-
accessible interfaces. T o make it clear , first the mechanism of a scanning-ba sed
interface and stand ardization problem should be un derstood. In principle, un like
direct selection (such as ty ping on a k e y board), the scanning -based interfac e
highlig hts it ems one- by -one on the computer screen, and the user activates the
switch whe n the desired item is highlighted. Between switch-accessibl e
interface and the switch, there is a switch adapter which is a d edicated device to
transform switch a ctivation signals into meaningful key board pr esses or m ouse
clicks. Following a sw itch activation, switch adapter emulates a spe cific
key board character o r a mouse click event (depending on the manufacturer of

switch interface) and send it to the computer in order to communicate with
switch-accessible int erface. But the main problem here is that there has not been
any commonl y a greed standard for the communication between switches and
switch-accessible interfaces; while some switch -a ccessible interfaces expect to
rece ive a specific ke yboard charac ter like space, the others expect to receive a
mouse c lick. This standardiz ation problem is pa rtial l y solved by a switch driver
software permitting the users to assign a specific character or mouse click —
following a switch activa tion— which is expe cted by the targ et switch -
accessible int erface. However , these switch driver software are onl y c ompatible
with a limited number of switch ada pters of specific brands, which makes them
partial solutions for the standardization problem. I n other words, each switch
adapter requires its specific switch driver software. T o the best of our
knowledge, there is not an y complete solution for this standardization problem
in literature.
2.3 Principles of the Software Switch Appr o ach
T o overcome the problems identified in Section 2.2, we propose our software s witch
approac h. T wo principles of the software swi tch approach are presented below
according to related problems :
• Requir eme nt of Dedicated Devices : As the first principle of our softwa re s witch
approac h, any int eraction techniques based on o ur approach should not r equire
any dedicated device be yond standard computer peripher als like a microphone or
a c amera . At thi s point , a s the onl y r easonable ex ception, we d ecided to ex clude
smartphones from d edicated devices list; because t he tot al number o f smartphones
—3.2 billion in 2019 [48]— g ot ahe ad of the total number of computers in recent
y ears worldwide [ 49], which makes them eas y to access for peopl e in even low-
income countries. Besides, smartphones are able to provide several services t o the
users unlike dedica ted de vices which are prod uced with a specific aim.
• Compatibility with Switch-accessible Interfaces: As the second principle of our
software switch approach, any interaction techniques based on our approach

should be configurable t o generate an y expected keyboard characters or mouse
clicks, which makes the m compatible with most switch -accessible interfaces. I n
this way , the y can provide a bett er solution to the standardization problem than
the current solution where a switch driver and a traditional switch are required to
purchase . In other words, the y are able to both detect a single -gesture like a
traditional switch and allow the users to ass ign the expected ke y board characters
or mouse clicks —which will be sent to the switch-accessible interfac e following
a detec t ed single-gesture— like a switch driver .
T o sum up, our softwar e switch approach has t wo principles: an interaction
technique based on soft ware switch appro ach (1) should not re quire an y dedicated
devices, and (2) should be configurable to be compatible with switch -accessible
interfaces.
2.4 Gestur e Selection
Although the software s witch approach is flexible enough to be employed with a ny
physica l gesture, withi n this thesis, we focus on the HC I techniques based on hands -
free g estures to provide comprehensive computer a ccess solutions for the o nes who are
only able to move f rom t he neck up. T o sele ct the proper t ypes o f hands -free gestures
which will be employ ed b y the proposed software switches, we be gin with the
evaluation of gesture t y pes of current hands -free int erac tion techniques given in S ection
2.1.
HCI techniques which require d edicated devices are left out of the focus in
accordance with the first principle of our software switch approach. Then, we exclud e
ey e- ga ze and facial ge stures (e.g., eye bli nk gesture), since the y mi ght be highl y a f fected
by Midas T ou ch problem [50] . For ex ample, if the e y e blink gesture is used to in terac t
with a compute r , it is h ard to distinguish whether the user blinked consciousl y a s a
trigger si gnal or it was just a regular e ye blink per formed un consciously . A lthough the
Midas T ouch problem could be resolved b y emplo y in g multi -modal inp uts such a s
performing an e y e blink with a n eye brow raisin g sim ultaneously as a tri gger si gnal,

employing a multi-modal input is not an option in the scope of this thesis since we aim
to solve the sin gle-gestu re based computer access problem. As another gesture t y pes
which don't require a dedicated device, the speech and non -speech gestures are also
omitted, bec ause the speech rec ognition is mostly preferred for more complica ted tasks
than just a single-g esture recognition. Unlike these gestures, we revea led that the
respiration (i.e., sip and puf f) and head moveme nt (e.g., a h ead tilt ) ge stures are not
af f ected b y Midas T ouc h proble m, and the y can be easil y recognized via simple
algorithms. Th erefore, we considered them as the most suitable hands-free gesture t ype s
among e xist ing solutions to interac t with a computer by following the principles of our
software switch approach.
2.5 Summ ary
In this ch apter , first w e be gin with a literature review of the current hands -free
interaction techniques in terms of the gestures u sed. Then, two m ajor pr oblems of
curre nt techniques are id entified.
T o overcome the identified problems, we propose the software switch approach
and detect the most suit able gestures. Afterwards, we also propose four nov el software
switches which are single-gesture ba sed human-comput er interaction (H C I ) techniques
—namely the Pu f fCam, the Puf fMic, the H eadCam, the HeadG y ro— b y following the
principles of our software switch approach. Unlik e the existing solutions, the proposed
software switches don 't require dedicated devices a nd c ompatible with most switch-
accessible interface s.

3 The PuffMic and the PuffCa m: Novel Interaction
T echniques Ba sed on a Single R espi ration-Gesture

3
The PuffMic and the PuffCam:
Novel Interaction T echniques Based on a Single Puff -Gestur e
In this chapter , we focus on how to devise a better method that ena bles a person to
interact with a computer by a sin gle puff-gesture. For the ones with a ver y limit ed motor
activity but a complete respiration activit y , interacting with a computer b y a single puff-
ge sture is considered a challenging task. T o ove rcome this challeng e , we p ropo se two
novel int erac tion techniques as software switches —the PuffCa m and the Puf fMic— for
single-gesture b ased CA T s by following th e principles of our so ftware switch approach.
Both software switches are respiration oper ated where a stron g puf f, detected non -
invasively b y a mi crophone or a modi fied camera, is considere d as a presse d switch. A
usability stud y —conducted with 46 participants with/out disabilities — reve als that the
accuracy , precision, reca l l a nd false positive rate of our interaction techniques ar e quite
impressive, and the PuffCa m perfor ms better than the Puf fMic for all metrics.
According to qu estionnaire findin gs, comfort ass essment of inter action techniques b y
participants is ra ted quite satisfactory .

3.1 Intr oduction
As state d in Section 1.1, there h ave been millions of motor-impaired individuals
worldwide who have difficulties to opera te a computer via conventional wa ys [ 4, 5] .
For the ones with a v ery limited motor activity but a complete respiration activit y ,
interacting with a computer b y a single resp iration -gesture is conside red as a
challenging task for computer a ccess.
The majority of current solutions addressing this challenge is based on invasive
sip-and-puf f devices [21, 23, 51, 52] . They are capable of recognizing the users ' sip and
puf f g estures to interact with a computer . Re cognition of these gestures is performed
invasively in a way that the users inhale (sip gesture) or exhale (puf f gesture) throug h a
tube/straw which is plac ed in his/her mout h. The n, air pres sure orig inated from the
user's sip or puff is recogniz ed b y th e devic e to serve as a double input switc h for switch -
accessible interfaces like CA T s. I n literature, as a dif ferent approach, we were able to
find just one solution called the BlowClick [47 ] which doesn't require a dedicated
device. It provides a no n -invasive solution in a wa y th at the us ers' puff gesture is
recognized b y a compute r after the user puffs on a standard mi crophone connected to
the computer .
Although current respira tion -based HCI solut ions are us eful to enhance peo ple's
quality of life in many aspects, we identif y two main problems of the existing solutions
below to be handled. T o overcome these p roblems, we propos e two novel HC I
techniques bas ed on th e user’ s puf f gesture —called the Puf fMic and t he Puf fCam
software switches— b y following the principles o f our software switch approach. T wo
problems of current inte raction techniques id entified and how we address them b y
applying our software switch approa ch are explained below :
• Requirement of Dedicated Devices: Whil e the BlowClick [ 47] doesn't require
any d edicated d evice; s ip -and-puff devices are comm ercial dedicated hardware
bey ond standard computer peripherals, which m akes them hard to af ford for
people with low - in come. Furthermore, the us ers might have h y giene prob lems
with sip-and-puf f devi ces, since a tube b etween mouth and environment mi ght

cause hygiene risks [53, 54] . I n accordance wi th the first principle of our
software switch approach, both proposed softw are switches depend on j ust
standard computer periphera ls (a microphone or a camera) like the BlowClick.
• Compatibilit y with Switc h-accessible Interfaces: S ip-and-puff devices are o nl y
compatible with a specific group of switch -accessible interfaces, while the
BlowClick is not compatible with an y switch-accessible int erface. In o ther
words, the Blowclick is unable to serve as a switch for an y other switch -
accessible interface. At t his point, the Puf fMic di f fers from the BlowClick as
being configurable to be c ompatible with switch-a ccessible inte rfaces, although
their input method is sim ilar . In line with the seco nd principle of our software
switch approac h; the Puf fMic and the PuffCam software switches are
config urable to generate an y expected ke yboard characters or mouse clicks,
which makes them compatible with most switch-accessible interfaces.
T o sum up, the proposed solutions are based on the a ssumption that individuals'
strong puf fs can be distinguished b y emplo y in g a standard microphone (the Puf fMic)
or a modified webcam (t he Puf fCam) via the prop osed software switch es running on a
standard computer . In this way , a puf f -gesture c aptured b y a standard microphone or a
webca m is considered as a pressed switch to m ake a selection in a switch -accessible
software like a CA T .
W e conduct ed a us ability stud y with 46 individuals (23 motor-impaired, 23 able-
bodied) to collect object ive and subj ective d ata b y employing the SITbench 1.0 [ 55]
(describe d in Chapter 6) and a five -point L ik ert scale questionnaire, respectivel y . As a
result of usability stud y , the Puf fCam method show ed better performance than the
Puf fMic method in all co nditions. Accuracy , precision, recall and false pos itive rate of
both interaction techniques were found quite impressi ve. Moreover , comfort assessment
results of a five-point L ik ert sca le que stionnaire were sa tisfactory . Th e idea to cont rol a
computer via breathing without purchasing any dedicated device was co nsidered ver y
promising by all partic ipants.

Studies within this chapter a re expected to sti mulate new studies b y motiv ating
researchers to place a greater emphasis on non -invasive respiration methods since the
non-invasive voluntar y r espiration is an underrated activity for computer access in
comparison with invasive s y stems. The proposed interaction techniques can be us ed
instead of traditional invasive sip-and-puff devi ces in man y cases. Considering that 80%
of the people with disabil ities live in low and middle income countries [4], the proposed
software switch es can help to meet the cost-free sw itch requirements of individuals with
motor -impairments worldwide in an open acc ess manner without any additional device.
There is not an y other alternative solut ion as a switch curre ntl y for th e ones who have
only respiration activit y and cannot affor d an y d edicated device. Th e y can b e integ rated
with any assistive sy st ems where voluntary respiration is utilized. For example, the
users can op erate a whe elchair . Similarl y , an assistive living s ystem mi ght be developed
where the users could interac t with smart hom e devices. On the other ha nd, the
application areas of the pr oposed interaction techniques can be quite flexible and should
not be considere d just for assistive technology ar ea. For example, it is possible to utilize
them in the entertainment are a like computer game industry be yond the a ssistive
technology area.
This chapter proceeds with int roducing the proposed software switches in
Section 3.2. Then, we evaluate both interaction te chniques by presenting objective and
su bjective evaluation results of our usabilit y study in Section 3.3. Finally , we conclude
and discuss our study in Section 3.4.
3.2 Softwar e S w itches
In thi s section, first we introduce th e user interface o f the P uf fMic and the PuffCam.
Then, we present both software switches.
3.2.1 The U ser Interf ace
Fig ure 3.1 illustrates the interface of both propo sed software switches during a puff
activity . It includes a pu ff meter where the green bars on it show th e puf f level of the

user , and the mi ddle red bar represents the threshold level detected durin g calibration
step (calibration steps of ea ch software s witches are ex plained in Sections 3.2.2 and
3.2.3.). The puff level de pends on how stron g th e user pu f fs . In principle, i f the puff
level exceeds the threshold value, a switch press is emulated. The user can moni tor the
puf fs on run-time via puf f meter , which helps the use r to estimate how strong s/he needs
to exhale.

Fig ure 3.1: The user inte rface of the Puf fMic and the PuffCa m.
The interface can b e con figured according to the selected softwar e switch . Fo r
both the PuffMic and the PuffCa m, an y ke y board character or mouse cl ick can be
assigne d in configuration for the targ et CA T . For ex ample, if the CA T waits to rece ive
enter character , the user assigns enter character to be sent via our software switch es .
Following a strong puff detected, th e proposed so ftware switch es s end enter character
to the tar get CA T . This w ay , the proposed software switches let the users to control any
compatible switch-ac cessibl e software . If the Puf fCam is selected to be con fig ured, the
colour of the tracked obj ect is also assigned b y the user . Both software switches were
developed under .NET 4 .5 framework, and they are compati ble with W indows -based
operating systems.
3.2.2 The P uffMic
For the first interaction technique proposed, a stan dard microphone is placed under the
user's nose or in front of the mout h in a way that puf f-gestures can be c aptured easil y
(Figure 3.2). Followin g positioning, the threshold level needs to be calibra ted since the
streng th o f puf f activity varies by person.

Fig ure 3.2: Camera, lapt op and microphone positions during the experiments.
The user first adjusts the scanning time of the software switch l ess than the
scanning time of the targ et switch -accessible software. For example, if the scanning
time of the tar ge t CA T is 2 seconds (i.e., if the tar get CA T is able to receive the trigger
signal in ever y 2 seconds), the user should assign the scannin g t ime less than two
seconds such as 0.9 second, and thus the software switch becomes capable of sending
the expected trigger signal to the CA T in every 0. 9 second. Then, the user assigns the
smoothing time to be a ppli ed on the audio sig nal during si gna l smoothing process.
Following the a ssignment of scanning and smoo thing time s, the software sw itch
records an audio wav e for assigned scanning tim e period as illustrated in Fig ure 3.3.
W ithin this time period, the user performs a strong puf f on a mic rophone in order to set
a threshold value. In this example, scanning time is c onsidered as 0.9 second, while the
smoothing time is set as 0.1 second.

Fig ure 3.3: Raw audio signal.

Then, the software switch ge ts the absolute value of audio signal ( Figure 3.4).

Fig ure 3.4: Absolute valu e of audio signal.
Afterwards, the software switch ex tracts the smoothed audio signal b y
calculating the a verage value for each smoothing time period (i.e., for each 0.1 second)
as can be seen in Figure 3.5. The peak puff level (between the 0.4 and 0.5 seconds) is
assigne d as th e threshol d value. Then, for the vi sual feedback, the p eak puff lev el is
repre sented by the middle red bar in th e puff m et er ( Figure 3.1) as a thresho ld indicator .

Fig ure 3.5: Smoot hed audio signal.
Following a suc cessful calibration, the PuffMic can serve as a puf f switch for a
switch-accessible software. T o do this, the user’s respiration activit y is tracked in real-
time . L ike in calibration s tep; (1) first the absolute value of the audio si gna l is extracted.
(2) Then, si gnal smoothing is applied to calculate the peak puff level depe nding on the
scanning time and the smoot hing time . (3) The peak puf f lev el is r epresented as green
bars in the puff m eter according to the str ength o f the puff. (4 ) Lastly , if the peak puff
level exceeds the threshold va lue following a strong puff, a switch pre ss is emulated by
the Puf fMic to send a trigger sig nal to the target CA T .
3.2.3 The P uffCam
The Puf fCam so ftware s witch basicall y tr anslates the motion of an object —as a result
of a strong puf f— captured b y a webcam into a trigger si gnal for a switch -accessible

software. A modified w ebcam with a post-it (or a piece of paper) that is placed just to
the opposite of the came ra lens b y me ans of an adhesive tape is emplo yed as can b e
seen in Figure 3.6. The o nly requisite of this modi fication is that the object t o be tr acked
(i.e., a black rec t angle fig ure drawn on post - it in our example) should be kept in the
visual field of the camera in an y case, even if th e user puffs very stron gly on the post-
it.

Fig ure 3.6: A standard webcam modified with a post - it.
Following a proper positioni ng as it is illustrated in F igure 3.2, the P uf fCam
should be ca librated. The user first adjusts the sc anning time of the software switch less
than the scanning time of the targ et switch- accessible software li ke adjusted i n PuffMic.
Then the real-time video motion tracking algorithm is employe d as follows :
• For scannin g time p eriod assigned, video fr ames are taken b y a w ebcam wi th a
frame rate of 15 frames per second and a frame siz e of 320x240 pix els (Figure
3.7 (a));
• Euclidean colour filte ring is applied for each vid eo frames acc ordin g to colo u r
of the tar get object assigned during configuration (Figure 3.7 (b));
• V ideo frames are converted to graysca le following Euclidean colour filtering
(Figure 3.7 (c));
• Object dete ction is performed on video frames via connected-component
labeling method. (Figure 3.7 (d));

• The position of th e det ected object on post -it is tracked throu gh run time (Figure
3.7 (e)).
• W ithin this scanning time period, the user puf fs strongl y on the post - it to change
the position of the detected object.
• The dif ference in pix els between th e position prior to puff and th e posit ion
during stron g puf f is considered as threshold value, and this threshold valu e is
repre sented b y the middle red bar o f the pu f f meter in Figure 3.1 as a visual
indicator .

Fig ure 3.7 : Main steps of motion tracking algorithm. (a) take video frames via camera;
(b) appl y Eu clidean colo ur filter for each frame; ( c) convert video frames to grayscale;
(d) detect the object; (e ) track the position of the detected object.
For filterin g and object detection, image processing librar y called AForg e.NET
was emplo y ed. Following a successful calibration (i.e., assignment of the threshold
value), the Puf fCam beco mes capable of recognizing the strong puffs to serve a s a puf f
switch. T o do this, every motion of the detected object is calcula ted for each scanning
time period and then re presented as green b ar s in the puf f meter depen ding on the
motion level. If the us er puf fs stron gly enough, threshold value is exceeded, and the
Puf fCam sends a trigger signal to the tar get CA T .

3.3 Evaluation
W e conducted experiments in T urke y at Medical F acult y of Izmir Katip Cel ebi
University in order to evaluate our int eraction te chniques b y colle cting objective and
subjective data. Th e study had been approved on 13.09.2017 b y the Ethical Comm ittee
of I z mir Katip Celebi University with a reference n umber of 179. The informed con sent
form was signed by a ll participants prior to the experiments.
In thi s section, firstl y we introduce the participants who are separated int o two
main groups as able- bodied and motor -impaired individuals. Therea fter , we prese nt the
appara tus employed t hroughout the experiments and the procedure to be applied on two
novel interaction techniques b y me ans of our evaluation software th e S ITbenc h 1.0 .
La stl y , th e experimental findings are shared.
3.3.1 P articipants
Overall, 46 participants including 24 females and 22 males took part in this stud y . Of
all participants, 23 (8 females, 15 mal es) had motor disabilities (hereafte r: disabil ity
group (DG)) whose age s ranged between 17 and 78. T wo of them can be seen in Fig ure
3.8. The participants without disabilities (here after: control group (CG)) included 23
people (16 females, 7 males) whose ages ran ge d between 17 and 71. Age statis tics of
all participants are sum m arized in T able 3.1. All participants with motor disabil ities,
who volunteere d to t est our interaction te chniques, had dif ficulties controlling their
hands. They were all receiving hospital t reatment for s everal mot or disabiliti es while
the experiments were conducted. Just five of the participants had previous experience
with switch interfac es. The majorit y of the pa rticipants of CG were volunteers
responding to the call made b y the Ph y sical Medicine and Rehabilitation Department as
well as the r elatives and friends of DG. The main characteristics of all parti cipants are
listed in T able 3.2.

Fig ure 3.8: T wo participants previous to the experiments.
T able 3.1 : Age statistics of the participants according to the groups.

Groups

Gender

Mean Age

Number of
Participants

Mix

47.5 (sd = 17.4)

Mix

Female

42.2 (sd = 16.2)

Mix

Male

53.3 (sd = 17.1)

Mix

52.9 (sd = 19.3)

Female

45.8 (sd = 24.2)

Male

56.6 (sd = 15.7)

Mix

42.3 (sd = 13.7)

Female

40.5 (sd = 10.9)

Male

46.3 (sd = 19.1)

T able 3.2 : Main characteristics of the participants.

The User

Age

Gender

Disability

DG1

Male

Hemipleg ia

DG2

Female

Hemipleg ia

DG3

Male

Hemipleg ia

DG4

Male

T etraplegia

DG5

Male

Hemipleg ia

DG6

Male

Hemipleg ia

DG7

Female

Hemipleg ia

DG8

Female

Neuromyelitis Optica

DG9

Female

Hemipleg ia

DG10

Female

Guillain-Barre Syndrome

DG1 1

Female

Hemipleg ia

DG12

Male

Hemipleg ia

DG13

Female

Hemipleg ia

DG14

Male

Hemipleg ia

DG15

Male

Hemipleg ia

DG16

Male

T etraplegia

DG17

Male

Hemipleg ia

DG18

Male

Hemipleg ia

DG19

Male

Hemipleg ia

DG20

Male

Hemipleg ia

DG21

Male

Hemipleg ia

DG22

Male

Hemipleg ia

DG23

Female

Hemipleg ia

CG24

Male

None

CG25

Female

None

CG26

Female

None

CG27

Male

None

CG28

Female

None

CG29

Female

None

CG30

Female

None

CG31

Female

None

CG32

Female

None

CG33

Female

None

CG34

Female

None

CG35

Male

None

CG36

Male

None

CG37

Male

None

CG38

Female

None

CG39

Female

None

CG40

Female

None

CG41

Female

None

CG42

Female

None

CG43

Female

None

CG44

Female

None

CG45

Male

None

CG46

Male

None

All partic ipants were required to mee t the following criteria in order to evaluate
our interac tion technique s. All participants are supposed to be able:
• to breath voluntarily ;
• to find a tar ge t on a grid;
• to follow a moving tar get;
• to maintain ga ze on a stable targ et;
• to stay focused on tests during e xperiments.
The Mini-Mental State Ex amination (MMSE), which is a 30-point questionnaire
to assess cognitive impairments in clinical studies, was applied to all participants in
order to validate whether they fulfil the co gnitive ability r elated requirements stated
above prior to the experiments.
3.3.2 A pparatus
The test apparatus consists of a laptop Lenovo G505S (CPU: AMD A8-4500 M 1.9 GHz;
RAM: 6 GB DDR3; S cree n: LCD 15.6; OS: W indows 10 64 bits; Resolution: 1600 ×

900), a Digicomm Headset 9088 (I mpedance: 32 ohms at 1kHz; Sensitivity : 105 dB /
mW ; Frequency ran ge: 8 ~ 22,000Hz) and an A4T ech V iewCam pro PK - 635M (Max
Digital V ideo Resolution: 640 x 480; I mage Sensor T y pe: 0.35 MP CMOS; Horizontal
Field of V iew: 54).
3.3.3 P r ocedur e
First of all, we ensure d that the participants and devices (i.e., microphone, webcam,
laptop) were positioned properly as illustrated in Figure 3.2 befo re starting the
experiments, since a good positioning allows:
• the stability to enhance mot or functions;
• easing abnormal reflexes due to voluntary movements ;
• being compatible with long-term sessions;
• preventing a signific ant increase in abnormal muscle tone.
A he adset-t y pe standard microphone is located as seen in Figure 3.2 in a wa y
that the s ystem ca n recognize the stron g puffs. In our e xperiment, we placed the
microphone approximatel y 5 cm ahead o f the us er's mouth. On the other hand, the
webca m should be positioned in a way that th e user can move the post -it easil y by
puf fing on it. The dist ance between w ebcam and the user's mouth, which depends on
the properties of attached post-it or paper (i.e., size, hardness, etc.), was adjusted as
about 40-45 cm. in our experiments.
After p roviding a good positioning, information was given to the particip ants
about the test, and we conducted some tria ls in counter balanced order until the y
understand the concept and become ready for te sts. This training proces s lasted for
approximately ten minutes for each participant.
Following positioning and training steps, we appl ied the first protot ype of the
T ie-Smile y Matching Game (TSMG) test of the S ITbe nch 1.0 described in Chapter 6 to
collect the obje ctive evaluation data. Each propos ed software switch was tested b y each
participant (n = 46) with the first thre e templates of TSMG where scan ning ti me was

1000 milliseconds. T ests were applied in counterbalanced o rder to avoid interaction
ef f ects due to learning and fatig ue. W e gave participants some time (1 to 5 minutes) to
rest during experiments so as to pre vent excessive mental or physical fatigue.
The quantitative subjective data of the propos ed interaction techniques was
collected b y appl y ing a comfort assessment questionnaire containing four statements
with a five-point L ikert s cale ( Fig ure 3.9) after co mpleting tests. The ratin g ranged from
1 to 5 (1 = stro ngl y disagree, 2 = disagree, 3 = nei ther agree nor disagree, 4 = agree, 5
= strongl y a gree) for the c omfort assessment questi onnaire during experiments. I n order
to collect the qualitative subjective data, we received responses of open-ended questions
and feedback b y p articipants about interaction techniques in addition to our
observations.

Fig ure 3.9 : The comfort assessment questionnaire with mean values (x -axis represents
the rating range).
3.3.4 Object ive Dat a based Results
Mean values of the proposed software switches t hrough evaluation metric s (acc urac y ,
precision, r ecall and fals e positive rate) for all participants are shown in Figure 3.10.
For all evaluation metrics, the P uf fCam showed better performance. Result s presented
in Figure 3.10 are listed below:
• In terms of accurac y mea n values, the Pu f fCam sh owed better performance with
a mean of 0.926 in comparison to the Puf fMic (m = 0.868). According t o the

Student’ s t-test for both techniques, p was found less than 0.05, which means
that dif ference amon g means in terms of accuracy is s tatistically significant;
• For precision mean values, the rankin g was the sa me as accurac y : th e PuffCam
(m = 0.885) and the P uf fMic (m = 0.859 ). As a result of the t -test fo r both
techniques, p was deter mined greater than 0.05, which means that the dif ference
among means in terms of prec ision is not statisti cally significant;
• Regarding recall mean values, the PuffCa m c ame first (m = 0.897) and the
Puf fMic followed it ( m = 0.806). In consequence o f the t -test for both
techniques, p was calculated as less than 0.05, which means that the difference
among means in terms of rec all is statisticall y significant;
• The P uf fMic (m = 0.089) was followed by the PuffCam (m = 0.065) based on
false positive ra te. Ac cording to the t-test for both techniques, p wa s detec ted as
greater than 0.05, which means that the difference among means in terms of
false positive rate is not statistically sig nificant.
Mean values of each int erac tion techniques depe nding on participant groups
(Mix, DG, CG) are pres ented in Figure 3. 11 . In terms of mean values o f accuracy ,
precision and recall evaluatio n metrics, CG member s performed bette r than DG
members for both so ftware switches except for th e Puf fMic in r ecall. Regarding false
positive rate score, D G ha d higher s cores than CG for both int erac tion techniques, which
means that DG members made false s elections mo re frequentl y when compared to CG
members. W e applied the Student’ s t-tests for both interaction techniques through all
evaluation metrics to s ee whether there is a significant dif f erence between the
performa nce of DG members and CG m embers. The dif ference among means between
DG and CG was found si gnificant in three conditions where: (1) metric t y pe = accurac y ,
interaction t y pe = the PuffCa m; (2) metric t y pe = precision, interaction t y pe = th e
Puf fMic; (3) metric ty pe = false positive rate, interaction type = the Puf fMic.

Fig ure 3.10 : Mean values of the propose d software switches for all participants through
evaluation metrics (acc ur acy , p recision, reca ll, false positive rate) (**** p < 0.0001).
3.3.5 Subject ive Dat a based R esults
The quantitative subjec tive da ta was collected fr om the filled -in questionnaire
containing fou r statements ranked in a five -point L ikert scale ( Figure 3.9), while the
qualitative subj ective data was collec ted b y the participants' responses t o the open-
ended questions, their feedback about interaction techniques and researche rs'
observations. The ratin g ranged from 1 'str ongly disa gr ee' to 5 ' str ongly agr ee' in the
questionnaire for comfo rt assessment during experiments. W ith reg ard to the four
statements, the mean value and standard deviation were calculated for all 46
participants. As can be observed in Fi gure 3.9, the statement 'I didn' t have any
r espiration fatigue' (m = 4.32) was rated low er than the statement 'S eating and
positioning wer e comfortable' (m = 4.76). Puf fing on a modified camera (m = 4.45) was
found easier than puf fing on a microphone (m = 4.32). Overa ll, all statements were
scored quite satisfactor y by th e pa rticipants. Prior to the ex periments, all participants
were quite confident that the y can handle it, and t hey seemed excited to experie nce it.

Fig ure 3. 11 : Mea n v alues of interaction tech niques through evalu ation metrics
(acc urac y , pr ecision, recall, false positive rate) according to the participant groups (Mix ,
DG, CG) (* p < 0.05; ** p < 0.01; *** p < 0.001).

After the experiments were completed, all pa rticipants a greed that both
interaction techniques were eas y to use and stated that the y would b e loo king fo rward
to operating a computer access application based o n these interaction techniques. The y
also all declared that the y li ked the techniques and that controllin g a comput er via
breathing onl y without purchasing an y dedicated device sounded v er y pro mising. Man y
participants stated that the respiration-based methods proposed would be an ef ficient
alternative for the peopl e with severe motor -impairments. On the other hand, some
participants unde rlined that respir ation-based s y stems might be a p roblem for eld erl y
people who might have r espiration dif ficulties. W e even had to exclude tw o volunt eers
because of their upper r espiratory infection. Mo st participants were pleased with the
scanning time in the eval uation software S ITbe nc h 1.0 (1 000 milli seconds), thoug h two
of them suggested to adjust it slower to track hig hli ghted objects easier .
3.4 Conclusion and Discussion
Most of respiration-based solutions for people with motor disabilit ies depend on
expensive dedicated hard ware called sip-and-puff switches be yond standard computer
peripherals. Moreover , ea ch sip-and-puf f switch emulates dif ferent ke yboard characters
or mouse clicks based on the decision of its manufacturer , which leads to a
standardization problem. A commonly a greed standard is lackin g on t he switch-
accessible software side as well ; while some s witch-accessible software expect to
rece ive enter character , t he othe rs expect to re ceive dif ferent characters. On the other
hand, non-invasive interaction techniques ba sed on r espiration activit y are un de r -
explored in comparison to invasive techniques like sip -and-puf f devices. In other words,
respiration operated int eraction techniques are mostl y limited to invasive si p-and-puf f
devices. Althou gh the y might meet some requirements of their users, sip -and-puff
devices are expensive s ystems and have tubes inside the use rs' mouth that have to be
changed regularl y due to h ygiene concerns. As a diffe rent approach, to the best of our
knowledge, there is just one solution called the B lowClick [47] which doesn 't require a
dedicated device. But it is unable to serve as a switch for a ny switch -accessible
interface.

In o rder to overcome th e above-mentioned proble ms, we propose two novel non-
invasive interaction tech niques as software switc hes (the Puf fCam and the P uf fMic)
based on the puf f gesture alone whe re a strong puf f detected by a microphone or a
modified camera is considered as a pressed switch. Furthermore, both software switches
provide the same functions of a tra ditional hardware switch and its software driver (i.e.,
an object on the screen can be selected, as it could be done b y means of a hardware
switch; and expected characters can be assigned, as the y could be assi gned by a softw are
driver) without the need for a dedicated hardware and its software driver . Alt hough the
the Puf fMic is similar to the BlowClick in terms of input method, the Puf fMic diffe rs
from the BlowClick as being configurable to be compatible with switch -accessible
interfaces.
The usabilit y stud y co nducted with 46 partici pants demonstrated that the
accuracy , precision, recall and false positive rate of the proposed interaction techniques
were quite impressive. For all evaluation metrics (accuracy , precision, recall and false
positive rate), the PuffMic exhibi ted the worst p erformance. The r easons behind this
based on our observations is considered as follows:
• Holding position of the headse t microphone stab le during the experiments is
very import ant since the threshold value is assigned in the calibration step and
highly depends on the initial position of the microphone. Any possible change
in the position (ang le or distance between the microphone and the use r's mouth
or nose) during the ex periments mi ght lead to a minor or a major change in the
air pr essure (ori ginated from the ex halation pressur e) applied on the
microphone, which is considered as the main reason of hi gh false positi ve rate.
Although high resolution Uni -Directiona l -Cardioid Microphones can e ase this
problem, such microphones mi ght be ver y ex pensive. Emplo y ing them is
definitely out of the scope of this stud y because they are not standard computer
peripherals;
In terms of m ean v alues of accurac y , precision and recall, CG mem bers
performe d better than DG members for both software switches except for the PuffMic

in reca ll. DG members had a higher sco re of fals e positive rate than CG members for
both int erac tion techniques, which means that DG members are more inclined to make
false selection.
W e have also collected the subjective data with a questionnaire containing four
statements ranked in a five-point Likert scale ( Figure 3.9 ), responses o f open-ended
questions, feedback from participants and observat ions of researchers. According to the
questionnaire for comfort assessment, all st atements were r ated quite satisfac tor y b y
participants. All particip ants agreed that the y en joye d using the proposed software
switches. Moreover , the y stated that the idea of controlling a compute r v ia breathing
without purchasing an y dedicated device sounded very promising. On the other hand,
each softw are switch is af fected b y th e external factors dif ferently . Whil e the PuffMic
is not affe cted b y the li ght level (i.e., it can even work in the darkness), the P uf fCam is
highly robust to the external voices. Additionally , the P uf fCam also is not af fected b y
the user’s speech.
This stud y p rovides a preliminary evidence o f our software switch approach.
Because non-invasive respiration-based s ystems are und erestimated cons idering the
limited number of previo us studies in compa rison with invasiv e respiration-based sip-
and-puf f devices, this study can also encourage the rese archers to plac e a greater
emphasis on non-invasive r espiration-based systems. Both software switches can be
replaced with inv asive puf f devices in man y cases. Because our syst em is cost-free
(except for a laptop with a standard w ebcam or a microphone), the budget alloca ted for
supplying expensive alternative a ssistive devices can be used for other requirements of
people with disabilities, improving cost-ef ficienc y for government soc ial servic es. As a
future stud y , we aim to compare the proposed software switch es with tradit ional sip -
and-puf f switches. It is also worth noting that, the application area of int erac tion
techniques we proposed should not be considered just for computer access sy st ems.
These interaction techniques c an be emplo y ed a nd integrated with a n y o ther s y stem
where a pu f f-gesture of t he users can be used. W e compared the performance of CG
with DG to see whe ther there is an y significant dif fe rence between the grou ps, because
we aim that our approach will be utilized b y able -bodied people as well. For ex ample,

bey ond assistive technology requir ements, computer-ga me industr y mi ght emplo y our
methods in a way that players use their respiration activity as a new input wa y while
playing.

4 The HeadCa m an d the HeadGyro: Novel Interaction
T echniques Ba sed on a Single Head -Gestur e

4
The HeadCam and the HeadGyr o :
Novel Interaction T echniques Based on a Single Head -Gestur e
W ithin this cha pter , we focus on how to devise a better method that ena ble s a person to
interact with a computer b y a sin gle head -gest ure. Head-oper ated CA T s are use ful
solutions for the ones with complete he ad control; but when it comes to the pe ople with
only reduced head contro l, computer access beco mes a very challenging t ask since the
users depend on a sin gle head-gesture like a head nod or a head ti lt to interac t with a
computer . Th erefore, two novel interaction te chniques called the HeadCam and th e
HeadGyro ar e proposed within thi s chapter . I n a nutsh ell, both interaction techniques
are b ased on ou r softwar e switch approach and can serve like traditional switches b y
recognizing head movements via a standard camera or a gyrosc ope sensor of a
smartphone to translate them into emulated switch presses. A usabilit y stu dy with 36
participants (18 motor -impaired, 18 able-bodied) is also conducted to colle ct both
objective and subjective evaluation data in this study . Whil e the HeadGy r o softwa re
switch exhibits slightly higher performance than the H eadCam for each objective
evaluation metrics, the HeadCam is rated better in subjective evaluation. All
participants agree that th e proposed interaction tec hniques are promising s olutions for
computer acce ss task.

4.1 Intr oduction
Head-opera ted CA T s are considered as one of the most ef ficient ex amples of assistive
solutions enabling hands-free computer a ccess. The y are generall y based on human -
computer interaction (H CI) te chniques where a mouse cursor is operated b y the user 's
complete head control abilit y . How ever , for the people who h ave onl y reduced head
control abilit y (i.e., the o nes who cannot operate the mouse cursor b y moving head o r
any other activity), computer access is a very ch allenging task since the users have to
interact with a computer by a single head- gesture. It is obvious that any new ef ficient
interaction techniques based on a sing le hea d - gesture will pla y an important role to
develop better CA T s for t he people with onl y reduced head control.
In accordance with ou r effor ts to find a solut ion for people with only reduced
head control to interact with a compute r by a single head -gesture, we reviewed the
curre nt h ead-operated s olutions in Section 4.2. W e noticed that the majorit y of
interaction techniques requires a c omplete head control ability . I n other wo rds, there are
limited solutions whic h are capable of suppo rting sing le head- gesture access for p eople
with reduced h ead movements. Considering these li mited solutions which are single
head-ge sture based HC I techniques that provide alternative means for computer access
task (given in Section 4.2.2), we identif y two main problems. T o overcome these
problems, we propose tw o novel int eraction techniques namely the HeadCam and the
HeadGyro b y following the principles of our s oftware s witch approach. The main
problems of the existing s ystems and how we add ress them b y appl ying our software
switch approach are explained below:
• Requirement of Dedicated Devices : T r aditional button switches are dedi cated
devices be yond standard computer peripherals, while software-based techniques
[56, 57] do not require any dedicated devic es. As low -cost solut ions, the
HeadCam and the HeadG y ro software switches are based on a standard ca mera
and a gyrosc ope sensor of a smartphone, respectively .

• Compatibilit y with Switch-accessible I nterfaces: Althou gh current software -
based techniques [ 56, 57] support single he ad-gesture and do not require an y
dedicated device, the y a re unable to serve as a switch for an y oth er switch -
accessible interfaces. In other words, the y are not compatible with an y switch -
accessible interfa ce. They c an only emulate mouse clicks within their interfac e.
Both interaction techniques proposed can be configurable to generate an y
expected keyboard characters or mouse clicks, which makes them compatible
with most switch-accessible interfaces.
In a nutshell, both interaction techniques c an serve like traditional switches by
recognizing the h ead movements via a stand ard camera or a gyroscope sensor of a
smartphone to translate them int o virtual switch presses. Furthermore, th e y don't require
a dedicated d evice, and the y are compatible with most of switch -accessible interfaces.
As low- cost alternatives, they ca n be re placed with e xpensive traditional head switches
for computer access. Cur rently , the y are the onl y options as switches for the ones with
a limited head control alone (i.e., the ones who have to use a switch -based system fo r
computer access) who cannot afford an y dedicated device. The y are also capable of
recognizing an y mot ion o f the other bod y parts su ch as the user 's shoulder o r leg, which
makes them quit e flexible switches. B y this way , dif ferent physical gestures can be
tar geted easily , when the user becomes tired. Besides, neither of the proposed software
switches require a ph y sical strength to be activated unlike ph y sical switch es; espec iall y
the HeadGyro c an even detect a minimal head movement to transfor m it into an
emulated switch press. Since the HeadG y ro software switch isn't affe cted by ex ternal
factors like lig ht or wind, it could be also employed for outdoor activities (e .g.,
operating a whee lchair).
A usabilit y stud y with 36 participants (18 motor-impaired, 18 able -bodied) was
conducted in order to e valuate the proposed so ftware switches. The S ITbench 1.0
benchmark [55] was employe d for objective evaluation. Besides, we also applied a
System Usability Scale (S US) [58] questionnaire for subjective evaluation. While the
HeadGyro showed slightly hi ghe r p erformance th an the HeadCam for each objective
evaluation metrics, the Hea dCam was rated bet ter than the HeadGyro in subjective

evaluation. All participants agreed that the idea of controlling a computer via a single
head-ge sture without requiring an y dedicated device sounded ver y promising .
This chapter pro ceeds wi th a literature r eview to summ arize the current head-
operated interaction te chniques for computer access in Section 4.2. S ubsequently , we
introduce our software s witches called the HeadGyro and the HeadCam proposed in
Section 4.3. Then, we evaluate both interaction te chniques by presen ting objective and
subjective evaluation results of our usabilit y study in S ection 4.4. Finally , we conclude
and discuss our study in Section 4.5.
4.2 Related W orks
In this sec tion, to look from a broader pe rspective, we review the current head-operated
HCI solutions that provide alternative me ans for computer access. W e preferred to
separa t e them into two main groups according to the condition whether the y have a
single head-ge sture access support.
4.2.1 Head- operated Interaction T echniqu es without a Single Head-
gestur e Access Support
Interaction techniques in this group require a co mplete head control abilit y fo r hands -
free computer access. In principle, the y translate the users' head movement s into mouse
cursor movements in severa l wa y s.
One of the most popul ar techniques is wearing inertial sensors suc h as a
gyroscope or an accelerometer on head (via a helmet or a cap) to control a mouse pointer
[17, 18, 59-67]. These inertial sensor-ba sed systems a re mostly combined with a
dif ferent sensor/switch t o perform a mouse click task (e.g., in a way that he ad
movements are d etected by inertial sensors to con trol mouse pointer , and mouse clicks
are performed b y a puf f switch). Another se nsor-based solution called Headmaster Plus
[68], which was evaluated in LoPresti et al.'s work [69] , consists of ult rasonic sensors.
Briefly , the user w ears a headset including th ree ult rasonic senso rs that wait an

ultrasonic signal fr om a stationary transmitter on the user's computer . In this way ,
ultrasonic sensors determine the orientation of the user 's head to conve rt them into
mouse pointer coordina tes.
Using a head pointer —a head-worn stick in principle— is another solution
which permits the users to control, press or touch any targ et [70] by head, although this
method is rarely prefe rred nowada y s. Similarly , head -operated jo y sticks are alternative
tools which enable the users to point a mouse cursor on the screen [30] .
On the other h and, a spe cific pa rt of the us er's face ( e.g., the tip of the nos e) or
the user's whole head can be tracked b y a stand ard camera in order to transform head
movements into mouse cursor movements on a co mputer screen[36-38, 71- 88]. Mouse
click tasks such as left o r right click s ar e generall y p erformed with dwelling method
(i.e., the user holds the mouse cursor ste ad y for a given amount of time to p erform click
tasks) or with multi-modal approaches by means of other gestures like e ye -blinks or
tooth-clicks.
In addition to abovementioned approa ches, he ad movements can also be
followed b y special camera-based s ystems to control a mouse cursor . In such s ystems,
the user w ears sm all reflective dots on his/her h ead/face o r an infrared LED (lig ht -
emitting diode) which is placed on a helmet or a pair of glasses. Thes e reflective dots
are illuminated by an infrared or near infrared li ght source, and then a stand ard camera
[89-91] or an infrared camera [19] tracks the position of tar g et sign als (comi ng from
reflective dots or an infrared LED) for mouse cursor pointing. L ikewi se, RGB- D
cameras as ne w visi on s ensor technolo gies are also able to do 3D mapping of h ead
position to control mouse pointer [92].
4.2.2 Head- operated I nteract ion T echni ques with a Single Head -
gestur e Access Support
For the ones with onl y reduced h ead control, the re hav e been limited solutions which
are able to support single head- gesture access. U sing a traditional button switch via a
scanning interface is a co mmon technique where a head switch is mounted close to the

user's head in a w a y that the user can hit it by tilt ing head (or b y any activit y moving
head) [ 16, 93]. I n addition to tra ditional hardwa re switches, there a re just a few
software-based technique s [56, 57] where a single hea d-gesture is employe d to perform
mouse clicks. I n such techniques, the users are enabled to navigate the mouse cursor to
the desired location b y vi sion-based head tracking methods, and then mouse clicks are
emulated according to the use rs' head- ge stures as an alternative to dwelling method.
4.3 Softwar e S w itches
This section beg ins with the introduction of the co mmon user int erface of both software
switch es proposed. A fterward, The HeadCam and the HeadG y ro software switches are
explained respective l y .
4.3.1 The U ser Interf ace
W e designed a user interface, as shown in Fi gure 4.1 (a), which is employ ed for both
software switches. Ga mi fication techniques w ere applied to make softwa re switches
more engaging and fun. An initial state of the interface —where the use r has a stable
head position— can be see n in Fig ure 4.1 (a). T he interface includes three d y namic
ga m e elements: (1) the earth, (2) the left and (3) the right red border line s. All three
elements can be controlled b y the user's head m o vements called pitch, y aw and roll as
illustrated in Figure 4.1 (b).

Fig ure 4.1: (a) The initial state of the interface of the HeadCam and the HeadGyro
software switches. (b) Rotational movements of a head.
The s ensitivity to control the game elements can be set according to the user's
head control capability . A s the sensitivity l evel gets higher , the user c an mov e the ga me
elements with a slower and minor head movement. The missi on of the game is to save
the earth from the gravity o f a bla ck hole b y moving these three game el ements until
the earth intersects with the red border lines. S witch press and switch release are
emulated accordin g to th is int ersection situation. In oth er wo rds, as soon as the e arth
intersects with the red border lines, a switch press is emulated until the end of
intersection; while a switch relea se is emulated once the intersection between the earth
and the red border lines is terminated. The interse ction (i.e., switch press) is followed
by a visu al or an a uditor y sensory feedba ck provid ed to the user .
At the beginning , the interface c an be configured t o emulate an y ke yboard press
or mouse action. Following an int ersection detected, the int erface sends th e expected
character s or mouse clicks to the targ et switch -accessible software like a CA T . This way ,
the proposed software switches let the users to control any compatible switch-accessible
software. If th e HeadCam is selected to be configured, the colo ur of the tracked object
(i.e., the us er’s head) is also assigned. Both software switches are com patible with
W indows-based opera ting s ystems and were developed under .NET 4.5 fr amework.

In order to calibrate the earth' s position, we simulated a gravit y function that
pulls the earth toward the black hole constantl y . The gravit y function becomes
inef fective durin g th e intersection (i.e., switch p ress). Once the int ersection is over (i.e.,
switch relea se), the gravity function is reactivated. I n this way , if the user keeps his/her
head stable for a while when there is not any intersection, the earth will be pulled to its
initial position eventually by gravity (i.e., to the c entre).
As it is illustrated in Figure 4.2, each of six dif ferent hea d - gesture s (i.e.,
rotational movements of the head) results in six dif fe rent int ersec tion states. While pitch
(Figure 4.2 (a)) and yaw (Fi gure 4.2 (b)) mov ements control the earth's p osition, roll
movements (Fi gure 4.2 (c)) operate the positi on of the right and the left red b order lines.
4.3.2 The Hea dCam
The HeadCam is base d on a re al-time video motio n trac king algorithm whic h is similar
with the Puf fCam descri bed in 3.2.3. In principle, the user's h ead is tracked by a built -
in camera or a standard webcam to translate the roll movements of the user' s head (as
can be seen in Fig ure 4.2 (c)) into emulated switch press es . The algorithm of the
HeadCam is listed step- by -step b elow:

Fig ure 4.2: Six dif fe rent intersection states of t he interf ace according t o rotational
movements of a head.

• V ideo frames are t aken b y a c amera with a frame rate of 15 frames p er s econd
and a frame size of 320x240 pixels (Figure 4.3 (a));
• Euclidean colour filtering is applied for e ach video frames according to assigned
colour of the tar get object to be tra cked during co nfiguration (Figure 4.3 (b));
• Following Euclidean colour filtering, video frames are converted to graysca le
(Figure 4.3 (c));
• All objects are detec ted in video frames (Fi gure 4.3 (d));
• The greatest obj ect is chosen if there is more than one obje ct detected ( Figure
4.3 (e));
• The greatest object is tracked in real-time (Figure 4.3 (f));
• Every motion of the greatest object is transformed into the motion of the right
or left red border line s as it is depicted in Figure 4.2 (c);
• Once the earth intersec ts with the red border lines, a switch press is emulated.

easily recognized b y the HeadCam, which makes our software switch ca pable of
supporting double switch inputs for switch-accessible interfaces.
4.3.3 The Hea dGyr o
The HeadG y ro interaction technique, basically , employs 3 -axis g y roscope data of a
smartphone —where the smartphone is placed on the user's head— to c onvert the
rotational movements of the user's head into emulated switch presses. The smartphone
can be placed on the user 's head in sev eral wa y s. For example, the us er can wear a cap
which is attached to the smartphone or a modifie d belt holding the smartphone as c an
be seen in Figure 4.4.

Fig ure 4.4: Th e placement of the smartphone on the user's head for the HeadG y ro
software switch.
The g y roscope is an important inertial sensor and mainly used to measure
ang ul ar velocit y of the s ensor in inertial space. In other wo rds, it measures the rate o f
change of the sensor’s orientation. T oda y , inertial sensors li ke gyroscope are based on
microelectromecha nical s ystem (MEMS) tec hnolog y . The y are e mplo ye d in modern
smartphones frequently since the y are small, cheap, light, and offe r low power
consumption. I n spite of all these adv antages, bec ause o f the el ectromagnetic
interference and the influence of semiconductor thermal noise, MEMS ba sed sensors
can ex hibit high frequenc y noise like jitter , which affec ts the acc urac y of the detected

ang ul ar velocit y . Th ere h ave been several filterin g methods used so far to reduce the
noise of gy roscope data su ch a s high/low-pass filter , forward linear filter , w avelet filter
and Kalman filter . W e preferred the Kalman filter to avoid jitter , considering the real -
time requirements and it s feasibilit y . W e also developed a mobile application depending
on Andr oid oper ating s ystem —which communicates with the computer i n a wireless
local ar ea n etwork (W LAN)— to conve y the stream g y roscope data to the computer .
The algorithm behind the HeadGy ro is basicall y describe d step - by -step below:
• Real-time gy roscope stream data o f the smartphone's 3 -axis gy r oscope sensor is
drawn by our Android application;
• The Android application conve y s this stream g y ro scope d ata wirelessl y to the
computer;
• A simple Kalman filter is applied to this stream data as shown in Figure 4.5;
• Every motion of the user' s hea d is translated into the motion of the g ame
elements as illustrated in Fig ure 4.2;
• Once the earth intersec ts with the red border lines, a switch press is emulated.
4.4 Evaluation
A usability stud y was conducted to colle ct objective and subj ective d ata. I n this section,
firstly w e int roduce the characteristics of p articipants. Then, we pr esent the apparatus
used within this study . Afterward, we brie fly explain the tests and the procedure applied
during the evaluation of the Hea dCam and the HeadGyro. At last, we conclude the
section with our experimental findings.
4.4.1 P articipants
Following th e approv al of the Ethics Committee of the Izmir Katip Celebi University
(T urk e y ) on 10.10.2018 (with a dec ision number: 332), the usability study was
conducted at Medica l Faculty of the Universit y . All participant s gave their informed
consent before the y parti cipated in the stud y . A total of 36 participants, including 18
females and 18 males, took part in the evaluation of the proposed s y stems. While,

disabilit y g roup (DG) comprises 18 participants (6 female s, 12 males) with motor -
disabilities whose ages ranged between 18 and 68, control g roup (CG) without
disabilities includes 18 people (12 females, 6 males) whose a ges ranged bet ween 18 and
59.

Fig ure 4.5: T wo different stream data graphs based on the x -axis of the gyroscope sensor
of two dif ferent pa rticipants when participants nod their head. Blue an d re d lines
repre sent unfiltered and Kalman filtere d g y roscope data, respectivel y .
In T able 4.1, age statisti cs of all participants are summ arized acc ordin g to
groups. M ain characteristics of the participants are li sted in T able 4.2. All participants
in DG we re under me dical treatment for severa l mot or disabilities, while the
experiments were condu cted. On the othe r hand, participants of CG were gene rally
accompan i es of DG or staf f workin g at the Phy s ical Medicine and Rehabilitation
Department. All particip ants met the following inclusion criteria: the y a re supposed to
(1) find a targe t on the scree n; (2) follow a moving tar ge t; (3) maintain g aze on a stable

tar get; (4) sta y focused on tests du ring experiments. As an inclusion criteria, all
voluntary participants in DG had several dif ficulties in controlling their hands and thus
couldn't operate a comput er with conventional wa ys (i.e., with a mous e and a keyboard).
Besides, th ere were fiv e participants in DG who have reduced he ad control. Prior to
experiments, we applied the MMSE —30- point questionnaire for cognitive
assessment— to va lidate whether the p articipants can meet the cognitive abilit y to
complete our tests.
T able 4.1: Age statistics of the participants according to the groups.

Groups

Gender

Mean Age

Number of
Participants

Mix

43.2 (sd = 15.3)

Mix

Female

39.3 (sd = 14.2)

Mix

Male

47.1 (sd = 15.7)

Mix

46.1 (sd = 17.3)

Female

38.0 (sd = 19.5)

Male

50.1 (sd = 15.4)

Mix

40.3 (sd = 12.8)

Female

39.9 (sd = 1 1.8)

Male

41.3 (sd = 15.7)

T able 4.2 : Main characteristics of the participants.

The User

Age

Gender

Disability

DG1

Male

Hemipleg ia

DG2

Female

Hemipleg ia

DG3

Male

Hemipleg ia

DG4

Male

T etraplegia

DG5

Male

Hemipleg ia

DG6

Male

Hemipleg ia

DG7

Female

Hemipleg ia

DG8

Female

Hemipleg ia

DG9

Female

Hemipleg ia

DG10

Female

Hemipleg ia

DG1 1

Male

Hemipleg ia

DG12

Male

Hemipleg ia

DG13

Male

Hemipleg ia

DG14

Male

Hemipleg ia

DG15

Female

Hemipleg ia

DG16

Male

Hemipleg ia

DG17

Male

Hemipleg ia

DG18

Male

Hemipleg ia

CG19

Female

None

CG20

Female

None

CG21

Female

None

CG22

Male

None

CG23

Male

None

CG24

Male

None

CG25

Female

None

CG26

Female

None

CG27

Male

None

CG28

Female

None

CG29

Female

None

CG30

Female

None

CG31

Female

None

CG32

Female

None

CG33

Female

None

CG34

Female

None

CG35

Male

None

CG36

Male

None

4.4.2 A pparatus
A laptop (Lenovo G505S; CPU: AMD A8-4500M 1 .9 GHz; R AM: 6 GB DDR3; screen:
LCD 15.6; OS: W indows 10 64 bits; resolution: 1600 x 900), an inte gra ted camera of
laptop (max digital video resolution: 1280 x 720; Image Sensor T y pe: 0.3 MP CMOS),
and a smartphone with g yrosc ope sensor (Son y Xperia XZ1 Compact; CPU: Qualco mm
Snapdragon 835; RAM: 4GB; OS: Android Oreo 8.0) were emplo y ed for experiments.
4.4.3 T es ts
W e used the S ITbe nch 1.0 benchmark —presented in Chapter 6— which helps
researchers to evaluate switch -based s ystems objectivel y . B y means o f thi s tool,
objective evaluation dat a can be collected and sa ved automaticall y with standardized
tests. T o this end, we emplo y ed the T ie -Smile y Matching Game (TSMG) and Hungr y
Frog Game (HFG) tests of the S I Tbench 1.0.
For subjective evaluati on, we applied the S ystem Usability S cal e (SUS)
questionnaire [5 8] which consists of ten statements with a five-point L ikert scale as can
be seen in T able 4.3. Scale values r ange from 1 to 5 (1 = strongl y disa gree, 2 = disagree,
3 = neither agree nor dis agree , 4 = agree, 5 = strongl y a gree). A SUS score ( ranging
from 0 to 100) is calculated based on scale value of the statements in a wa y that: (1)
score contributions of each statement are summed where the score contribution is the
scale value minus 1 for statements 1, 3, 5, 7, 9; the score contribution is 5 minus the
scale value for statemen ts 2, 4, 6, 8, 10; (2) the sum of the score contributions is
multiplied b y 2.5 to c alculate the SUS score.

T able 4.3: Statem ents of the S US questionnaire with av erage scale values of all
participants. ∆ s ymbol is replaced with the He adGyro and th e HeadCam, respe ctivel y ,
during a ssessments.

Statements

The Hea dG yro
A verage Scale

The Hea dC am
A verage Scale

1. I think that I would like to use ∆ frequentl y

4.1 1

4.07

2. I found ∆ unne cessarily complex

1.16

1.14

3. I thou ght ∆ was ea s y to use

4.41

4.30

4. I thi nk that I would need the support of a
technical person to be able to use ∆

2.22

1.52

5. I found the various functions in ∆ were we ll
integrated

4.30

6. I thought there was to o much inconsistency
in ∆

1.19

1.22

7. I would imag ine that most people would
learn to use ∆ very quickly

4.33

8. I found ∆ ve r y cumbersome to use

1.41

1.1 1

9. I felt ver y confident using ∆

3.97

4.27

10. I ne eded to learn a lot of things be fore I
could get g oin g with ∆

1.13

4.4.4 P r ocedur e
At the beginning, the participants were informed about the test verbally . Then, we
ensured that the participants and devices were posit ioned properly . Followi ng a proper
positioning, we let them to practice the tests (in a counterbalanced order) under our
guidance, until they feel confident to star t the tests. Afterwards, we applied two tests of
the S I Tbench 1.0 to collect objective data: (1) TSMG: each software switch was tested
by each participant (n = 36) with the first three templates of TSMG where sc anning time

was 1000 milliseconds. ( 2) HFG: each software switch was tested b y each participant
(n = 36) with the f irst three scenarios of HFG.
W e applied the tests in the counterbalanced order to avoid learning and repetition
ef f ects. I n order to prevent the mental or physical fatigue, we allowed the participants
to get rest up to 5 minutes betwee n the ex periments. At the end of the SITbenc h 1.0
experiments, we also applied the SUS questionnaire to the participants for quantitative
subjective evaluation. B esides, we collected the qualitative subjec tive data via our
observations and participants ’ responses of open- ended questions about t wo software
switches proposed within this stud y .
4.4.5 Object ive Dat a based R esults
As can be seen in Figure 4.6, acc ording to the results of TSMG experiments, the
HeadGyro demonstrated slightly better performance than the HeadCam in all
performa nce evaluation metrics (accurac y , precision, recall, and false -posit ive rate). In
terms of accurac y , mean value of the HeadG y ro (m = 0.938) was greater than the
HeadCam (m = 0.904 ), and the dif ference among mean values was found statisticall y
significa nt ( p < 0.05) a ccording to S tudent's t -test for both software s witches. F or
precision, the HeadG y ro (m = 0.921) ex hibited better performance than the HeadCam
(m = 0.872), and there was a sig nificant dif f erence a mong means ( p < 0.05). Regarding
reca ll, the H eadGyro (m = 0.910) was followed by the H eadCam (m = 0.863) with a
significa nt differe nce among means ( p < 0.05) of both interac tion te chniques. For false -
positive rate, the HeadCam (m = 0.077) was ahe ad of the Head G yro (m = 0.048), and
the dif ference among means wa s si gnifica nt ( p < 0.05).

Fig ure 4.6: Mean values of interaction te chniques acquired from all participants through
evaluation metrics of TSMG including acc uracy , prec ision, recall, and f alse positive
rate (* p < 0.05).
Fig ure 4.7 presents the mea n values of eac h software switches for TSMG
depending on the participant g roups (Mix, DG , CG). CG members performed better
than DG members for b oth software switches according to the mean values through
accuracy , precision and recall evaluation metrics. In false positive rate score, DG had
higher s cores th an CG for software switch es, which means that DG members made false
selections more frequentl y when compared to C G members. The Student ’s t-tests for
both interaction techniques throu gh all evaluation metrics was applied to check whether
there is a significant difference between th e per formance of DG memb ers and CG
members. The dif ference a mong means between DG and CG wa s not significant for a ll
metrics.
Likew is e, the HeadG y ro prove d a better performance in compar ison to the
HeadCam for all evaluation metrics of HFG ( Figure 4.8) (av erage press ti me, average
release ti me, the fastest press ti me, the slowest press time, the fastest release time, and
the slowe st relea se time) . Mea n values of both in teraction techniques were pre sented in
T able 4.4 depending on HFG ex periments. Accordi ng to p-values based on the S tudent's
t-test results of all partici pants for both inte raction techniques, it is demon strated that
there is a statis tically significant difference among th e means of the HeadGyro and the
HeadCam through a ll evaluation metrics.

Fig ure 4.7 : Mean v alues of the software switches through evaluation metric s (ac curac y ,
precision, recall, false positive rate) according to the participant groups (Mix , DG, CG) .

Fig ure 4.8: Mean v alues of two softwa re switch es for all participants throu gh evaluation
metrics of HFG ( average press time, the fastest press time, the slow est press ti me,
average release time, the fastest release time, and the slowest release time) (* p < 0.05;
** p < 0.01).
T able 4.4: M ean values o f the HeadG y ro and the HeadCam through evaluation metrics
of HFG (avera ge press t ime, the fastest p ress ti me, the slowest press time, avera ge
release time, the fastest relea se time, and the slowest release time) for all participants.

Metric T y pe

The Hea dG yro

The Hea dC am

A verage Press

0.514

0.582

The Fa st est Press

0.402

0.424

The Slowest Press

0.670

0.775

A verage Release

0.204

0.255

The Fa st est Release

0.140

0.176

The Slowest Release

0.281

0.342

4.4.6 Subject ive Dat a based R esults
Results of the SUS questionnaire as quantitative s ubjective data a re listed i n T able 4.3.
The average sc ale values a cquired from a ll particip ants are represented according to the

HeadGyro and th e Head Cam. The average SUS scores w ere c alculated as 85.0 and 87,9
for the HeadGyro and the HeadCam, respectively .
According to the SUS adjective rating scale [ 94], both SUS scores can be
considered as ex cellent. After the experiments, all participants a greed that both
proposed interaction techniques are promisi ng s olutions for computer access tasks.
They also declared that th ey were looki ng forward to experience both software switches
to control a computer . Regarding to experiments with the SITbench 1.0, five participants
stated that the y would p erform better if the s canning time/speed of TSMG test was set
to a slower value, while four participants suggeste d to increase the siz e of smiley s. All
participants were plea sed with the visual and audit ory senso r y feedback pro vided to the
user during tests once the switch is activated or the targ et is appea red. W hil e 31 of all
participants declared that they would pr efer to use the Hea dCam for computer access, 5
of them chose the HeadGyro as their favourite s oftware switch. The y all agree d that
ga mification techniqu es made so ftware switches more engaging. N one of th e
participants experienced any fati gue during tests.
4.5 Conclusion and Discussion
Hands-free computer ac cess via head moveme nts is already a challen ging task in
comparison to conventional wa y s, but when it co mes to the people who have li mited
head control, computer access becomes a more challenging task sinc e the users are
obligated to inte ract with a computer by a sing le head- gesture like a head nod or a head
tilt. On the other hand, high-cost of dedicated dev ices —employed by the majority of
curre nt head-operated HCI solutions — creates a new barrier , although t he aim of
universal access is to brea k the barriers to enable equal opportunity and acc ess for
people with disabilities.
Alternative computer access methods c an provide seve ral useful s ervices fo r the
ones with motor disabilit ies in every part of life such as communica tion and education.
Any ne w interaction techniques enablin g computer access w ith minimal head
movements will obviously help to enha nce the quali ty of life and the self -su f ficiency of

people with re duced h ead control ability alone. Therefor e, w e proposed two novel
interaction techniques call ed the HeadG y ro and the HeadCam which dep e nd on the
gyroscope s ensor o f a smartphone and a standa rd camera, respectively . B oth int erac tion
techniques are based on our software switch appr oach that provides a co mprehensive
solution to the following problems of the current single head -gesture based interaction
techniques: (1) requirement of dedicated devi ces, (2) compatibilit y with switch -
accessible interfaces. I n acc ordance with two principles of our software switch
approac h, the H eadGyro and the HeadCam software switches (1) don't require any
dedicated devi ces; and ( 2) are configurable to be compatible with switch -accessible
interfaces. In a nutshell, both software switches c an serve like traditional switches by
recognizing head movements via a standard camera or a g y roscope sensor of a
smartphone to transfor m them into virtual switch presses.
According to the evaluation data of conduct ed usabilit y stud y with 36
participants, HeadGyro showed slightl y better performa nce than th e HeadCam in
objective eva luation, while the HeadCam was rated be tter than the HeadGy ro in
subjective evaluation. F urthermore, 31 o f all participants dec lared that they would
prefer to use the HeadCam for computer access, w hile 5 of th em selected th e HeadG y ro.
Base d on our obs ervations, the reasons behind this situation are considered as follows:
(1) The head control abi lity is the ke y factor for thi s situation. The ones who have
complete head control a bilit y (31 participants) rated the HeadCam, while the ones with
reduce d head control (5 participants) pr eferred th e HeadG yro since th e HeadGyro is
more sensitive and thus capable of r ecognizing tiny head movements. (2) T he ones with
complete head control can ea sil y activate the software switch via a standard camera. As
it was expected, we aring a smartphone on the head was found an unnecessar y solution
by the p articipants as long as th eir head control c apabilit y remains unimpaired or their
head moveme nts can be detec ted by the He adCam. However , the He adG y ro can be
advantage ous if (1) the users cannot move their head enough to be recognized by a
camera, or if (2) the external factors (e.g., low/hi gh light or an y moving ob ject behind
the user) cannot be tolerated b y c amera-based tra cking. As can be conclud ed from the
results of objective evaluation, the HeadG yro works in a mor e sensit ive wa y in
comparison to the HeadCam. On the other hand, CG members performed sli ghtly b etter

than DG members for b oth software switches through accuracy , precision and r ecall
metrics. A ccording to false positive rate score, DG members are more inclined to make
false selections.
Both software switche s can serve as the onl y low-cost options for the ones with
limited head control who cannot af ford the systems depending on high -cost dedicated
devices. Be y ond the head mot ions, the proposed software switch es can b e quite flex ible
by recognizing the other body motions to transform them into emulated switch pre sses.
This flexibility also permit s the user to change the tar geted body motion once the user
became tired. On the oth er hand, we didn’t find a ny significant differe nce between the
performa nce of D G and CG members, which means that the proposed software switches
could be emplo y ed for able -bodied people effic ientl y . The y can be emplo yed in multi-
modal systems as new input techniques beyond the assistive technology area (e.g., as a
new input fo r a computer vid eo game). As another application domain, the HeadG yro
software switch might be pre ferred du ring outdo or activities, since it is quite durable
against the external factors like low li ght, high noise, and air conditions. As a future
work, any other ph y sical gesture — which is well-controlled by the user — can be
tar geted to e valuate the e f ficiency a nd usabilit y of the proposed interaction techniques.

5 The GLOSTER 1.0: A new Singl e-sw itch Accessib le CA T
with a Novel Mouse Pointing T echn ique

5
The GLOSTER 1.0 :
A new Single-switch Accessible All - in -one CA T with a Nov el
Mouse Pointing T echnique
In this chapter , we fo cus on how to devise a better CA T that enables a person to
control a computer with a single -gesture. Therefore, we propose a new single -switch
accessible C A T called the GLOSTER 1.0 with a novel mous e point ing te chnique called
the CoorP . By means of the GL OSTER 1.0, trigger signals comin g from the switches
can be translate d into meanin gful commands to control a computer . As an all - in -one
solution, it provides all three functions —pointing, clicking, and typing—
conventionally pe rformed b y a mouse and a ke y board. A usabilit y stud y con ducted with
20 participants suggests that the CoorP provides a better solution than the CrossHair
technique —whi ch is the most popular mouse pointing technique in literature — to the
single-switch bas ed mouse pointing problem. Furthermore, the overall SUS score as
86,1 also demonstrates the high usability of the GL OSTER 1.0 in a real-li fe scenario.

5.1 Intr oduction
In conventional use, computer access is achieved by means of three basic f unctions as
pointing, clicking, and t yping ; where pointin g and clicking are performed by a mouse,
and t y ping is p erformed by a ke yboard. In case the users cannot access a computer
through thes e conv entional input methods because of motor-disabilities, the CA T s are
employe d as useful solutions which allow the users to perform these thr ee functions
indirectly . Sev eral types of CA T s, whi ch var y according to the CA T s' abilit ies based on
these three functions, ha ve been developed so far . For ex ample, while all - in -one C A T s
are c apable of providin g all three functions [ 3, 95-99], some CA T s can only provide a
specific function such as t y ping [100-107].
In CA T s, t y ping and clicking ar e generall y carried out using switches via a
scanning method, while mous e pointing is mostly performed in a way th at the user's
head or e ye movements are translated into the mouse cursor mov ements. But, if the user
has an onl y sing l e-gesture unim paired, computer acce ss becomes a h arder task since all
three func tions have to be performed with a single- gesture/single-switch. Espec ially ,
single-switch based mouse pointing is a very challenging task in CA T s.
W e review th e existing CA T s which allow single -switch based mouse pointing
in Section 5.2. W e notice that they all depend on scanning techniques for mouse cursor
pointing where the whole screen is scanned i n several directions (i.e., vertical,
horizontal or rotational) until the desired location is re ached. Besides, th ere is just one
non-commerc ial CA T allowing sin gle-switch based mouse pointing, which means that
the majority of existing solutions are not accessible for the ones who have financial
dif ficulties to af ford such CA T s.
W ithin this chapter , we present a new cost-free single- switch accessible all- in -
one CA T c alled the G LOSTER 1.0. As a dif fe rent approach, we p ropose a novel single-
switch based mouse point ing technique called the C oorP . I t aims to improve the most
popular mous e pointing technique called the C rossHair (des cribed in Sect ion 5. 2) b y
navigating the mouse cu rsor to the targe t point m ore accuratel y at the sam e scan -line
sensitivit y . A usabilit y stud y was conducted with 20 participants to e valuate the

proposed CA T with th e CoorP . W e also develope a new test tool ca lled the
PointingChalleng e for this usabilit y stud y . Th e results of the us ability stu dy revealed
that the CoorP pe rformed more accurate mouse pointing than the CrossHair . On the
other hand, for the ones who have di f ficulties to speak, the G LOSTER 1.0 also includes
a speech-generator module as an AAC tool which converts the text ty ped by the user to
speech.
This chapter proceeds with a section that review s the existing solutions.
Afterwards, we introduce the proposed CA T with the novel C oorP mouse pointing
technique in S ection 5.3. Then, we evalu ate the G LOSTER 1.0 in Section 5. 4 according
to the conducted usability stud y . Lastly , we conclude and discuss our study in Section
5.5.
5.2 Related W orks
In this section, w e summarize the ex isting CA T s t hat allow single-switch b ased mouse
pointing in T able 5.1 acc ording to the following three properties:
Is All- in -one: (Y es / No) The all- in -one CA T s allow the users to perform all three
functions —pointing, clicking, and t yping— b y u sing a single -switch, while the other
CA T s do not support single-switch based typing. In such c ases, the us er h as to employ
an on-screen key board fo r typing.
Is Commer cial: (Y es / No) The ACA T [95] is the only cost-f ree C A T amon g the ex isting
ones described in T able 5 .1.
Scanning T echnique: There have been six diffe rent scanning techniques which are
employe d b y the current single-switch based mouse pointing techniques . In principle,
they help to navigate the mouse cursor position to the desired point on the screen:
The CrossHair: Th is is the most preferr ed mou se pointing technique b y th e
ex isting CA T s. I t is generally performed in 4 steps:

(1) A horiz ontal scan-line scans up/down the screen continuously .
(2) The user presses the swit ch to stop the scanning when the horizontal scan-
line intersects with the tar get point.
(3) Then, a vertical scan-line starts to sc an the screen from the left/right side to
the opposite side.
(4) The next switch press sto ps the vertical scan- line when it intersects with th e
tar get point.
The Inverse CrossH air: Although this scanning technique is sim ilar with the
CrossHair method described above, horizontal and vertical scanning depend on
the press time of the switch.
(1) A horizontal sc an-line scans up/down the screen as long as the user presses
the switch.
(2) The user r elease s the switch to stop the s canning when the horizontal scan-
line intersects with the tar get point.
(3) The user holds the switc h pressed in ord er for a v ertical s can-line scans the
scree n from the left/right side to t he opposite side.
(4) The next switch release stops the vertical scan-line when it intersects with
the tar get point.
The R adarMouse : This technique scans the screen li ke a radar as described in
steps below:
(1) A line arrow rotat es clockwise/counter -clockwise around the centre of the
scree n.
(2) The user pre sses the switch to stop the rotational scanning when the line
arrow inter sects with the tar get point.
(3) Then, the mouse cursor moves in the direction of the arrow .
(4) A final switch press stops t he mouse cursor at the targ et point.
The Divide 'nConquer: Th is technique aims to approach the targ et point b y
dividing the scr een several times. For example, initiall y the entire screen is

divided int o two sub-parts. These two sub -parts ar e highli ghted one- by -one unti l
the user presses the switch to select the sub-part covering the tar get point. T hese
divisions are repeated until the last divided scree n is small enough.
The SelectDirectionFirst: The user presses the switch to se lect a direction
of fered in a scanning me nu according to the t ar get poin t. These directions are
ge nerall y r epresented by arr ows. Following the f irst switch pre ss, the mouse
cursor starts to move thr ough the sele cted direction until the user presses the
switch to stop it . If the mous e cursor is not at the target point, these st eps are
repea ted b y the user .
The I nverse SelectDirectionFirst: I n a similar way with the SelectDirectionFirst
technique, the user first presses the switch to selec t a direction of fered in a
scanning menu according to the tar get point. Th en the user keeps the switch
pressed to move th e mous e cursor throu gh the selected direction until the user
releases the switch to st op the mouse cursor . These steps are repeated by the
user , if the mouse cursor is not at the tar ge t point.
T able 5.1: Summary of the ex isting CA T s allowing single-switch bas ed mouse pointing.

Name

Is All- in -one

Is Commercial

Scanning T echnique

ACA T [95]

y es

CrossHair

Grid 3 [96]

y es

RadarMouse

SwitchXS [3]

y es

RadarMouse,
SelectDirec tionFirst

EZ Keys [97]

y es

CrossHair ,
RadarMouse

W I NSCAN 3.0 CS
[98]

y es

Inverse
SelectDirec tionFirst

QUALI WOR LD
Basic [99]

y es

CrossHair ,
SelectDirec tionFirst

CrossScanner [108]

y es

CrossHair ,
Inverse CrossHair

ScanBuddy [2]

y es

Divide'nConquer

GUS [109]

y es

CrossHair ,
RadarMouse

5.3 Design of the GLOSTER 1.0
In this section, first, the user interface of the proposed CA T is introduced. Then, we
explain how pointing, clicking and typing func tio ns a re achieved to control a computer
by a sin gle-switch.
5.3.1 The U ser Interf ace
As can be se en from the Figure 5.1, the GLOSTER 1.0 includes 4 modules to perform
the pointing, clickin g and typing fun ctions. The int erface locks itself to the right side of
scree n once the computer is started. At the beginnin g, in the config uration step, the
expected ke y can be assigned a ccording to the trigg er signal which is sent by the
preferre d s witch. Fo r mouse cursor point ing fun ction, in the configuration st ep, the user
config ures the scan-line s ensitivity which is the time required in milliseconds to sca n 1
pixel. For ex ample, if th e scan -line sensitivit y is set to 10 milliseconds/pixel, then the
vertical/horizontal scan-line moves one pixel for each 10 milliseconds. I n o ther words,
the sc an-line scans 100 pixel for each second. Th e scanning time is also a ssigned in the
config uration step which defines how fast the icons, the letters and the numbers are
highlig hted in the modules.

Fig ure 5.1 : The use r interfa ce of the G LOSTER 1 .0 including 4 modules. (a) The main
menu module to select the pointing, clickin g, an d t y ping functions. (b) T he numbers
module to t y pe the numbe rs and define the coordin ates of the target point. (c) The speller
module to t ype the letters. (d) The speech- generator module to convert the t y ped texts
to speech.
In the main menu modul e (Figure 5.1 (a) and F igure 5.2), pointing, clicking, and
typing functions are represented b y icons. An automatic linear scanning method is
employe d to select these functions. I n thi s method, first, the repre sented functions are
highlig hted one - by -one o n screen for a scannin g ti me; then, the user activates the switch
if the highlighted icon is what s/he intends to select.

(a)

(b)

(c)

(d)

Fig ure 5.2 : The main menu module to select the pointing, clicking, and t y ping functions.
(a) Mouse left-click. (b) Mouse ri ght-click. (c ) Mouse point ing function. (d) Mouse
double-click. (e) Mouse middle-click. (f) T yping f unction.
By means of the numb ers ( Figure 5. 1 (b)) and the speller (Figure 5.1 (c ))
modules, the user can type the letters and the numbers; while the speech -generator
module (Figure 5.1(d)) enables to convert the text typed by the user to speech.
5.3.2 P ointing: The Coor P T echnique
The mouse cursor point ing function of the G L OSTER 1.0 is based on our novel CoorP
technique. The user first selects the arrow icon (Figure 5.2 (c)) —which represents the
mouse pointing function— in the main menu. Then, the CoorP technique is employed
to naviga te the mouse cursor . I n principle, the CoorP improves the CrossHair tec hnique
in 7 steps. The first 3 steps reduc e the w hole screen to a more manag eable size a nd help
to approach to the targe t point more quickly than the CrossHair technique.
(1) At first, the whole screen (except the G L OSTER 1.0 interfac e) is divi ded into
100 sub-parts accordin g to X-Y coordinate plane as can be seen in Figure 5.3 .
Thus, the user ca n reco gnize the tar get sub-part which includes the tar get point.
(2) Then, the user defines the X -coordinate of the targ et sub-part (between 0 and 9)
in the numbers module (Figure 5.1 (b)) b y p ressing the switch if the highlighted
number is the desire d one. The numb ers are highlig hted according to the
scanning time assigned at t he configuration step.
(3) Afterwards, the second s witch press defines the Y -coordinate of the target sub-
part. Thus, the ta rg et s ub -part can be focused in the following steps by
employing the CrossHa ir technique.

(a)

(b)

(c)

(d)

(e)

(f)

(4) Following the identification of the tar get sub-part, a vertica l scan-line starts to
scan the identified sub- part from left to the right according to the scan-line
sensitivit y assigned in the configuration step.
(5) The thi rd switch press stops the vertical scan -line when it intersects with the
tar get point. Thus, the X- coordinate of the tar get point is identified.
(6) Th en, a horizontal scan-line scans the identified su b-part fr om the bottom to the
top.
(7) The user presses the switch when the horizontal scan -line intersects with the
tar get point. This fourth switch press identifies the Y -coordinate of the targ et
point. At the end of this step, the mouse cursor is na vigated to the identified
tar get point.

Fig ure 5.3 : The user interface of the GLOSTER 1.0 which is divided int o 100 sub -parts
once the mouse c ursor po inting function is selected.
5.3.3 C licking
Clicking is performed b y selecting the desired mouse click function wit hin the main
menu via the automatic linear scanning technique. For example, once the user selects
the mouse left-click (Figure 5.2 (a)), the GL OSTER 1.0 emulates a left -c lick at the
curre nt position of the m ouse cursor on the screen.

5.3.4 T yping
For ty ping function, the user selects the A BC icon (Figure 5.2 (f)) which repre sents the
speller module from the main menu. Followin g this selection, row -column scanning
method begins in the spell er module. First, t he ro ws are hi ghlighted one by one from
the top to the bottom until the user selec ts the desire d row . Then, the letters are
highlig hted one- by -on e. If the highlighted letter is the tar geted one, the u ser hits the
switch to type that letter into the speech -generator module or an y where receiving tex t
input on the screen such as the address bar of a web browser . The user can also pass to
the numbers module from the speller menu to ty pe the numbers.
5.4 Evaluation
In thi s section, we evaluate the GLOSTER 1.0 and th e proposed mouse pointing
technique CoorP in accordance with the results of a usabilit y stud y condu cted b y us .
Firstly , the participants are int roduced. Then, we pres ent the apparatus used in the
usability stud y . A fterward, the tests and the procedure applied during the evaluation are
explained. La stl y , w e conclude the section b y sha ring th e obje ctive and the subjective
evaluation data.
5.4.1 P articipants
The usability study was c onducted at Me dical Fa cult y of the I zmi r Katip Celebi
University (T urke y ) after the approval of the Ethics Committee of the University on
10.10.2018 (with a decision number: 334). Informed consent s of all participants were
rece ived prior to the experiments. A total of 20 p articipants (1 1 males and 9 females )
whose ages ranged between 21 and 65, took part in the evaluation of the proposed
systems.
T able 5.2 summarizes the age statisti cs of all partic ipants according to
participants ’ gender . All voluntary p articipants ha ve seve ral dif ficulties in controlling
their hands and thus couldn' t operate a computer with conventional ways. Besides, the y

were all under medic al treatme nt for seve ral mot or disabil ities, while the experimen ts
were condu cted. All parti cipants c an (1) move thei r head; (2) find a t ar get o n the screen;
(3) follow a moving target; (4) maintain gaze on a s table ta rg et; ( 5) sta y focused on tests
during experiments. W e also applied the MMS E prior to experiments to validate that
the participants have suf f icient cognitive ability to complete our tests.
T able 5.2 : Age statistics of the participants according to the genders.

Gender

Mean Age

Number of
Participants

Mix

44.8 (sd = 13.2)

Female

42.7 (sd = 12.0)

Male

46.6 (sd = 14.4)

1 1

5.4.2 A pparatus
A laptop (Lenovo G505S; CPU: AMD A8-4500M 1 .9 GHz; R AM: 6 GB DDR3; scree n:
LCD 15.6; OS: W indows 10 64 bi ts; resolution: 1600 x 900) and a smartphone with
gyroscope sensor (Son y Xperia XZ1 Compact; CPU: Qualcomm Snapdragon 835;
RAM: 4GB; OS: Androi d Oreo 8.0) were emplo ye d for the ex periments.
5.4.3 T es ts
W e developed a new test tool called Point ingChallenge for the ex periments to collect
the objec tive evaluation data. By means of this tool, we compare the Cros sHair and the
proposed CoorP techniqu es through a standardized test.
The user interface of the PointingChalleng e can be se en in Figure 5.4 . There are
3 dif fere nt square box es in a white panel whose size is 1000 pi xel s in width and 600
pixels in height. Siz es of the r ed, green and blue square boxes are 30x30, 25x 25 and
20x20 pixels, respectively . The test tool has three input parameters to be configured
previous to start the test. The first one is the mouse pointing tec hnique to be tested (i.e.,
the Crosshair or the CoorP), while the s econd one is the sc an-line sensitiv it y which is
described in Section 5.3.1. The last input paramete r is the scanning time which defines

how fast the numbers are highlig hted if the CoorP poi nting technique is test ed. A fter the
test is completed, it prov ides a performance evaluation metric as an output called the
mispoint count. I n prin ciple, the mispoint count is the sum of the wrong m ouse cursor
pointings during the test.

Fig ure 5.4 : The use r interface of the Point ingChallenge with three box es in diffe rent
sizes and colors.
In the test, the user aims to navigate the mouse cursor onto the tar get sq uare
boxes in an order . The test i s performed as follows:
(1) The test begins once the start button is clicked.
(2) Firstly , the red box is tar geted. T o do this, the test tool scans the white p anel
according to th e employed mous e pointing technique. If the user su cceeds to
point the mouse cursor onto the red box, the step 3 begins. But, if the use r m akes
a wron g mouse c ursor p ointing, the step 2 is repea ted until the user succeeds.

During the test, the test tool counts all wrong mouse pointings as the mispoint
count.
(3) Next, the green box is targ eted. If the user succeeds, the step 4 begins.
Otherwise, the step 3 is repeated.
(4) Lastly , the blue box is targe ted. If the user succeeds, the test stops. Otherwise,
this step is repeated.
At the end of the test, the mispoint count can be seen on the interface .
In the test, the CoorP technique is employed as it is described in Sect ion 5.3.2,
while the CrossHair tec hnique is emplo y ed as below:
(1) A horiz ontal scan- line begins to scan the white panel from the bottom to the
top according to the scan-line sensitivit y .
(2) The user presses the switch if the horizontal scan-line intersec ts with the
tar get box . The first switch press defines the Y - coordinate of the tar get box
on the screen.
(3) A vertical scan-line scans the white panel fr om the left side to the right side.
(4) The user presses the switch to stop the scanning when the vertical scan-line
intersects with the tar get box. Th is switch press defines the X- coordinate.
For subjective evaluation , we emplo y a SUS questionnaire which c onsists of ten
statements with a five-point L ikert scale as can be seen in T able 5.3.
T able 5.3 : The statements of the SUS questionnaire according to the GLOSTER 1.0
with average sca le values of all participants.

Statements

Scale

1. I would like to use the G LOSTER 1.0 fr equently .

4.01

2. I found the G LOSTER 1.0 unnecessarily complex.

1. 11

3. I found the G LOSTER 1.0 eas y to use.

4.23

4. I would need the support of a technical person to be able to
use the GLOSTER 1.0.

1.64

5. I found th e various functions in the GL OSTER 1.0 were well
integrated.

4.10

6. I thought there was too m uch inconsistency in the GLOSTER
1.0.

1.34

7. I would imagine that most people would lea rn to use the
GLOSTER 1.0 very quickly .

4.29

8. I found the GLOSTER 1.0 ve r y c umbersome/awkward to use.

1.12

9. I felt ver y confident using the G LOSTER 1.0.

4.22

10. I n eed to learn a lot of things befo re I could get going with
the GLOSTER 1.0.

1.20

5.4.4 P r ocedur e
First, the participants were informed about the test. Then, the y practiced th e tests in a
counterbalance d order under our guidance. For all tests, the HeadGy ro software switch
which is describe d in Section 4.3.3 was employed. W hen the pa rticipants felt confident
enoug h to start the tests, we applied the tests as foll ows:
(1) The CoorP and the Cross Hair techniques were tested by each participant (n=20)
twice for two different scan-line sensitivit y v al ues as 10 and 20, wher e the
scanning time is 1000 milliseconds via the PointingChallenge tool . The te sts
were applied in the counterbalanced order to a void learning and repetition
ef f ects. The participants were also allowed to get rest up to 5 minutes between
the tests.
(2) Then, we asked the participants to complete a computer access task. I n thi s task,
the participants open the web-brows er and search for the dail y weather cast b y
using the GL OSTER 1.0 unde r our guidance. W e didn’t collect an y qu antitative
data such as mispoint cou nt in this task. After the p articipants complete th is task,
we applied the SUS questionnaire to the partic ipants for quantitative subjective
evaluation about the G LOSTER 1.0. Be sides, we also collected the qualitative

subjective data via our observations and participants ’ re sponses of open-en ded
questions.
5.4.5 Object ive Dat a based Results
According to the results of the PointingChallenge test, the CoorP performe d
considerably b etter th an t he CrossHair for both scan -line sensitivit y values in terms o f
the mispoint count. The diffe rence among the means between two mouse pointing
techniques was found s tatisticall y si gnificant for both scan-line sensitivity valu es
according to Student's t-t est. As ca n be seen in F igure 5.5, once the s can-line se nsitivit y
is doubled from 10 to 20, the mispoint count show s a noteworth y decrea se both for the
CoorP and the CrossHair .

Fig ure 5.5 : Mean values of the two mous e pointing techniques in ter ms of mispoint
count according to the two dif ferent scan-line sensitivi ty level whe re in (a) t he scan-line
sensitivit y is 10, (b) th e scan-line sensitivit y is 20. (*** p < 0.001; **** p < 0.0001).
5.4.6 Subject ive Dat a based R esults
T able 5.3 shows the resul ts of the SUS questionnaire as quantitative subj ective data. It
repre sents the average sc ale values acquired from all participants after the y completed
the computer a ccess task by usin g the GLOSTER 1.0 and the HeadG yro. The av erage
SUS score was calculate d as ex plained in Section 4.3.3. The score was calculated as
86.1 which is considere d as quit e impressive.

All participants declare d that the y would prefer the C oorP technique for mous e
cursor pointing inst ead of the C rossHair . 12 partic ipants also stated that the CrossHair
was overwhelming, since it scans the whole screen which leads to a long waiting time.
They declared that the long wait ing times ca uses them to lose their attention.
5.5 Conclusion and Discussion
In this cha pter , we proposed a new single-switch a ccessible CA T called the GLOSTE R
1.0 with a novel mouse pointing technique. The GLOSTER 1.0 is an a ll- in -one solution
which allows th e user t o perform the three mai n functions for computer access as
pointing, clicking, and t yping . In other words, it p rovides all three functions which can
be performed by a mouse and a keyboard in conventiona l way . By m eans of the
GLOSTER 1.0, trigger sig nals coming from the switches can be translated into
meaningful commands to control a computer .
W e also proposed a novel single-switch based mouse pointing technique c alled
the CoorP within this chapter . T o evaluate the proposed technique and compare it wi th
the CrossH air tec hnique —which is the most pre ferred te chnique by t he e xist ing
CA T s— we developed a test tool called the P ointi ngChallenge . A usabilit y stud y with
20 participants was conducted b y emplo y ing the PointingChalleng e for the CoorP and
a SUS questionnaire for the GLOSTER 1.0.
According to the usability stud y results, t he proposed mouse pointing technique
CoorP provided a better solution than the CrossHair technique to the single-s witch
based mouse pointin g problem. Moreover , the overall SUS score as 86,1 also
demonstrated the high usability of the GL OSTER 1.0 in a real-lif e scenario.
The mi spoint count would be decrease d fu rther if the scan-line sensitivit y was
increased more in the experiments. But, in this case , the scan-line would scan the screen
more slowl y , which l eads to a de crease in the atten tion level of the use r's a s it is alre ad y
declared b y th e p articipants during the experiments. As a future work, more ex periments
should be performed to define the optimum scan-l ine sensitivit y .

W e need to note in here that t he cu rrent t yping m ethod of the GLOSTER 1.0
was designed ver y simpl e. T o im prove the s ystem, t y ping rate mi ght be accelerated b y
designing a better alphab et in the speller menu acc ording to the frequency o f the letters.
Furthermore, a word prediction algorithm might be implemented where a list of possible
words is of fered to the user a s a new letter t y ped by the user .
A further futur e work w ould be devising a mark -up lang ua ge which allows to
perform pr e-defined task s in GL OSTER 1.0. For e xample, the user can check his/her e-
mail once the user types the comma nd <e-mail>.
W e strongl y b elieve in t he G LOSTER 1.0 and the proposed software switches
to have a mark ed impact on people's lives, since t hey are to gether capable of enablin g
the single-gesture bas ed hands-free computer acc ess without purchasing any so ftware
or dedicated device s.

6 The SITbench 1.0: A Nov el Sw itch- Based Interaction
T echnique Be nch mark
6
The SITbench 1.0:
A Nov el Switch-Based Interaction T echnique Be nch m ark
W ithin this cha pter , we focus on how to devise a better tool that enables objective
evaluation of a S IT . Evaluation process of a S IT re quires an interdisciplinary t eam ef fort
and takes a considerable amount of time. Collecting subje ctive evaluation data from the
users is a very common approa ch, but the subjective evaluation data alon e mig ht be
manipulated and unreliable for compa ring pe rformances in many cases. Thus,
therapists generall y can not succee d in determining the optimum SIT setup (i.e.,
determining the most appropriate combination of setup variables such as the switch type
or switch site) at first attempts since it is hard to evaluate the me asurable performance
by collectin g subjective data inst ead of objective data. On the other hand, the existing
objective evaluation met hods in li terature are far from being a benchmark . T o make
performa nce evaluation of S IT s b y using a number of standard tests and empirical
attributes, a benchmark application is also requi red. Therefore , we propose a novel
benchmark for performance evaluation called the SITbe nch 1.0 that p rovides a quicker
and more a ccurate switch evaluation p rocess b y collecting and saving the ob jective data
automatically . W e condu ct a usabilit y study with eight participants and demonstrate that
the objective data collected via the S ITbench 1.0 helps to determine the optimum SI T
setup accurately .

6.1 Intr oduction
As we stated in Section 1.1, there have been man y people with motor- disabilities
worldwide who dep end on SI T s for computer access, communication or an y other
reasons. Therefore, plenty of dif ferent SI T s have been develop ed var y ing from
traditional hardware switches to software switches as proposed within this thesis . I n
principle, t hey assist the users to interact with their environment. For example, a user
can selec t a targe t on computer screen b y hitti ng a single switch [23], or an elec tric
wheelchair can be operated via multiple switches [ 1 10]. I t is obvious that an ef fici ent
evaluation process of a SIT pla y s a vital rol e to develop better S IT s and det ermine the
optimum SI T setup for motor-impaired pe ople.
There are man y va riables in a S I T setup such as switch ty pe, scan ning ti me,
switch site, the use rs' posture, activation method, etc. For ex ample, even a sim ple button
switch can be used in sev eral wa y s: it can be activated b y hand or an y other body part.
Likew is e, the users can be positioned in dif fere nt postures during switch usage, which
might af f ect the performance dramaticall y . The main aim of a SIT evaluation is to
determine the optimum S I T setup which is the most suitable combination of these
variables for the users to interact with their environment. T o this end, a c onsiderable
time and ef fort is needed by an interdisciplinar y team that includes many tria ls with
dif ferent variables of S IT setup. On the other han d, assistive technolog y professionals
require a b enchmark appli cation [1 1 1], which is compatible with most S I T s, to make a
better comparison and ev aluation automatically with standardized tests under the same
conditions. Considering the increasing number of the S I T us ers, an y tool th at allows a
more accurate and quick er S IT evalu ation process becomes an importa nt requirement
day b y da y .
Currently , S IT evaluatio n is performed in three wa y s: (a) collecting subjective
data [ 1 12-1 16] via questionnaires, observations and interviews b y an interdisciplinary
team; (b) colle cting obj ective data [12, 25, 28, 40, 46, 1 17-120] via performance tests;
(c) c ollecting both subjec tive and objective data [121, 122] .

Because the subjective data alon e mi ght be unreliable and manipulated easil y
for performance evaluation, it might be hard to s ucceed in determining th e optimum
switch setup on the first attempts in many cases by therapists. They might need several
attempts by re-appl y ing questionnaires or making ne w observ ations. For each
unsuccessful attempt, serious time and ef fort are required to collect a new subjective
data. Thus, collecting subjective data is not a proper wa y to evaluate the measura ble
performa nce of a S I T . W ithout a performance evaluation, it might be very challenging
to achieve the opti mum S I T setup with subjective evaluation alone. On the other hand,
although collecting obje ctive data is the most a ppropriate method for p erformance
evaluation, current objective evaluation methods in litera ture are far fr om being a
benchmark. These methods are mostl y d esigned to evaluate just a specific SIT , which
makes them ineligible to be a benchmark wh ere the other S IT s could be evaluated via
standardized test. T o the best of our knowledge, th ere are onl y two evaluation
applications in literature [1 18, 1 19] which a re close to be a b enchmark for S IT
evaluation. The y can pro vide quantitative dat a to evaluate computer access skills and
help ther apists to choose the switch type a nd position. But both a pplications have some
common limitations that we aim to over come with our novel tool the SITbench 1.0:
• Incompatibility: Switch-accessible applications might require differe nt
key board characters or mouse clicks from switches to work. Furthermor e, each
switch might emulate and send dif fere nt ke y board charac ters or mouse clicks
depending on its manufac turer . Unfortunately , com monl y agre ed standard is not
available. Fo r example, while some switch -accessible applications mi ght expect
to rece ive a ke yboard space character , other applications mi ght expect to receive
a mouse right- click. Both applications expect to receive a mouse left - click to
work. I n other words, they are only compatible with switches which are able to
emulate mouse left-click. The r emaining switches are excluded, which means
that just a minorit y of S IT s are compatible and could be evaluated with these
applications. Therefore, we consider that the y are far from being a proper
benchmark for S IT evalu ation. Our novel tool the S I Tbench 1.0 is compatible
with all switches, which can emulate an y mouse -clicks or keyboard ch aracters,
since it allows therapists to assig n the expected characters from an y switch.

• Limite d nu mber of switches: The y a re only capa ble of evaluating single switch
systems. Doubl e switch support is also required, since double switch usa ge is
widely used as an alternative interaction method. The S ITbe nch 1.0 is capable
of allowing both single and double switch e valuation.
• Limite d number of tests: Both applications empl o y o nl y one test that measures
press time (i.e., the time from the prompt to the when switch is pressed) and
release time (i.e., the time from when the switch i s pressed until it is released)
of a switch. The SI Tbench 1.0 includes two mor e additional tests to evaluate
SIT s with single and double switch.
• Database r equir ement: They have some r eporting fun ctions for the test results.
However , we considered that a well-structured data base would be us eful to s hare
the results and apply some querie s or statistical te sts. I n addition to re porting
function, the S ITbench 1.0 a lso allows to save the t est results automatically i nto
a Microsoft Access database.
• Sufficient atte ntion sp an r equir ement: Suf ficient attention spa n via both
applications mig ht not be a chieved especially by infants, since the y can become
distracted and lose their attention easily durin g l ong and borin g sessions. W e
applied gamification techniques while designing th e S I Tbench 1.0 t ests with the
intent to make evaluations more engag ing and fun.
Therefore, we propos e a novel S IT evaluation too l namely the S ITbench 1. 0 as
a benchmark a pplication which helps to determine the optimum S I T setup with the aim
of providing a quicker and more accurate SIT evaluation process. T o c ollect the
objective data, the S ITbench 1.0 includes three dif fere nt games which can be pla y ed via
single or double switch. It measures and save s the performance metrics (accuracy ,
precision, recall, fa lse positiv e rate) a utomaticall y at the end of each trial.
A usabilit y stud y with eight par ticipants was conducted as a pa rt of this work in
order to test and demonstrate the proposed benchmark application. W e identified two
dif ferent switch sites to be tested by the users u nder the same conditions in order to
determine the most suitable switch site. T o this end, we collected the objective data via
the S I Tbench 1.0. Results revealed that the S I Tbe nch 1.0 could help to determine the

optimum switch setup accura tel y . W e also applied a SUS [58] questionnaire to evaluate
the SITbench 1.0 itself, and the re sults were quite satisfactory .
More potential S IT users can b e se rved at the same time period with the same
workforce since a quicker and more a ccurate SIT evaluation process is provided by th e
SITbe nch 1.0, which might prevent governments t o spend high amounts of money as a
result of bette r cost and s chedule management. As a benchmark a pplication, it allows to
make objective comparisons with standardized tests under the same conditions by
collecting the perform ance data o f SIT s for assistive technology community
automatically . Thus, it p rovides extra time for therapists to observe mor e subjective
aspects of c lient needs. O n the other hand, it might be used to eva luate fine-motor skill s
of clients as a clinical to ol. Occupational therapis ts can tr ack the patients ' progress b y
the S ITbe nch 1.0 th at all ows to measure and record clients' fine motor p erformance and
reflexes automaticall y in the form of quantitative objective data. The S ITbe nch 1.0
might also h elp to improve the contingenc y awareness of the ones with pr ofound and
multiple lea rning disabilit ies, or it might b e use ful for pupils with severe le arning
dif ficulties to assess their auditory and visual attention.
This chapter p roceeds with Section 6.2 that presents the design and
implementation of our no vel switch evaluation too l. Then, in the S ection 6.3, we sh are
the objective results of our usability stud y and the questionnaire results of the SITbe nch
1.0. Finally , w e conclude our study and dis cuss our future work in Section 6.4.
6.2 Design of the SITbench 1.0
The SITbench 1.0 is desi gned as a novel benchmar k application for assistive technolog y
and healthcare prof essionals to determine the most appropriate S I T setup. I t helps to
collect and s ave the obje ctive data automaticall y with the aim of opti mum S I T setup.
T o this end, three diffe rent switch-accessible games, depending on single or double
switch, were desi gned within the SI Tbench 1.0 namely T ie-Smile y Matching Game ,
Non-stop Driver Game and Hungry Fr og Game respectively .

The S I Tbench 1.0 welco mes the users with a ver y s imple int erfa ce (Figure 6.1 )
when it is initi alized. I n welcoming screen, the u sers can sele ct the games (i.e., tests)
and open the key assignment module to assign the expected keys from switches.

Fig ure 6.1: W elcoming screen of the SITbench 1.0.
As can be seen in Fi gure 6.2, single and double switch settings can be confi gured
according to expected ke y (i.e., a ke y board character or a mouse -click) fro m any S IT s
to be tested via the SI Tbench 1.0. I n this way , the S I Tbench 1.0 becomes compatible
with the majorit y o f assi stive switches, sinc e almost all switches on the marke t can
emulate a key board character or a mouse-click.

Fig ure 6.2: Ex pected key assig nment module.
6.2.1 T ie-Smiley M atching Gam e (TSMG)
TSMG is a single-switch accessible game b ased on indi rect selection with automatic
linear scanning method. As it is e xemplified in Fig ure 6.3, an indirect sel ection with
automatic linear scanning method can be summa rized in three steps: (1) letters in a
scanning array (En glish alphabet as a selection se t) are hi ghlighted one - by -on e on the
scree n for an equal duration t units of time where t repre sents scannin g ti me, i.e., time
interval between two successive states; (2) until the end of each state, the us er is allowed

to make a selection b y hit ting a switch or sending a ny kind of si gna l det ected by a s ensor
(e.g., a blink); (3) if the highlighted letter is the targ et (i.e., what the user intends to
select), the user sends a selection signal such as blinking.

Fig ure 6.3: T ime-state m odel of an automa tic linear scanning sample.
There are five differe nt t emplates which could be tested via TSMG. Fig ur e 6.4
shows an initial form of template 1. The scanning arra y of each template consists of
y ellow and red smile y s (26 smiley s in total). Red smileys are tar gets to be se lected, and
they are set in a dif ferent order for each template in order to avoid repetition. T ar gets
are seen by the user before star ting and during the test.

Fig ure 6.4: Initial form of TS MG in template 1.
TSMG is based on auto matic li near scannin g where each smile y is highlighted
for a given time period (i.e., scannin g ti me) one- by -on e. The user should activate th e
switch once the highlighted smile y is red on e. The user also hears a click sound as an
auditory prompt , as soon as the targ et is highlighted. When the switch is activated, it
sends the ex pected ke y to the S ITbench 1.0 as a selection signal. Once the e xpected ke y
is received, the SITbench 1.0 gives a senso r y feedback b y swapping the bac kground

colour of the inte rface like a blink. I n expert mode ( Figure 6.5), th erapists can enter
some de tails about the user . The y can also select the template and set the scanning time
(in milliseconds).

Fig ure 6.5: Initial form of TS MG in expert mode.
The user aims to match e ach smile y with a ti e in a wa y that smi ley and its tie are
in the same colour (e.g ., red smileys with red ties). T o this end, the user should select
all red smile y s but y ellow ones vi a a switch. A sample view of results after the user
completed a trial without any mistake can be seen in Figure 6.6.

Fig ure 6.6: A view of TS MG in the end of a trial foll owing a user performance without
any mistake.
At the end of each trial, confusion matrix variabl es (true positives (TP), false
positives (FP), false ne gatives (FN), true ne gatives (TN)) are c alculated and assigned
automatically as can be seen in Figure 6.7 according to count and colour o f ti es in a wa y
that: TP represents count of red ti es; FP represents count of orange ties; FN represents
count of gree n ties; TN represents count of yellow ties. All performance evaluation

metrics (accurac y , precision, recall and false posit ive rate) are measured b y S ITbe nch
automatically at the end of trial b y using the following formulas:
𝑎𝑐𝑐𝑢𝑟𝑎𝑐𝑦 = 𝑇𝑃 + 𝑇𝑁
𝑇𝑃 + 𝑇𝑁 + 𝐹𝑃 + 𝐹𝑁
𝑝𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 = 𝑇𝑃
𝑇𝑃 + 𝐹𝑃
𝑟𝑒𝑐𝑎𝑙𝑙 = 𝑇𝑃
𝑇𝑃 + 𝐹𝑁
𝑓𝑎𝑙𝑠𝑒 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑟𝑎𝑡𝑒 = 𝐹𝑃
𝑇𝑁 + 𝐹𝑃
The SITbench 1.0 allo ws therapists to save th e results and all data i nto a
structured database, and i t has also a reporting fun ction to print or save the results as a
document.

Fig ure 6.7: A general view from TS MG in the end of a trial following a user
performa nce with several mistakes (i.e., with false negatives and false positives).
6.2.2 N on-stop Drive r Gam e (NDG)
NDG can b e played with single or double switc h . While the single- switch accessible
version is called SS -NDG, double-switch accessible version is calle d DS -NDG. Figure
6.8 shows the initial form of SS-NDG. Ga me objects are labelled with blue numbers in

Fig ure 6.8 to int roduce them: label 1 shows left signal (i.e., o range square box); 2 is
right signal; 3 represe nts green car; 4 is finish line.

Fig ure 6.8: The initial form of SS -NDG where 1 shows left signal (i.e., or ange squar e
box); 2 is right signal; 3 represents green car , 4 is f inish line.
Before starting the game, therapists can select the track and adjust the scan ning
time. The aim of the user is to reach the finish line as soon as possible wit h minimum
crash into the walls. SS- NDG depends on automatic scanning method with a sin gle
switch where sign als (i.e., car's left and right signals which are illust rated as orange
boxes) are flash ed one by one for a given t ime period (i.e., scan ni ng time in
milliseconds). After the game starts, the car begins to move and never stops until
reac hing the finish line. T o turn the car left, the us er hits the switch once car 's left signal
is flashed; and hits the s witch once right sig nal is flashed to turn the car r ight. The only
dif ference between SS-NDG and DS-NDG is that the c ar in DS- NDG doesn’t have left
and ri ght signals ( Figure 6.9) since it is controlled with double switch. The u ser a ctivates
the first switch to turn it left and the second switch to turn it right.

Fig ure 6.9: The initial form of DS -NDG in track 2.
W e also assigned two diffe rent sounds to the S ITbench 1.0 as auditor y prompts
according to left and ri ght signals. I n other word s, the user hears two dif ferent sounds
when signals are fl ashed during game. Once the user hits the switch, expected ke y is
rece ived and the S ITbench 1.0 provid es a sensory f eedback visuall y b y swapping the
background colour of the interface. Th ere have been five dif ferent tracks where each
track has a different finish line loca tion from each other to avoid learn ing ef fect. A
sample view in the end of a trial is shown in Figure 6.10 where the user comp leted game
via double switch in track 3 without any crash. Completion time (in seconds) and crash
count are measured automaticall y as pe rformance metrics at the end of each trial. The
SITbe nch 1.0 enables ther apists to save all re sults and data into a database and to report
them as a document. It also depicts a black tracking li ne of the car (Fi gure 6.10) and
allows therapists to save the screenshot of the interface as a separate image file.

Fig ure 6.10: A vie w of DS -NDG in th e end of a t rial whe re th e use r r eached to the finish
line in track 3 without any crash.
6.2.3 Hungr y Fr og Game (HF G)
HFG is a single-switch accessible application to measure the user's switch performance.
At the beginning of each trial, therapists can select the scenario and enter the user’s
details. Each trial of the game consists of ten tasks. As it is il lustrated in Figure 6. 11 ,
each task in a trial is achieved in a wa y th at: (a) t he user w aits until a fly is appeared;
(b) the user activates the switch as soon as a fly is see n; (c) the frog eats the fl y once the
user ac tivates the switch.

As soon as the fly appears, the user hears a click sound as an auditor y prompt.
When the expected ke y is rece ived f rom the switch, background colo ur of the S I Tbench
1.0 is swapped like a blink to g ive a sensor y feedback. After the user completes ten
tasks, the S I Tbench 1.0 measures average press time (i.e., th e aver age time from wh en
the fl y appears to the when switch is pressed) and averag e rel ease ti me (i.e., the averag e
time from when the switch is pressed until it is released) a utomaticall y . The fastest and
the slowest pre ss time and relea se time amon g ten tasks are also detected. HFG has five
dif ferent scenarios to avoid repetition and learning af fect. For each scenar io, waiting
times (i.e., the time from when the user starts to wait to the when the fl y appears, and
not more than 6 seconds) of each task in a trial are set dif ferent from ea ch other . Figure
6.12 shows the view of int erfac e in the end of each trial. Six performance metrics
(measure d in seconds) can be saved into a database and reported via the S ITbench 1.0 :
(1) average press ti me; (2) average release time ; (3) the fastes t press time; (4) th e
slowest press time; (5) the f astest release time; (6) the slowest release time.

Fig ure 6.12: A view of HFG in the end of a trial.
6.3 Evaluation
W e conducted a us abilit y stud y as a demonstration of the S I Tbench 1.0. W e identified
two diffe rent switch sites (Figure 6.13 (c)) to be tested: forefinger distal pulp (hereafter:

FDP) and forefinger prox imal interphalangeal joint (hereafter: FPI J) . FPD was
considered as a proper switch site to activate a s witch easi l y in contrast t o FPIJ. W e
aimed to demonstrate that the SITbench 1.0 c an determine the most proper switch site.
T o thi s end, the users performed tests by using two dif ferent switch sites. A
questionnaire wa s also a pplied to evaluate the SITbench 1.0 itsel f.

Fig ure 6.13: Positions of forefinger durin g ex periments according to two switch sites
FDP (r epresented by x) and FPI J (represented b y z ): (a) switch press with FDP; (b)
switch relea se with FDP; (c) switch press with FPI J; (d ) switch release with FPIJ.
In this section, firstly we introduce th e participants. Then, we pre sent the
appara tus used and the procedure applied. At last, we share the experimental findings.
6.3.1 P articipants
Eight able-bodied p articipants (mean age = 30.2, standard deviation = 3.1) , including
four females and four m ales, took part in this study . Just two of the participants were
familiar with switch-accessible applications befor e experiments.
6.3.2 A pparatus
A laptop (Model: Lenovo G505S; CP U: AMD A8 -4500M 1.9 GHz; RAM: 6 GB DDR3;
Screen: LCD 15.6; OS: W indows 10 64 bit s; Resolution: 1600 x 900) was emplo ye d
within this stud y for experiments.

6.3.3 P r ocedur e
At the beginning, the pa rticipant is positioned in front of a laptop in a wa y that the
participant is able to access laptop's keyboard easily . Enter k e y on ke y board was
considered as a switch.
Participants were informed about the S ITbe nch 1.0 and tests, and then the y
practised the S ITbench 1.0 in counterbalanced ord er until they b ecome r ead y for tests.
This prac ticing step too k 20 minutes approximately for each p articipant. Following
positioning and practicing steps, three tests were applied to particip ants to colle ct
objective performance data:
• TSMG: Each switch site (FDP and FP I J) was t est ed b y each participant (n=8 )
for each template (n=5) two times where scanning time is 500 milliseconds (i.e.,
each participant pe rform ed 20 trials in total with TSMG).
• SS -NDG: Each switch site (F DP and F P I J) was tested by each participant (n=8 )
for each t rack (n=5 ) w here s canning time is 500 milliseconds (i.e., each
participant performed 10 trials in total with SS -ND G).
• HFG: Each switch site (FDP and FPI J) was tested b y each participant (n=8 ) for
each scenario (n=5) (i.e., each participant performe d 10 t rials in total with HFG).
All tests were applied in counterbalanced order to avoid learning and repetition
ef f ects. The participants were also allowed to rest (1 to 5 mi nutes) during experiments
to prevent excessive mental or phy sical fatigue.
At the end of experiments, a SUS questionnaire [58], which is an industr y
standard, w as applied to the participants to evaluate the usabilit y of the SITbench 1.0
application. The S US inc ludes ten statements ( T ab le 6.1) with a five- point Likert scale.
Scale value of statements is ran ging from 1 (strongly disa gree) to 5 (stron gly agree). W e
modified SUS statements according to the S ITbench 1.0 to clearly descri be it. On the
other hand, qu alitative subjective data was collected via our obser vations and
participants' re sponses of open-ended questions about the S I Tbench 1.0.

T able 6.1: Modified stat ements of a S US questionnaire with avera ge scale values of all
participants.

Statements

Scale

1. I would use the S ITbench 1.0 for S IT ev aluation tasks
freque ntl y .

4.00

2. I found the S ITbench 1.0 unnecessarily complex .

1.37

3. I found the S ITbench 1.0 eas y to use.

4.12

4. I would need the support of a technical person to be able to
use the SITbench 1.0.

1.75

5. I found the various functions in the S ITbench 1.0 were well
integrated.

4.37

6. I thought there w as too much inconsistenc y in the S I Tbench
1.0.

1.37

7. I would imagine that most people would learn to use the
SITbe nch 1.0 ve r y quickly .

4.25

8. I found the S ITbench 1.0 ver y cumbersome/awkward to use.

1.50

9. I felt ver y confident us ing the SITbench 1.0.

4.12

10. I need to learn a lot before I c an use t he SITbench 1.0.

1.25

6.3.4 Object ive Dat a based R esults
FDP as a s witch site showed quit e impressive per formance in comparison with FP I J in
all three tests (TMSG, SS -NDG, HFG) as it is expected at the beginning. It is
demonstrated that the S I Tbench 1.0 succeeded to determine the most appropriate switch
site as FDP .
According to results of TS MG ( Figure 6.14), FDP was better than FP I J in all
performa nce evaluation metrics (accuracy , p recision, recall and false positive rate).
SS -NDG results (Figure 6.15) also suggested that FDP performed better than
FPI J in t erms of the completion time and the crash count.

Fig ure 6.14 : Mean v alues of two switch sites for all participants through eva luation
metrics of TSMG (accuracy , pr ecision, recall, false positi ve rate). (* p < 0.05; *** p <
0.001; **** p < 0.0001).

Fig ure 6.15 : Mean values of two switch sites for all participants according to evaluation
metrics of SS -NGD as (a) the completion time and (b) the cra sh count. (* p < 0.05; *** p
< 0.001).
La stl y , HFG results (Fi gure 6.16) prov ed that FDP is by far th e best switch sit e
in all evaluation metrics (ave rage press time, average release ti me, the fastest press time,
the slowest press time, the fastest release time, the slowest release time). W e also applied
Student’s t-test for both switch sit es through all ev aluation metrics in all three tests. In
consequence of t-tests, it is proved that ther e is a signi ficant dif ference between the
performa nce of FDP and FPIJ for all e valuation metrics.

Fig ure 6.16 : Mean v alues of two switch sites for all participants through eva luation
metrics of HFG (average pr ess time, av erage release time, the fastest pr ess ti me, the
slowest press time, the fa stest release time, the slowe st release time) (**** p < 0.0001).
6.3.5 Subject ive Dat a based R esults
Results of a SUS questionnaire are li sted in T able 6.1. Scale column holds the avera ge
scale values (1 to 5) of e ach statement fo r all participants. A vera ge SUS sc ore for all
participants was c alculated as 84. According to adjective rating scale [ 94], overall SUS
score (84) of the S ITben ch 1.0 was rated as excellent, and SUS scores of e ach
participant ranged from good to excellent. Prior to experiments, all participants were
excited for ex periments. Just two of them had a previous experience with S I T s. The y all
declared that the S ITbe nch 1.0 would be a very useful tool for assisti ve tec hnolog y
community . One participant stated that he c ould have performed better if scan ning ti me
was slower . T wo of the participants suggested to increase the size of smiley s in TSMG.
All participants declared that FDP is definitely mo re proper than FP I J as a switch si te.
None of the participants experienced fatig ue durin g tests.
6.4 Conclusion and Discussion
Evaluation process is one of the most important tas ks in order to reach the opti mum S I T
setup. Because the optimum SIT setup plays a vital role for people with motor

100

disabilities to interact with their environment, an y t ool to achieve the optimum SIT setup
for having a better cost and schedule mana gement becomes a ver y important
requirement considering the increa sing number of the S I T users.
Determining the optimum swi tch setup b y colle cting the subjective data might
be challenging, since the subjective data alone might be unre liable and manipulated
easily for pe rformance evaluation. Therapists might have to re-apply questio nnaires and
make new observations severa l time s. A serious ti me and ef fort are ne eded for these
repea ted trials to collec t subjective data. Therefore, subjective data colle ction instead of
objective data doe sn't seem a proper method for performance evaluation of a S I T .
On the other hand, current evaluation methods based on collecting objective data
in literature are far from being a benchmark. The se methods are generally e mployed to
evaluate just a specific SIT . In other words, the y are not designed to evaluate the other
SIT s, whi ch makes th em ineligi ble to be a benchmark. T o the b est of our knowledge,
there have b een just two applications [1 18, 1 19] in literature which are close to be a
benchmark. The main limitations of these applications and solutions we proposed with
the S ITbe nch 1.0 are as follows: (a) The y onl y work with the SI T s that can emulate
mouse lef t-click, which makes them c ompatibl e with just a minority of S I T s for
evaluation. The S ITbench 1.0 as a benchmark a llows to assign an y expected characters
or mouse-clicks from any S I T . By this way , al l SI T s which can emulate keyboard
character s or mouse-clicks could be evaluated and compare d via the SI Tbench 1.0 with
standardized tests; (b) They onl y support sin gle switch based s ystems. Be cause double
switch usage is a widely preferred int eraction technique, the S ITbench 1.0 supports
double switch evaluation as well; ( c) The y have on ly one test to measure pre ss t ime and
release time of a switch. The SITbench 1.0 has two more performance tests to evaluate
SIT s; (d) The y don't allow to save th e results into an external dat abase, although the y
have some reporting functions. The S ITbench 1.0 supports to save the resul t
automatically int o a data base to share or anal yse it for further studies. Th erefore, we
propose the S ITbench 1.0 as a benchmark appl ication that helps to determine the
optimum S I T setup to provide a quicker and mor e accurate S I T evaluation process b y
collecting and savin g the objective da ta automaticall y .

101

W e conduc ted a us ability stud y as a demonstr ation with eight participant s to
evaluate th e usa ge of di f ferent switch sites. T o this end, objective data was collected via
the SITbenc h 1.0. FDP performed better performance than FP I J in all tests as it is
expected. Findin gs d emonstrated that th e SITbench 1.0 is capable of determin ing the
most proper switch site with the aim of optimum S I T setup. Result of a SUS
questionnaire to eva luate the SITbench 1.0 itself was also quite satisfactory .
A quicker and mo re a ccurate SIT evaluation via the SITbench 1.0 he lps to serve
more potential S I T users at the same ti me period with the same workforce. As a result
of a better cost and schedule mana gement, the SITbench 1.0 might p revent governments
from unne cessary expen ses and human -resource allocations. But, future studies with
the S ITbench 1.0 are required to verif y that the SITbench 1.0 is capa bl e o f doing this.
On the other hand, it might be also emplo ye d b y therapists and assistive technology
professionals to measure the fine-motor skills and reflexes of th e use rs as a clinical tool.
They can track the progress of the user 's skill via the SITbench 1.0, since it is able to
measure and s ave the pe rformances automatically as a quantitative objectiv e data. The
SITbe nch 1.0 can also b e utilized to improve the conting enc y awareness of the ones
with profound and multiple learning disabilities. Besides, it might be employ ed as a tool
to assess auditory and visual attention of people with severe learning dif ficu lties.
In ord er to improve the SITbench 1.0 and overcome some of its li mitations, some
future studies would be quite useful. W e int end to include new tests depending on
severa l scannin g methods. So as to test the ef ficiency of the S ITbench 1.0 better , we
aim to extend the participant group with motor-impaired peopl e. Since the SITbench
1.0 is currently compatible with only desktop computers, it mi ght be mo dified to be
compatible with mobile s y stems such as smartph ones and tablets to extend the target
group. W e also aim to include some tests such as a speller to evaluate the use rs' computer
access activities. Emplo y ing a group of therapists and assistive technolog y
professionals to evaluate and demonstrate the SITbench 1.0 would be also quite useful.

102

103

7 Conclusion and O utlook

7
Conclusion and Outlook
Throug h thi s thesis, we addressed the sin gle-gesture based hands-free com puter access
problem of the people with severe motor -neuron impairments to i mprove their self -
suf ficiency . I n thi s chapter , we summ arize and discuss the main contributions in Section
7.1 under thre e domains through chapters b y revisiting the research questions.
Subsequently , an overv iew of the possible future expansions is g iven in Section 7.2.
Finally , we con clude the thesis with remarks about the work presented within this thesis.

104

7.1 Summ ary of the Thesis and Contributions
Softwar e Switches: Single-gestur e based Hands-fr ee Interaction T echniqu es
Chapter 2:
Our ef forts to provide ef ficient solut ions to single-gesture bas ed hands-free computer
access problem led us to focus on the following researc h question:
Q1: How to devise an efficient appr oach enabling singl e-gestur e based hand s-fr ee HCI?
W e started with a literature r eview to handle this question. T o have a bro ader
point of view , we reviewed the current hands-free interaction techniques —instead of
reviewing just the single -gesture base d solutions — in terms of the ph y sical gestures
used. This review helped us to identify two major p roblems of the current single-gesture
based hands-free interaction techniques. The first problem is that the majority of cu rrent
single-gesture based hands -free HC I techniques depend on ex pensive dedicated devic es
bey ond standard computer peripherals. Unfortun ately , high- cost of d edicated devices
became a new ba rrier for the people with motor -disabilities. Many people liv ing in poor
and middle income countries have dif ficulties to af ford such devi ces [7 -9]. Thus, as the
only reasonable exception, we decided to exclude smartphones from dedica ted devices
list because the total num ber of smartphones —3.2 billion in 2019 [48]— got ahead of
the tot al number of computers in recent y ears w orldwide [49] . Smartphone s became
easy to access devices li ke computer peripherals today e ven for the on es live in low -
income countries. Furthermore, unlike the dedicated devices, smartphones can also b e
used to achieve several s ervices be y ond assistive purposes.
The se cond problem is that the majorit y of current single -gesture b ased hands -
free solutions for computer access in literatu re a re onl y compatible with specific switch -
accessible interfaces. W hil e some switch-a ccessible interfa ces expect to re ceive a
specific k eyboard character like space, the othe rs ex pect to r eceive a mouse click.
Unfortunately , a commonly agreed standa rd on this is lacking. T o overcome these two
major problems, we prop osed our novel software s witch approach. In brie f, the softwa re

105

switch approach has two principles: an int erac tion technique based on our software
switch approach (1) should not require an y dedicated devices, and (2) should be
config urable to be compati ble with the other switch-accessible interfac es.
After w e propos ed our s oftware switch app roach, in Chapter 2, we also aimed
to identif y the most suita ble t y pes of hands-free gestures which will be us ed with the
proposed software switches. T o do this, we b egan with the evaluation of gesture t ypes
of current hands -free int erac tion te chniques given in Section 2.1. W e omi tted the HC I
techniques which re quire dedica ted devices in accorda nce with the first principle of our
software switch approach, and we foc used on t he techniques which don't r equire
dedicated devices.
Afterwards, we excluded e ye-gaze and facial gestures (e.g., eye bli nk gesture),
since the y might b e hi ghly affe cted b y Mid as T ouch problem [50] . For ex ample, if the
ey e blink is used to interact with a computer , it is hard to distin guish whet her the user
blinked voluntary as a trigger signal or it was just a regular e y e bli nk performed
unconsciously . Although the Midas T ouch problem could be resolved by emplo ying
multi-modal inputs such as performing an e ye blink with an e y e b row raising
simultaneously as a trigger signal, emplo y ing a mult i -modal input was not an option in
the scope of this thesis since we focused on the single-gesture based hands-free
computer acce ss problem.
As another gesture t ype s i n literature, the speech an d non-speech based solutions
were also omitted b y us, because the spe ech recognit ion algorithms are mostl y pr eferred
for more complicated tasks than just a single- ge stu re. Unlike thes e gesture s, we revealed
that the re spiration (i.e ., sip and puf f) and head movement (e.g., a he ad tilt) g estures are
not af fected b y Midas T ouch problem, and the y can be e asily recognized via si mpl e
algorithms. Th erefore, we considered them as the most suitable hands-free gesture t y pes
among ex isting solutions to interact with a computer accordin g to the software switch
approac h.

106

Chapters 3 and 4:
Following the dete ction of the most suitable g es tures, next step was to de vise
ef fici ent interac tion methods by following the principles of our software switch
approac h to interact with a computer b y a single puf f or head ge sture. T o do this, we
began with addressing the following research questi ons:
Q2 : How to de vise a better technique that enables a person to interact with a computer
by a single puff-gestur e?
Q3: How to devise a bette r technique that e nables a person to interact with a computer
by a single head-ge stur e?
T o answe r Q2, we designed two no vel interac tion techniques called the PuffMic
and the PuffCa m softw are switches in Chapter 3 . Both software switches are based on
a puf f-gesture whe re a st rong puff, detected b y a microphone or a modifie d camera, is
considered as a pr essed s witch. In Chapter 4, in accordance with our effor ts to answer
Q3, we designed two more novel interaction techniques called the Hea dC am and the
HeadGyro software switches. Both software swit ches are capable of recognizing head
ge stures ( e.g., a head tilt ) via a standard c amera or a gyroscope sensor of a smartphone
to consider them as pressed switch es .
T o sum up, all software s witches proposed within t his thesis can allow to int erac t
with a computer b y a single-gesture in a w ay that the y receive the use r's gesture as an
input signal to tr anslate them into emulated swit ch presses. Furthermore, the y don 't
require an y dedic ated de vice, and the y are compatible with the other switch -accessible
interfaces in accordance with our software switch approach. The high usabilit y of the
proposed switches w as demonstrated b y two di f ferent studies conducted with 82
participants with/out disabilities in total. As a r esult of these us ability st udies via the
TSMG test of the S I Tbench 1.0, we demonstrated that the perf ormance of the proposed
software switch es through evaluation metrics —accuracy , precision, recall and fa lse
positive rate— w ere qu ite im pressive. For each evaluation metrics, t he Puf fMic
exhibited the worst performance, whereas th e H eadGyro was the b est. According to our

107

observations, the reasons why the PuffMic performed the worst is considered as
follows:
• The position of the hea dset mi crophone should be stable during e xperiments
since the threshold valu e is assi gned in the c alibration step and hi ghly d epends
on the initial positi on of the microphone. Any po ssible change in the posi tion
(angle or distance between microphone and the user's mouth or nose) during the
experiments mig ht lea d to a minor or a major c hange in the air pr essure
(orig in ated f rom the ex halation pressur e) applied on the mi crophone, which is
considered as the main reason of hig h false positive rate.
Besides, each software switch exhibited diffe rent reactions to the external
factors. While the HeadG y ro and the PuffMic are not af f ected b y the li ght level (i.e.,
they can even work in the darkness), the Puf fCam and the HeadCam are a ffe cted. As a
microphone based solut ion, the Puf fMic can be a f fected b y high external vo ices and the
user's speech unlike the other software switches. Additionally , the HeadG y ro is more
sensitive than the Hea dC am since it is capa ble of recognizing tiny head movements.
The GLOSTER 1.0: A Sin gle-gestur e Acc essible Hands -fr ee CA T
Chapter 5:
On the other hand, the proposed software switches were not functional alone to
control a computer , since they can only serve like tra ditional switches by recognizing a
single head or puff gesture to translate them into emulated switch press es. I n other
words, they ca n onl y allow to interac t with a computer .
As a genera l approa ch, c omputer access depends on three functions: point ing,
clicking, and t y ping. I n c onventional use, pointing a nd clicking functions ar e performed
by int eracting with a computer via a standard m ouse, while t yping is achieved b y a
key board. B ut, i f the us er has an onl y single -ge sture unimpaired, all three func tions
have to be performed with that single-gesture/single-switch. T o do this, the proposed
software switches require a CA T with a scanning in terface which converts the emulated

108

switch pre sses into commands (e.g ., mouse clicks) in a wa y that a c omput er can
understand. In acc ordance with this requirement, we focused on the following re search
question in Chapter 5:
Q4: How to devise a better CA T that enables a person to contr ol a computer with a
single-gestur e?
This question led us t o a new sin gle-switch accessible CA T called the
GLOSTER 1.0 with a n ovel mouse pointing t echnique . It allows the users to easil y
control a compute r b y emplo y ing the p roposed software switches o r an y othe r
traditional switches. The proposed mouse pointing technique called the CoorP also
provided a more accurate solut ion to the sin gle-switch based mouse point ing problem
than the most popular mouse pointing tec hnique in literature.
The SITbench 1.0: An Evaluation T ool
Chapter 6:
In the evaluation stage o f the proposed software s witches, we realized the lack
of a reliable evaluation t ool. Evaluation process of a S IT —like the prop osed software
switches— requires an interdisciplinar y team ef for t and takes a considerable amount o f
time, since a SIT setup depends on man y variables suc h as switch t y pe, switch position
or switch site. Optimum S I T setup (i.e., the most appropriate combination of setup
variables) could not be achieve d at first attempts.
Although subjec tive evaluation is a very common approac h, we considered that
the subjective evaluation data alone might be manipulated and unreliable for comparing
performa nces in man y cases. It is hard to evaluate the measurable p erformance b y
collecting subjective data instead of objective data. Unfortunately , the existing objective
evaluation methods are far from bein g a benchmark where the other S IT s could be
evaluated via standardize d test. They are mostl y designed to ev aluate just a specific S I T .
Requirement of a useful tool to evaluate our software switches objectively led us to
focus on the following resea rch question in Chapter 6:

109

Q5: How to devise a better tool that enables objective evaluation of switch -based
interaction techniques?
In an effor t to address Q5, we proposed a nov el benc hmark tool for objective
evaluation called the SITbench 1.0. It was demonstrated b y a usabilit y study with 8
participants that th e SITbench 1.0 provides a quicker and more acc urate switch
evaluation process by col lecting the objective da ta automaticall y.
7.2 Outlook
W ithin the scope of this thesis, novel hands-free i nterac tion techniques an d alternative
computer access solutions were presented in accordanc e with our ef forts to answer the
research questions in Section 1.3. However , there are sti ll possible f uture ex pansions
for the proposed studies. W e identify pot ential research di rections below that can be
followed in the future .
The proposed software s witches are m ainl y bas ed on sim ple audio and video
processing alg orithms to recognize the user' s gesture. As a future research direction,
machine learnin g te chniques ca n be emplo y ed t o recognize these g estures. Such a
direction would allow to ac hieve better recognition performance or the opposit e.
While the P uf fMic and the PuffCam s erve as alternatives to traditional puf f
switches, the HeadCam and the HeadGyro are alternatives to traditional he ad switches.
In another future work, a usabilit y stud y to compare the proposed softw are switches
with the traditional alternatives would be useful to see how usable the p roposed software
switches.
Employ ing a unidir ectional-cardioid microphone was out of option within this
thesis because the y are ex pensive and beyond the standard computer peripherals. In a
future study , the y would be emplo y ed to reject sounds from other directions if high-cost
of unidirectional-cardioid microphones ca n be tolerated.

1 10

W e compared the performance of CG with DG to see whether there is an y
significa nt diffe rence b etween these groups, because we aimed that our approach will
be utilized b y able-bod ied people as well. The application area of t he proposed
interaction techniques is quite flex ible and should not be considered just for assistive
technology area, although we aimed to provide comprehensive solutions for motor-
impaired people withi n this thesis. For ex ample, in a mul ti-modal computer video game,
one can try to emplo y ou r software switch es in a way that pla yers use their puff/head
ge sture as a new interacti on method during the game-play .
W ithin thi s thesis, we employe d the proposed software switches in the S I Tbench
1.0 and the GLOSTER 1.0. As a nother future work, the proposed softwa re switches
would be employed in a dif ferent sin gle-switch accessible inter face to provide a fur ther
validation in terms of their compatibilit y in real- life scenarios. Another future work
would be making the p roposed software switches and int erfaces compati ble with the
other operating systems like Ma cintosh to improve the acce ssibilit y . Currentl y , the y are
only compatible with Mi crosoft-based operating sy stems.
It is also worth noting that the HeadCam and the HeadG y ro are also capable of
recognizing an y motion of the other bod y pa rts such as the user's shoulder or leg be y ond
head movem ents. In a f uture wor k, dif ferent phy sical movements can b e t ar geted
depending on the user’s unim paired physical abilit y to evaluate the flexibilit y of the
proposed software switches.
The GLOSTER 1.0 is ba sed on an automatic row -column scanning metho d to
type tex t in a way that the user has to type all letters of a word. Current text entr y method
can be accelerated by a prediction tec hnique whe re a list of possible words is of fere d to
the user as a new letter ty ped. Moreover , the alphabet in the speller menu ca n be
designe d mor e wisel y according to the frequ ency of the letters. By this way , the use r
would achieve a better t ext i nput ra te.
A further futur e expansion would be de vising a mark-up languag e which allows
to perform pre-defined ta sks in GL OSTER 1.0, for example the command “< browser> ”

111

typed b y the use r will open the web browser and etc. Such commands would increase
the usability of the CA T .
The SITbench 1.0, in its curr ent for m, is ca pable of providing stand ardized tests
with single and double switch access. Development of new standardized tests with
multi-switch acc ess wo uld be another research direction to im pr ove the proposed
evaluation benc hm ark.
Besides, it would be interesting to emplo y th e S ITbench 1.0 for clinical purposes
to evaluate the us abilit y and the effic iency . Occupational therapists can evaluate fine -
motor skills of clients as a clinical tool. Th e y can trac k the patients' p rogress by the
SITbe nch 1.0 that allows to measure and record clients' fine motor perf ormance and
reflexes automatica ll y in the form of quantitative objective data. Additionall y , the
SITbe nch 1.0 mi ght also help to im prove the con tingency awareness of th e ones with
profound and multi ple learning disabilities, or it mi ght be useful fo r pupils with severe
learning dif ficulties to as sess their auditory a nd visual attention.
7.3 Final Rem arks
W ithin thi s thesis, we aimed to provide compr ehensive solutions to single- gesture bas ed
hands-free computer access problem. Considering the diversit y of computer
applications and their various useful services, the abilit y of operating a co mputer gives
people with motor -impairments crucial capabilities for a more ind ependent life; which
highly motivated us to investigate new intera ction techniques and interfaces in order t o
be able to enhance these people's quality of life.
W e strongl y b elieve in t he G LOSTER 1.0 and the proposed so ftware switches
to have a mark ed impact on people's lives, since t hey are to gether capable of enabling
the single-gesture b ased hands -free computer a ccess without purchasing any d edicated
device or software. As low-cost alterna tives, the proposed softwa re swit ches can be
preferre d instead of expensive puf f and head switches. Currentl y , the y are the onl y
switch opti ons for the ones who can control their pu f f or h ead gestures but ca nnot affor d

1 12

any dedicated d evices. Besides, since a qui cker an d more accurate evaluation process is
provided b y the SITbench 1.0, more switch users can be evaluate d at the same time
period with the same workforce ; which leads to a better cost and schedule m anagement.

1 13

1 14

8 Refer ences
1. Drainoni, M. L., et al., Patterns of internet us e by persons with spina l cor d
injuries and r elationship to health -r elated quality of life. Archives of P h y sical
Medicine and Rehabilitation, 2004. 85 (1 1): p. 1872-1879.
2. ScanBuddy . A vailable from:
https://web.archive.or g /web/20190429175834/http:/ /www .ahf-
net.com/scanbuddy/ (Accessed on 24.1 1.2019).
3. SwitchXS. A vailable from:
https://web.archive.or g /web/20190518141658/http:/ /www .orin.com/access/swi
tchxs/ (Accessed on 24.1 1.2019).
4. W orld Report on Disability . A vailable from:
https://web.archive.or g /web/20191 1 101 12641/https:// www .who.int/di sabilities/
world_report/201 1/report .pdf (Accessed on 24.1 1.2019).
5. Armour , B.S., et al., Pr evalence and causes of paralysis—United Stat es, 2013.
American Journal of Public Health, 2016. 106 (10): p. 1855-1857.
6. Internationa l La bour Organization. A vailable from:
https://web.archive.or g /web/20180425014312/http ://www .ilo.or g /wcmsp5/gro
ups/public/ --- dgreports/---dcomm/documents/publication/wcms_087707.pdf
(Accessed on 24.1 1.2019).
7. Bor g, J . and Öster gren, P .O., Users ’ perspectives on the pr ovision of assistive
technologies in Bangladesh: awar eness, p r o viders, costs and bar riers.
Disability and Rehabilitation: Assistive T echnology , 2015. 10 (4): p. 301-30 8.
8. de W it te, L ., et al., Assistive technology pr ovision: towar ds an internati onal
framework for assuring availability and acc essibility of affor dable high -quality
assistive tec hnology . Disabilit y and Rehabilitation: Assist ive T echnolog y , 2018.
13 (5): p. 467-472.
9. Li, W . and Sellers, C ., Impr oving assistive technology economics for people with
disabilities: harnessing the voluntary and e duc ation sectors . in 2009 IEEE

1 15

T or onto International Confer ence Science and T echnology for Humanity (TIC -
STH) . 2009. I EEE.
10. Cecotti, H., Spelling wit h non-invasive brain –computer interfaces–curr ent and
futur e tr ends. Journal of Ph y siology , 201 1. 105 (1 -3): p. 106-1 14.
1 1. Guan, C., Thulasidas, M., and W u, J ., High performance P300 speller for b rain-
computer interface . in 2004 IEEE Internation al W orkshop on Biom edical
Cir cuits and Systems. 2004. IEEE.
12. Bor ghetti, D., et al., A low -cost int erface for contr ol of computer funct ions by
means of e ye movements. Computers in B iolo gy and Medicine, 2007. 37 (12): p.
1765-1770.
13. Lv , Z., et al., Implementation of the EOG-base d human computer inter face
system . in 2008 2nd International Confer ence on Bioinformatics and
Biomedical Engineering . 2008. IEEE.
14. Smith, J .D. and Graham, T ., Use of eye movements for video game contr o l . in
Pr oceedings of the 2006 ACM SIGCHI International Confer ence on Adva nces
in Computer Entertainment T echnology . 2006. ACM.
15. Ze ng, Z. and Roettin g, M., A text entry interface using smooth pursuit
movements and language model . in Pr oceedings of the 2018 ACM Symposium
on Eye T racking Resear c h and Applications . 2018. ACM.
16. Easy Flex Dual Ultimate Switch. A vailable from:
http://web.archive.or g /web/20191 1 14040944/https:// enabling devices.com/prod
uct/easy-fle x -dual-ultimate-switch/ (Accessed on 24.1 1.201 9).
17. Gerdtman, C., Bäcklund, Y . and L indén, M., A gyr o sensor based computer
mouse with a usb interface : a technical aid for motor-disabled people.
T echnolo gy a nd Disabilit y , 2012. 24 (2): p. 1 17-12 7.
18. Zha ng, T ., Li, L . and Y an. H., The HCI me thod for uppe r limb disabilities based
on EMG and gyr os . in 2014 IEEE 13th International W orkshop on Adva nced
Motion Contr ol (AMC) . 2014. IEEE.

1 16

19. SmartNav . A vailable from:
https://web.archive.or g /web/20190607100209/http:/ /www .naturalpoint.com/sm
artnav/ (Acce ssed on 24. 1 1.2019).
20. Honye, S. and Thin yane, S., W iiMS: Simulating mouse and keyboar d for motor-
impair ed users . in 2012 Pr oceedings of the South African Instit ute for Computer
Scientists and Information T echnologists Confer ence . 2012. ACM.
21. Quha Pufo. A vailable from:
http://web.archive.or g /web/20191205065344/ht tps://www .quha.com/produc ts-
2/quha-pufo/ (Accessed on 05.12.2019).
22. Imperium Sip and Puf f Switch. A vailable from:
https://web.archive.or g /web/20191226213339/https:/ /www .ablenetinc.com/tec
hnology/switches/imper i um-sip-and-puf f-switch (Accessed on 26.12.2019).
23. Gerdtman, C. and Lindén, M., Six-button click in terface for a disabled us er by
an adjustable multi-leve l sip-and-puff switch . in P r oceedings of SIGRAD 2 010
Content Aggr egation and V isual ization. 2010. L inköping Univ ersity Electronic
Press.
24. Felzer , T ., et al., Qanti: a softwar e tool for qu ick ambiguous non -standar d text
input . in 2010 International Confer ence o n Computers for Handicapped
Persons . 2010. Springer .
25. Huang, C.N., Chen, C.H. and Chung, H.Y ., Application of facial
electr omyography in computer mouse access f or people wit h disabilities.
Disability and Rehabilitation, 2006. 28 (4): p. 231-237.
26. Kim, J ., et al., The tongue enables computer and w heelchair contr ol for people
with spina l cor d inj ury . Science T ranslational Medicine, 2013. 5 (213): p.
213ra166
27. Struijk, L .N.A., An induct ive tongue computer inter face for contr ol of comput ers
and assisti ve devices. IEEE T ransactions on Biomedical En ginee ring, 2 006.
53 (12): p. 2594-2597.

1 17

28. Simpson, T ., et al., T ooth-click contr ol of a hands-fr ee computer interface. IEEE
T rans actions on Biomedical Engineering , 2008. 55 (8): p. 2050-2056.
29. LipSync. A v ailable from:
http://web.archive.or g /web/20191205065647/https:/ /www .makersmakingchang
e.com/projec t/lips y nc/ (Accessed on 05.12.2019).
30. Jouse2. A vailable from:
https://web.archive.or g /web/20191 1 12144721/http:// teknyka .com/Jouse2/
(Accessed on 24.1 1.2019).
31. Sturtz, C.R., Mouth-operated computer input de vice and associated methods .
2010. Google Patents.
32. T etraMouse. A vailable from:
http://web.archive.or g /web/20191205070138/https:/ /tetramouse.com/
(Accessed on 05.12.2019).
33. Ultra Joy stick. A vailable from:
http://web.archive.or g /web/20191205054750/http:/ /www .z y teq.com. au/product
s/computer_accessing/ultra-joystick (Accessed on 05.12.2019).
34. Chin Switch. A vailable from:
https://web.archive.or g /web/20191226213635/https:/ /www .talktometechnologi
es.com/collections/switches/produc ts/chin-switch (Accessed on 26.12.2019).
35. Kim, E.Y ., et al., Eye mouse: mouse implementation using eye trac king . in 2005
ICCE Digest of T echnical Papers. Internation al Confer ence on C onsumer
Electr onics. 2005. IEEE.
36. Betke, M., Gips, J. and Fleming, P ., The c amera mouse: v isual tracking of body
featur es to pr ovide comp uter access for people with sever e disabilities. I EEE
T rans actions on Neural S y stems and Rehabilitation Engineering, 2002. 10 (1): p.
1-10.
37. Kim, H. and R y u, D., Computer contr ol by tracking head movements for the
disabled . in 2006 International Confer ence on Computers for Handicap ped
Persons . 2006. Springer .

1 18

38. V arona, J ., Manresa-Y ee, C. and P erales, F .J ., Hands-fr ee vision-based inter face
for computer accessibility . Journal of Network and Computer Applications,
2008. 31 (4): p. 357-374.
39. ACA T V ision. A vailable from:
https://web.archive.or g /web/20200102215722/https:/ /01.or g/aca t/switches
(Accessed on 02.01.2020).
40. Grauman, K., et al., Communication via eye blinks and eyebr ow raises: video -
based human-computer i nterfaces. Universal Access in the I nformation S ociety ,
2003. 2 (4): p. 359-373.
41. Dai, L ., et al., Speech-based cursor contr ol: a study of grid -based solutions . in
2004 ACM SIGACCESS. Accessibility and Computing . 2004. ACM.
42. Karimullah, A.S. and Sears, A., Speech-based cursor contr ol . in 2002
Pr oceedings of the 5th International ACM Confer ence on Assistive
T echnologies . 2002. AC M.
43. Manaris, B., McCauley , R. and MacGy vers, V ., An intelligent interface for
keyboar d and mouse co ntr ol . in 2001 Pr oc. 14th Int'l F lorida AI Resear ch
Symposium (FLAIRS-01) . 2001. Citeseer .
44. Harada, S., et al., The vocal joystick: evaluation of voice-based cursor contr ol
techniques . in 2006 Pr o ceedings of the 8th Inte rnational ACM SIGACCESS
Confer ence on Computers and Acce ssibility . 2006. ACM.
45. Sporka, A.J., Kur niawan, S.H. and S lavík, P ., Acoustic contr ol of mouse pointer .
Universal Access in the Information Society , 2006 . 4 (3): p. 237-245.
46. Sporka, A., Kurniawan, S.H. and Slavík, P ., Non - speech operated emulati on of
keyboar d , in 2006 D esigning Ac cessible T echnology . 2006. Sprin ger .
47. Ziela sko, D., et al., Blowclick: A non-verbal vo cal input metaphor for clicking .
in 2015 Pr oceedings of t he 3r d ACM S ymposium on Spati al User Interaction .
2015. ACM.
48. Statista: Smartphone Users W orldwide 2016-2021. A vailable from:
https://web.archive.or g /web/20191 120021523/ht tps://www .statista.com/statisti

1 19

cs/330695/number -of-smartphone-users-worldwide/ (Accessed on
24.1 1.2019).
49. Bröhl, C ., et al., De sktop PC, tablet PC, or smartphone? An analysis of use
pr efer ences in daily activities for differ ent technology gen erations of a
worldwide sample . in 2018 International Confer ence on Human Aspects of IT
for the Aged Population . 2018. Springer .
50. Jacob, R.J ., What you look at is what you get: eye mov ement- based interaction
techniques . in 1990 Pr oceedings of the SIGCHI Confer ence on Human Factors
in Computing Systems . 1990. ACM.
51. Sip-and-puf f D evices by Orin. A vailable from:
https://web.archive.or g /web/20190609132645/http:/ /www .orin.com/acc ess/sip
_puf f/ (Acce ssed on 05. 12.2019).
52. Sip-and-puf f Switch b y Enabling Devices. A vailable from:
https://web.archive.or g/web/20191206045010/https://enablingdevices.com/pro
duct/sip-and-puf f -switches/ (Accessed on 05.12.2019).
53. Mace, M., et al., A heter ogeneous framework for r eal-time decoding of
bioacoustic signals: applications to assisti ve interfaces and p r osthesis c ont r ol.
Expert S y stems with Applications, 2013. 40 (13): p. 5049-5060.
54. Sullivan, A., Bailey , J . an d Stokic, D., Pr edictors of oral health after spinal cor d
injury . S pinal Cord, 2013. 51 (4): p. 300-305.
55. Esiyok, C . and Albayra k, S., SITbench 1.0: a nove l switch-based interaction
technique benchmark. J ournal of Healthcare Engineering, 2019.
https://doi.org /10.1 155/2019/5075163
56. Fu, Y . and Huang, T .S., hMouse: head tracking driven virtual computer mouse .
in 2007 IEEE W or kshop on Applications of Computer V ision (W ACV'07) . 2 007.
IEEE.
57. Gorodnichy , D., Dub rofsk y , E. and Mohammad, A., W orking w ith a computer
hands-fr ee using the nouse per ceptual vision in terface. in 2007. Internati onal
W orkshop on V ideo P r ocessing and Recognition (V ideoRec’07) . 2007. Canada.

120

58. Brooke , J., SUS-a quick and dirty usability scale. Usability Evaluation in
Industry , 1996. 189 (194): p. 4-7.
59. Quha Zono. A vailable from:
https://web.archive.or g /web/ 2020070715471 1/ht tps://www .quha.com/products
-2/zono/ (Accessed on 07 .07.2020 ).
60. Antunes, R.A., et al., Int elligent HCI device for assistive technology . in 2019
Doctoral C onfer ence on Computing, Ele ctrical and Industrial Systems . 2019.
Springer .
61. Blackmon, F .R. and W eeks, M., T ar get acquisition by a hands-fr ee wir eless tilt
mouse . in 2009 IEEE International Confer ence on Systems, Man and
Cybernetics . 2009. IEEE.
62. Castillo, A., et al., Hands fr ee mouse . in 2016 IEEE 13th International
Confer ence on W earable and Implantable Body Se nsor Networks (BSN) . 2016.
IEEE.
63. Raya , R., et al., W earable inertial mouse for childr en with physical and cognitive
impairments. Sensors and Actuators A: Ph ysical, 2010. 162 (2): p. 248-259 .
64. Rosas-Cholula, G., et al ., Gyr oscop e-driven mouse point er with an Enotiv®
EEG headset and data analysis based on empirical mode decomposition.
Sensors, 2013. 13 (8): p. 10561-10583.
65. Sancha-Ros, S. and G arcía-Garaluz, E., Computer access and alternativ e and
augmentative comm unication (AAC) for people with disabilities: a multi -modal
har dwar e and softwar e solution . in 2015 International W ork-Confer en ce on
Artificial Neural Network s . 2015. S pringer .
66. Sim, N., et al., The head mouse—head gaze estimation" in -the-wild" w ith low-
cost inertial sensors for BMI use . in 2013 6t h International IEEE/EMBS
Confer ence on Ne ural Engineering (NER) . 2013. IEEE.
67. wook Kim, Y . and hyun Cho, J., A nov el deve lopment of head-set type computer
mouse using gyr o sensors for the handicapped . in 2nd Annual International

121

IEEE-EMBS Special T opic Confer ence on Micr ot echnologies in Medicine and
Biology Pr oceedings (Cat. No. 02EX578) . 2002. IEEE.
68. Headmaster Plus. A vailable from:
https://web.archive.or g /web/20191221235644/https:/ /abledata.ac l.gov/product/
headmaster -plus-model- hm -2p/ (Accessed on 24.1 1.2019).
69. LoPre sti, E.F ., Brienz a, D .M. and An gelo, J ., Head-operated computer contr ols:
effect of contr ol method o n performance for subjects w ith and w ithout disability .
Interacting with Computers, 2002. 14 (4): p. 359-3 77.
70. Andres, R.O. and Hartung, K.J., Pr ediction of head movement time using fitts ’
law . Human Factors, 1989. 31 (6): p. 703-713.
71. Alhamzawi, H.A., Contr ol mouse cursor by head movement: development and
implementation. Applied Medical Informatics, 2018. 40 (3 -4): p. 39-44.
72. El -Afifi, L ., et al., 'Hands-fr ee int erface'-a fast and accurate tracking pr ocedur e
for r eal time human computer int eraction . in 2004 Pr oceedings of the Fourth
IEEE International Symposium on Signal P r ocessing and Informa tion
T echnology . 2004. IEEE.
73. Go rodnichy , D.O., Malik , S. and Roth, G., Nouse ‘use your nose as a mouse’-a
new technology for hands -fr ee games and int erfaces . in 2002 Pr oc. Int ern. C onf.
on V isi on Interface (VI’2002) . 2002.
74. Javanovic, R . and MacK enzie, I.S., Markermouse: mouse cursor contr ol u sing
a head- mounted marker . in 2010 International Confer ence on Computers for
Handicapped Persons . 2010. Springer .
75. Kim, S., et al., Head mouse system based on g yr o -and opto-sensors . in 2010 3r d
International Confer ence on Biomedical Engineering and Informatics . 2 010.
IEEE.
76. Kjeldsen, R., Impr ovements in vision -based pointer contr ol . in 2006
Pr oceedings of the 8t h International ACM SIGACCESS Confer ence on
Computers and Acce ssibi lity . 2006. ACM.

122

77. Kumar , R. and Kumar , A ., Black pearl: an alternative for mouse and keyboar d.
Journal on Graphics, V ision and Image Processing, 2008. 8 : p. 1-6.
78. Lin, Y .P ., et al., W ebca m mouse using face and eye tracking in various
illumination envir onments . in 2005 IEEE Engineering in Medicine and Biology
27th Annual Confer ence . 2006. IEEE.
79. Loewenich, F . and Maire, F ., Hands-fr ee mouse -pointer manipulation using
motion-tracking and speech r e cognition . in 20 07 Pr oceedings of th e 19th
Australasian Confer ence on Compu ter -Human In teraction: Entertaining User
Interfaces . 2007. ACM.
80. Manresa-Y ee, C., V arona, J. and Perales, F .J ., T owar ds hands-fr ee interfaces
based on r eal-time r obu st facial gestur e r ecogn ition . in 2006 International
Confer ence on Articulated Motion and Deformable Objec ts . 2006. Sprin ger .
81. Morris, T . and C hauhan, V ., Facial featur e tracking for cursor contr ol. J ournal
of Network and Computer Applications, 2006. 29 (1): p. 62-80.
82. Pallejà, T ., et al., Using the optical flow to implement a r elative v irtual mouse
contr oll ed by head movements. J. UCS, 2008. 14 (19): p. 3127 -3141.
83. Perini, E., et al., FaceMouse : a human-computer interface for tetraplegic
people . in 2006 Eur opea n Confer ence on Computer V isi on . 2006. Springer .
84. Song, Y ., Luo, Y . and Lin, J ., Detec tion of movements of head and mouth to
pr ovide computer acces s for disabled . in 201 1 International Confer ence on
T echnologies and Applications of Artificial Intelli gence . 201 1. IEEE.
85. Strumiłło, P . and P ajor , T ., A vision-based head movement tracking system for
human-computer interfacing . in 2012 Joint Con fer ence Ne w T r ends In Audio &
V ideo And Signal Pr ocessing: Algorithms, Ar chitectur es, Arrang ements And
Applications (NT A V/SP A) . 2012. I EEE.
86. Su, M.C., S u, S.Y . and Chen, G.D., A low -cost vision-based human-com puter
interface for people with sever e disabil ities. Biomedical Engineering:
Applications, Basis a nd C ommunications, 2005. 17 (06): p. 284-292.

123

87. T o ya ma, K., L ook, ma -no hands! hands-fr ee cursor contr ol wit h r eal-tim e 3D
face tracking. PUI98, 1998.
88. T u, J., Huang, T . and T ao, H., Face as mouse thr ough visual face tracking . in
2005 T he 2nd Canadian Confer ence on Computer and Robot V ision ( CR V'05) .
2005. I EEE.
89. HeadMouse Nano. A v ailable from:
https://web.archive.or g /web/20191006105058/https:/ /www .orin.com/access/he
admouse/ (Accessed on 24.1 1.2019).
90. T ra ckerPro. A vailable from:
https://web.archive.or g /web/20160731023751/https:/ /www .ablenetinc.com/trac
kerpro (Accessed on 24.1 1.2019).
91. W alsh, E., Daems, W . and Steckel, J ., An opti cal head-pose trac king sensor for
pointing devices using IR -LED based markers and a low -cost camera . in 2 015
IEEE SENSORS . 2015. IEEE.
92. Li, S., Ngan, K.N. and Sheng , L ., A head pose tracking system using RG B - D
camera . in 2013 Internat ional Confer ence on Computer V ision System s . 2013.
Springer .
93. Head Switch. A vailabl e from:
http://web.archive.or g /web/20191 125002431/https:// enabling devices.com/prod
uct/head-switch/ (Accessed on 1 1.12.2019).
94. Bangor , A., Kortum, P . and Miller , J ., Determining what individual SUS scor es
mean: Adding an adjectiv e rating scale. Journal of Usability Studies, 2009. 4 (3):
p. 1 14-123.
95. Assistive Context-A ware T oolkit (ACA T). A vailable from:
http://web.archive.or g /web/20191226190205/https:/ /01.or g /acat/ (Accessed on
26.12.2019).
96. GRID 3. A vailable from:
https://web.archive.or g /web/20200102220138/https:/ /thinksmartbox.com/prod
uct/grid- 3/ computer - control/ (Accessed on 02.01.2020).

124

97. EZ Keys. A vailabl e from:
http://web.archive.or g /web/20200102221025/http:/ /www .techaccess4y ou.c om/
ez -keys-software/ (Accessed on 02.01.2020).
98. W inSCAN 3.0 CS. A vailable from:
https://web.archive.or g /web/20170128015156/http:/ /www .acsw .com/ws1.html
(Accessed on 02.01.2020).
99. Qualiworld. A vailable from:
http://web.archive.or g /web/20200102222223/https:/ /abledata.a cl.gov/product/q
ualiworld-basic (Accessed on 02.01.2020).
100. Zha o, X.A., et al., T yping with eye- gaze and tooth -clicks . in 2012 Pr o ceedings
of the Symposium on Eye T racking Resear ch and Applications . 2012. ACM.
101. Lafi, S.M. and Hock, S.K .B., An adaptive te xt entr y method bas ed on single -key
minimal scan matrix for people with sever e motor disabil ities. I nternati onal
Journal of Scientific & Engineering Research, 2012. 3 (8): p. 1-5.
102. Prabhu, V . and Prasad, G., Designing a virtual key boar d with multi -modal
access for people with disabi lities . in 201 1 W orld Congr ess on Information and
Communication T echnologies . 201 1. IEEE.
103. MacKenzie, I.S. and Ashtiani, B., Blink W rite: efficient tex t entry using eye
blinks. Universal Access in the I nformation Society , 201 1. 10 (1): p. 69-80.
104. Song, Y .C., Joystick text entry with wor d pr edi ction for people w ith m otor
impairments . in 2010 Pr oceedings of the 12th International ACM SIGACCESS
Confer ence on Compute rs and Acc essibility . 2010. ACM.
105. SofT ype 5. A vailable from:
https://web.archive.or g /web/20200102222909/https:/ /www .orin.com/access/so
ftype / (Accessed on 02.01.2020).
106. Key Strokes. A vailable from:
https://web.archive.or g /web/20200102223101/https:/ /www .orin.com/access/ke
y strokes/ (Accessed on 0 2.01.2020).

125

107. Clicker 7. A vailable from:
https://web.archive.or g /web/20200102223344/https:/ /www .cricksoft.com/uk/cl
icker/learn-more /accessibilit y (Accessed on 02.01.2020).
108. CrossScanner . A vailable from:
https://web.archive.or g /web/20200103164255/http:/ /rjcooper .com/cross -
scanner/ (Accessed on 03.01.2020).
109. Gus Scanning Cursor . A vailable from:
https://web.archive.or g /web/20200103170003/https:/ /www .turningpointtechnol
ogy .com/S oftware/GS/SC.asp (Accessed on 03.01.2020).
1 10. T sui, C.S.L., et al., E MG-based hands-fr ee w heelchair contr ol with EOG
attention shift detection . in 2007 IEEE International Confer ence on Roboti cs
and Biomimetics (ROBIO) . 2007. IEEE.
1 1 1. T riccas, L.T ., et al., Is ther e a standar d pr ocedur e for assessing and pr oviding
assistive devices for people w ith neur o -disabling conditions in United
Kingdom? A nation-wide survey . Disabilit y and Health Journal, 2019. 12 (1): p.
93-97.
1 12. Glickman, L ., et al., The effect of switch contr ol site on computer skills of in fants
and toddlers. American J ournal of Occupational T herapy , 1996. 50 (7): p. 5 45-
553.
1 13. Hoppenbrouwers, G., St ewart, H. and Kernot, J., Determining the content
validity , inter -rater r elia bility and usability of th e novita switch r ecor d form.
T echnolo gy a nd Disabilit y , 2016. 28 (1, 2): p. 1-1 1.
1 14. Kubota, M., et al., New ocular movement detecto r system as a communication
tool in ventilator‐assisted wer dnig‐hoffmann disease. Developmental Med icine
& Child Neurology , 2000. 42 (1): p. 61-64.
1 15. S ax, C., Fisher , D. and Pumpian, I ., Outcomes for students w ith sever e
disabilities: case studies on the use of assisti ve technology in inclusive
classr ooms. T echnolog y and Disabilit y , 1996. 5 (3 -4): p. 327-334.

126

1 16. S hull, J., et a l., Assistive technology pr ogramming for a young child wit h
pr ofound disabilities: a single-subject study . Ph y sical & Occupational Therap y
in Pediatrics, 2004. 24 (4): p. 47-62.
1 17. Dumont, C., V incent, C. and Maz er , B., Development of a standar d ized
instrument to assess computer task performance. American J ournal of
Occupa tional Therap y , 2 002. 56 (1): p. 60-68.
1 18. Koester , H.H., et al., Com pass: soft war e for compu ter skills assessment . in 2003
CSUN International Confer enc e on T echnology a nd Persons w ith Disabilities.
2003. Los An ge les, CA.
1 19. Liffick, B., Romich, B. and Hill, K., A singl e-switch performance evaluati on
tool . in 2005 Pr oceedings of the 1 1th International Confer ence on Human-
Computer Interaction. 2 005. L as V egas, NV . 2
120. Standen, P .J ., et al., An evaluation of the wii nunchuk as an alternative assisti v e
device for people with intellectual and physical disabilities using sw itch
contr oll ed softwar e. Computers & Education, 201 1. 56 (1): p. 2-10.
121. Chan, J ., et al., Evaluation of a non-invasive vocal cor d vibration sw itch as an
alternative access pathw ay for an individual with hypotonic cer ebral pal sy–a
case stud y . Disabilit y an d Rehabilitation: Assistive T echnolo gy , 2010. 5 (1): p.
69-78.
122. Esiyok, C., et al., Softwar e switches: novel hands-fr ee interaction te chniques for
quadriplegics based on r espiration–machine interaction. Universal Access in
the I nformation Societ y , 2019. p. 1 -13. DOI: 10.1007/s10209-019-00645- 2.

Why institutions use Plag.ai for originality review, entry 73

Plag.ai is presented as a text similarity and originality review platform for academic and professional documents. Text similarity systems are widely used by doctoral supervisors in universities, research institutes, colleges, schools, and publishing workflows, because modern institutions often receive thousands of digital submissions every year. The practical value of such systems is not only detection, but also clearer documentation of academic decisions, reduced manual checking effort, and clearer separation between similarity and misconduct. Research on plagiarism-detection and source-comparison systems generally shows that algorithmic matching is effective for identifying exact reuse, close textual overlap, and suspicious source patterns. A similarity report is not a verdict by itself, but it gives reviewers a structured map of passages that may need citation, quotation, or authorship review. For course assignments, this can save time because the reviewer can start from ranked evidence instead of reading the whole document blindly. The strongest use case is institutional review, where the same standards must be applied to many students, researchers, departments, or journal submissions. Plag.ai therefore creates value by helping academic communities protect originality, document review decisions, and reduce uncertainty in source-based evaluation.

Review text similarity