Human Computer Interaction Via BrainWaves for Disable People

HUMAN COMPUTER INTERACTION VIA BRAINWAVES FOR DISABLED PEOPLE LOURENÇO BARBOSA DE CASTRO Faculty of Engineering – University of Porto September 2013 HUMAN COMPUTER INTERACTION VIA BRAINWAVES FOR DISABLED PEOPLE Thesis submitted in partial fulfillment of the requirements for the degree of Master of Science in Biomedical Engineering. Faculty of Engineering – University of Porto DEVELOPED BY Lourenço Barbosa de Castro Bachelor of Science in Biological Scien ces Catholic University of Portugal ORIENTATION João Manuel R.S. Tavares Associate Professor Faculty of Engineering – University of Porto, Department of Mechanical Engineering ACKNOWLEDGEMENTS To my family and friends, for all the support given throughout this project’s duration. To Fraunhofer Portugal, Prof. João Manuel Tavares and Prof. Ana Maria Mendonça for giving me the opportunity to be a part of this project. Additional acknowledgement to Fraunhofer Portugal for providing all the required resources and funding. To Luis Rosado, for all the help, support and availability during this last year. ABSTRACT The idea of interacting with computers using only the human brain quickly emerged after the discovery of electroencephalography in 1929. Although plenty of research has been done related to this subject, and several consumer based products are already being sold, little importance has been given to mobile devices, which are rapidly dominating the personal computing scene. With this in mind, an Android application was developed in this project, attempting to facilitate the usage of mobile computers for disabled people. Using exclusively electric activity acquired from a brain computer interface device, different types of interactions were experimented, expecting to improve the mentioned people's autonomy and quality of life. The users of the application were able to control several features just by winking, which achieved over 87% success rate after a short calibration session. Additionally, the application was able to discriminate the users' state of mind when subjected to different conditions. Four situations were tested, all of them exhibiting unique neural oscillations patterns, making the application accurate in understanding how the user was feeling. To conclude, the main contribution of this project was creating a type of interaction that was not previously available, and, although more limited, presented a higher success rate than the chosen device's native application. RESUMO A ideia de estabelecer interação com computadores recorrendo apenas ao cérebro humano surgiu rapidamente após a descoberta da eletroencefalografia, em 1929. Apesar de ter sido realizada bastante investigação sobre este tema, e de diversos produtos estarem comercialmente disponíveis, pouca importância tem sido dada à sua adaptação a dispositivos móveis, dispositivos esses que estão rapidamente a dominar o mercado dos computadores pessoais. Com isto em mente, uma aplicação Android foi desenvolvida neste projeto, com o intuito de facilitar a utilização de computadores móveis a pessoas incapacitadas. Utilizando apenas sinais elétricos obtidos por um dispositivo de comunicação entre o cérebro e o computador, diferentes tipos de interações foram testadas, na tentativa de melhorar a autonomia e qualidade de vida das pessoas anteriormente referidas. Os utilizadores desta aplicação podem controlar várias funcionalidades apenas pestanejando, o que, após uma curta sessão de calibração, atingiu uma taxa de sucesso superior a 87%. Adicionalmente, a aplicação desenvolvida foi capaz de diferenciar o estado de espírito dos utilizadores, quando sujeitos a diversas condições específicas. Neste caso, quatro situações foram testadas, em que todas elas apresentaram padrões de atividade cerebral distintos, tornando esta aplicação eficaz a perceber como o seu utilizador se sente. Concluindo, a principal contribuição deste projeto foi criar um tipo de comunicação que não era possível anteriormente, entre um dispositivo de interação cérebro-computador e um tablet Android. Apesar de mais limitado do que aplicação nativa desse mesmo dispositivo, apresentou uma maior taxa de sucesso nas funcionalidades implementadas. II LOURENÇO BARBOSA DE CASTRO – 2013 INDEX 1. INTRODUCTION 1 1.1. MOTIVATION 2 1.2. OBJECTIVES 3 1.3. STRUCTURE 4 1.4. CONTRIBUTIONS 6 2. BRAIN-COMPUTER INTERFACES 7 2.1. INTRODUCTION 8 2.1.1.RESEARCH 8 2.1.2 INDUSTRY 10 2.2. BRAIN 12 2.2.1. LOBES 12 2.2.1.1. FRONTAL LOBE 12 2.2.1.2. PARIETAL LOBE 13 2.2.1.3. OCCIPITAL LOBE 13 2.2.1.4. TEMPORAL LOBE 13 2.2.2. NEURONS 14 2.3. ELECTROENCEPHALOGRAPHY 15 2.3.1. INTRODUCTION 15 2.3.2. ELECTRODES 15 2.3.3. POSITIONS 16 2.3.4. NOISE 17 2.3.5. NEURAL OSCILLATIONS 18 2.3.5.1. DELTA 18 2.3.5.2. THETA 18 2.3.5.3. ALPHA 19 2.3.5.4. MU 19 2.3.5.5. BETA 20 2.4. ELECTROMYOGRAPHY 20 2.5. SIGNALS DISCRIMINATION 21 2.5.1. EVENT-RELATED POTENTIALS 22 2.5.2. P300 POTENTIAL 22 2.5.2.1. METHODOLOGY 22 2.5.2.2. IMPLEMENTATION 23 2.5.3. COGNITION/EXPRESSIONS 24 HUMAN COMPUTER INTERACTION VIA BRAINWAVES FOR DISABLED PEOPLE III 2.6. TYPES OF BCIS 25 2.6.1. INVASIVE 25 2.6.2. NON-INVASIVE 26 2.7. SUMMARY 27 3. EMOTIV EPOC 29 3.1. INTRODUCTION 30 3.2. HARDWARE 31 3.3. TYPES OF INTERACTION 32 3.3.1. EXPRESSIV SUITE 32 3.3.2. COGNITIV SUITE 33 3.3.3. AFFECTIV SUITE 33 3.4. SUMMARY 33 4. DEVELOPED APPLICATION 35 4.1. TARGET USERS 36 4.2. ANDROID OPERATING SYSTEM 37 4.2.1. OPERATING SYSTEM 37 4.2.2. NEXUS 7 TABLET DEVICE 37 4.3. FEATURES 38 4.3.1. TIME DOMAIN FEATURES 38 4.3.2. FREQUENCY DOMAIN FEATURES 38 4.3.3. SPATIAL FEATURES 39 4.4. SOFTWARE 39 4.4.1. DATA ACQUISITION 39 4.4.2. EVENTS DETECTION 40 4.4.3. FUNCTIONALITIES 40 4.4.3.1 EYES NAVIGATION 41 4.4.3.2. KEYBOARD 41 4.4.3.3. CONTACTS 42 4.4.3.4. CALCULATION 43 4.4.3.5. BRAINWAVES 44 4.4.3.6. CONFIGURATION 45 4.5. APPLICABILITY 46 4.5.1. COMPUTER CONTROL 46 4.5.2. COGNITIVE ORIENTATION 47 4.5.3. BRAINWAVES MONITORING 48 4.5.3.1. ALERTS 48 4.5.3.2. FOCUS 48 IV LOURENÇO BARBOSA DE CASTRO – 2013 4.5.3.3. ENGAGEMENT 48 4.6. SUMMARY 49 5. RESULTS AND DISCUSSION 51 5.1. WINKS 52 5.2. BRAINWAVES 54 5.3. SUMMARY 60 6. CONCLUSION AND FUTURE IMPROVEMENTS 63 6.1. CONCLUSION 64 6.2. FUTURE IMPROVEMENTS 64 BIBLIOGRAPHY 67 HUMAN COMPUTER INTERACTION VIA BRAINWAVES FOR DISABLED PEOPLE V TABLES TABLE 1 32 EMOTIV EPOC'S HARDWARE SPECIFICATIONS (EMOTIV, EMOTIV SYSTEMS WEBSITE 2013) TABLE 4 54 ELECTRIC ACTIVITY MEASURED WHILE THE SUBJECT WAS WINKING THEIR LEFT EYE. TABLE 5 55 RESULTS FOR THE RIGHT WINKING TRIALS, SHOWING THE RATE OF SUCCESS FOR EACH CALIBRATIONS AMOUNT TABLE 6 55 RESULTS FOR THE LEFT WINKING TRIALS, SHOWING THE RATE OF SUCCESS FOR EACH CALIBRATIONS AMOUNT TABLE 7 63 RESULTS OF THE BRAINWAVES TRIALS, COMPARING THE TOP ROW WITH THE LEFT COLUMN STATE. BOTH HIGH AND LOW PEAKS ARE DENOTED, AS WELL AS OTHER INTERESTING FEATURES. FIGURES FIGURE 1 14 REPRESENTATION OF A NEURON. 1: DENDRITES; 2: CELL BODY (SOMA); 3: NUCLEUS; 4: AXON (FORESMAN 2009). FIGURE 2 16 EXAMPLE OF THE INTERNATIONAL 10-20 SYSTEM, DISPLAYING THE LAYOUT OF THE ELECTRODES ON A SUBJECT'S HEAD (FIELDTRIP 2013). FIGURE 3 18 REPRESENTATION OF 1 SECOND OF EEG ACTIVITY FROM THE OZ POSITION, FILTERED TO DISPLAY ONLY DELTA WAVES (GAMBOA 2005). FIGURE 4 18 REPRESENTATION OF 1 SECOND OF EEG ACTIVITY FROM THE OZ POSITION, FILTERED TO DISPLAY ONLY THETA WAVES (GAMBOA 2005). FIGURE 5 19 REPRESENTATION OF 1 SECOND OF EEG ACTIVITY FROM THE OZ POSITION, FILTERED TO DISPLAY ONLY ALPHA WAVES (GAMBOA 2005). FIGURE 6 20 REPRESENTATION OF 1 SECOND OF EEG ACTIVITY FROM THE OZ POSITION, FILTERED TO DISPLAY ONLY BETA WAVES (GAMBOA 2005). FIGURE 7 25 REPRESENTATION OF THE BRAINGATE TECHNOLOGY, A TYPE OF INVASIVE BCI (BRAINGATE 2009). FIGURE 8 26 EMOTIV EPOC, A TYPE OF NON-INVASIVE BCI EQUIPMENT (EMOTIV, EMOTIV SYSTEMS WEBSITE 2013). CHAPTER 1 INTRODUCTION 4 LOURENÇO BARBOSA DE CASTRO – 2013 conditions can inflict, in order to present efficient solutions based on brain-computer interaction. Trials and future prospects Lastly, since this field is still heavily underdeveloped considering its already demonstrated potential, this project aimed to study the future possibilities of interacting with mobile computers employing exclusively bioelectric signals. Also, by developing and demonstrating some basic brain-computer interface functionalities, this project hopes to help opening doors to more elaborate and life-changing creations. STRUCTURE 1.3 HOW THE THESIS IS ORGANIZED Chapter 1 This thesis is initiated with an introductory chapter, in which it is presented the motivation behind the subject’s choice, as well as its objectives and contributions to the state of the art. This chapter was created as succinct as possible, leaving most theoretical concepts to be described on the subsequent chapters. Chapter 2 Brain-Computer Interfaces In chapter 2, the topic of brain-computer interfaces is introduced. The initial experiments regarding this type of interaction are described, mentioning some of the most prominent people involved. It is also given emphasis to the commercially focused devices, both available and in development. Then, the required precursor technologies and concepts are described, including brain regions and corresponding functions, electrodes positioning and neurophysiological signals and their adequate detection techniques. CHAPTER 1 INTRODUCTION HUMAN COMPUTER INTERACTION VIA BRAINWAVES FOR DISABLED PEOPLE 5 Chapter 3 Emotiv EPOC Afterwards, the specific device used in this work is presented, in contrast with other devices and projects. The main objective of this chapter is to situate the headset used in this project along with the current state-of-the-art brain-computer interfaces technologies, describing all of its features, both physical and digital, and arguing why it was selected among other available products. Chapter 4 Developed Application The next chapter is dedicated to the developed application. Initially, the usability factors are addressed, explaining for whom the application was designed and the types of disabilities and diseases that should benefit from it. The different types of useful signal features are introduced, along with their detection and classification procedures. Then, both the Android platform and the device used in this project are briefly introduced, explaining their specifications and why they were chosen. Lastly, the software characteristics are discussed, explaining the data acquisition methods and how the electrophysiological signals were used to control the application’s functions, finishing with practical examples of how this technology could be used. Chapter 5 Results and Discussion Following is the results and discussion chapter, where firstly it is explained how the technology was tested and which results were obtained. The validity and accuracy of the results are hereby discussed Chapter 6 Future Improvements and Conclusions The last chapter explains the conclusions of this project, and makes use of such assertions to comprehensively prospect possible future improvements to the developed application and, on a broader extent, to the brain-computer interfaces field. CHAPTER 1 INTRODUCTION 6 LOURENÇO BARBOSA DE CASTRO – 2013 CONTRIBUTIONS 1.4 WHERE DOES THIS PROJECT STAND COMPARED TO THE STATE OF THE ART Adaptation to mobile devices The main contribution of this thesis is the adaptation of functional brain-computer interfacing to mobile systems, resulting in an interaction between an electroencephalography (EEG) headset and an Android tablet computer. By contributing as a proof-of-concept of brain-computer interface technology applied to mobile computers, this project hopes to motivate the development of similar and more complex implementations using alternative devices. Additionally, by experimenting with basic forms of BCI interaction, it is expected that more elaborate and useful techniques can surface in this greatly promising field. Rehabilitation features Although some available devices already have tools to connect to mobile computers, they are all considerably more limited than the equipment used in this project, making the interaction consequentially less interesting. Even though very efficient and well-designed software is available for said devices, the implemented features do not take rehabilitation in consideration, leaving much to be desired. This project is directly related to that subject, attempting to build functionalities that could help disabled people regain their autonomy. Brain-Computer Interfaces efficiency Furthermore, a major electronics manufacturer is testing the possibilities of using braincomputer interfaces with their own mobile devices, which should vastly increase the public interest in this subject. However, their study is still on very early stages of trials, and high grade EEG headsets are being used. On the other hand, this project’s implementation is already functional, using a cheaper device already available on the market for a few years. 7 CHAPTER 2 BRAIN-COMPUTER INTERFACES This chapter introduces the discovery of electroencephalography, which made interacting with computers using brainwaves possible. Next, research and commercial approaches are described, mentioning important scientists and companies that have been increasingly developing the BCI field. The human brain is briefly described, with the most relevant structures and physiological aspects being mentioned. Also, both EEG and electromyography are specifically explained, as well as some other neurophysiological signals and their interpretations. 2.1. INTRODUCTION 8 2.2. BRAIN 12 2.3. ELECTROENCEPHALOGRAPHY 15 2.4. ELECTROMYOGRAPHY 20 2.5. SIGNALS DISCRIMINATION 21 2.6. TYPES OF BCIS 25 2.7. SUMMARY 27 CHAPTER 2 BRAIN-COMPUTER INTERFACES 8 LOURENÇO BARBOSA DE CASTRO – 2013 INTRODUCTION 2.1 THE BEGINNING OF BRA IN-CO MPUTER INTERFACES. IMPORTANT PEOPLE AND COMPANIES. After the discovery of electroencephalography in 1929, by Hans Berger (Berger 1929), the possibility of using brainwaves to communicate or even interact with the environment became a reality. The so called brain-computer interfaces are best described as communication systems which do not depend on the brain’s regular pathways, such as peripheral nerves or muscles (Wolpaw, et al. 2000). This creation led to diverse research milestones, as well as several, well established BCI companies. Both of these fields are described in this chapter. RESEARCH 2.1.1 The earliest publications directly related to brain-computer interfaces were developed by José M. R. Delgado, during the decade of 1960. His work consisted in stimulating and recording the electrical activity of the brain in unrestrained monkeys and chimpanzees, using bilaterally implanted electrodes assemblies to read the electrophysiological signals and a radio frequency system to induce stimulation. This allowed for a better understanding of cerebral behavior and spontaneous activities while the brain was not suffering any kind of disturbance to its typical functioning. The motivation behind this study was to better understand brain dysfunctions associated with behavioral abnormalities, which were diagnosed using restrictive methods that would not produce sufficiently realistic results (Delgado, et al. 1968). In 1980, focusing on the neurophysiology of paralyzed patients, a study elaborated by Edward M. Schmidt tried to implement long-term connections to the central nervous system, using microelectrodes. This could allow for direct stimulation of paralyzed limbs from the motor cortex, ideally restoring movement. Although it was concluded that additional improvements would be required for feasible implementations, the obtained results indicated information transfer rates only moderately slower than in the regular physiological pathway (Schmidt 1980). CHAPTER 2 BRAIN-COMPUTER INTERFACES HUMAN COMPUTER INTERACTION VIA BRAINWAVES FOR DISABLED PEOPLE 9 The first published demonstration of a brain-controlled device came in 1999, developed by a group led by Miguel A. L. Nicolelis. Initially, brain activity of a group of mice was recorded while they pressed a lever to position a robotic arm that allowed them to obtain water. Next, this activity pattern was associated with the robotic arm control, triggering it when the brain signals were recognized. Finally, a subgroup of the studied mice started to routinely and exclusively use this brain activity to reach their goal, and, as the training progressed, resorting to the initial lever diminished or stopped. This study’s findings raised the possibility for paralyzed patients to control external devices or even their own muscles using only electrophysiological patterns (Chapin, et al. 1999). During 2008, the first brain-computer interfaces mainstream report was presented by Columbia Broadcasting System (CBS), on an episode of their news program 60 Minutes. Several interviews were conducted to people whose lives were changed by this technology. A man diagnosed with amyotrophic lateral sclerosis (ALS), who lost all types of movement control except for his eyes, managed to start communicating again. This was possible using a non-invasive system developed by Jonathan Wolpaw, by selecting characters on a screen using only EEG information. Another presented major development was the Braingate project, created by John Donoghue’s team, consisting in implanting a grid of electrodes on the patient’s motor cortex. This allowed for a stroke victim, who lost control of her body, to move a computer cursor using brain activity readings, which was later adapted to the steering of an electric wheelchair (CBSNews 2008). Recently, the already mentioned Braingate project presented another breakthrough: again using a microelectrode array implanted in the motor cortex, two patients were able to control a robotic arm with three-dimensional reach and grasp functions. Such implementation allowed one of them to autonomously drink coffee from a bottle, task which would be impossible otherwise. This was achieved by recording neural activity while the subject was imagining the referred movements, and afterwards triggering the robotic arm control when the same patterns were detected. John P. Donoghue’s team, once again, developed this work (Hochberg, et al. 2012). Lastly, and still only reproducible in mice, Miguel A. L. Nicolelis group published in 2013 the results of an experiment where brain-to-brain information was shared. The cerebral activity CHAPTER 2 BRAIN-COMPUTER INTERFACES 10 LOURENÇO BARBOSA DE CASTRO – 2013 of one rat was recorded while performing a simple visual identification that would produce a reward. Then, this activity was transmitted, via intracortical microstimulation, to a second rat, making it perform the correct identification without any visual cues. This study hoped to help the development of new types of social interaction and biological computing devices (Pais-Vieira, et al. 2013). INDUSTRY 2.1.2 From a commercial perspective, several companies have been established on the braincomputer interfaces field, mainly focusing on low cost devices for common users. The main goal of this approach is to introduce BCI technology as a gaming controller option, which can require less EEG quality and present low risk interactions. Even though said companies exist since, at least, 2004, none of them have obtained a mainstream position, fact that retains brain-computer interfaces in a “science-fiction” status. However, the low price tags and increasingly decent quality of this grade of devices attracted the attention of the research community, bringing us closer to a future of easily accessible neural rehabilitation tools. Following are a few of those companies, worth mentioning for their initiative, impact and philosophy. NeuroSky As previously mentioned, in 2004, one of the first BCI dedicated companies joined the field: NeuroSky. Although its primary focus was developing dry sensors for industry partners, this company currently has a few devices of their own. Using a single dry sensor, both raw brainwaves and neural oscillation bands can be accessed, and, by means of a dedicated online store, several games and other applications can be used. This company provides both low cost and medical grade EEG technology, making a wide range of partnerships possible, from gaming and toys makers to academic research groups (NeuroSky 2012). CHAPTER 2 BRAIN-COMPUTER INTERFACES HUMAN COMPUTER INTERACTION VIA BRAINWAVES FOR DISABLED PEOPLE 11 InteraXon In 2007, after working together on brainwave-controlled art installations, a small group of people founded Interaxon, a company simply dedicated to commercialize thought controlled computing, trying to make it a part of everyday life. Currently, they sell a relatively discreet headband, featuring 6 dry sensors, which is mainly used as a neural coach to monitor and to optimize the brain's electrical oscillations (InteraXon 2010). Emotiv Lastly, the Australian company Emotiv was founded in 2003 by a small group of scientists and entrepreneurs. Their device was designed with gaming in mind, using 14 electrodes to detect features such as emotional states, facial expressions and even conscious thoughts. It is also the only consumer-grade BCI equipment that includes a two-axis gyroscope. Additionally, as referred on their official website, the company wants to “democratize” brain research, providing more accessible tools to everyone, objective which they are slowly achieving (Emotiv 2013). Currently, more devices are being developed, which, soon, should be comparable to medical grade equipment, helping computers understand more efficiently how we think and who we really are. Also, major advances are taking place on the field of human brain emulation, making the creation of even better and more complex BCI devices possible. Another point worth noting is that major companies on this field are using an online applications store approach for their devices and making development kits available to the general public. This should help boosting the technology even further, as people with little to no BCI knowledge can easily gain interest or even take part in this subject (Emotiv 2013, NeuroSky 2012). CHAPTER 2 BRAIN-COMPUTER INTERFACES 12 LOURENÇO BARBOSA DE CASTRO – 2013 BRAIN 2.2 The brain is the most complex structure known to man. It is the organ where most of the exterior stimuli are processed and translated into functions. Although apparently presented as a mostly homogeneous mass of neurons, the brain is divided in distinct regions, each with considerably different purposes. LOBES 2.2.1 Brain lobes designate paired structures, symmetrically appearing on each hemisphere. Originally used exclusively as anatomic divisions, they are now also used to classify the functional regions of the brain, with names given according to their location. Therefore, the electric activity measured near each pair of lobes should be correlated with their function, which helps the development of new types of brain-computer interaction. Likewise, absent or lowered activity should help diagnosing medical disorders originated on these anatomical structures (Yalçin 2000, Stuss 1992). Frontal Lobe 2.2.1.1 The frontal lobe is a structure associated with higher cognitive functions, such as intelligence and voluntary actions, located frontally in each hemisphere. This lobe suffered a very noticeable enlargement on humans as a consequence of evolution, being usually correlated to our cognitive superiority compared to lesser primates (Semendeferi, et al. 2002). This is the region that manages attention, comprehension and language, which also implies a connection to diseases like Attention Deficit Hyperactivity Disorder (ADHD), schizophrenia and bipolarity. Moreover, damages to the frontal lobe result in various changes in emotional response and personality, such as mood disorders, adversities planning the future and reduced attention span (Stuss 1992). CHAPTER 2 BRAIN-COMPUTER INTERFACES HUMAN COMPUTER INTERACTION VIA BRAINWAVES FOR DISABLED PEOPLE 13 Parietal Lobe 2.2.1.2 Behind the frontal lobe, on the top section of the brain, is positioned the parietal lobe. This region integrates different senses in order to build a coherent picture of the surrounding world. Using the ventral and dorsal visual pathways, it can compute vision for perception and vision for action, respectively, which, consequentially, makes it possible for human beings to coordinate their movements with the environment. This is also the region that processes somatosensory information, such as perceiving temperature or pain. Disorders related to this brain lobe include loss or deterioration of the previously mentioned functions and also the ineptitude to tell from right or left and the inability to recognize faces or the surrounding environment (Culham 2006). Occipital Lobe 2.2.1.3 Located in the rear section of the human skull is the smallest of the paired lobes, the occipital lobe. This structure is the primary visual area of the brain, involved both in high and low level visual processing, including recognition of objects and faces and their spatial localization. Different information, like color, orientation and motion, is processed separately by different groups of neurons. This region is strongly correlated with photosensitive epilepsy, which causes abnormal responses in brain activity to visual stimulation, like television flickering (Yalçin 2000). This is also an important region to extract information related to visual evoked potentials (Weinstein 1977), which is a powerful tool for brain-computer interaction (Allisona, et al. 2008). Temporal Lobe 2.2.1.4 Beneath the frontal and parietal lobes, appears the temporal lobe, which consists of a large number of substructures with a wide range of functions. The most notable section is the hippocampus and its adjacent regions, essential for long-term declarative memory (Squire 2001), although it is acknowledged that this is not the only brain structure related to memory. Additionally, it is hypothesized that this region presents a strong involvement in CHAPTER 2 BRAIN-COMPUTER INTERFACES 20 LOURENÇO BARBOSA DE CASTRO – 2013 The Event-related Desynchronization of this wave may be used in BCIs. Groups of neurons at rest tend to fire in synchrony, so when a user is requested to perform a specific action, the resulting desynchronization can be analyzed by a computer (Nyström, et al. 2010). Beta (13-30 Hz) 2.3.5.5 This activity (Figure 6) represents an index of cortical arousal, being related to attention, perception and cognition, presenting a desynchronization shortly after an external stimulus takes place. Also, the beta oscillation is a well-known indicator of movement preparation. It is found on both sides with symmetrical distribution, most evident frontally (Rangaswamya, et al. 2002, Zhang, et al. 2008). Figure 6 - Representation of 1 second of EEG activity from the Oz position, filtered to display only beta waves (Gamboa 2005). ELECTROMYOGRAPHY 2.4 Muscle cells activity depends greatly on their electrical activation, which is generally provoked neurologically. This activity releases electrical potential, which can be detected and measured using appropriate tools. There are currently two methods to perform this measurement, intramuscular and surface EMG. Although inserting a needle directly inside a patient’s muscle can result in high quality characterization of the electric activity, it can be considered too invasive in terms of simple human-computer interaction. Additionally, this method may even be too precise outside medical research purposes, since it gathers information from only a few fibers. As an alternative, making these measurements on the surface of the muscle of interest’s region turns out to be sufficiently effective in order to detect CHAPTER 2 BRAIN-COMPUTER INTERFACES HUMAN COMPUTER INTERACTION VIA BRAINWAVES FOR DISABLED PEOPLE 21 activity, even though the high power of the EMG signals tend to make them propagate far from their origin (Gomez-Gil, et al. 2011). Most EEG based systems pick up, inevitably, electrical myography signals, generated in the facial and neck muscles. However, these are usually treated as artifacts, being carefully removed from the final outcome. Systems such as the Emotiv EPOC are trying to make use of that data, interpreting it in order to detect facial expressions such as blinks, winks, frowns and smiles. The usefulness of this type of detection is diverse. Combinations of facial muscles can be translated into computer commands, both for disabled people with movement impairments and for avid technology users, searching for novel forms of interaction. Specific facial expressions, like smiling, can be associated with mental states, such as joy or happiness, helping computers understand one’s behavior. SIGNALS DISCRIMINATION 2.5 Ideally, a BCI system would directly detect every wish, intention and reaction of its user, based on its brain activity. To allow for discrimination of different neurophysiologic signals or to map such signals to movements, users have to acquire conscious control over their brain activity. Two fundamentally different approaches exist to achieve this. In the first, subjects perceive a set of stimuli displayed by the BCI system and can control their brain activity by focusing onto one specific stimulus. The changes in neurophysiologic signals resulting from perception and processing of stimuli are termed event-related potentials (ERPs). In the second approach, users control their brain activity by concentrating on a specific mental task, e.g., the imagination of hand movement can be used to modify activity in the motor cortex. In this approach, feedback signals are often used to help subjects learning the production of easily detectable patterns of neurophysiologic signals. CHAPTER 2 BRAIN-COMPUTER INTERFACES 22 LOURENÇO BARBOSA DE CASTRO – 2013 EVENT-RELATED POTENTIALS 2.5.1 ERPs are stereotyped, spatio-temporal patterns of brain activity, occurring time-locked to an event, e.g. after presentation of a stimulus, before execution of a movement or after the detection of a novel stimulus. Traditionally, ERPs are recorded with EEG and have been used in neuroscience for studying the different stages of perception, cognition and action. It is worth noting that event-related changes can also be measured with other signal acquisition techniques like the MEG or fMRI. P300 POTENTIAL 2.5.2 The P300 is a positive deflection in the human EEG of about 2-5uV, appearing approximately 300ms after the presentation of a rare or surprising, task-relevant stimuli. This is an endogenous event-related potential, which means it is caused by a late and conscious processing of stimuli, and depends mainly on the stimulus content. It can be reliably measured, generally observed over centro-parietal brain regions (Fz, Cz and Pz regions [Figure 2]), and its waveform can be influenced by various factors. If the user is not concentrated enough, though the P300 wave might disappear completely, and if the task is too difficult, the latency of the wave increases while its amplitude decreases. Although P300 can be evoked using all five basic human senses, the most used, for practical reasons, are auditory and visual. It is also worth mentioning that this kind of potential can be detected on nearly all healthy users, requiring almost no training. Methodology 2.5.2.1 Generally, to evoke P300, subjects are asked to observe a random sequence of two kinds of stimuli. One stimulus type (the oddball, or target stimulus) appears only rarely in the sequence, while the other stimulus type (the normal, on non-target stimulus) appears more often. In short, whenever the target stimulus happens, P300 can be observed in the EEG. CHAPTER 2 BRAIN-COMPUTER INTERFACES HUMAN COMPUTER INTERACTION VIA BRAINWAVES FOR DISABLED PEOPLE 23 Another method is to use three types of stimuli, in which one of them is a, so called, distracter stimulus, which also appears in a sequence with target and non-target stimuli, but less frequently. Also, the subject is usually not informed about this stimulus, to increase the element of surprise of a non-target element. After presentation of many distracter stimuli, the related wave’s amplitude decreases, while the target P300 wave remains unaffected. This stimuli interaction requirement presents as the main drawback of the P300 potential, since the user must pay attention to a specific source to perform brain-computer communication. Several negative and positive components precede the P300, being P300 the highest positive deflection of these components. The P300 peak is inversely related to the probability of the evoking stimulus, being 10% a good probability value to evoke this potential. Also, if many nontarget stimuli precede the target stimulus, a higher amplitude is observable in this potential. The amplitude of the P300 wave is positively correlated to the interstimulus interval, i.e. the amount of time between two consecutive stimuli. Theoretically, longer intervals might yield better results, because the P300 amplitudes are larger. On the other hand, this requires longer duration of runs, requiring longer periods of concentration, which might prove to be difficult for some disabled subjects. Overall, longer interstimulus intervals might decrease P300 amplitude and classification accuracy. Moreover, higher bitrates are obtained with shorter intervals, but if the stimuli are made too short, subjects with cognitive deficits might have problems to detect all the targets. In conclusion, an optimal interstimulus interval for P300-based BCIs can only be determined experimentally. Additionally, the P300 system works in such way that the user doesn't have to perform any training at all, besides being instructed to keep their focus on a specified stimulus. Implementation 2.5.2.2 Farwell and Donchin were the first to employ the P300 as a control signal in a BCI (Farwell and Donchin 1988). They described the P300 speller in which a matrix containing symbols CHAPTER 2 BRAIN-COMPUTER INTERFACES 24 LOURENÇO BARBOSA DE CASTRO – 2013 from the alphabet is displayed on a screen. Rows and columns of the matrix are flashed in random order, and flashes of the row or column containing the desired symbol constitute the oddball stimulus, while all other flashes constitute nontarget stimuli. Since the seminal paper of Farwell and Donchin, many studies about P300-based BCI systems have appeared, usually associated with machine learning techniques such as support vector machines or neural networks (Campbell, et al. 2010). This potential has also been linked to subject specific factors such as gender, age or brain diseases. Systems using this potential are relatively slow forms of brain-computer interaction, since it is usually required for a stimuli to occur several times after confirming the user's intention. Due to the low power of the resulting electric wave, signal to noise ratio is very low, making its detection very difficult, resulting in the need of high resolution equipment. This condition also increases the signal processing complexity required for a successful detection. Additionally, as previously mentioned, the user must focus their attention on a specific stimuli, which can be difficult for some types of people. Finally, since this system is constantly transmitting information and scanning for specific signals, it is complicated to make pauses, either voluntary or not. COGNITION/EXPRESSIONS 2.5.3 HOW CAN COGNITION AN D FACIAL EXPRESSIONS BE USED TO INTERACT WITH COMPUTERS Using cognitions or facial expressions is also a possibility in this field. EEG and EMG patterns can be read and interpreted, and afterwards assigned to specific computer commands. This methodology requires a solid and time-consuming calibration, in order for the user's actions to be detected accurately. Also, from a daily usage perspective, performing several facial expressions in public should make some people uncomfortable, and might even be impossible for impaired patients. However, after sufficient calibration is performed, using conscious thoughts and facial expressions should result in fast brain-computer interactions, although some currently available systems have some limitations in terms of using multiple types of thoughts. Addition- CHAPTER 2 BRAIN-COMPUTER INTERFACES HUMAN COMPUTER INTERACTION VIA BRAINWAVES FOR DISABLED PEOPLE 25 ally, when considering muscular electric activity, noise becomes less relevant, as facial expressions usually produce high amplitude signals. Another advantage of this approach, over pure EEG signals, is that momentaneous interruptions are possible, since permanently keeping focus is not essential, helping this system to become a better option for daily usage. TYPES OF BCIS 2.6 DESCRIPTION OF INVAS IVE AND NON-INVASIVE METHODOLOGIES INVASIVE 2.6.1 Invasive BCI systems usually consist of arrays of electrodes implanted directly on the patient’s brain, either reading activity from single neurons or from groups of these cells. Due to the aggressiveness of such implementation of brain-computer interfacing, this method is only considered if it provides significant benefits over noninvasive technology. The main advantages are higher information bitrates and signal resolution, making it a better choice for delicate rehabilitation projects, such as the BrainGate system which can be seen on Figure 7 (BrainGate 2009). In order to select an appropriate implantation region, techniques such as functional resonance imaging, magnetoencephalography and other imaging tools are used, although the most common choice is the motor cortex. It is generally the information from this region that allows paralyzed people to find alternative pathways to communicate and interact with their surroundings. Figure 7 - Representation of the BrainGate technology, a type of invasive BCI (BrainGate 2009). CHAPTER 2 BRAIN-COMPUTER INTERFACES 26 LOURENÇO BARBOSA DE CASTRO – 2013 Another important factor to ponder is the simultaneous amount of neurons needed to obtain a meaningful signal, since this technology requires risky medical procedures that need to be justified by a decent information transfer rate. The research community’s opinion is inconclusive in this subject, with values ranging from 2 cells up to 100, leading to different types of approaches, some of which previously described in section 2.1 (Wolpaw, et al. 2000). The use of invasive technology on human beings is also conditioned by the period of time that the specific electrodes arrays can be kept inside a patient’s brain, while transmitting stable recordings. Recently, the BrainGate implant proved to be efficient even after a period of nearly 3 years, making it the most viable option currently available (Simeral, et al. 2011). Additionally, after long periods of time, this type of BCI can become so intrinsic to the patient that their brain completely maps its activity to the computer interface, creating a true human-machine hybrid (Hockenberry 2001). NON-INVASIVE 2.6.2 Noninvasive brain-computer interfaces depend on electrodes mounted on a user’s scalp, detecting electric activity that manages to travel through bone and skin. This means that, contrary to invasive technology, this method doesn’t require any kind of surgery. However, this results in significantly reduced signal quality and bitrates, while being more prone to different types of noise. Thereby, noninvasive approaches are commonly used in association with sensory stimulation techniques, using the brain’s feedback as a communication tool. Figure 8 - Emotiv EPOC, a type of non-invasive BCI equipment (Emotiv, Emotiv Systems Website 2013). CHAPTER 2 BRAIN-COMPUTER INTERFACES HUMAN COMPUTER INTERACTION VIA BRAINWAVES FOR DISABLED PEOPLE 27 In order to improve signal conductivity, a specialized gel or water solution is used, although the current tendency is to adopt dry-based sensors. The information transfer rate is generally slower, compensated by the fact that starting using such technology is as simple as wearing a specialized cap or headband (Middendorf, et al. 2000). The accessibility and affordability of this approach makes it the best target for entertainment developments, like mind controlled games or personalized entertainment systems. Every consumer-based BCI company develops noninvasive devices, usually offering relatively simple features, such as detecting the user’s focus or mental stress levels. However, recent advances in this field have improved its bitrate sufficiently for noninvasive BCIs being considered as neural rehabilitation tools. SUMMARY 2.7 This chapter explained the foundations of this project, mentioning the discovery of EEG by Hans Berger in 1929 and the methodology behind this technique. The most used electrodes types are metal disk and cup electrodes, usually built with tin, silver, gold or surgical steel, with sizes ranging from 4 to 10 mm. Although most devices are trying to avoid this characteristic, it is commonly required the use of a special gel or saline solution for better conductivity. The 10-20 electrodes positioning system is widely adopted by EEG and BCI communities, creating a standard for better results comparison and validation. The low signal amplitude verified in EEG makes it highly prone to noise, mostly caused by muscular artifacts or power supply line noise. This aspect makes noise removal a very important factor in BCI analysis. Neural oscillations can be divided in different ranges, which can be associated with different behaviors and mindsets. Although not precisely known, lowest frequencies are usually associated with relaxation and deep sleep, and disruptions in this range are seen in conditions such as ADHD or depression. Higher frequencies can be correlated with attention, creativity and active cognitive processing in general. CHAPTER 2 BRAIN-COMPUTER INTERFACES 28 LOURENÇO BARBOSA DE CASTRO – 2013 Electromyography can be described as the detection and analysis of the electric activity generated by muscular activation. This study is generally and more accurately elaborated using needles inserted in the muscles of interest, although it can also be measured on the surface of the skin with sufficient precision. These signals, otherwise seen as artifacts, are being used as features by some BCI devices, creating associations with computer commands or being studied for a better understanding of users’ emotions. Following, several interpretations of neurophysiological signals are described. Event-related potentials are brainwave patterns associated to a specific event. The P300 potential is an EEG deflection verified after a surprising stimuli, which is being used to help disabled people select options on a computer screen. Cognitive patterns and facial expressions, based both on EEG and EMG, are another source of electric signals that can be translated on a computer, after sufficient calibration, which can make computers adapt to their users. Finally, the two generic BCI types were explained, invasive and non-invasive interfaces. While introducing electrodes directly on the brain’s surface provides great improvements in signal quality and resolution, the risks involved make this approach only considerable when no other viable options are available. On the other hand, non-invasive technology is more likely to be affected by noise or low resolution effects, but it is much more affordable and completely safe, measuring the brain’s activity over the scalp. 29 CHAPTER 3 EMOTIV EPOC This section focuses on the EEG device used in this project, the Emotiv EPOC. Describes its creation, purpose and their current position on the market. Following is a comparison between this device and the competition, explaining advantages and disadvantages, justifying why this model was chosen. To finalize, Emotiv’s native software is presented, along with its functionalities. 3.1. INTRODUCTION 30 3.2. HARDWARE 31 3.3. TYPES OF INTERACTION 32 3.4. SUMMARY 33 CHAPTER 4 DEVELOPED APPLICATION 36 LOURENÇO BARBOSA DE CASTRO – 2013 TARGET USERS 4.1 TO WHOM THIS APPLICATION WAS DESIGNED One of the main purposes of this work is to implement features that can help overcoming difficulties on common daily tasks. These difficulties can originate in several different sources, such as diseases or accidents, which may cause severe muscular disabilities that strongly reduce affected people’s capabilities and autonomy. Locked in syndrome One of the most frequently targeted disease on BCI research is amyotrophic lateral sclerosis (also known as Lou Gehrig’s disease), which essentially locks patients inside their own bodies, progressively deteriorating muscle functions. Affected people usually die of respiratory failure, also due to muscular degeneration, after living a mean time of 43 months with this disease (Turner, et al. 2003). Strokes Another cause for muscular paralysis are strokes, affecting 15 million people worldwide every year, while killing 5 million of those and leaving another 5 permanently disabled (World Health Organization 2002). These people have their lives greatly impaired, losing most or all of their autonomy. Spinal cord injury Spinal cord injuries can also leave affected patients unable to move, with either paraplegia or tetraplegia. Currently, there are no restorative treatments for such conditions, meaning that developing alternatives for their muscular related activity is a desired option (King, et al. 2013). Ageing Additionally, as people age, they tend to lose muscle control and strength, increasing their dependency on family members or healthcare professionals. CHAPTER 4 DEVELOPED APPLICATION HUMAN COMPUTER INTERACTION VIA BRAINWAVES FOR DISABLED PEOPLE 37 Most of these conditions leave one’s ability to reason intact, making them a good target for brain-computer interfaces. By providing an alternative method to interact with computers, it might be possible to return some of these people’s autonomy, enabling them to communicate or perform basic tasks. ANDROID OPERATING SYSTEM 4.2 OPERATING SYSTEM 4.2.1 BRIEF INTRODUCTION TO THE ANDRO ID PLATFORM Android was started in 2003, as a Linux-based mobile operating system, making it opensource and, thus, easily modifiable. The original company was bought by Google (U.S.A.) in 2005, preserving its name and philosophy, while keeping it away from the general public. It was only 5 years after the initial developments, in 2008, that the first smartphone running Android, version 1.0, was released. The biggest advantage of this operating system was its unrestricted availability to manufacturers, allowing them to focus solely on hardware development, which greatly helped Android growing exponentially (Kovach 2013). Currently, this is the most widely used mobile operating system, with nearly 80% market share (International Data Corporation 2013), making it an exceptional target for innovative accessibility applications development. NEXUS 7 TABLET DEVICE 4.2.2 TABLET DEVICE U SED IN THIS PROJECT The Nexus 7 was chosen mainly for 3 aspects: USB Host, good price to quality relationship and appropriate screen size. The USB Host application programming interface (API) is a possibility first launched on Android 3.1 (Honeycomb), version released in May 2011. Not all devices currently support this feature, in favor of better battery life, but an increasingly number of modern devices is embracing this technology (Google, USB Host 2013). As of CHAPTER 4 DEVELOPED APPLICATION 38 LOURENÇO BARBOSA DE CASTRO – 2013 the start of this project, this mobile computer was the only known tablet device to have USB Host enabled, which was crucial to interact with the Emotiv EPOC. Considering all the available options, this tablet showed the best price to quality ratio, featuring a quad-core processor, 1280x800 screen resolution and 1GB of RAM, which was useful to quickly perform the required calculations and display the sensors data, all in real-time. The device’s weight, 340g was slightly above the average of the same sized tablets, but was still light enough to be easily carried, even with only one hand. Additionally, the screen size of 7 inches was sufficiently large to properly display keyboard characters and contacts pictures (Asus 2012). FEATURES 4.3 TIME DOMAIN FEATURES 4.3.1 These features are related to changes in the amplitude of the signals, over a period of time after the presentation of a stimuli or after the actions of a user. P300 potentials can be characterized using time domain features. The number of spikes occurring in a certain time interval is also a time domain feature. FREQUENCY DOMAIN FEATURES 4.3.2 These domain’s features are related to changes in oscillatory activity, which are useful because the phase of the oscillation is usually not related to the time of the stimuli presentation or the action of the user. Features like the synchronization of signals from different brain regions are also of the frequency domain. CHAPTER 4 DEVELOPED APPLICATION HUMAN COMPUTER INTERACTION VIA BRAINWAVES FOR DISABLED PEOPLE 39 SPATIAL FEATURES 4.3.3 When data from more than one electrode is available, the extracted features have to be combined in an efficient way. This efficiency is the goal of spatial features extraction. One method is to use only electrodes carrying useful information for a specific set of tasks, i.e., electrodes placed over regions of the brain related to specific cognitive functions or event potentials. These electrodes can be selected manually or using algorithms that automatically choose an optimal sensor subset. Algorithms like the common spatial patterns algorithm or the independent component analysis can perform such tasks (Blankertz, et al. 2008, Li, et al. 2009). SOFTWARE 4.4 DATA ACQUISITION 4.4.1 HOW THE DATA IS ACQUIRED FROM THE HEADSET As previously mentioned, the developed application relies on the Android’s USB Host API. Using a USB on-the-go (OTG) cable, the Android tablet could communicate with Emotiv EPOC’s USB dongle, therefore establishing a connection between both devices (Google 2013). The application actively detects the connection of new USB devices with characteristics specific to the EPOC. When the equipment of interest is connected to the tablet, its serial number is stored, for data decryption purposes, and the proper communication pathway, the USB endpoint, is opened. This endpoint is the channel used to receive large amounts of data from the headset, more precisely a 32 bytes packet containing the electric values captured by all the sensors and their contact quality, gyroscope orientation and the device’s battery level. The data packet is retrieved at the EPOC’s sampling rate, 128 Hz, followed by pre-processing to remove the direct current (D.C.) component and round the values to a single decimal place (this last step is done because, as previously mentioned, the EPOC’s CHAPTER 4 DEVELOPED APPLICATION 40 LOURENÇO BARBOSA DE CASTRO – 2013 resolution has a least significant bit of 0.5 μV, making every following figure insignificant (Emotiv 2013)). Finally, these corrected values are distributed to the appropriate active Java class, using Android’s PropertyChangeListener API (Google 2013). EVENTS DETECTION 4.4.2 After the acquisition and pre-processing, the application interprets the electric signal, looking for specific patterns which are associated with user actions. The actions that the software currently detects are winks and horizontal eye movements, being used to trigger commands in the tablet device. In order for this detection to occur, a calibration is required. This process consisted in detecting the peak activity for an event-specific electrode, while also recording the output of the sensor on the opposite side of the head. After several recordings, a range of values could be established for both electrodes, creating thresholds for their actual values as well as their ratios. This way, when a peak was detected in a channel of interest, the detection process was initiated. Using this channel and its symmetrical sensor’s values, ratios were calculated and compared with the saved data. If it fit inside the trained ranges, a positive detection was triggered, otherwise nothing happened. FUNCTIONALITIES 4.4.3 With accessibility in mind, several features were implemented to try to help disabled people overcome their daily difficulties. The actual tasks targeted in this project were typing, simple mathematical operations, selecting phone contacts and general application navigation. Additionally, there is a feature to monitor neural oscillations across the entire brain and a configuration page where calibration takes place. This calibration is user-dependent, saving the trained data on separate profiles, thus making the application personalized and operational for different people. CHAPTER 4 DEVELOPED APPLICATION HUMAN COMPUTER INTERACTION VIA BRAINWAVES FOR DISABLED PEOPLE 41 Each section is accompanied by a help button, located on the top right part of the screen. Essentially, the help provided is a short version of the following descriptions, therefore enabling the user to quickly learn how to operate the application. Eyes navigation 4.4.3.1 This feature simulates horizontal swipes on the screen, replacing them with sideways looks, which enables the user to switch the current section without touching the device. There is an option to quickly toggle this function in order to avoid conflicts with other application parts where looking sideways is detected. Keyboard 4.4.3.2 In order to provide an alternative for paralyzed or movement impaired people to be able to type messages on a computer screen, this section was developed. Using only winks, the user can efficiently select characters of interest until a sentence is built, with option for spaces and deleting mistakes (Figure 10). After initiating this section of the application, the first right eye wink will activate the first row, changing its color. Subsequent right winks will cycle through all the available rows, also changing their color. When reaching the row containing the target character, a left wink will select it, fixating the current color and changing the right wink command. Afterwards, right winks cycle through all the columns, creating an intersection with the previously selected row. The letter that falls in this intersection is selectable, again, using a left wink, which types it on a specified field on the bottom of the screen. Additionally, when no characters are selected, a left wink deletes the previous typed letter. To summarize the controls, winking the right eye moves the cursor, while the left eye selects the current character or deletes the previous letter. CHAPTER 4 DEVELOPED APPLICATION 42 LOURENÇO BARBOSA DE CASTRO – 2013 Figure 10 - Keyboard section of the application, showing the selection of the letter L, while the word "HELLO" was previously typed. Contacts 4.4.3.3 Making use of the Emotiv EPOC’s gyroscope, this section is straightforward. Moving the user’s head sideways will cycle through all available contacts, on the same direction as the device was moved. When the contact of interest is reached, quickly nodding will select it, simulating a phone call. The current selection will be more prominently displayed, while the remaining show a fading effect, as can be seen on Figure 11. Finally, nodding again will terminate or cancel the simulated call. CHAPTER 4 DEVELOPED APPLICATION HUMAN COMPUTER INTERACTION VIA BRAINWAVES FOR DISABLED PEOPLE 43 Figure 11 - Contacts section, exemplifying the selection of a contact. It is also displayed the current position of the gyroscope. Calculation 4.4.3.4 The calculation section was implemented as a mathematics mini-game, to display an alternative form of interaction with the EEG device. Using events triggered by looking sideways, it is possible to attempt to select a correct answer among two different options. Each answer is a simple mathematical operation, and the result of one of them is shown on the middle of the screen. If the user looks in the correct direction, the answer will turn green. Otherwise, it will change to red (Figure 12). A timeout was implemented, automatically switching the alternatives after 5 seconds, and there is also the possibility to change the difficulty level, resulting in more complicated operations. CHAPTER 4 DEVELOPED APPLICATION 44 LOURENÇO BARBOSA DE CASTRO – 2013 Figure 12 - Calculation section, showing a correct selection during the second level of difficulty. Brainwaves 4.4.3.5 The brainwaves section is used to visualize EEG frequencies across the user’s brain, separated by the 4 neural oscillation bands that the EPOC can capture, Delta, Theta, Alpha and Beta. One chart for each sensor is used, positioned according to the location on the headset, simulating the comprised region of the human brain. Additionally, one can click on a sensor of interest, alternating to a single, larger, chart which displays a more detailed frequency information (Figure 13). For a better frequency analysis, the charts are only refreshed every 8 seconds, providing a larger sample pool, which enables the detection of broader ranges of frequencies. CHAPTER 4 DEVELOPED APPLICATION HUMAN COMPUTER INTERACTION VIA BRAINWAVES FOR DISABLED PEOPLE 45 Figure 13 - Detailed oscillation information, displayed after clicking on a sensor of interest. Configuration 4.4.3.6 The most visible aspect of the configuration section is the real-time display of the acquired electric signals. Using the list of the available sensors on the top of the screen, it is possible to select which channels to visualize. Also, on this same list, the channel’s name will change its color according to the contact quality, with values from 0 (no contact) to 100 (full contact quality) being translated into colors ranging from red to green, while black indicates there is no contact established (Figure 14). This is also the section where all of the previously described actions, such as winks, can be calibrated. For this purpose, the user selects the desired action on a list on the bottom right part of the screen, and then presses the “Train” button (which promptly changes to a “Stop” button). The user should then perform the selected action, finishing with a click on the new “Stop” button. All of this action-specific data can be loaded on the screen or fully deleted. CHAPTER 5 RESULTS AND DISCUSSION 52 LOURENÇO BARBOSA DE CASTRO – 2013 WINKS 5.1 For the winks detection, the subject (healthy male, 26 years of age) was asked to sit and relax, while wearing the EEG headset and looking forward to the tablet's screen. These trials were divided in two parts, one for each eye, with all the following procedures being fully repeated for each side. The experiment consisted in a series of training phases, while testing their effectiveness in between. Each sequence required an increasing number of calibrations, although the number of required successful attempts remained the same. More precisely, 4 phases took place, with 5, 10, 15 and 20 training calibrations, respectively, with a goal of 10 successful attempts. Each of these phases was repeated 3 times, and the final results consisted of the average of these trials. Also, the results were gathered sequentially, adding 5 new calibrations to the previous set, not discarding any values. After 20 calibrations were reached and successful attempts calculated, a new dataset was initiated. It should be noted that the results obtained in this section were much better and consistent than using Emotiv’s native application, which often provided apparently random results, even after proper calibration. The following graphic shows how the channel of interest for the left eye was chosen among all the others available. The process was similar, except symmetrical, for the right eye winks. Table 2 – Electric activity measured while the subject was winking their left eye. -50 0 50 100 150 7 8 9 10 11 12 13 14 15 16 17 SAMPLES AF3 F7 F3 FC5 T7 P7 O1 O2 P8 T8 FC6 F4 F8 AF4 CHAPTER 5 RESULTS AND DISCUSSION HUMAN COMPUTER INTERACTION VIA BRAINWAVES FOR DISABLED PEOPLE 53 While observing the activity during an eye wink, choosing the appropriate sensor was an obvious choice, and the denoted high electric activity can be strongly correlated with EMG, due to its range. As shown on Table 3, the chosen channel was F7; for the opposite side, the selected sensor was F8. For this measurement, the subject was asked to stand still and relaxed, with their head facing the tablet’s display, all of this in order to reduce the possibility of muscular artifacts. Each eye's winks calibration results are presented on tables 5 and 6. Table 3 - Results for the right winking trials, showing the rate of success for each calibrations amount Table 4 - Results for the left winking trials, showing the rate of success for each calibrations amount 33,72% 54,50% 75,55% 87,45% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 510 15 20 Success Calibrations 53,45% 50,79% 94,44% 93,94% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 510 15 20 Success Calibrations CHAPTER 5 RESULTS AND DISCUSSION 54 LOURENÇO BARBOSA DE CASTRO – 2013 Observing these results, can be inferred that at least 15 calibrations should be performed with this method. It should be noted that no preloaded data was included in the trial, fact verified on the native Emotiv desktop application. In order to make this possible, a large dataset needed to be gathered and analyzed, which was not possible during this project’s timeframe due to the amount of time required to adapt the EEG device to the Android platform. However, each winking calibration had a very short duration, usually under 2 seconds, making 15 trains very accessible, and using this approach could make the user experience very personalized. Due to time limitations, this dataset was gathered from a single subject, although the implemented method should be dynamic enough to adapt itself to different users. Also, comparing this method to Emotiv’s native application was not possible, due to random outcomes in the left and right winks detection, even with different users. Finally, these results proved to be consistent when used over different sessions while keeping the same datasets. This means the same calibration data can be reused every time the application is needed. BRAINWAVES 5.2 For the brainwaves analysis section, the subject was asked to sit and look straight to the tablet device while wearing the EPOC. Additionally, the subject was asked to sit as still as possible, in order not to affect the highly sensitive brainwave frequencies. Afterwards, four different series were studied, each trying to induce a different mindset. Every series consisted of 30 seconds of brainwaves recording, analyzing exclusively neural oscillations on the delta, theta, alpha and beta bands. Every series was repeated 3 times, and the results were averaged. The first series consisted in the user relaxing, not thinking about anything in particular, while keeping their eyes open. The second one was similar in every aspect, except that the CHAPTER 5 RESULTS AND DISCUSSION HUMAN COMPUTER INTERACTION VIA BRAINWAVES FOR DISABLED PEOPLE 55 user was asked to attentively read a previously unknown text, with the purpose of increasing their focus levels. Third, the user was asked to close their eyes and maintain a relaxed state, while listening to relaxing music. Lastly, loud and fast music played for the whole duration of the trial, while the user also kept their eyes closed. In every trial, there was a short adaptation period prior to the actual recording, in order for the brainwaves to settle on the levels of interest. Also, if sudden movements were detected, the corresponding trial was discarded. The results of this phase are presented in a comparative form, quantifying the absolute differences from each series relative to all the others, using absolute values. This was necessary since the obtained values’ range and scale of magnitude was not exactly known, due to uncertainties acquiring data on the tablet device. Moreover, the standard deviation was calculated by the sum of the standard deviations of each compared set of values, and it is also worth noting that this value was considerably low throughout most samples. Each results’ chart is presented and discussed separately, in order for a better understanding of the impact of each mindset on the subject. Figure 15 - Comparing the activity of a normal and relaxed state with a normal and focused state. Comparing a relaxed and distracted state of mind with an active and focused brain (Figure 15), it is noted that the overall activity is slightly lower, as expected. However, the decrease -40 -30 -20 -10 0 10 20 30 40 AF3 F7 F3 FC5 T7 P7 O1 O2 P8 T8 FC6 F4 F8 AF4 Delta Theta Alpha Beta Eyes opened, relaxed Eyes opened, focused CHAPTER 5 RESULTS AND DISCUSSION 56 LOURENÇO BARBOSA DE CASTRO – 2013 of Delta waves’ activity is less evident relatively to the other registered EEG bands, which can be explained by the frequent association of this kind of waves with relaxed brains and deep sleep phases. It is also possible to observe higher activity changes on the FC5, FC6 and T7 regions. Figure 16 - Comparing the activity of a normal and relaxed state with an eyes closed with soft and relaxing music state. When comparing two relaxed states, one with open eyes and the other with both closed eyes and relaxing music playing (Figure 16), it is apparent that these last conditions stimulated more brainwave activity. More precisely, the frontal region of the right hemisphere (FC6 and F4) showed high electric activity while relaxing music was playing, whereas lower activity was visible on the symmetrical part of the brain (F7 and F3). -40 -30 -20 -10 0 10 20 30 40 AF3 F7 F3 FC5 T7 P7 O1 O2 P8 T8 FC6 F4 F8 AF4 Delta Theta Alpha Beta Eyes opened, relaxed Eyes closed, relaxing music CHAPTER 5 RESULTS AND DISCUSSION HUMAN COMPUTER INTERACTION VIA BRAINWAVES FOR DISABLED PEOPLE 57 Figure 17 - Comparing the activity of a normal and relaxed state with an eyes closed with loud and fast music state. The relaxed state with eyes open, relative to having eyes closed while listening to loud and fast music (Figure 17), also shows lower overall activity. The exceptions were the frontal left and the parieto-occipital regions. Also, the highest increase in activity, while fast music was playing, was in the right frontal section of the brain, on the FC6 and F4 sensors. Figure 18 - Comparing the activity of a normal and focused state with a closed eyes state with soft and relaxing music playing. -40 -30 -20 -10 0 10 20 30 40 AF3 F7 F3 FC5 T7 P7 O1 O2 P8 T8 FC6 F4 F8 AF4 Delta Theta Alpha Beta -40 -30 -20 -10 0 10 20 30 40 AF3 F7 F3 FC5 T7 P7 O1 O2 P8 T8 FC6 F4 F8 AF4 Delta Theta Alpha Beta Eyes opened, relaxed Eyes closed, fast music Eyes opened, focused Eyes closed, relaxing music CHAPTER 5 RESULTS AND DISCUSSION 58 LOURENÇO BARBOSA DE CASTRO – 2013 Switching from a focused situation to a relaxed one, with the eyes closed and soft music playing (Figure 18), presented an almost symmetrical distribution of the brainwaves activity, with higher increases on the left frontal region for the focused state of mind, while the right corresponding part of the brain was more active when the subject was relaxing. Figure 19 - Comparing the activity of a normal and focused state with a closed eyes state with loud and fast music playing. As expected, playing loud and fast music with the subject’s eyes closed (Figure 19), generated more activity across most sensors, when compared to a simple focused state. Higher emphasis on the frontal region (FC6 and F4) and on the parieto-temporal region (T7 and P7), both on the right hemisphere. A symmetrical but relatively smaller increase of activity was visible for the focused condition, on the electrodes F7, F3 and P8. -40 -30 -20 -10 0 10 20 30 AF3 F7 F3 FC5 T7 P7 O1 O2 P8 T8 FC6 F4 F8 AF4 Delta Theta Alpha Beta Eyes opened, focused Eyes closed, fast music CHAPTER 5 RESULTS AND DISCUSSION HUMAN COMPUTER INTERACTION VIA BRAINWAVES FOR DISABLED PEOPLE 59 Figure 20 - Comparing the activity of two closed eyes states, one with soft relaxing music, other with fast loud music playing. Lastly, comparing both closed eyes states (Figure 20), the most likely outcome was visible, with the loud and fast music producing higher activities, with the exception of the right occipital area. The impact was more visible on the left frontal region, over the F7 and F3 electrodes. The most important aspect learned from this experimental phase is that each tested mind state produced a particular pattern of brainwaves activity, which can be effectively differentiated. For an accessibility approach, detecting when the user switches from a calm and relaxed situation to a more stressed and nervous condition might be an important alert signal for family members or health professionals. Additionally, from an autonomy perspective, an impaired user could see their computer adapt to situational needs, such as displaying help topics when frustration is detected, or playing a warning sound when focus is being lost. Table 5 summarizes these results. -40 -30 -20 -10 0 10 20 30 40 AF3 F7 F3 FC5 T7 P7 O1 O2 P8 T8 FC6 F4 F8 AF4 Delta Theta Alpha Beta Eyes closed, fast music Eyes closed, relaxing music CHAPTER 5 RESULTS AND DISCUSSION 60 LOURENÇO BARBOSA DE CASTRO – 2013 Table 5 - Results of the brainwaves trials, comparing the top row with the left column state. Both high and low peaks are denoted, as well as other interesting features SUMMARY 5.3 This project’s trials were performed to assess the effectiveness of the winks detection and the brainwaves variations under different conditions. The first segment of trials was dedicated to detecting winks using exclusively Emotiv EPOC’s electric outputs. The selection of appropriate channels was fairly obvious, with explicitly higher activity shown during each wink event on the channel F7 for the left side and channel F8 for the right side. Then, an algorithm to create a personalized threshold range was developed, which proved to be at least 75% accurate after 15 calibrations with an average duration of 2 seconds each. Eyes opened, Relaxed Eyes opened, Focused Eyes Closed, Relaxing music Eyes Closed, Fast music Eyes opened, Relaxed - More activity Lower Delta increase. High: FC5 Low: T7 Increase: F7, F3 Decrease: F4, FC6 High: FC6, F4 Low: P8, AF4 Lower delta increase Eyes opened, Focused Less activity High: T7 Low: FC5 - High: FC6, F4 Low: F7, F3 Other regions decreased High: T7, FC6, F4 Low: P8 Eyes Closed, Relaxing music High: F4, FC6 Low: F7, F3 High: F7 and F3 Low: FC6, F4 and T7 - Overall increase High: F7, F3 Low: O2 Eyes Closed, Fast music High: P8, AF4 Low: FC6, F4 High: P8 Low: T7, FC6, F4 Overall decrease High: O2 Low: F7, F3 - CHAPTER 5 RESULTS AND DISCUSSION HUMAN COMPUTER INTERACTION VIA BRAINWAVES FOR DISABLED PEOPLE 61 For the second part of trials, brainwaves were measured under four circumstances: while the user was relaxing, reading an unknown text, listening to slow and relaxing music and, lastly, listening to loud and fast music. These results showed distinct brainwave patterns in each condition, which could be used to differentiate states of mind and make computers adapt to their users. BIBLIOGRAPHY 68 LOURENÇO BARBOSA DE CASTRO – 2013 Breen, Randy. "Demonstration of Brain Computer Interface using the Emotiv Epoc." Computer Systems Colloquium. Stanford University. April 2, 2008. Campbell, Andrew, Choudhury, Tanzeem; Hu, Shaohan; Lu, Hong; Mukerjee, Matthew K.; Rabbi, Mashfiqui; Raizada, Rajeev D.S. "NeuroPhone: brain-mobile phone interface using a wireless EEG headset." Proceedings of the second ACM SIGCOMM workshop on Networking, systems, and applications on mobile handhelds. New York, NY: Association for Computing Machinery, 2010. 3-8. Castermans, Thierry. "Detecting biosignals with the Emotiv EPOC headset: a Review." Tangible Feelings : a Symposium on EEG. Brussels: TCTS Lab, 2011. CBSNews. "Harnessing The Power Of The Brain." CBSNews - 60 Minutes. Denise Schrier Cetta. November 2, 2008. http://www.cbsnews.com/stories/2008/10/31/60minutes/main4560940.shtml (accessed July 25, 2013). Chapin, JK, KA Moxon, RS Markowitz, and MA Nicolelis. "Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex." July 1999: 664-70. Clarke, Adam R, Robert J Barry, Rory McCarthy, and Mark Selikowitz. "EEG-defined subtypes of children with attention-deficit/hyperactivity disorder." Clinical Neurophysiology, November 2001: 2098–2105. Cofre, J.P., G. Moraga, C. Rusu, and I Mercado. "Developing a Touchscreen-based Domotic Tool for Users with Motor Disabilities." Ninth International Conference on Information Technology: New Generations. Las Vegas, NV: Conference Publishing Services, 2012. 696 - 701. Culham, Jody C, and Kenneth F Valyear. "Human parietal cortex in action." Current Opinion in Neurobiology, April 2006: 205-212. Delgado, JM; Mark, V; Sweet, W; Ervin, F; Weiss, G; Bach-Y-Rita, G; Hagiwara, R. "Intracerebral radio stimulation and recording in completely free patients." October 1968: 329-40. BIBLIOGRAPHY HUMAN COMPUTER INTERACTION VIA BRAINWAVES FOR DISABLED PEOPLE 69 Emotiv. Emotiv Emerges from Stealth to Revolutionize. Press Release, San Francisco, CA: Technology Venture Partners Pty., 2007. Emotiv. Emotiv Systems Website. 2013. www.emotiv.com (accessed June 26, 2013). Farwell, LA, and E Donchin. "Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials." Electroencephalography and clinical neurophysiology, December 1988: 510-523. Fetz, EE. "Operant Conditioning of Cortical Unit Activity." Science 163, no. 3870 (February 1969): 955-58. Fieldtrip. Donders Centre for Cognitive Neuroimaging. January 23, 2013. http://fieldtrip.fcdonders.nl/template/layout (accessed September 28, 2013). Foresman, Pearson Scott. Wikimedia. September 13, 2009. http://commons.wikimedia.org/wiki/File:Neuron_(PSF).png (accessed September 28, 2013). Gamboa, Hugo. December 2005. http://en.wikipedia.org/wiki/File:Eeg_delta.svg (accessed September 28, 2013). Gamboa, Hugo. December 2005. http://en.wikipedia.org/wiki/File:Eeg_theta.svg (accessed September 28, 2013). Gamboa, Hugo. December 2005. http://en.wikipedia.org/wiki/File:Eeg_alpha.svg (accessed September 28, 2013). Gamboa, Hugo. December 2005. http://en.wikipedia.org/wiki/File:Eeg_beta.svg (accessed September 28, 2013). Ganapati, Priya. "Winter Olympics to demo lighting controlled by thoughts." Wired Magazine. Condé Nast. March 2, 2010. http://www.wired.com/gadgetlab/2010/02/thought-controlled-lights/ (accessed September 10, 2013). BIBLIOGRAPHY 70 LOURENÇO BARBOSA DE CASTRO – 2013 Gomez-Gil, Jaime, Israel San-Jose-Gonzalez, Luis Fernando Nicolas-Alonso, and Sergio Alonso-Garcia. "Steering a Tractor by Means of an EMG-Based Human-Machine Interface." Sensors, 2011: 7110–7126. Google. "PropertyChangeListener." Android Developers Website. 2013. http://developer.android.com/reference/java/beans/PropertyChangeListener.html (accessed September 3, 2013). Google. "USB Host." Android Developers Website. 2013. http://developer.android.com/guide/topics/connectivity/usb/host.html (accessed September 3, 2013). Haglund, MM, MS Berger, M Shamseldin, E Lettich, and GA Ojemann. "Cortical localization of temporal lobe language sites in patients with gliomas." Neurosurgery, April 1994: 567-576. Hochberg, Leigh R.; Bacher, Daniel; Jarosiewicz, Beata; Masse, Nicolas Y.; Simeral, John D.; Vogel, Joern; Haddadin, Sami; Liu, Jie; Cash, Sydney S.; Smagt, Patrick van der; Donoghue, John P. "Reach and grasp by people with tetraplegia using a neurally controlled robotic arm." Nature, May 17, 2012: 372-375. Hockenberry, John. "The Next Brainiacs." Wired Magazine. August 2001. http://www.wired.com/wired/archive/9.08/assist_pr.html (accessed September 5, 2013). Hoffmann, Ulrich. Bayesian machine learning applied in a brain-computer interface for disabled users. Lausanne: École Polytechnique Fédérale de Lausanne, 2007. Hoffmann, Ulrich, Jean-Marc Vesin, Touradj Ebrahimi, and Karin Diserens. "An efficient P300-based brain-computer interface for disabled subjects." Journal of Neuroscience Methods, January 15, 2008: 115-125. InteraXon. InteraXon - Though Controlled Computing. 2010. http://interaxon.ca/ (accessed June 26, 2013). BIBLIOGRAPHY HUMAN COMPUTER INTERACTION VIA BRAINWAVES FOR DISABLED PEOPLE 71 International Data Corporation. "Apple Cedes Market Share in Smartphone Operating System Market as Android Surges and Windows Phone Gains, According to IDC." IDC Website. August 7, 2013. http://www.idc.com/getdoc.jsp?containerId=prUS24257413 (accessed September 15, 2013). King, Christine E, Po T Wang, Luis A Chui, An H Do, and Zoran Nenadic. "Operation of a brain-computer interface walking simulator for individuals with spinal cord injury." Journal of Neuroengineering and Rehabilitation, 2013: 77. Kovach, Steve. "How Android Grew To Be More Popular Than The iPhone." Business Insider. August 13, 2013. http://www.businessinsider.com/history-of-android-20138?op=1 (accessed September 15, 2013). Li, Kun, Ravi Sankar, Y. Arbel, and E. Donchin. "Single trial independent component analysis for P300 BCI system." Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Minneapolis, MN, 2009. 4035-4038. Li, Shan. Mind reading is on the market. Los Angeles, CA: Los Angeles Times, 2010. Middendorf, M, G McMillan, G Calhoun, and KS Jones. "Brain-computer interfaces based on the steady-state visual-evoked response." Transactions on Rehabilitation Engineering, June 2000: 211-214. NeuroSky. NeuroSky official website. 2012. http://www.neurosky.com/ (accessed September 2, 2013). Niedermeyer, Ernst, and Fernando Lopes Silva. Electroencephalography: Basic Principles, Clinical Applications and Related Fields. Fifth edition. Lippincott Williams and Wilkins, 2004. Nyström, Pär, Therese Ljunghammar, Kerstin Rosander, and Claes von Hofsten. "Using mu rhythm desynchronization to measure mirror neuron activity in infants." Developmental Science, June 18, 2010: 327-335. BIBLIOGRAPHY 72 LOURENÇO BARBOSA DE CASTRO – 2013 Pais-Vieira, M, M Lebedev, C Kunicki, J Wang, and MA Nicolelis. "A Brain-to-Brain Interface for Real-Time Sharing of Sensorimotor Information." February 2013: 1-10. Palva, Satu, and J. Matias Palva. "New vistas for alpha-frequency band oscillations." Trends in Neurosciences, April 2007: 150-158. Rangaswamya, Madhavi; Porjesz, Bernice; Chorliana, David B; Wanga, Kongming; Jonesa, Kevin A; Bauerb, Lance O; Rohrbaughf, John; O’Connorc, Sean J; Kupermand, Samuel; Reiche, Theodore; Begleitera, Henri. "Beta power in the EEG of alcoholics." Biological Psychiatry, October 15, 2002: 831–842. Schmidt, E.M. "Single neuron recording from motor cortex as a possible source of signals for control of external devices." Annals of Biomedical Engineering, July 1, 1980: 339-349. Schmidt, EM, JS McIntosh, L Durelli, and MJ Bak. "Fine control of operantly conditioned firing patterns of cortical neurons." September 1978: 349-69. Semendeferi, K, A Lu, N Schenker, and H Damasio. "Humans and great apes share a large frontal cortex." Nature Neuroscience, March 2002: 272-276. Simeral, J D, S-P Kim, M J Black, J P Donoghue, and L R Hochberg. "Neural control of cursor trajectory and click by a human with tetraplegia 1000 days after implant of an intracortical microelectrode array." Journal of Neural Engineering, April 2011. Squire, L.R., and S. Zola-Morgan. "The medial temporal lobe memory system." Science, September 20, 2001: 1380-1386. Stuss, Donald T., Catherine A. Gow, and C. Ross Hetherington. ""No longer Gage": frontal lobe dysfunction and emotional changes." Journal of Consulting and Clinical Psychology, June 1992: 349-359. BIBLIOGRAPHY HUMAN COMPUTER INTERACTION VIA BRAINWAVES FOR DISABLED PEOPLE 73 Tam, Wing-Kin, Kai-yu Tong, Fei Meng, and Shangkai Gao. "A Minimal Set of Electrodes for Motor Imagery BCI to Control an Assistive Device in Chronic Stroke Subjects: A MultiSession Study." Transactions on Neural Systems & Rehabilitation Engineering, December 2011: 617-627. Turner, MR, A Al-Chalabi, MJ Parton, CE Shaw, and PN Leigh. "Prolonged survival in motor neuron disease: a descriptive study of the King's database 1990-2002." Journal of neurology, neurosurgery, and psychiatry, July 2003: 995-997. Wachspress, Samuel. "EEG." University of Illinois - Engineering Wiki. December 15, 2010. https://wiki.engr.illinois.edu/display/BIOE414/EEG (accessed September 7, 2013). Weinstein, G W. "Clinical aspects of the visually evoked potential." Transcriptions of the American Ophthalmological Society, 1977: 627–673. Wolpaw, Jonathan R.; Birbaumer, Niels; Heetderks, William J.; McFarland, Dennis J.; Peckham, P. Hunter; Schalk, Gerwin; Donchin, Emanuel; Quatrano, Louis A.; Robinson, Charles J.; Vaughan, Theresa M. "Brain-computer interface technology: a review of the first international meeting." Transactions on Rehabilitation Engineering, June 2000: 164 - 173. World Health Organization. "The world health report 2002 - Reducing Risks, Promoting Healthy Life." 2002. Wright, Frankling Pierce. "Emochat: Emotional instant messaging with the EPOC headset." Edited by University of Maryland. 2010. http://search.proquest.com/docview/851703167 (accessed August 28, 2013). Yalçin, AD, A Kaymaz, and H Forta. "Reflex occipital lobe epilepsy." Seizure, September 2000: 436-441. Zhang, Yan, Yonghong Chen, Steven L. Bressler, and Mingzhou Ding. "Response preparation and inhibition: The role of the cortical sensorimotor beta rhythm." Neuroscience, September 22, 2008: 238-246.