Brain-Computer Interface: comparison of two control modes to drive a virtual robot

BRAIN-COMPUTER INTERFACE: COMPARISON OF TWO CONTROL MODES TO DRIVE A VIRTUAL ROBOT Ron-Angevin Ricardo (PhD) ETSI Telecomunicación, University of Málaga, Spain Debeyre Aurélie, Marquet Yvan ENSC, Bordeaux INP, France Lespinet-Najib Véronique (PhD), Andre Jean Marc (PhD) Team CIH, Laboratory IMS CNRS UMR 5218, France Abstract: A Brain-Computer Interface (BCI) is a system that enables communication and control that is not based on muscular movements, but on brain activity. Some of these systems are based on discrimination of different mental tasks; usually they match the number of mental tasks to the number of control commands. Previous research at the University of Málaga (UMA-BCI) have proposed a BCI system to freely control an external device, letting the subjects choose among several navigation commands using only one active mental task (versus any other mental activity). Although the navigation paradigm proposed in this system has been proved useful for continuous movements, if the user wants to move medium or large distances, he/she needs to keep the effort of the MI task in order to keep the command. In this way, the aim of this work was to test a navigation paradigm based on the brain-switch mode for ‘forward’ command. In this mode, the subjects used the mental task to switch their state on /off: they stopped if they were moving forward and vice versa. Initially, twelve healthy and untrained subjects participated in this study, but due to a lack of control in previous session, only four subjects participated in the experiment, in which they had to control a virtual robot using two paradigms: one based on continuous mode and another based on switch mode. Preliminary results show that both paradigms can be used to navigate through virtual environments, although with the first one the times needed to complete a path were notably lower. Key Words: Brain-Computer Interface (BCI), virtual robot, switch mode, motor imagery (IM) 1-Introduction: A brain-computer interface (BCI) is based on the analysis of the brain activity, such as electroencephalographic (EEG) signals, recorded during certain mental activities, in order to control an external device. One of its main uses could be in the field of medicine, especially in rehabilitation. It helps to establish a communication and control channel for people with serious motor function problems but without cognitive function disorder (Wolpaw, Birbaumer, McFarland, Pfurtscheller & Vaughan, 2002). Amyotrophic lateral sclerosis (ALS), brain or spinal cord injury, cerebral palsy and numerous other diseases impair the neural pathways that control muscles or impair the muscles themselves. Some patients suffering this kind of diseases can neither communicate with the outside world nor interact with their environment. In this case, the only option is to provide the brain with a new and non-muscular communication and control channel by means of a BCI. EEG activity includes a variety of different rhythms that are identified by their frequency and their location. Mu (7-13Hz) and central beta (18-26Hz) rhythms are focused over sensorimotor cortex and recorded from the scalp over central sulcus. Sensorimotor rhythm-based BCIs (SMR-BCI) are based on the changes in mu and beta rhythms, which can be modified by voluntary thoughts through such specific mental tasks as motor imagery (MI), (Kübler & Müller, 2007); i.e. when a person performs a movement (or merely imagines it), it causes a synchronization/desynchronization in the neuron activity (event related synchronization/desynchronization, ERS/ERD) which involves a mu rhythm amplitude change (Neuper & Pfurtscheller, 1999). This relevant characteristic is what makes SMR suitable to be used as input for a BCI. Many BCI applications based on mental task discrimination allow the user to control simulated (Tsui, Gan & Roberts, 2009) or real mobile robots (Barbosa, Achanccaray, Meggiolaro, 2010), (Millán, Renkens, Mourino, & Gerstner, 2004). The vast majority of BCI system to control external device match the number of commands to the number of mental tasks. Having a higher number of commands implies higher information throughput and makes it easier for the subjects to navigate through the environment, since they have more choices to move. However, some studies proved that the best classification accuracy is achieved when only two classes are discriminated (Kronegg, Chanel, Voloshynovskiy, & Pun, 2007). One of the main objectives of the BCI research at the University of Málaga (UMA-BCI) is to provide a BCI system to freely control an external device (robot, wheelchair) based in the discrimination of only two classes. To obtain this objective, different paradigms have been proposed. In (RonAngevin, Velasco-Álvarez, Sancha-Ros & Da Silva-Sauer, 2011), subjects performed one MI task to extend a rotating bar that pointed to four possible commands in order to select them; two mental tasks are mapped this way into four navigation commands, allowing carry out discrete movements. On a later experiment (Velasco-Álvarez, Ron-Angevin, Da Silva-Sauer & SanchaRos, 2010), the same navigation paradigm was used to provide continuous movements: after the selection of a command, the movement was kept while the MI task was above certain threshold. Both paradigms have been used to control a virtual and a real robot (Ron-Angevin, Velasco-Álvarez, SanchaRos & Da Silva-Sauer, 2011), (Velasco-Álvarez, Ron-Angevin, da SilvaSauer & SanchaRos, 2013), and a virtual (Velasco-Álvarez, Ron-Angevin, Da Silva-Sauer & Sancha-Ros, 2010) and real wheelchair (Varona-Moya et al., 2015). Although a wheelchair controlled through a BCI system should provide continuous movements, in some situations this paradigm could have some disadvantages. If the user wants to move forward during a long period in order to cover medium or long distances, he/she needs to keep the effort of the MI task in order to keep the virtual wheelchair moving forward. A smart solution to this problem could be to apply the concept of a Brain-Switch (Mason, & Birch, 2000) to this paradigm. A BCI based on a brain-switch offers only an on/off control and only distinguishes between a predefined state and one specific mental task, therefore it fits the paradigm operating mode. In this way, for large distance, instead of keeping the ‘forward’ command active continuously, this one could be activated by a switch control. Once the subject decides to stop the movement, he/she deactivates the ‘forward’ command through another switch control action. This approach has been used by others BCI groups (Solis-Escalante, Müller-Putz, Brunner, Kaiser & Pfurtscheller, 2010), (Müller-Putz, Kaiser, Solis-Escalante & Pfurtscheller, 2010). The aim of the present study is to check the usefulness of this brainswitch mode for controlling a virtual robot. In order to obtain comparative results, subjects also control the virtual robot in continuous mode. 2Methods: 2.1Subjects and Data acquisition: Twelve naïve subjects (aged 21.52.2 years) participated in the study. As a design criterion, a maximum value of 30% in the error rate was considered to allow an efficient control of the paradigm. In the present study, only subjects who performed under this threshold in the calibration session (see section 2.2) continued with the navigation sessions. Finally, six out of the twelve subjects accomplished this criterion, being the others six subjects discarded due to their lack of control in the training sessions. The EEG was recorded using gold disc electrodes from two bipolar channels over left and right central areas. Channels were derived from two electrodes placed 2.5cm anterior and posterior to positions C3 and C4 (right and left hand sensorimotor areas, respectively) according to the 10/20 international system. The ground electrode was placed at the FPz position. Signals were amplified by a 16 channel biosignal g.BSamp (Guger Technologies) amplifier and then digitized at 128 Hz by a 12-bit resolution data acquisition NI USB-6210 (National Instruments) card. 2.2Initial training and signal processing: Before using the system to test the two paradigms, subjects had to follow an initial training that consisted of two sessions: a first one without feedback and a second one providing continuous feedback. As we have indicated in the previous section, those subjects who obtained a low error rate in the first session continued with the experiment. These two training sessions were used for calibration purposes. This training used the paradigm proposed by our group (UMA-BCI) in (Ron-Angevin & Díaz-Estrella, 2009), based on that proposed by the Graz group (Guger et al., 2001), in which subjects immersed in a virtual environment (VE) had to control the displacement of a car to the right or left, depending on the mental task carried out, in order to avoid an obstacle (a puddle), see Fig. 1. The training entailed discriminating between two mental tasks: mental relaxation and imagined right hand movements (right hand MI). The subjects did not receive any feedback in the first session, which was used to set up classifier parameters for the second session, in which continuous feedback was provided. In this first session, subjects were instructed to carry out four experimental runs consisting of 40 trials each. After a break of 5–10 min, the time necessary to do the offline processing (see (Ron-Angevin & Díaz-Estrella, 2009) for details) to determine the parameters for the feedback session, subjects participated in the second session. This feedback session consisted of one experimental run, intended to check the effectiveness of the chosen parameters and the ability of the subject to control his or her EEG signals. Figure 1: Timing of one trial of the training with feedback. The same parameters obtained were used to calibrate the system for the virtual environment (VE) navigation sessions. This processing is based in the procedure detailed in (Pfurtscheller, 2003), and consisted of estimating the average band power of each channel in predefined, subject-specific reactive frequency (manually selected) bands at intervals of 500 ms. In the feedback session, the movement of the car was computed on-line every 31.25 ms as a result of a Linear Discriminant Analysis (LDA) classification. The trial paradigm and all the algorithms used in the signal processing were implemented in MATLAB. 2.3Navigation Paradigm: The main objective of the BCI research at the University of Málaga is to provide an asynchronous BCI system (UMA-BCI) which, by the discrimination of only two mental states, offers the user several navigation commands to be used in a VE. An asynchronous (or self-paced) system must produce outputs in response to intentional control as well as support periods of no control (Schlögl, Kronegg, Huggins, & Mason, 2007); those are the socalled intentional control (IC) and non-control (NC) states, respectively. Both states are supported in the study presented in this paper: the system waits in a NC state in which an NC interface is shown (Fig. 2a). The NC interface enables subjects to remain in the NC state (not generating any command) until they decide to change to the IC state, where the control is achieved through the IC interface (Fig. 2b). Figure 2: a) NC interface (left) and b) IC interface (right) The NC interface consists of a semi-transparent vertical blue bar placed in the centre of the screen. The bar length is computed every 62.5 ms as a result of the LDA classification: if the classifier determines that the mental task is right-hand MI, the bar extends; otherwise, the bar length remains at its minimum size. In order to change from the NC to the IC state, the subject must accumulate more than a “selection time” with the bar over the “selection threshold”. If the length is temporarily (less than a “reset time”) lower than the selection threshold, the accumulated selection time is not reset, but otherwise it is set to zero. The IC interface is similar to the one presented in (Ron-Angevin, DíazEstrella, & Velasco-Álvarez, 2009): a circle divided into four parts, which correspond to the possible navigation commands (move forward, turn right move back and turn left), with a blue bar placed in the centre of the circle that is continuously rotating clockwise. The subject can extend the bar carrying out the MI task to select a command when the bar is pointing at it. The way the selection works in this interface is the same as in the NC interface, with the same selection and reset time and the same selection threshold. In the IC interface, another threshold is defined: stop threshold, which is lower than the selection threshold, and not visible to the subject. When it is exceeded, the bar stops its rotation in order to help the subject in the command selection. The rotation speed was fixed to 24 degrees every second, so it took 9 s to complete a turn if there was not any stop. Subjects receive audio cues while they interact with the system. When the state changes from IC to NC they hear the Spanish word for ‘wait’; the reverse change is indicated with ‘forward’, since it is the first available command in the IC state. Finally, every time the bar points to a different command, they can hear the correspondent word (‘forward’, ‘right’, ‘backward’ or ‘left’). In the next two sections, the two paradigms to be compared will be described, which are based in the interfaces explained above. 1) Continuous Mode Once a command is selected, the bar changes its color to red and the virtual robot starts moving forward or backward, or turning left or right at a fixed speed. The movement is maintained as long as the bar length is above the selection threshold (this means that the subject is still carrying out the MI mental task). If the bar is temporarily under this threshold (less time than the reset time), the movement stops, but the system allows the subject to continue the same movement if the bar again exceeds the selection threshold. While it happens, the bar keeps its red color to indicate this possibility to the subject. In the case that the bar remains under the selection threshold longer than the reset time, the bar changes its color to blue and continues rotating (if it is under the stop threshold) so that the subject can select a command again. The position of the rotating bar does not change; it takes its rotation up again from the same point at which it last stopped to select a command. In this way, the subject can select the same command several times in a row, in case the reset time passes without the subject wanting to stop the movement. 2) Switch Mode Once a command is selected, the movement starts (as it happened in the previous case) and the bar color is set to green. The main difference is that, in the present case, when the bar is shortened under the selection threshold the movement does not stop, but it is kept until the user enlarges the bar length above the selection threshold again (carrying out a MI mental task); at that moment the robot stops. Besides, as it was the case of a command selection, if the bar still remains above the threshold for the same “selection time”, the command is unselected and the bar turns blue and continues its rotation. If the time that the bar is above the threshold is lower than the “selection time”, the movement of the robot starts again. 2.4Experimental Procedure: This experiment consisted of controlling a virtual robot through a group of corridors which formed a sort of small maze. This proposed virtual robot Wolpaw, J.R., Birbaumer, N., McFarland, D.J., Pfurtscheller, G., & Vaughan, T.M. (2002). Brain-computer interfaces for communication and control. Clinical Neurophysiology 113(6), 767-791.