Repositorio Institucional de Documentos

Tesis de M´aster M´aster en Ingenier´ıa de Sistemas e Inform´atica Curso 2009-2010 Sistema de Telepresencia controlado por una Interfaz Cerebro-Computador: Pruebas iniciales con pacientes de ELA Carlos Escolano Marco Septiembre de 2010 Director: Javier M´ınguez Zafra Departamento de Inform´atica e Ingenier´ıa de Sistemas Centro Polit´ecnico Superior Universidad de Zaragoza i Resumen Las interfaces cerebro-ordenador (BCIs, siglas de su t´ermino ingl´es Brain-Computer Interfaces) proporcionan a sus usuarios comunicaci´on y control ´unicamente con su actividad cerebral. ´ Estas no dependen de los canales de salida habituales del cerebro de nervios perif´ericos y m´usculos, abriendo un nuevo y valioso canal de comunicaci´on para personas con enfermedades neurol´ogicas o musculares severas, tales como la esclerosis lateral amiotr´ofica (ELA), infarto cerebral, par´alisis cerebral, y da˜nos en la m´edula espinal. La combinaci´on de las interfaces cerebro-ordenador con la rob´otica puede dotar a los usuarios de una entidad f´ısica personificada en un entorno real (en cualquier parte del mundo con acceso a Internet) preparada para percibir, explorar, e interaccionar, controlada ´unicamente con la actividad cerebral. Adem´as, ha sido sugerido que este tipo de sistemas podr´ıa proporcionar beneficios en patientes de ELA dentro del contexto de neurorehabilitaci´on o mantenimiento de la actividad neural. Esta tesis fin de m´aster presenta el proceso completo de desarrollo de un prototipo inicial de un sistema de telepresencia basado en BCIs y su evaluaci´on con usuarios sanos, y su posterior redise˜no (para cubrir las necesidades de pacientes reales) y evaluaci´on con pacientes de ELA. Los resultados mostraron la viabilidad de esta tecnolog´ıa en pacientes reales. 1 Abstract Brain-computer interfaces (BCIs) provide their users communication and control with their brain activity alone. They do not rely on the brain’s normal output channels of peripheral nerves and muscles, opening a new valuable communication channel for people with severe neurological or muscular diseases, such as amyotrophic lateral sclerosis (ALS), brainstem stroke, cerebral palsy, and spinal cord injury. The combination of braincomputer interfaces with robotics can provide the users with a physical entity embodied in a real environment (anywhere in the world with Internet access) ready to perceive, explore, and interact, and controlled only by brain activity. Furthermore, it has been suggested that this kind of systems could provide benefits for ALS patients in the context of neurorehabilitation or maintainment of the neural activity. This master thesis reports the entire process of development of an initial prototype of a brain-actuated telepresence system and its evaluation with healthy users, and its later redesign (in order to address the real patients needs) and evaluation with ALS patients. The results showed the feasibility of this technology in real patients. 2 Index 1 Introduction 7 1.1 Project scope and document structure . . . . . . . . . . . . . . . . . . . . 8 2 Prototype 1: Technology 10 2.1 Brain-Computer System . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.1.1 Neurological protocol and instrumentation . . . . . . . . . . . . . . 11 2.1.2 Graphical interface . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.1.3 Pattern recognition strategy . . . . . . . . . . . . . . . . . . . . . . 13 2.2 RoboticSystem................................. 14 2.2.1 Navigation technology . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.3 Integration and execution protocol . . . . . . . . . . . . . . . . . . . . . . 15 3 Prototype 1: Validation on healthy users 16 3.1 Experimental Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 3.1.1 Participants............................... 16 3.1.2 Experiment Design and Procedures . . . . . . . . . . . . . . . . . . 16 3.1.2.1 Phase I: Screening and Training Evaluation . . . . . . . . 16 3.1.2.2 Phase II: Brain-Actuated Telepresence Evaluation . . . . . 18 3.2 ResultsandEvaluation............................. 18 3.2.1 Technical Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.2.1.1 Global Evaluation . . . . . . . . . . . . . . . . . . . . . . 19 3.2.1.2 Brain-Computer System . . . . . . . . . . . . . . . . . . . 20 3.2.1.3 Robotic System . . . . . . . . . . . . . . . . . . . . . . . . 21 3.2.2 Users’ Behavior Evaluation . . . . . . . . . . . . . . . . . . . . . . . 22 3.2.2.1 Execution Analysis . . . . . . . . . . . . . . . . . . . . . . 22 3.2.2.2 Activity Analysis . . . . . . . . . . . . . . . . . . . . . . . 23 3.2.2.3 Psychological Assessment . . . . . . . . . . . . . . . . . . 24 3.2.3 Variability Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 4 Prototype 2: Technology 28 4.1 Requirements .................................. 28 4.1.1 Functional Requirements . . . . . . . . . . . . . . . . . . . . . . . . 28 4.1.2 Non-Functional Requirements . . . . . . . . . . . . . . . . . . . . . 29 4.2 Development .................................. 29 4.2.1 Brain-Computer System . . . . . . . . . . . . . . . . . . . . . . . . 30 4.2.1.1 Neurological protocol and instrumentation . . . . . . . . . 30 3 4.2.1.2 Graphical interface . . . . . . . . . . . . . . . . . . . . . . 31 4.2.1.3 Pattern recognition strategy . . . . . . . . . . . . . . . . . 31 4.2.2 RoboticSystem............................. 32 4.2.3 Integration and execution protocol . . . . . . . . . . . . . . . . . . 34 5 Prototype 2: Validation on ALS patients 37 5.1 Experimental Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 5.1.1 Participants............................... 37 5.1.2 Experiment Design and Procedures . . . . . . . . . . . . . . . . . . 37 5.1.2.1 Phase I: Screening and Training Evaluation . . . . . . . . 38 5.1.2.2 Phase II: Brain-Actuated Navigation and Exploration Evaluation............................. 38 5.1.2.3 Phase III: Brain-Actuated Global Evaluation . . . . . . . 39 5.2 Results and Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 5.2.1 PhaseI ................................. 40 5.2.2 PhaseII................................. 41 5.2.3 PhaseIII ................................ 42 6 Conclusions 43 Bibliography 44 4 List of Figures 1.1 Projectscope .................................. 9 2.1 Prototype 1: Technology overview . . . . . . . . . . . . . . . . . . . . . . . 10 2.2 P300 visual evoked potential . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.3 Prototype 1: Graphical interface of the brain-computer system . . . . . . . 12 2.4 P300 r2distribution in a screening trial . . . . . . . . . . . . . . . . . . . . 14 3.1 Prototype 1 evaluation: Brain-Actuated Telepresence Tasks . . . . . . . . . 17 3.2 Prototype 1 evaluation: Psychological assessment . . . . . . . . . . . . . . 25 4.1 Prototype 2: Technology overview . . . . . . . . . . . . . . . . . . . . . . . 29 4.2 Prototype 2: Graphical interface of the brain-computer system . . . . . . . 32 4.3 Prototype 2: Robot device and graphical interface of the robotic system . . 33 4.4 Prototype 2: Communication system design . . . . . . . . . . . . . . . . . 36 5.1 Prototype 2 evaluation: Brain-Actuated Navigation and Exploration Evaluation ...................................... 39 5.2 Prototype 2 evaluation: EEG desynchronization example . . . . . . . . . . 40 5 Table Index 3.1 Metrics to Evaluate the Global Performance of Prototype 1 . . . . . . . . . 19 3.2 Metrics to Evaluate the Brain-Computer System of Prototype 1 . . . . . . 20 3.3 Metrics to Evaluate the Robotic System of Prototype 1 . . . . . . . . . . . 21 3.4 Metrics for the Execution Analysis of Prototype 1 . . . . . . . . . . . . . . 22 3.5 Metrics for the Activity Analysis of Prototype 1 . . . . . . . . . . . . . . . 24 3.6 Metrics for the Variability Study of Prototype 1 . . . . . . . . . . . . . . . 26 3.7 Intrauser Variability of Prototype 1 . . . . . . . . . . . . . . . . . . . . . . 26 3.8 Interuser Variability of Prototype 1 . . . . . . . . . . . . . . . . . . . . . . 26 5.1 Metrics to Evaluate the Classifier Accuracy of Prototype 2 . . . . . . . . . 40 5.2 Metrics to Evaluate the Telepresence System Performance of Prototype 2 . 41 6 1. Introduction Restoring human motor functions has been a fascinating and yet frustrating area during the last century. The possibility of interfacing the human nervous system with mechatronic devices, and then employ these devices to restore neurological and motor functions has long fascinated scientists. The paradigmatic scenario is a patient following a medulla lesion or a chronic neuromuscular disease where the flow of motor neural information has been interrupted. In this direction recently there has been a great impulse towards research and development of brain-actuated devices due to the great advances in braincomputer interaction. Brain-computer interfaces (BCIs) provide their users communication and control with their brain activity alone. They do not rely on the brain’s normal output channels of peripheral nerves and muscles, opening a new valuable communication channel for people with severe neurological or muscular diseases, such as amyotrophic lateral sclerosis (ALS), brainstem stroke, cerebral palsy, and spinal cord injury. ALS is a progressive neurological degenerative disease that leads to the locked-in syndrome (LIS), which is characterized by complete motor paralysis, except for eye movement, with intact cognition and sensation [1]. The ability to work with non-invasive recording methods, being the most popular the electroencephalogram (EEG), is one of the major goals for the development of brainactuated systems for humans. Some examples of EEG-based applications include the control of a mouse on the screen [2], communication like an speller [3], Internet browsers [4], etc. The first brain-actuated control of a physical device was demonstrated in 2004 [5] and since then, research has been mainly focused on wheelchairs [6, 5], manipulators [7], small-size humanoids [8] and neuroprosthetics [9], to name a few. All these developments have a property in common: the user and the robot are placed in the same environment. Limited research has been focused in applications where the human and the robot are not co-located such as in robot teleoperation. The ability to brain-teleoperate robots in a remote scenario could provide disabled patients with telepresence by means of a physical device embodied in a real environment (anywhere in the world) ready to perceive, explore, manipulate, and interact; controlled only by brain activity. Furthermore, it has been suggested that the engagement of ALS patients in using such BCIs could lead to a neurorehabilitation effect and/or a maintainment of the neural activity avoiding or delaying this way the extinction of thought, hypothesized to happen in those patients [1]. In this direction, the author developed an initial prototype of an EEG-based brainactuated telepresence system to provide the user with presence in remote environments through a mobile robot, with access to the Internet [10]. The principle of operation was a person using a BCI and concentrated on a visual display, which showed the environ- 7 2. Prototype 1: Technology 2.2 Robotic System Figure 2.4: r2values for each location in the interval [0 - 800] ms after the stimulus target onset. Values are displayed in a color scale. The higher values are found at a latency of roughly 300 ms. 2.2 Robotic System The robot was a commercial Pioneer P3-DX. It was equipped with a laser sensor, a camera, back wheels (that worked in differential-drive mode), wheel encoders (odometry) and a network interface card. The main sensor was a SICK planar laser placed on its frontal part, which operated at a frequency of 5 Hz with a 180◦field of view and 0.5◦resolution (361 points). The camera, placed on the laser, was a pan/tilt Canon VC-C4 with a ±100◦ pan field of view and a 90◦/−30◦tilt field of view. The initial camera orientation was set to 0◦pan and −11.5◦tilt in the experimental sessions, which provided a centered perspective of the environment starting roughly one meter in front of the robot. The robot was equipped with a computer, an Intel @ 700MHz with Linux OS. This computer managed all the computational tasks, provided access to the hardware elements through the player platform [19], and integrated the autonomous navigation system. 2.2.1 Navigation technology The objective of the navigation technology was to drive the robot to given destinations while also avoiding collisions with the obstacles (static and dynamic), detected by the laser sensor [20]. This technology incorporated real-time adaptative motion planning and modeling construction, and it was able to deal with unknown (non-preprogramed) and dynamic scenarios. It was composed by two modules: (i) a model builder, and (ii) a local planner. The model builder integrates the sensor measurements to construct a local model of the environment (static and dynamic parts) and to track the vehicle location. This module corrects the robot’s position as well as it constructs and updates the map to detect and track the moving objects around the robot [21]. On the other hand, the local planner computes the local motion based on a hybrid combination of tactical planning and reactive collision avoidance. It used an efficient dynamic navigation function (D∗Lite planner [22]) to compute the tactical information (i.e. main direction of motion), which 14 2. Prototype 1: Technology 2.3 Integration and execution protocol is well suited for unknown and dynamic scenarios. The final motion of the vehicle is computed using the ND technique [23], which has the distinct advantage of being able to cater to the complex navigational tasks such as maneuvering in the environment within constrained spaces (i.e. passage through a narrow doorway). 2.3 Integration and execution protocol The communication system performed the integration between the brain-computer system and the robotic system. The design of this system allows to teleoperate the robot in any remote environment with Internet access. The software architecture was based on the TCP/IP protocol and the client/server paradigm. It was composed of two clients (one for the brain-computer system and one for the robotic system) and a link server that concentrated the information flow and conferred scalability to the system. The BCI client was multiplexed in time with the BCI system, with a period of roughly 30 ms, and communicated with the link server through an Internet connection. On the other hand, the robot client communicated with the link server by means of an ad-hoc local wireless connection. The upper boundary of the information transfer was set by the video transfer rate. The images captured by the camera were compressed to the jpeg standard format, obtaining an image size of approximately 30 Kbytes. In the experimental sessions 10 images/s were transferred, thus obtaining a transfer rate of approximately 300 KB/s, adequate for the typical bandwidth order of Internet networks. The user interacts with the telepresence system using the functionalities provided by the graphical interface according to the execution protocol. It is modeled as a finite-state machine: (i) the BCI graphical interface develops a stimulation process (flashing) over all the possible options following the P300 oddball paradigm; (ii) the pattern recognition strategy detects the target the user was attending; (iii) once the desired target is selected, the user must subsequently select the validation option to send the target to the robotic system (this redundancy minimizes the probability of sending incorrect orders to the robotic system although BCI errors happen); (iv) the robotic system executes the order (this will be referred as a mission); (v) while the mission is being performed the robot sends live video to the user station. Then, when the mission has finished, the loop starts again. 15 3. Prototype 1: Validation on healthy users This chapter reports the evaluation of the initial brain-actuated telepresence prototype on healthy users: the experimental methodology (Section 3.1) and the results of the experiments (Section 3.2). 3.1 Experimental Methodology An experimental methodology was defined to carry out a technical evaluation of the system, to assess the degree of adaptability to the users, and the degree of homogeneity and variability of the system. The experimental sessions were performed by able-bodied users in real settings. The recruitment of the participants and the experimental protocol are discussed next. 3.1.1 Participants The participants were recruited following a set of inclusion and exclusion criteria in order to obtain conclusions over a homogeneous population. Five healthy, 22 years old, male and right-handed students of the University of Zaragoza were recruited. They had neither utilized the telepresence system nor participate in BCI experiments before. The study was approved by the University of Zaragoza Institutional Review Board. After being informed about the entire protocol, all participants signed informed consent. 3.1.2 Experiment Design and Procedures The study was accomplished in the BCI Laboratory of the University of Zaragoza. It was divided into two phases: (i) a screening and training phase, and (ii) a brain-actuated telepresence phase. Each phase lasted one week. 3.1.2.1 Phase I: Screening and Training Evaluation This phase was composed by two tasks: (i) a screening task to study the P300 response and to validate the graphical interface design, and (ii) a training task to calibrate the system and to measure the BCI accuracy. Initially, the visual aspects of the graphical interface were selected adapting the results of a parallel study [6]. Furthermore, the images were captured in black & white to preserve a high saliency of stimuli; and the initial camera 16 3. Prototype 1: Validation on healthy users 3.1 Experimental Methodology (a) Task 1 (b) Task 2 (c) Task 1 trajectories (d) Task 2 trajectories Figure 3.1: (a) The objective of Task 1 was to drive the robot from the start location to the goal area. In the exploration area (E.A. in the figure), the participant had to search for two signals located in the yellow cylinders 2.5 m above the floor. Then, if both signals were equal, the participant had to avoid the yellow triangle by turning to the right-hand side, or if otherwise, by turning to the left-hand side. (b) The objective of Task 2 was to drive the robot from the start location to the goal area. In the exploration area, the participant had to search for one signal located in the yellow cylinder 2.5 m above the floor. The participant then had to continue navigation to the right or left direction of the two cylinders, as specified by the signal. All measures are in meters and the robot is to scale. (c) and (d) Maps generated by the autonomous navigation system (black zones indicate obstacles, white zones known areas and gray zones unknown areas). The robot trajectories of one trial per participant are shown. orientation was set to 0◦pan and −11.5◦tilt to provide a centered perspective of the environment starting roughly one meter in front of the robot. The final aesthetic factors of the visual display are shown in Figure 2.3. The scheduling of the stimulation process was also set for both tasks according to the parallel study. The inter-stimulus duration was set to 75 ms, the stimulus duration was set to 125 ms, and the number of sequences was set to 10. The objective of Task 1 was to validate the design of the graphical interface with real participants, studying whether the P300 response was elicited or not. In order to achieve that objective, an experimental session of 8 screening trials was designed. In each trial, the participants had to attend to a predefined sequence of 10 targets. After its execution, they were asked to fill in neuropsychological and cognitive assessment forms. Regarding Task 2, the system was trained with the previously recorded EEG. Then, a battery of online trials was designed to check whether the accuracy of the system was greater than a threshold value set to 90%, qualifying the participant for the second phase. For each participant this phase lasted 3 hours. 17 3. Prototype 1: Validation on healthy users 3.2 Results and Evaluation 3.1.2.2 Phase II: Brain-Actuated Telepresence Evaluation This phase consisted of a battery of online experiments with the telepresence system in order to carry out a technical evaluation of the system, to assess the degree of adaptability to the participants, and the degree of homogeneity and invariability of the system. The experiments were accomplished the week of June 23, 2008, between the BCI Laboratory at the University of Zaragoza (Spain) and the University of Vilanova i la Geltr´u (Spain), separated by 260 km. Two tasks were designed, which jointly combined navigation and visual exploration in unknown scenarios and under different working conditions. Each participant had to perform two trials for each task. Task 1 involved complex navigation in constrained spaces with the active search of two visual targets. Task 2 involved navigation in open spaces with the active search of one visual target. The maps of the circuits designed for each task are shown in Figure 3.1. Note that those maps were the only information of the remote environments shown to the participants and they had never been physically there. Regarding the scheduling of the stimulation process, the interstimulus duration was set to 75 ms, the stimulus duration was set to 125 ms, and the number of sequences was customized for each participant according to the results thrown by the classifier in the calibration process. They were set to the minimal number of sequences that allowed the participant to achieve a theoretical accuracy higher than 90%. After each trial, they were asked to fill in neuropsychological and cognitive assessment forms. For each participant this phase lasted 4 hours. 3.2 Results and Evaluation This section reports the obtained results in the experimental phases. Phase I was composed by a screening and a training task. Regarding the screening task, it was found by visual inspection of the EEG data recorded that the P300 potential was elicited for all participants. Furthermore, participants reported high satisfaction in the psychological assessments. Thus, the graphical interface was validated. Regarding the training task, the pattern recognition strategy was trained and the participants performed the online tests. All of them achieved more than 93% BCI accuracy; and thus, all of them were qualified to carry out the next phase. Phase II consisted of the execution of the predefined telepresence tasks, jointly combining navigation and visual exploration. Firstly, the participants performed 4 offline trials to gather EEG data to train the pattern recognition strategy. Then, the experiments were performed. On the basis of these experiments a technical evaluation of the teleoperation system, a users’ behavior study, and a variability analysis among trials and participants are described next. The overall result is that all participants were able to complete the designed tasks reporting no failures, which shows the robustness of the system and its feasibility to solve tasks in real settings where jointly navigation and visual exploration are needed. Furthermore, the participants showed a great adaptation as well as a high homogeneity. 18 3. Prototype 1: Validation on healthy users 3.2 Results and Evaluation 3.2.1 Technical Evaluation The technical evaluation is composed of a global evaluation of the brain-actuated telepresence system and a particular evaluation of its two main systems (brain-computer system and robotic system). 3.2.1.1 Global Evaluation Following [24, 6], the subsequent metrics were proposed: (i) task success; (ii) number of collisions of the navigation system; (iii) time taken to accomplish the task; (iv) path length traveled by the robot; (v) number of missions 1to complete the task; (vi) BCI accuracy; (vii) BCI selection ratio (i.e. ratio of time the user was selecting orders); and (viii) navigation ratio (i.e. ratio of time spent in navigation mode). The results are summarized in Table 3.1. Table 3.1: Metrics to Evaluate the Global Performance of Prototype 1 Task 1 Task 2 min max mean std min max mean std Task success 1 1 1 0 1 1 1 0 # collisions 0 0 0 0 0 0 0 0 Time (s) 685 1249 918 163 706 1126 910 154 Path length (m) 10.99 13.53 11.84 0.90 19.68 21.83 20.68 0.63 # missions 12 19 13.9 2.30 10 15 11.70 1.64 BCI accuracy 0.83 1.00 0.92 0.07 0.78 1.00 0.89 0.07 BCI select. ratio 46.28 60.47 52.85 4.51 44.62 59.11 52.12 4.64 Nav. ratio 64.81 76.61 70.90 3.58 70.43 85.55 78.28 4.43 All the participants succeeded to perform all the trials reporting no collisions, which denotes the robustness of the system. The time, path length and number of missions were very similar for all the participants indicating a similar performance among them (those metrics will be further discussed from the point of view of the participants in the users’ behavior section). The real robot trajectories are displayed in Figure 3.1. Although the variations in the BCI accuracy, the BCI interaction was satisfactory since the BCI accuracy was always higher than 78%, achieving a mean performance of 90%. The BCI selection ratio was 52% on average, which indicates the great importance that has the BCI accuracy in the global system performance. Regarding the ratio of usage of both operation modes (exploration ratio is the complementary of navigation ratio), both operation modes were used to complete the tasks. It can also be inferred that the system adapted to the different working conditions of the tasks. Task 1 presents a higher exploration ratio than Task 2 because it involved a more complex visual exploration, whereas Task 2 presents a higher navigation ratio than Task 1 because it involved navigation in open spaces and a more simple visual exploration. In summary, the results were very encouraging because they showed the feasibility of the technology to solve tasks in which jointly navigation and visual exploration were needed, and under different working conditions. Furthermore, it must be noted that 1Missions are defined in the execution protocol (section 2.3). 19 3. Prototype 1: Validation on healthy users 3.2 Results and Evaluation the system was calibrated in less than an hour, the participants had never used a BCI, and they only received a short briefing of the system operation before performing the experiments. 3.2.1.2 Brain-Computer System The brain-computer system evaluation was divided into an evaluation of the pattern recognition strategy performance (BCI accuracy) and an evaluation of the visual display design. Based on [24, 6], the following metrics were proposed to assess the pattern recognition strategy performance: (i) total errors; (ii) useful errors; (iii) real BCI accuracy; (iv) useful BCI accuracy (i.e., the BCI accuracy computed using the correct selections plus useful errors); (v) number of selections per minute; (vi) usability rate (number of selections per mission); (vii) number of missions per minute; (viii) number of sequences; and (ix) information transfer rate (ITR) according to the Wolpaw definition [25] 2. The results are summarized in Table 3.2. Table 3.2: Metrics to Evaluate the Brain-Computer System of Prototype 1 Task 1 Task 2 min max mean std min max mean std # total errors 0.00 7.00 3.50 2.88 0.00 11.00 4.9 3.7 # useful errors 0.00 2.00 0.60 0.84 0.00 5.00 1.20 1.81 Real BCI acc. 0.81 1.00 0.90 0.08 0.73 1.00 0.86 0.09 Useful BCI acc. 0.83 1.00 0.92 0.07 0.78 1.00 0.89 0.07 # selections/min 3.39 5.49 4.41 0.72 3.40 4.77 4.16 0.46 Usability rate 2.11 3.08 2.54 0.34 2.36 3.40 2.80 0.39 # missions/min 1.17 2.27 1.77 0.39 1.00 2.02 1.53 0,33 # sequences 6 10 8 1.33 8 10 8.4 0.84 ITR (bits/min) 9.97 21.73 16.05 3.83 9.86 20.62 14.32 3.33 # misunderstandings 0 0 0 0 0 1 0.10 0.32 Real BCI accuracy was very high, above 85% on average. In some cases, although BCI detects a non-desired target for the participant, this target is reused to complete the task (this is a common situation in open spaces, where a task can be solved in many different ways). These errors are referred to as useful errors because they do not increment the time to set a mission to the system. Useful errors turn the useful accuracy higher than the real one. Useful accuracy was 90% on average. The BCI system set only two incorrect missions in all executions, representing 0.78% of all missions (the theoretical probability of this situation was 0.3%). Regarding the stimulation process, the number of sequences was customized for each participant, and all of them were in the range from 6 to 10. The number of sequences determined the number of selections per minute, which was approximately 4. The usability rate was slightly greater than 2 (ideally it is equal to 2, i.e. a mission needs at least 2 selections when no BCI detection errors occur) due to BCI errors and interface misunderstandings. The number of missions per minute, determined by the number of selections per minute and the usability rate, was 1.65 on average. The 2B= log2N+Plog2P+ (1 −P) log2 1−P N−1where B is the number of bits per trial (i.e. bits per selection), N is the number of possible selections, and P is the probability that a desired selection will occur. 20 3. Prototype 1: Validation on healthy users 3.2 Results and Evaluation ITR of the BCI system was set to 15 bits/min on average. Note that low ITRs is a common problem of all event-related potential approaches, which is in part overcame by the taskrelevant options provided by the graphical interface and the automation capabilities of the robotic system. The second part of the brain-computer system evaluation is the visual display design. Based on [24, 6] the following metrics were proposed: (i) number of errors caused by misunderstandings in the interface; (ii) usability rate (number of selections per mission); (iii) option utility (option usage frequency). The results are also summarized in Table 3.2. The design of the interface was correct since it allowed the participants to achieve the tasks with a short briefing of its functionalities. There was only one incorrect selection due to misunderstandings in the interface, which arose at the very end of one trial (a participant set an unreachable mission, located behind the goal wall). The option utility for all the options and participants is not reported, but it was always greater than zero for all options and participants, thus indicating that were no useless options. Furthermore, it also suggests that the participants found it simple. In summary, these results show a good integration between the visual display and the designed stimulation process since they allowed the participants to successfully complete all the trials achieving high BCI accuracies. Furthermore, the graphical interface was usable, easy to understand, and showed a great integration with the robotic system. 3.2.1.3 Robotic System Based on [24, 6], a set of metrics was proposed to evaluate the two operation modes of the robotic system: (i) number of navigation missions; (ii) length traveled per mission; (iii) mean velocity of the robot; (iv) minimum and mean distance to the obstacles; (v) number of exploration missions; and (vi) total angle explored by the camera. The results are summarized in Table 3.3, which is divided into two sections, each one relevant to a operation mode. Table 3.3: Metrics to Evaluate the Robotic System of Prototype 1 Task 1 Task 2 min max mean std min max mean std # missions nav. 7 12 9.00 1.6 7 11 8.7 1.2 Length(m)/mission 1.06 1.61 1.34 0.18 1.90 2.81 2.41 0.29 Velocity (m/sec) 0.05 0.07 0.06 0.01 0.08 0.10 0.10 0.01 Clearance min (m) 0.89 1.12 1.03 0.07 1.09 1.19 1.14 0.03 Clearance mean (m) 2.22 2.47 2.40 0.07 3.16 3.23 3.20 0.02 # missions exp. 4 7 4.9 1.2 2 5 3 1.1 Exploration (rad) 1.21 6.37 2.79 1.56 0.16 0.88 0.37 0.25 Regarding the navigation mode, a total of 177 navigation missions were carried out without collisions, traveling 325 meters with a mean velocity of 0.08 m/s (10 times less than usual human walking velocity). The mean velocity and the length traveled per mission were greater in Task 2 than in Task 1, which denotes that the navigation system adapted to the different conditions of the environments, obtaining a velocity increase 21 3. Prototype 1: Validation on healthy users 3.2 Results and Evaluation in open spaces (Task 2) and a reduction when maneuverability became more important (Task 1). The mean and minimum clearances show that the vehicle carried out obstacle avoidance with safety margins, which is one of the typical difficulties in autonomous navigation [20]. Regarding the exploration mode, a total of 79 missions were carried out, exploring a total angular distance of 3.2 radians. In general, the performance of the robotic system was remarkable since the navigation missions were successfully executed reporting no failures, and the exploration system provided a good enough visual feedback of the remote environment and enough functionalities to actively explore it. 3.2.2 Users’ Behavior Evaluation An evaluation of the users’ behavior was carried out to measure the degree of adaptability of the brain-actuated telepresence system to the participants. Three studies were defined to achieve that objective: (i) an execution analysis to study the participants’ performance; (ii) an activity analysis to study the interaction strategy with the robot; and (iii) a psychological assessment to study the participants’ workload, learnability and level of confidence. 3.2.2.1 Execution Analysis The next metrics, based on other studies [24, 6], were defined: 1. Task success. 2. Number of missions. 3. Path length. 4. Execution time. 5. Useful BCI accuracy. The results are summarized in Table 3.4, which shows the average of the two trials executed in each task for each participant. Table 3.4: Metrics for the Execution Analysis of Prototype 1 Task 1 Task 2 P1 P2 P3 P4 P5 P1 P2 P3 P4 P5 Task success 1 1 1 1 1 1 1 1 1 1 # missions 12.5 15.5 13.5 13.0 15.0 12.5 11.5 11.5 12.5 10.5 Path length (m) 11.6 12.3 11.3 12.3 11.6 20.9 21.3 20.2 20.4 20.5 Exec.time (s) 746 1135 880 1001 826 964 1070 784 1011 717 BCI acc. (%) 93 92 93 85 99 90 87 91 81 98 The number of missions is an indicator of the intermediate steps required to complete the tasks. Although this metric is strongly related to the interaction strategy (discussed in next subsection), one can infer that some participants presented a more efficient mission 22 3. Prototype 1: Validation on healthy users 3.2 Results and Evaluation selection. Participants 1 and 4 showed the more efficient mission selection in Task 1; while participants 2, 3 and 5 did in Task 2. This metric suggests that there are two groups of participants which adapted in a different way to the environment conditions. One group adapted better to the constrained environment of Task 1, and the other one adapted better to the open spaces in Task 2. Path length is another metric of the individual performance in the use of the telepresence system. Participants 3 and 5 presented the lower path lengths in both tasks, showing a better adaptation to the automation capabilities of the system. Execution time is mainly conformed of two factors, the BCI accuracy and the mission selection performance, which are the factors that can increase the number of selections required to complete the tasks. The variability among participants in the number of missions was lower than the variability in BCI accuracy, which translates the higher BCI accuracies in the best execution times. Participants 1, 3 and 5 presented the higher BCI accuracy and consequently the best execution times. In general, the fact that all participants succeeded in solving the complete tasks with zero collisions suggests a high adaptability to the potential users. 3.2.2.2 Activity Analysis This section studies the interaction strategy of the participants in the teleoperation of the robot. Regarding those robotic devices that provide automation facilities, there are two types of interaction strategies that can be applied: the supervisory-oriented interaction and the direct control-oriented interaction [26]. Supervisory-oriented interaction extensively explores the automation capabilities (mainly trajectory planning and obstacle avoidance in navigation mode) minimizing the user intervention; whereas direct controloriented interaction is characterized by an increased user intervention minimizing the use of the automation capabilities. In the concrete case of the system developed, supervisoryoriented interaction will be characterized by a high number of far destinations in the navigation, and the direct control-oriented interaction will be characterized by a higher number of near range destinations or left/right turn selections. The following metrics, adapted from [24, 6], have been defined to study whether the participants followed any of the previously described interaction strategies:  Activity discriminant (DA): ratio of goal selections minus total of turn selections to the total number of selections. DA=]Dest. −]Turns ]Selections  Path length per mission (PM).  Robot motion time per mission (TM).  Control activity descriptor (CA): ratio of turn selections to the total number of selections. CA=]Turns ]Selections 23 4. Prototype 2: Technology 4.2 Development The next subsections outline the main systems that compose the brain-actuated robotic telepresence system (brain-computer system, robotic system, and integration), focusing on the new functionalities with regard to the initial prototype. 4.2.1 Brain-Computer System The brain-computer system was redesigned in order to add the next functionalities:  To provide the user with communication or interaction capabilities with the robot station by means of a set of pre-configurable sentences, binary responses or alarms.  To incorporate the possibility to pause the system (the system is referred to be paused when the stimulation process is stopped while visual and auditive feedback from the remote environment is received).  To play the sound captured by the microphone in the robot station. These functionalities were incorporated within the BCI2000 platform. This platform is based on events and runs on Windows OS. It is composed by three modules, which perform the online data acquisition, the signal processing, and the feedback to the user, respectively. Regarding the second functionality, the events system of the BCI2000 platform was modified to allow periods of time in which no visual stimulation is developed and the information of the remote environment is received. Regarding the third functionality, the sound is played using the multimedia library of the Windows API, which imposes a sound data format, and must be taken into account in the integration of both systems (see subsection 4.2.3). In order to allow the user to select the new options (first functionality ones and the pause option), the graphical interface was modified. The different components of the brain-computer system are next described. 4.2.1.1 Neurological protocol and instrumentation The brain-computer system relied on the same neurophysiological protocol, the P300 visual-evoked potential, which has been successfully used for communication in ALS patients [11, 27, 28]. The main advantage of using this protocol with ALS patients is the short-time of the calibration process. The BCI instrumentation was also based on the gTec commercial system, however, more sensors were decided to use because of the objectives of the experimental sessions (see subsection 5.1.2). Finally, 24 EEG electrodes were placed on FP1, FP2, F3, F4, C3, C4, P3, P4, T7, T8, CP3, CP4, Fz, Pz, Cz, OZ, FC3, FC4, F7, F8, P7, P8, FCz and CPz according to the international 10/20 system (this configuration covers all the positions used in the experimental sessions of prototype 1). The ground electrode was positioned on the forehead (position Fz) and the reference electrode was placed on the left earlobe. The EEG was amplified, digitalized with a sampling frequency of 256 Hz, power-line notch filtered and bandpass-filtered between 0.5 and 30 Hz. 30 4. Prototype 2: Technology 4.2 Development 4.2.1.2 Graphical interface The brain-computer system of the initial system incorporated a graphical interface with two functionalities: (i) allowed the user to control the robotic system and to receive a visual feedback by means of the visual display, and (ii) elicited the P300 potential developing a stimulation process. Regarding the first functionality, the graphical interface was redesigned in order to allow the users to interact with the people in the vicinity of the robot by sending preconfigurable sentences, binary responses or alarms. Furthermore, a new option to enable the user to pause the system was incorporated. Thus, it was decided to arrange the options in a 4 ×4 matrix and to incorporate a new interface, the interaction interface (Figure 4.2). The robot navigation mode and the camera exploration mode maintained the same functionalities of the initial prototype, but the number of options in the central panel were reduced from a (3 ×5) matrix to a (3 ×3) matrix. In the robot navigation mode those options were set to (1.5m, 2.5m, 4m)×(−10◦,0◦,10◦,). The row of icons in the lower part of the display represented the following options, from left to right: (i) turn the robot 45◦left; (ii) validate the previous selection; and (iii) turn the robot 45◦right. In the camera exploration mode, the row of icons in the lower part of the display represented the following options, from left to right: (i) align the robot with the horizontal camera orientation and change to navigation mode 1; (ii) validate the previous selection; and (iii) set the camera to its initial orientation. In the interaction mode the visual display showed a 2D set of options that the user could select to communicate with the remote scenario: five primary alarms (to express breathing problems, movement requirement, pain, inadequate room temperature, toilet need), two emotional states (feels happy or sad), two binary responses (yes, no), and two options to express the willingness to establish or finish a communication with anybody in the vicinity of the robot. The option to enable the user to pause the system was represented by the lower icon of the lateral panel. The way the users interacted with the graphical interface to use the different options is modeled as a finite-state machine (see subsection 4.2.3). Regarding the second functionality, the Farwell & Donchin paradigm [17] was also followed, obtaining in this redesigned system 8 stimulations (4 rows plus 4 columns) per sequence. The visual aspects of the elements shown on the visual display, as well as all the scheduling of the stimulation process was kept constant. The combination of this kind of interface with the P300 neurological protocol provides the users with task-relevant options, minimizing this way the user intervention to control the telepresence system, thus avoiding exhaustive mental processes. 4.2.1.3 Pattern recognition strategy The same two-step supervised pattern recognition technique was used to recognize the P300 visual-evoked potential. It was composed by two steps: (i) feature extraction following Krusienski et al. study [16], and (ii) classification algorithm. This classification algorithm was consequently adapted to the new stimulation process, obtaining two subproblems of classification, each one of 4 classes. This classification algorithm has been 1Alignment option could be useful, for example, if the user watches a door while exploring the environment and desires to pass through it. 31 4. Prototype 2: Technology 4.2 Development (a) Navigation interface (b) Exploration interface (c) Interaction interface Figure 4.2: The BCI graphical interfaces of each operation mode. All interfaces are composed by a common lateral panel in the right side of the display, a row of icons in the lower part of the display, and a 3 ×3 central panel. The lateral panel allows changing among the operation modes; furthermore, the lower option allows pausing the system for a configurable amount of time. The options are stimulated (flashed) by means of rows and columns displaying a blue circle on them. An example of a flashing of the second row options is shown for each interface. extensively studied for P300 classification problems obtaining very good results in online communication for ALS patients using visual stimulation [27, 28]. 4.2.2 Robotic System The robot device used was the same commercial Pioneer P3-DX, equipped with the same sensors (laser, camera, odometry), with a network interface card, and a embedded computer (Intel @ 700MHz with Linux OS). The navigation capabilities, provided by the navigation technology, were suitable for the patients needs to navigate in dynamic and 32 4. Prototype 2: Technology 4.2 Development (a) (b) Figure 4.3: (a) Robot device in the brain-actuated redesigned telepresence system with a laptop placed on it. (b) Graphical interface of the robotic system. unknown environments. The visual exploration capabilities were also suitable to actively explore the environment. The new functionalities in the design of the new brain-actuated telepresence were:  To provide the people in the vicinity of the robot with the interaction orders selected by the user. These orders should be visually displayed and will throw alarms (sounds).  To provide the people in the vicinity of the robot with a feedback of the users’ decisions. This feedback should be visually displayed.  To record live sound to provide the user with an auditive feedback of the environment. Finally, it was decided to incorporate a second computer to implement those functionalities, which will be referred as multimedia computer, in order to provide scalability to the system. This computer, which is a laptop (Intel Core2 Duo @ 2.10GHz with Linux OS) placed on the robot, displays a visual feedback through its screen, and incorporates a microphone to record sound and speakers to play sounds (Figure 4.3a). This design does not overload the robot computer and it does not interfere with the real-time operation of the navigation technology. The audio capabilities are implemented thought the ALSA audio library. The sound was recorded in format 11025 Hz, mono, 16-bit raw PCM. The sound was packet into 4-second segments (it is described in the system integration). The laptop displays a graphical interface using the SDL libraries. The graphical interface (Figure 4.3b) of the robotic system provides the next information to the people in the vicinity of the robot:  The operation mode of the system.  The action that the user is performing (also the interaction option selected). 33 4. Prototype 2: Technology 4.2 Development  The user emotional state.  Whether video and audio is being sent to the user station.  In the case the robot device is sending video and audio, it displays that video and the resting time to stop the transfer. 4.2.3 Integration and execution protocol The communication system performs the integration between the brain-computer system and the robotic system, which operates as the link between them, managing all the tasks related with the synchronization and information flow. The software architecture is based on the TCP/IP protocol and it is composed of a server, which concentrates the information flow and makes the system scalable for further additions; and three clients, one for the BCI system, one for the robot computer in the robotic system, and one for the multimedia computer in the robotic system (Figure 4.4). The BCI client was integrated within the BCI2000 platform in the user station, it was multiplexed in time with a period of 30 ms, and communicated with the link server through an Internet connection. The link server ran in a dedicated computer, an Intel Core2 Duo @ 2.10GHz with Linux OS, equipped with an Ethernet and Wireless network card. The robot computer client and the multimedia computer client were configured in the same network. The robot computer encapsulated the navigation system, synchronized the orders to the camera and to the navigation system, and communicated with the link server through a peer-to-peer wireless connection (ad-hoc wireless connection). This computer also communicated with the robot hardware controllers using the player platform [19]. Since the autonomous navigation system is a time-critical task, it was integrated in a thread-based system with time-outs to preserve the computation cycle (200 ms). The multimedia computer managed of the tasks relevant to audio (play and record sounds), visual display, and communicated with the link server through an ad-hoc wireless connection. The maximum bandwidth requirements were imposed by the image and audio transfer rates. The images were captured with a resolution of 640×480 pixels and were compressed in the jpeg standard format, obtaining an image size of 30 KBytes. In the experimental sessions the maximum transfer rate was set to 10 images per second, so the upper boundary of the video information transfer was close to 300 KBytes per second. The audio was recorded in format 11025 Hz, mono, 16-bit raw PCM; and in 4-second packets, thus obtaining an audio packet size of 70 KBytes. The usage of big audio packets simplifies the problem of audio transfer and allows obtaining an enough sound quality to understand sentences. These maximum bandwidth requirements are adequate for the typical bandwidth order of Internet networks. The execution protocol is modeled as a finite-state machine: (i) the BCI graphical interface develops a stimulation process (flashing) over all the possible options following the P300 oddball paradigm; (ii) the signal processing strategy detects the target the user is concentrated on (if the pause option is selected a counter is increased, which denotes the number of minutes that the system will be paused if the validation option is subsequently selected; otherwise the counter is reset to zero); (iii) once the desired target is selected, the user must subsequently select the validation option to send the target to the robotic 34 4. Prototype 2: Technology 4.2 Development system (this redundancy minimizes the probability of sending incorrect orders to the robotic system although BCI errors happen); (iv) the robotic system executes the order (this will be referred as a mission); (v) while the mission is being performed in navigation and exploration modes the robot sends live video and audio, in the interaction mode video and audio are sent for 30 seconds in order to allow short periods for a binary conversation. This time was empirically proved to be enough for a simple but successful communication. 35 4. Prototype 2: Technology 4.2 Development Figure 4.4: First row represents the two stations. Second row represents the computer hardware. A simplified event trace of the information flow between the hardware components for a navigation order is described. The BCI infers the user desired goal location (8 Bytes of information), which is transferred via Internet to the link server. Via the ad-hoc wireless connection, the link server transfers the goal to the robot computer and sends a flag to the multimedia computer to start recording sound. The robot computer makes the location available for the navigation system. Within a synchronous periodical task of 200 ms, the navigation system reads the location of the robot from the motor control system and the laser sensor, requests the robot odometry, executes the mapping and planning module, and sends the computed translational and rotational velocities to the controllers. While the robot is navigating, the robot computer iteratively requests images to the camera, and the multimedia computer records sound packets, which are transferred to the BCI. Finally, when the robot reaches the final location, the navigation system triggers a flag and the image transfer stops. Then, the robot computer sends three variables to the BCI needed to display the augmented reality 2D map: the map model (400 Bytes), the model location (12 Bytes), and the robot location within the map (12 Bytes). Once those variables are sent to the link server, it sends a flag to the multimedia computer to stop recording sound. 36 5. Prototype 2: Validation on ALS patients This chapter reports the experimental methodology (Section 5.1) and the evaluation of the second brain-actuated telepresence prototype on ALS patients (Section 5.2). 5.1 Experimental Methodology A experimental methodology was defined to carry out a technical evaluation of the system by ALS patients. The recruitment of the participants and the experimental protocol are discussed next. 5.1.1 Participants Three individuals suffering from amyotrophic lateral sclerosis (ALS) participated in the study. They were in the range 40 - 55 years old. The study was approved by the Ethical Review Board of the Medical Faculty of the University of Tubingen in Germany. The telepresence experience was performed being the patients in their home (South Germany) and the brain-actuated robot in the University of Zaragoza (Spain). 5.1.2 Experiment Design and Procedures The study was divided in three phases: (i) a screening and training phase, (ii) an online phase to perform a goal-oriented telepresence predefined task, and (iii) an online phase to freely explore all the functionalities of the telepresence system. The objectives of the study were to evaluate whether the P300 response was elicited in ALS patients by the graphical interface of the system, to measure whether it can be detected with a minimum of 70% accuracy (suggested as a predictor for satisfactory communication [1]), and to explore the boundaries of the telepresence system and its real usefulness for ALS patients. Furthermore, it was decided to acquire biosignals and EEG from more positions than the needed for a P300-based BCI experiment (usually 16 EEG positions are considered enough). It was due to the two additional objectives of the experimental session:  To perform offline source location data analysis. Source location techniques estimate the active brain areas from the EEG data recorded in the scalp [29]. These techniques usually require 32, 64 or more electrodes to get an accurate estimation. 37 5. Prototype 2: Validation on ALS patients 5.1 Experimental Methodology It was decided to use all the possible EEG electrodes with the available equipment in the laboratory.  To perform offline data analysis of the patients’ physiological responses using biosensors to measure user relaxation, comfortability, etc. Then, those metrics should be correlated to the system operation. Five biosensors were decided to use: temperature, skin conductivity, respiration, blood volume pulse, and electrocardiogram. Thus, the maximum number of EEG electrodes was limited by the number of amplifiers (2 gUSBamp amplifiers were available, having a total of 32 channels). 5 channels were dedicated to the biosensors, and since they require different ground and reference inputs 1, 8 channels were assigned to biosensors. Thus, 24 channels were available for EEG. It must be noted that the experiment design was very ambitious because the selected participants were used to shorter P300-based experiments using simpler systems (i.e. usually a speller). This methodology had to be adapted for each participant during the experiments according to their motivation and tiredness. It was included the possibility to attend the participants in a second session (if possible due to participants and researchers availability) to complete the three phases. The results of the source location techniques and biosensors are outside the scope of this project and they are not reported. The tasks, procedures and objectives of each phase are next detailed. 5.1.2.1 Phase I: Screening and Training Evaluation This phase was composed by two tasks: (i) a screening task to study the P300 response, and (ii) a training task to calibrate the system and to evaluate the online BCI accuracy. In these tasks the participants had to attend a predefined set of targets in the graphical interface. The number of sequences and all the scheduling of the stimulation process, mainly the inter-stimulus interval (ISI) and stimulus duration, were customized for each task. In the screening task the participants had to attend to 3 targets. The number of sequences was set to 5, the ISI to 1 s (to avoid the P300 overlapping) and the stimulus duration to 125 ms. Each target selection totaled 45 s and the duration of the trial was 2.5 min. In the training task the participants had to perform 4 training trials to calibrate the system and online trials to evaluate whether they were able to achieve a minimum of 70% accuracy. In each trial, a sequence of 8 targets had to be attended (the even targets in 2 trials and the odd ones in the other 2 trials to cover all the row and columns). The number of sequences was set to 10, the ISI to 75 ms and the stimulus duration time to 125 ms. Each target selection totaled 16 s and the duration of each trial was 3 min. The complete phase lasted 25 min for each participant. 5.1.2.2 Phase II: Brain-Actuated Navigation and Exploration Evaluation The objective of the phase was to evaluate the online BCI accuracy, the navigation and exploration capabilities of the system, its usefulness and its easy of use in a goal-directed predefined task. The participants had to accomplish two trials of a complex task that 1Each gUSBamp amplifier provides 4 isolated blocks of channels (with different ground and reference inputs), each one with 4 channels 38 5. Prototype 2: Validation on ALS patients 5.1 Experimental Methodology Figure 5.1: The objective of the task was to drive the robot from the start location to the goal area. In the exploration area (E.A. in the figure), the patient had to look for two yellow cylinders, in which a sign 2.5m above the floor on each cylinder was placed. Then, if both signals were equal, the patient had to avoid the yellow triangle by turning to the right-hand side, or if otherwise, by turning to the left-hand side. Red line shows the real trajectory of the patient in the first trial of the second session. jointly involved navigation in constrained spaces and the active search of two visual targets (Figure 5.1). The number of sequences and all the scheduling of the stimulation process was set to the same values as the training task in Phase I. The study was accomplished between the patient’s home (South Germany) and the University of Zaragoza (Spain), where the robot was placed, both connected via Internet. The only information of the remote scenarios shown to the patients prior to the experiment was the plan referenced above. Note that the same task was performed successfully by five healthy users in the evaluation of the initial prototype (chapter 3), although here we do not intend to compare healthy versus ALS affected individual performance. After this phase, the participants were presented with a battery of neuropsychological questionnaires like the Questionnaire for Current Motivation (QCM) and ”Scales for the assessment of quality of life”, described in the section 2.4 from [30], to study motivation and mood. A cognitive assessment form was used to analyze their feelings using the device during the task. This entire phase consisting on telepresence experience and questionnaires lasted about 1.5 hours for each participant. 5.1.2.3 Phase III: Brain-Actuated Global Evaluation The objective of the phase was to evaluate the usefulness of the overall telepresence, focusing more on the interaction mode. The participants had to freely use the system functionalities for at least 25 minutes with the only requirement of using the interaction interface to communicate with any of the BCI team researchers in the University of Zaragoza at least once. The number of sequences and all the scheduling of the stimulation process was set to the same values as the training task in Phase I. After this phase, the participants were presented with the same battery of neuropsychological questionnaires of Phase II. 39 6. Conclusions [24] L. Montesano, J. Minguez, M. Diaz, and S. Bhaskar. Towards an Intelligent Wheelchair System for Cerebral Palsy Subjects. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2010. [25] J. R. Wolpaw, N. Birbaumer, W. J. Heetderks, D. J. McFarland, P. H. Peckham, G. Schalk, E. Donchin, L. A. Quatrano, C. J. Robinson, and T. M. Vaughan. Braincomputer interface technology: a review of the first international meeting. IEEE Trans Rehabil Eng, 8(2):164–173, June 2000. [26] M.Baker, R.Casey, B.Keyes, and H.A.Yanco. Improved interfaces for human-robot interaction in urban search and rescue. Systems, Man and Cybernetics, 2004 IEEE International Conference, 3:2960–2965, 2004. [27] F. Piccione, F. Giorgi, P. Tonin, K. Priftis, S. Giove, S. Silvoni, G. Palmas, and F. Beverina. P300-based brain computer interface: Reliability and performance in healthy and paralysed participants. Clinical neurophysiology, 117(3):531–537, 2006. [28] E.W. Sellers and E. Donchin. A P300-based brain-computer interface for people with amyotrophic lateral sclerosis. Clinical Neurophysiology, 117(3):538–548, 2006. [29] J. M. Antelis and J. Minguez. Dynamic solution to the EEG source localization problem using Kalman Filters and Particle Filters. International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2009. [30] S.C. Kleih, F. Nijboer, S. Halder, and A. K¨ubler. Motivation modulates the P300 amplitude during brain-computer interface use. Clinical Neurophysiology, In Press, 2010. [31] Jesse M. Cedarbaum and Nancy Stambler. Performance of the Amyotrophic Lateral Sclerosis Functional Rating Scale (ALSFRS) in multicenter clinical trials. Journal of the Neurological Sciences, 152(Supplement 1):s1 – s9, 1997. 46