The Possibilities of Kinect as an Access Device for People with Cerebral Palsy A Preliminary Study

The Possibilities of Kinect as an Access Device for People with Cerebral Palsy A Preliminary Study Isabel María Gómez1, Alberto Jesús Molina1, Rafael Cabrera1, David Valenzuela2, and Marcelo Garrido1 1 Electronic Technology Department, Universidad de Sevilla, Spain {igomez,almolina}@us.es, [email protected] 2 ASPACE, Sevilla, Spain [email protected] Abstract. Cerebral palsy (CP) is a general term for a group of permanent, nonprogressive movement disorders that cause physical disability in development, mainly in the areas of body movement but it might also affect intellectual capabilities. Among all this diversity of profiles, we find that, for some of them, access to a computer application is almost impossible in spite of the great variety of commercial devices based of different technologies. Kinect might be a viable possibility in order to facilitate access to games and computer applications that help users improve their skills or communication. Keywords: Access Device, Kinect, Cerebral Palsy, Middleware Software. 1 Introduction The use of Kinect in the case of CP can be found mainly in the area of rehabilitation. In [1] a preliminary study to evaluate the usefulness of games based on Kinect is done. In [2] participants played 4 video games and a quantitative evaluation of its potential for physical activity promotion and rehabilitation is explained. In [3] games are specifically designed to promote physical rehabilitation and include information of progress for the therapist. In [4] a specific application called Move it to improve it (Mitii) was used to determine if 20 weeks of intensive training can improve upperlimb activity (unimanual and bimanual), occupational performance and cognitive skills in children and adolescents with CP compared to standard care. In [5] the work is focused on how to design the game in order to be both fun and efficient in terms of motor skills development or attitude change, emphasizing the adaptation to physical skill a motivating social participation. Furthermore Kinect can be used as an access device based in gestures that allows the control of computer applications in general. There are several tools that translate gestures into mouse or keyboard events which allow the use of applications that can be controlled by such events. Two of them are based exclusively in hands: Tip-TepMouse [6] and Winect[7]. In [8] the Flexible Action and Articulated Skeleton Toolkit (FAAST), a middleware software framework for integrating full-body interaction with virtual environments, video games, and other user interfaces is described. A case study is presented that is made up of in improving accessibility for people with mobility difficulties. However some problems are encountered such as tool configuration and gesture recognition. 2 Methodology Used In this paper four applications have been tested. Three of them are described in [6,7,8]. They are available for free download. The fourth one is of our own design. This tool is called KIVIAD (Kinect Virtual Access Device). The advantage of KIVIAD is that it has been designed to deal with specific necessities that have been observed during several training sessions in ASPACE (Association of people with cerebral palsy of Seville). With this tool we have two kinds of control: one of them is with a precise gesture and the other one generates the event choosing any voluntary movement, eliminating the need to maintain the same position and repeat the same pattern of movement. The event is generated based only in the velocity of movement. 2.1 Subjects We have conducted the experiment using two groups of users: •Healthy people. Three individuals aged between 29 and 47, two males and one female tested the software. •Adults with cerebral palsy. Five users, two males and three females were chosen. The trials were carried out in a daycare center of ASPACE. The users had different profiles. Two of them could not speak and used a notebook of pictograms to communicate. Subjects were asked which body movement they had more control over. Three subjects selected head movement, one selected their left hand and finally the last selected their right hand. 2.2 Tests Description The tests designed had to be established in accordance with the tools under evaluation. Tip-Tep and Winect translate specific hand gestures to a mouse event (including click and displacement). With these tools gestures cannot be configured. In FAAST any kind of gesture can be chosen and translates them into all kinds of events (mouse and keyboard). To do the configuration, this tool has a series of menus. KIVIAD has a different orientation in its design, being designed for people with mobility problems, works alone in discrete mode. This means this tool translates between gestures and mouse click or keyboard key_press but not in cursor mouse movements. So, two kinds of tests have been considered: •The test in continuous mode, Tip_Tep, Winect and FAAST have been used. An activity that requires cursor movement and mouse clicks is proposed. The cursor began in the central point of the screen. Users must do the following action in a computer: open Virtual keyboard (VK); open Web Browser; write bbc and press INTRO in the browser bar using VK; displace mouse cursor to the first link and select it; then scroll the selected web page for its reading. In order to do these actions, we need two gestures that are different depending on the tool used. Time spent in performing each task and mistakes were measured. Users answered a small test related with tools usability. •Test in discrete mode. In this case activity only requires a mouse click. A communication application based on pictograms has been used for tests. This tool works by scanning, this being controlled by the mouse click. Users must transmit three ideas built by pictograms. The application is prepared to receive clicks only and does not work properly if cursor movement is transmitted. So in this test only FAAST and KIVIAD were used because both of them can be configured to transmit only the click. It is very important to determine the gesture to be used for the event generation corresponding to this click, especially with subjects with cerebral palsy. It is not possible to ask for a specific gesture, since they do not have control over the movement. The only way to widen the user profile and that they work comfortably is to base the movement on speed without restriction of direction and ask them to choose the right hand, left hand or the head. If no errors are made, the time required to transmit each idea depends on the position of the icons on panels so, only errors were measured. An error was made when the icon is not selected when the scanning reaches it. False clicks can also take place when the skeleton is not tracked properly. KIVIAD and FAAST are different in their configuration. KIVIAD is especially designed for people with disabilities and FAAST has a broader spectrum. For this reason, KIVIAD design has not considered a great variety of input gestures and neither a wide variety of output events. It is not necessary more because this is not used in the environment of its use. Furthermore, events are generated in a lower level than in FAAST. There are some kind of software that cannot operate with events in the upper level, in fact some games designed for the same people where XNA Game Studio has been used cannot be controlled using FAAST but work properly with KIVIAD. It was considered important to obtain point of view of therapist about usability of these tools. 2.3 Method Healthy subjects did both tests, continuous and discrete. Three sessions with each tool were held for each of them. In this form, the experience acquired can be evaluat-ed. Users with cerebral palsy only did the test in discrete mode. Three sessions were held with each user and the therapist. Continuous mode was discarded after talking to the therapist and noted that it was not possible for users to have the required accuracy in the movements. 3 Results In continuous mode Tip-Tep presented the best results(less time, less errors and the best answer in usability). In discrete mode, KIVIAD and FAAST were functionality equivalent. But KIVIAD is more intuitive and presented better configuration. 4 Conclusions In cerebral palsy is more common to consider Kinect as a tool for supporting rehabilitation therapies. There are very few studies which have considered it as an access device. Besides testing Kinect applications for CP users, this work also aims to conduct an assessment of this system as an access device itself and remark what this technology may contribute with respect to other established solutions. Special emphasis has been taken into account in those cases where appropriate solutions for computer access have not been found yet. The proposed of Kinect as access device raises questions because users must perform control actions with nothing tangible and without seeing his image. However users that did the trials have gotten the operation of the device and have managed to complete the tasks that were established. References 1. Luna-Oliva, L., et al.: Kinect Xbox 360 as a therapeutic modality for children with cerebral palsy in a school environment: A preliminary study. Neuro Rehabilitation 33(4), 513–521 (2013), doi:10.3233/NRE-131001 2. Howcroft, J., et al.: Active Video Game Play in Children With Cerebral Palsy: Potential for Physical Activity Promotion and Rehabilitation Therapies. Archives of Physical Medicine and Rehabilitation 93(8), 1448–1456 (2012) 3. Roy, A.K., Soni, Y., Dubey, S.: Enhancing Effectiveness of Motor Rehabilitation Using Kinect Motion Sensing Technology. In: Global Humanitarian Technology Conference: South Asia Satellite (GHTC-SAS). IEEE (2013) 4. Boyd, R.N., et al.: Move it to improve it (Mitii): study protocol of a randomised controlled trial of a novel web-based multimodal training program for children and adolescents with cerebral palsy bmjopen-2013-002853 (2013) 5. de Greef, K., van der Spek, E.D., Bekker, T.: Designing Kinect games to train motor skills for mixed ability players. In: Games for Health: Proceedings of the 3rd European Conference on Gaming and Playful Interaction in Health Care, pp. 978–973. Springer Fachmedien Wiesbaden (2013) ISBN: 978-3-658-02896-1 6. http://tiptep.com/index.php/products 7. http://xorastudios.com/portfolio/winect 8. Suma, E., Krum, D., Lange, B., Koenig, S., Rizzo, A., Bolas, M.: Adapting user interfaces for gestural interaction with the flexible action and articulated skeleton toolkit. Computers & Graphics 37(3), 193–201 (2013)