TetraNauta: a intelligent wheelchair for users with very severe mobility restrictions

PWqs of t!m 2032 IEEE lntemahal September 18-x).2W2* Glssgow, Scotland, U.K. confEmlKa MI canbol Appliitims TETRANAUTA: A INTELLIGENT WHEELCHAIR FOR USERS WITH VERY SEVERE MOBILITY RESTRICTIONS Vicente Din, S*., Amaya Rodriguez C., Diaz del Rio, F. , Civit Balcells, A., Cagigas Muiiiz, D. E.T.S. Ingenieria Infodtica. University of Seville. AV&. Reina Mercedes dn. Seville. Spain. I. INTRODUCTION Some electric wheelchair users have strong motion impairments restricting the movements needed to drive a wheelchair. They need steering devices ((joystick, mouthstick, chinstick, keyboards, etc.) specially adapted to drive the wheelchair with parts of their body they can control in some way. Problems arise when they drive along lengthy paths: they can get tired as force, accuracy and movement control needed to use the steering device must be maintained for a long time. Several automatic control systems have been developed to free users fiom continuously attend the wheelchaii movement. Some of them have lead to very interesting solutions based on autonomous motion wheelchairs, but with serious drawbacks when porting from research prototypes to commercial wheelchau. The fust drawback is their technical complexity, which makes them too expensive to implement in practice [4], [5]. On the other hand, they are usually based on modifications in the wheelchair control or power unit, and any attempt to modify these parts of a commercial wheelchair invalidates its homologation and warranty. The TetraNauta project [I], [2], [6] is a low cost, filly~open steering system to allow people with heavy motor impairments to move in a known environments (hospitals, schools, home, etc.). It minimizes the steering effort, especially in long paths, while caring for user safety. In such locations, most wheelchair motions repeatedly follow a limited number of trajectories along a fairly controlled environment. TetraNauta tries to reduce user efforts limiting the degrees of freedom in wheelchair motion: it can go over predefmed paths following track marks painted on the floor. This is a first level of safety as the path to follow is, in principle, supposed to be obstacles bee. As this cannot be guaranteed in practice, TetraNauta provides a second safety level using an infiared-based obstacle detection system. TetraNauta system can be divided in two parts: Traflc management subsystem. It is a central computer that communicates with TehaNauta wheelchairs to automatically obtain traffic state (position, motion information, etc.). Moreover, system operator or field sensors can report it some incidences that affects to traffic: temporary restricted areas, closed doors, obstacles, etc. Wheelchair motion control subsystem. Every TetraNauta wheelchair has an embedded computer, the motion control module, and several functional modules controlling different parts of the wheelchair: power module, steering device module, etc. Modules inside the wheelchair are connected by the DX serial bus [IO], so that any new wheelchair function could be included by connecting a new module to the bus and modifying the central control unit software. Hardware and software changes involve just to modules directly implicated in that function. The DX specification also defines software interface between modules, so that a module can communicate with any other connected to the bus. This gives us a completely open system where any system module can be easily adapted or replace according to user needs (Fig. I). Fig. 1. Modular diagram of TetraNauta system. II. MOTION CONTROL OPERATION Every TetraNauta wheelchau has a general map of the environment wherein it navigates in automatic mode. It also knows its actual position in the map using on board sensors that detect absolute positioning marks. User must select the destination point using the user interface to start navigation. Next step is to create a navel plan: given an origin and destination locations, the Traffic Management Subsystem fmd the better path attending to several factors: shortest journey, deadlocks prevention, high traffic avoidance, etc. If wheelchair cannot communicate with the Traffic Management Subsystem, it generates itself the travel plan using the last received traffic state information. After the Motion Control Subsystem receives the navelplan, it executes it making necessary actions to reach destination. Tasks executed in TetraNauta can be arranged using a multilevel organization (Fig. 2.) common in traditional mobile robot systems [3]: These levels, applied to TetraNauta, are: Task planning. User performs this level in TeWauta, as he/she always decides destinations. Trajectory planning. To generate a high level description (graph) of the path to follow is not a complex task, as initial absolute position is known and the system has a 0-78w-738631MFs17.00 8 2002 IEEE ?70 complete environment map. It can be done both by the Traffic Management or Motion Control Subsystem. Trajectory generation. The Motion Control Subsystem must generate trajectory topological descriptions ftom their graph description. A significant part of this work is already done, as trajectories follow lies and marks painted on the floor. The system performs trajectories by following lines and detecting cross and intersections with a CCD camera at the bottom part of the wheelchair. Robot control. This level controls the wheelchair power unit to keep the wheelchair moving through the topological trajectoy defined by the track marks. Semiautomatic navigation deals with user intentions at this level. In this paper, we propose techniques based on viitualpotentiaifieids (VPF) to do it. AWOMXTICSYSEM - OPERATOR I Fig. 2. Tasks involved in automatic navigation Next section describes how VPFs can be applied both to automatic and semiautomatic navigations. 111. NAVIGATION As functionality concerns, TetraNauta has several operation modes: it is intended not only as an automatic navigation wheelchair, but also as training and learning platform for novice users. User can switch among operation modes according to hidher needs or skills. There are three operation modes: Automatic navigation. The wheelchair follows predefined paths, so that user (or assistant) just selects the desired destination. The motion control unit performs all the necessary actions to arrive at destination. User cannot directly control any intermediate movement once destination bas been given to the wheelchair. Semiautomatic navigation. The purpose of this mode is to help user in learning to operate a conventional electric wheelchair. TetraNauta works as a training system, makiig the painted track marks behave as rails or barriers. It performs an automatic like navigation, but allowing the user to manually interfere on wheelchair movements. The system applies a shared control: users can modify movements generated by automatic navigation with the steering device (i.e. the joystick). Different levels of semiautomatic navigation can be used according to user skill. As practice increases, he/she can switch to a level with more responsiveness to user actions. Semiautomatic navigation can also be used to help user in crossing doors or narrow corridors. Manual navigation. In this mode, TetraNauta behaves like a conventional electric wheelchair, taking all direction and speed commands from user. Virmal potential field (VPF) control techniques give us a simple and intuitive way to implement the shared control needed for semiautomatic navigation. It is based on the idea that any painted line generates a virtual potential field that attracts or repels the wheelchair, according to line colour. VPFs generate virtual forces affecting to the wheelchair and guiding its motion. User intentions, taken with the steering device, are treated as one more force by navigation system (supposing that we are operating in semiautomatic navigation). "User virtual force" is added to virtual forces generated by VPFs associated to painted track marks. The resulting force contains both user intentions and automatic navigation intentions. In automatic navigation, the only considered virtual forces are those coming ftomVPFs. A. Guiding TennNauta with VPFs For following a line, at least an AVPF (attractive VPF), with the line as the attractive element, is needed. It affects to robot motion so that line is finally reached. We pretend not just to reach the line, but also to follow it. And additional AVPF is needed, but with a line point as attraction element, attracting the robot toward a destination location in the line. This seem a good solution for implementing automatic navigation mode of TetraNauta. But semiautomatic navigation also needs lines acting as repulsive elements. So we also need RVPF (repulsive VPF) where line act as a repulsive element. U,,, and U,,, are the names of the potential fields amacting to a line and a destination point (the end of the line). In automatic navigation, we combine their effects to get U,, the total VPF influencing the wheelchair. In semiautomatic navigation, an additional rype of WF, U,, is used for modeling repulsive effects of lines. In following sections, a generic definition of these VPFs is presented, followed by their adaptation to OUI coordinated system. E. AVPF toward the line (Um,lm). This VPF attracts the wheelchair toward the line. Two important problems must be solved The first is to choose a defining function for the AVPF. The simplest functions of those defined in literature are parabolic [7] or conic [8] functions. The main benefit of conic functions is that they are constants in all the working space (except at destination point), while parabolic function has excellent stabilization features. We use a combined model [9] mixing parabolic and conic functions. It is the most satisfactory for TemNauta as once the wheelchair bas reach the line, it needs to keep it over the line to make a good tracking. On the other hand, a comfortable navigation, with constant velocity and he of brusqueness, is very important because wheelchair users are disabled persons. Our combined AVPF function is eiven bv: where q is the robot location. We shall call union distance to the positive constant po . k,, is a positive proportionality constant. Finally, we have to define the function e,,,. (q) . Definition of P,,(q) is the second problem to solve. The original definition of a VPF states that a point is the VPF generating element: the drain point. P(q) is defined as the Euclidean distance to between the drain point and q. Now we have not a single drain point, but a drain line. To manage this, P,,(q) is defined as the minimum Euclidian distance between the wheelchair and the line. The force generated by the AVPF can be computed by: qini = (25, OT and qkM = (25, SOT. Parameters of (1 j are kpo =O.l v.p.u./cm2 and po = lOcm . Fig. 3. Attractive Virmal potential field generated by a line. Potential values are in virtual potential units (v.P.u.) C. AVPF toward the destination point This AVPF is defined so that the destination point generates an attracting force to the wheelchair. The destination point is over the line to follow, and usually matches with the end of its visible segment We use a combined AVPF definition function very similar to It differs only in the target position to reach: it is a single point qdnt. The AVPF function is given by: Where pl is again the union distance and k,, is a positive proportionality constant. We also define Pden(q) as the Euclidian distance between the wheelchair and the desired final destination. Fig. 4 shows the shape of a sample destination point AVPF in a 50x50 cm2 zone. qds, = (25, SOT, and parameters of (3) are kpl=0.1v.p.u./cm2 and p,=lOcm. Wbere qis the nearest point of the line. Fig. 3 shows the shape of a sample AVPF generated by a line in a 50x50cm2 zone. The line is perpendicular to the x axis, 780 Fig. 4. Attractive Virtual potential field generated by a point D. Line generated RVPF (UvJhe) A line generating a RVPF may have two different configurations: It may be completely repulsing, creating an impassable obstruction to the wheelchair, so that it can only walk around it. If the line is defined as partially repulsing the wheelchair can cross it, but feeling some resistance. The degree of resistance can be also configured. We ought to use a computationally inexpensive RVPF function expressing both partially and completely repulsion. A RVPF function [6] meeting these requirements is: Where q-is the robot's nearest obstacle point, po is the obstacle (line) influence distance and A is the obstacle maximum repulsion potential. Moreover, poh (9) is the minimum distance from the robot to the line. Fig. 5. shows the RVPF shape defined hy (4) in a 50x50 cm' zone. It is generated by a line between points q,,,, = (25, Oy and qdm = (25, SOY, and its parameters are A = 5 U.P.V. and po =15cm. Fig. 5. Repulsive Vi1 potential field generated by a line. N. AUTOMATIC NAVIGATION IN TETRANAUTA In automatic navigation, the user just tells the final destination to system and it makes all required movements to reach it. Wheelchair control must be completely automatic, avoiding any user contribution to its local movements' (apart from stopping to change a new destination or select a new navigation mode). In TetraNauta, this kind of navigation uses local sensing of lines painted on the floor. The sensing system is based in a low cost CCD color camera. Analyses of camera images get topological parameters used in VPF functions for the guiding system. Fig. 6 shows a sample situation to demonstrate how the guiding system uses VPF. Captured Light dopc imager deviation Very cimilu 'lope ".IYSI Fig. 6. Sample of a sequence of scanned images in automatic navigation. ' That is. all movements needed to follow the user selected mute. 781 We combine effects of two attractive VPF to meet both objectives: a line AWV makes the wheelchair to move toward the line, while an point AVPF moves it toward the end of the line. The global effect of both VPFs is that wheelchair moves towards the desired location, not in any way but following the lie, because the line AVPF has not effect once the wheelchaii is aligned with the line. AWFs defined in (I) and applied in (3) can be used to get the virtual force applied to the wheelchair, as: Um(q) = jhe (9) + Um,dur (q) (5) An4 Fm (9) = Fatb. (9) + pm,dcn (9) (6) Their defining functions (2) and (3) are continuous, positive, and derivable, so the resulting AVPF keeps its components properties. On the other hand, the AVPF has an absolute minimum in q- (Um(qdur)=O), because both Ll,,d,,(gd,)= 0, and Umjme(q) = 0, Vq inside the line with the same slopping, including qdm . The force due to the total AVPF also has good stabilization features because it converges to 0 when the robot approaches to qdm . Fig. 7. shows the AVPF shape in a 50x50cm2 zone, when (5) is used in a line defined by points q,,,, = (25,49y and qdu, = (25, Oy . Parameters are kpo,kp, = 0.1 v.pu./cm’ and po,pI=lOcm. DO Fig. 7. Attractive virtual potential field used for automatic navigation. Potential values are in virtual potential units (V.P.U.). V. SEMIAU~OMATIC NAVIGATION. In semiautomatic navigation painted marks behave like rails or baniers. It is useful when user operates wheelchair for the very first time because a wheelchair operation is an unusual task hdshe needs to learn. Moreover, learning process could be difficult if user skills are restricted by some kind of disable. In this mode, user and navigation system share wheelchair guidance. Navigation system is also based in WFs generated virtual forces that affects to movements; A simple and natural way to treat user intentions, taken with some input device2, is to generate with then an additional “virtual force” in the navigation system. All virtual forces are adding up to drive wheelchair movement including both user intentions and automatic navigation. A standard joystick can be used to compute user virtual force. We can read and computed the x/y joystick position. Navigation system virtual force is defined by the navigation submode. Two have been defined for semiautomatic navigation: rail submode and learning submode. User can change submode anytime when the system is in semiautomatic navigation A. Rail submode. It starts working once the wheelchair is on a main line3. The line attracts it aligning the wheelchair to its direction. Then the line acts like a rail: user is able to go forward or backward, or change velocity, but he/she cannot leave the line. User virtual force is compute from the y-axis projection of joystick position and the orientation of the scanned line section. The user virtual force angle is equal to the scanned line inclination, so that it has the same direction that navigation system force. The user can navigate on any line controlling velocity. HeIShe uses input device to command wheelchair for turning back or choosing direction when it meets a line fork. Wheelchair operation can be tiring in long courses because user has to keep joystick position toward marching direction to keep wheelchair moving. A minor variation of rail submode, sloped rail submode, is provided to alleviate user from this. In sloped rail submode, user only needs joystick to choose direction in fork options, to turn back or to stop motion, while system keeps the wheelchair moving. B. Learning submode. Learning mode is primary intended to use TetraNauta as a mining platform, hut it also helps wheelchair steering in some circumstances (crossing narrow doors, corridors, etc.). Joystick, mouthstick, chinstick, etc. TeuaNauta implements line searching algorithms to ease this operation. 782 The training environment contains attracting and/or repulsing lines (painted with different colors). The lines modify user commands to help wheelchair control. There are four different types of line defined for this submode4: Color I. Repulsive line. Repulsion level depends on user skill. The maximum level is used for novel users, becoming impassable lines. Repulsion level decreases as user improves. Repulsive lines can be used to make virtual corridors (Fig. 8). Color 2. Impassable repulsive line. It is used to disallow user to leave training environment. Color 3. Attractive line. The AVPF acts like a virtual valley along the line. It has no effect on user force if he/she directs joptick parallel to line. If joystick is directed toward lie, system boosts user force effects (the wheelchair is “lowering the valley”) while system weakens user force effects (the wheelchair is “rising from the valley”) if joystick is directed off the line. Attraction level also depends on user skills. These lines are useful for crossing doors or corridors (Fig. 8). Color 4. Rail line. Wheelchair behaves line rail submode. It is also used for crossing narrow places. If system does not detect any line it behaves lie a standard electric wheelchair. So, user can activate learning mode as default operation mode and directly guide wheelchair movements except in dangerous or difficult zones (unevenness, stairs immediacy, doors, etc), where marks can help navigation. Repulsive lines Fig. 8. Attractive and repulsive lines effect in semiautomatic navigation In Teb-aNauta prototype, colors are green red, blue and black, but they can be changed in the image processing module to adapt them to floor background color. CONCLUSIONS This paper have presented Virtual Potential Fields as a good tool for implementing automatic guidance in the in the intelligent electric wheelchair area, and these can extent to the teleoperated mobile robots area, in general. Vimal Potential Fields are inhlitive and easy to use. It bas been shown that small software variations can greatly increase system functionality, making an automatic navigation system into a training platform for wheelchair operation. REFERENCES A. Civit, J. Abascal. “TetraNauta: A Wheelchair Controller for Users with Very Severe Mobiliry Resm’ctions”. Improving the Qualify of Lge for the European Citizen, I. Placencia, E. Ballabio (Eds.). ISBN 1383-813X. 10s Press, Netherlands, (1998). Pg. 336-341. A. Civit, F. Diaz, S. Vicente, C. Amaya, G. Jimbnez, J.L. Sevillano. “Arquitechrra de controladores de sillas de ruedas”. Novitica, Monografia: idodtica y discapacidades. Pg. 16-20. Nov-Dec 1998. I.J. Cox. “Blanche- An Experiment in Guidance and Navigation of an Autonomous Robot Vehicle”. IEEE Transactions on Robotics and Automation. Vol. 7, N4?,April1991. I. Craig, PI Nisbet. “The Smarf Wheelchair; An Augmentative mobility tool!#. 2nd ECART Cod. Stockholm, Sweden, 1993. R. Gelm, J.M. Detriche. “The sprint of COACW. IEEE International Conference on System, Man and Cybernetic. Le Touquet, Fance, 1993. S. Vicente Diaz. “UM aportacibn al guiado de sillas de ruedas elicm’cas en entomos estnrchrrados”. Ph. D. Thesis. Universidad de Sevilla, July 2001. 0. Khatib. “Real-Time Obstacle Avoidance for Manipulators and Mobile Robotr”. International Joumal of Robotics Research. Vol. 5, NY, Primavera 1986, Pig. 90-98. J.C. Latombe. “Robot Motion Planning”. Kluwer Academic, Boston, 1991. J.R Andrews, N. Hogan. “Impedance control as a framework for implementing obstacle avoidance in a manipulator”. Editores: D.E. Hardt y W.J. Book, Control Of Manufacturing Processes And Robotic Systems, ASME, 1983. PBg. 243-251. Dynamic Controls, Ltd. PBgina web. hnp:llwww.dynamiccontrols.co.nz 703