scieee AI-readable full text Open interactive document viewer

Intuitive Human-Robot Interaction by Intention Recognition

Awais, Muhammad

Full text

1 Intuitive Human-Robot Interaction by Intention Recognition Der Universität Bayreuth zur Erlangung des Grades eines Doktors der Naturwissenschaften (Dr. rer. nat.) genehmigte Abhandlung von Muhammad Awais aus Faisalabad 1. Gutachter: Prof. Dr. Dominik Henrich, Universität Bayreuth 2. Gutachter: Prof. Dr. Klaus Schilling, Julius-Maximilians-Universität Würzburg Tag der Einreichung : 28.10.2012 Tag des Kolloquiums: 25.01.2013 2 3 ACKNOWLEDGEMENTS First of all, I want to express my deepest gratitude to my supervisor, Prof. Dr. Dominik Henrich, for his continuous and caring help, advice, and encouragement during the three and half years of my PhD studies at the University of Bayreuth. I thank him for being very patient with my questions and my progress, for the countless lessons on writing and presenting technical materials, on doing research in general, and also for the English lessons, even though he is not a language teacher. His courtesy must be appreciated as he always politely answered my questions, passing through the corridors or while preparing his coffee in the kitchen. I am also thankful to my second supervisor Prof. Dr. Klaus Schilling. I am very grateful to all of my colleagues and friends in the Department of Applied Computer Science III, Robotics and Embedded Systems, University of Bayreuth, for many helpful discussions, the pleasant working environment, and many beautiful memories. I am very thankful to Dr. Stefan Khun for his support concerning image processing and code optimization. I am also thankful to Dr. Thorsten Gecks for his all round support specifically concerning Robot Programming. I had many useful discussions with Christian Groth and Maria Hänel. I am thankful to both of them as they always welcomed me for discussion. I am thankful to Mark Schuetz for his all time availability for trouble shooting the problems concerning software and hardware. I will remember the nice time passed along with Tobias Werner in Waischenfeld. I would also like to thank Markus Fischer, Johannes Baumgartl, Katharina Barth and Antje Ober Gecks for their supportive and descent behaviour. I could not be able to finish my thesis without financial support from HEC – Higher Education Commission, Pakistan and DAAD - Deutscher Akademischer Austausch Dienst, Germany. I am very much thankful to my parents and my family for supporting and encouraging me to complete my PhD. I am thankful to my father for taking care of my family back in Pakistan for the Six and half years as I stayed that period of time in Germany for the completion of Master and PhD studies. I am thankful to my wife and my two sons who had to wait patiently for six and half years. I owe a lot to both of my sons as they were deprived of the father’s love and care for this period of time. Specially, my elder son Muhammad Abul-rehman Maaz who missed me a lot and always asked me when will I come back home? I am thankful to the moral support continuously provided by my mother and wife. I am also thankful to my uncle Muhammad Muslim for his support and encouragement during PhD. I am thankful to ALLAH almighty that he helped me in all kinds of difficulties in my PhD studies and in-sha-allah will help me in the life to come. Bayreuth, January 2013-01-27 4 5 Zusammenfassung Damit zwei Menschen miteinander interagieren können, um eine gemeinsame Aufgabe zu erfüllen, müssen sie die Erwartungen, die sie während der Interaktion aneinander haben, kennen. Betrachten wir das Beispiel eines Obers und eines Gastes. Kippt der Kellner eine Flasche, um dem Gast ein Getränk anzubieten, so kann er zwei mögliche Reaktionen des Gastes erwarten. Entweder reicht ihm der Gast sein Glas, um es füllen zu lassen oder er zieht es zurück um anzudeuten, dass er kein Getränk will. Hält er dem Kellner das Glas hin, so kann dieser damit rechnen, dass der Gast sein Glas solange an einem bestimmten Ort hält, bis er das Glas füllt. Zieht der Gast dagegen das Glas weg, so rechnet er damit, dass der Kellner sein Glas nicht füllen wird. Im Falle eines Missverständnisses kann ein Missgeschick geschehen. Für fast alle Fälle von Mensch-Mensch-Interaktion gilt, dass die Erkennung der Absicht eine Schlüsselrolle spielt. Für die Mensch-Roboter-Interaktion ist sie genau so wichtig. Mit zunehmender Forschung auf dem Gebiet der Robotik sind und werden Roboter mehr und mehr Teil des menschlichen Lebens. Damit Roboter ein erfolgreicher Teil des menschlichen Lebens werden müssen sie nützlich für den Menschen sein. Hierfür sollen sie sich nach dem Menschen richten. Versucht der Roboter, einem Menschen zu helfen, ohne die Absicht der interagierenden Person zu kennen, so kann der Roboter selbst zu einem Problem werden, statt die Lösung der Probleme zu sein. Daher ist es notwendig, dass ein Roboter die Absicht eines Menschen, mit dem er interagieren soll um ihn zu unterstützen, kennt. Das Ziel dieser Arbeit ist es, eine Lösung vorzuschlagen, die die intuitive Mensch-Roboter- Interaktion intuitiv macht. Um die Mensch-Roboter-Interaktion intuitiv zu machen sollte dem Roboter die Absicht des Menschen bekannt sein. Es wird ein wahrscheinlichkeitsbasierter Ansatz zur Erkennung der menschlichen Absicht eingeführt. Der Ansatz nutzt endliche Zustandsautomaten. Jeder endliche Automat stellt eine menschliche Absicht dar und besitzt einen Wahrscheinlichkeitswert, der als Gewicht des endlichen Automaten bezeichnet wird. Aus diesem Gewicht bestimmt der Roboter die momentane Absicht des Menschen. Da es nicht möglich ist, alle möglichen Absichten, die der Roboter erkennen muss, in den Roboter einzubetten, bedarf es einer Maßnahme, damit der Roboter neue menschliche Absichten lernen kann. Für diesen Zweck wird ein Ansatz diskutiert. Damit die Mensch-Roboter-Interaktion intelligent ist sollte der Roboter schnell in auf die menschliche Absicht reagieren. Hier wird ein Ansatz für eine schnelle (proaktive) Reaktion des Roboters beschrieben. Der Ansatz diskutiert auch das Szenario einer mehrdeutigen menschlichen Absicht. Dabei handelt es sich um eine Absicht, die mehr als einer menschlichen Absicht entspricht. Es ist möglich, dass der Mensch eine völlig neue Intention hat, die der Roboter noch nicht kennt und auch noch nicht gelernt hat. In diesem Fall gibt es offensichtlich keine Mensch- Roboter-Interaktion. Für die Bewältigung dieses Problems wird ein Ansatz diskutiert, der es dem Roboter ermöglicht, eine geeignete Aktion auszuwählen, um mit dem Menschen zu interagieren. Darüber hinaus wird ein Ansatz zur Verallgemeinerung der menschlichen Absicht diskutiert. Dadurch kann der Roboter seine Reaktion dem menschlichen Willen entsprechend ausweiten. Die Ausweitung der Reaktion bedeutet, dass der Roboter diejenigen Aktionen nimmt, die er nicht beauftragt wurde, bei einer menschlichen Intention zu nehmen. 6 Abstract For two humans to interact with each other to perform a common task, they need to know the expectation of each other during interaction. For example if we consider an example of a waiter and a guest. If the waiter tilts the bottle to offer a drink to the guest then he may expect two actions from the guest, i.e., either the guest will forward his glass to get it filled or he will take his glass backward for not accepting the drink. If the guest forwards his glass then the waiter expects that the guest will keep his glass at a certain point until he pours the liquid into the glass. Similarly if the guest takes its glass backward then he expects from the waiter not to pour the liquid into his glass. In any case of misunderstanding an accident can occur. It applies to almost all the instances of human-human interaction. The recognition of the intention plays a key role in human-human interaction. It is equally important in human-robot interaction. With the increase of research in the field of robotics, the robots are and will be becoming more and more part of human life. For the robots to be the effective part of the human life they should be helpful to the human. For a robot to be helpful to the human he should act according to the human. In case if the robot tries to help the human without knowing the intention of the interacting human then the robot can be itself a problem rather than a solution to the problems. Therefore it is necessary for a robot to know the intention of the human with whom the robot is supposed to interact to facilitate him. The aim of this work is to propose a solution to make the human robot interaction intuitive. For making the human-robot interaction intuitive the intention of the human should be known to the interacting robot. A probabilistic approach is introduced to recognize the human intention. The approach uses the finite state machines. Each finite state machine representing a unique human intention carries a probabilistic value that is called the weight of the finite state machine. That weight tells the robot about the current human intention. Since it is not possible to embed all the possible intentions into the robot that the robot may need to recognize. Thus, there should be a measure that the robot can learn new human intentions. An approach is discussed for this purpose. For the human-robot interaction to be intelligent the robot should be quick in his response towards the human intention. An approach is described that addresses the issue of quick (proactive) response of the robot. The proposed approach also discusses the scenario concerning the ambiguous human intention. An ambiguous intention is a human intention that apparently corresponds to more than one human intention. There may be a scenario in which the human has a totally new intention that the robot does not know already and also has not learned that intention. In this case, apparently there is no humanrobot interaction. In order to cope with this problem an approach is discussed that enables the robot to select an appropriate action to interact with the human. An approach concerning the generalization of the human intention is also discussed. By generalizing the human intention, the robot can extend its response according to the human intention. The extension of the response means that the robot takes those actions that were not instructed to him to be taken concerning the human intention. 7 8 Table of Contents 1. Introduction 13 1.1 Motivation 15 1.1.1 Safety in Human Robot Interaction (HRI) 16 1.1.2 Robot as a tool 17 1.1.3 Adaption 18 1.1.4 Robots in Small and Medium Enterprise 18 1.2 Goals 19 1.2.1 Intuitive HRI by intention recognition 20 1.2.2 Online intention learning by scene observation 20 1.2.3 Early intention estimation 20 1.2.4 Interaction in unknown human intention scenario 21 1.2.5 Rule-based intention generalization 21 1.3 Demarcation 22 1.4 Overview 23 2. Related work 25 2.1 Overview 25 2.2 Social HRI 26 2.3 Robot as an assistant 27 2.4 Tactile HRI 30 2.4.1 Skin sensors 31 2.4.2 Tactile HRI 32 2.5 Conclusion 34 3. Intention recognition 35 3.1 Problem definition and Motivation 35 3.2 Related work 36 3.3 Finite State Machines (FSMs) 38 3.2.1 Recognition of explicitly communicated intentions 42 3.2.2 Recognition of implicitly communicated intentions 43 3.4 Intention recognition algorithm 44 3.5 Experiments 47 3.6 Summary 53 9 4. Intention learning 55 4.1 Problem definition and Motivation 55 4.2 Related work 57 4.3 Intention learning 58 4.3.1 Finite State Machine construction 59 4.3.2 Mapping actions to the intention 59 4.3.3 Mapping actions to the scene information 61 4.3.4 Mapping using the scene changes 62 4.4 Experimental results 63 4.5 Summary 67 5. Proactive interaction 69 5.1 Problem definition and Motivation 69 5.1.1 Safety in HRI 70 5.1.2 Importance of proactiveness in intuitive HRI 71 5.2 Related work 72 5.3 Trigger state determination 73 5.4 Online update of local transition weight 77 5.5 Experiments 83 5.6 Summary 86 6. Interaction in unknown scenarios 89 6.1 Problem definition and Motivation 89 6.2 Related work 91 6.3 Interaction in an unknown intention scenario 92 6.4 Probabilistic action selection 93 6.4.1 Action probability 94 6.4.2 Action prediction 94 6.4.3 Weighting of the predicted actions 95 6.4.4 History based actions prediction 96 6.4.5 Combination of action aspects 97 6.5 Particle Filter based action selection 97 6.6 Experiments 101 6.7 Summary 108 7. Intention generalization 109 7.1 Problem definition and Motivation 109 16 Another reason that the robots are not employed in the industry to work in cooperation with the humans is that the robots do not take into account what the human is currently doing, what is his task, and what he will be doing in few moments. Mostly robots work like simple machines performing the already programmed tasks with very little flexibility. Figure 1.5: Industrial robots separated from humans by fences [129] For a robot to work with the human the robot needs to be flexible but also needs to be aware of what the interacting human intends to do so that both the human and the robot can work in collaboration. We motivate the importance of intention recognition in HRI by addressing the following issues concerning HRI, i.e. safety in HRI, robot as a tool, adaption, and robot in Small and Medium Enterprises (SME). 1.1.1 Safety in HRI In the industry, HRI safety is a significant issue that restrains the human and the mighty industrial robot from interaction. The range and the vision sensors can be used to monitor the HRI workspace. With the presence of human, the speed of the robot may be decreased, the robot may be completely stopped or the robot’s path from the source to the destination can be reconsidered and planned to avoid human robot collision in HRI workspace. Decreasing the speed of the robot or simply stopping the robot effects the efficiency of operation. The HRI is negatively affected due to slowing or stopping the operations of the robot. The changing and reconsidering of the path to avoid the collision between the human and robot is acceptable, but it is not risk-free. There may be a situation while the human and robot are moving in the HRI workspace that one or more parts of human body are occluded by the robot. Thus there may be a collision between the human and the robot due to the improper monitoring of the HRI workspace. The situation may be improved by predicting the human locations in HRI workspace, i.e., the robot can anticipate the future human actions and thus the robot can plan the path avoiding any expected collision. In order to anticipate the future human actions, the robot needs to know the human intention, i.e., what the human intends to do. Then the robot can infer in which direction the human can move, stay, bend, etc. Taking into account all the virtually occupied locations the robot can plan its collision-free path. Moreover, while path planning; the robot can consider the locations as virtually occupied that are frequently visited 17 by the human during HRI. This can considerably improve the safety measurements but it can not fully guarantee the risk-free safe HRI. 1.1.2 Robot as a tool In manufacturing industries, there may be tasks that require enormous power, intelligent decision making, and excellent sensors with efficient inference. The robots can help the humans with enormous power, but intelligent decision making and excellent sensors with efficient inference may not always be guaranteed by the robots in all the cases. The human can not perform such tasks alone too. Therefore the human and robot need to work together. In almost all such cases the robot is used as a tool by the human instead of an intuitive coworker. As a tool the robot is very expensive unless the task is impossible without the robot. There exists other less intelligent machines that can be applied instead of the robot, e.g., in assembly line there exist less intelligent devices that help the coworkers to move the heavy objects, e.g., doors of the vehicles, dashboards, seats etc to the desired places as shown in Figure 1.6. These less intelligent machines are called CoBots [11]. They are used to assist the human coworkers on an assembly line. Figure 1.6: CoBots. Left: Seat assembly [34]. Right: Door assembly [33] The robot can only be appreciated in such conditions if the robot can perform that task with least human input as compared to the less intelligent devices, i.e., if the robot performs the task automatically recognizing the human intention and bring him the required component and cooperate intuitively to install that component into the vehicle. The tasks of moving, assembling, and installation of the heavy components are repeatedly performed in the manufacturing industries. The intuitive execution of these tasks by the robots cooperating in accordance with the human intention can improve the efficiency of the human workers. The intelligent tool behaviour of the robot can be accepted in HRI if the robot acts according to the human intention for a task in the given situation. For example, consider a robot that can perform more than one operation. The robot interacts with the human while performing certain task and executes the specific operations according to the human intentions to complete the task. The robot as an intuitive tool with multiple capabilities is valuable if the robot selects and switches between the available capabilities according to the intention of cooperating human. 18 1.1.3 Adaption As an intuitive and intelligent machine the robot should also adapt to the small changes in HRI. The adaption may correspond to the workspace of HRI and to the cooperating human. Adaption to the workspace is to remember the knowledge gained in the workspace concerning intuitive HRI and to apply that knowledge in the next HRI situations in order to be more intuitive and helpful to the cooperating human. The adaption to the human coworker corresponds to adapt towards the human intention. There may be more than one aspect for adaption towards the human intention. For example, adaption may correspond to the solution of the conflict between the two nearly similar human demonstration concerning different intentions. Similarly the adaption aspect may also involve the robot adaption to the routine human tasks in the HRI workspace. If the robot does not have the adaption capability then the robot needs to be explicitly programmed or the robot requires adding or update of related modules. In this case the difference between an intelligent robot and a simple machine is reduced. In every robot related industry making manual updates for small changes in HRI workspace is less acceptable for robots. Update for the robots will require extra trained manpower, stopping of production and extra costs. This is further problematic if the update is required to be performed after short intervals. Thus the capability of adaption is necessary for an intuitive robot for HRI. The capability of adaption enables the robot to alter its response in HRI without the explicit human clarification and robotic expert intervention. In response to the little changes in the HRI the robot needs to adapt to the changes intuitively by performing accordingly. The recognition of the human intention is the basic ingredient to adapt according to the interaction human. For example if the human has one of the two intentions while working in the HRI workspace. Then the collaborating robot can only adapt according to the human if he can recognize both of the intentions. Next time the robot can proactively interact with the human based on the adaption. 1.1.4 Robot in Small and Medium Enterprise (SME) A SME consists of limited resources relating to manpower and finances. The production rate is also low due to the lack of resources and less demand. There may be a few or no robot experts in SME. The robotic tasks in the SME are quite different as compared to big manufacturing industries. In big manufacturing industries the robots are mostly working as automated machines without human interference, whereas in SME almost all the tasks are performed directly by the human workers or under the direct supervision of human workers. Thus the robot present in SME must have the capability to work in an environment concerning HRI. In order to justify a robot to be present in SME it must work as intelligent and intuitive machine. It must not require reprogramming for small amendments in different tasks, possessing the capability of adaption. The robot must be adaptive towards the small changes in the HRI workspace regarding human intention. For better HRI regarding intuition and adaption it should anticipate the intention of the cooperating human. The ability of robot of being proactive is an extra advantage for HRI in SME. Similarly a robot with intuitive interacting capabilities with the human can act as helper for a craftsman and mechanic in their related workshops. 19 In industry ranging from SME to big manufacturing industry mostly the manufacturing pattern remains the same for quite a time. In big manufacturing industries like vehicle industries the manufacturing setup is established for longer time as compared to the SMEs. The production speed is increased by introducing the robots as well as less intelligent machines. The automated robots work mostly very fast, independent from each other. However, all the sections of the industry big or small do not contain the automated robots. The tasks in such sections are performed directly by the humans or under the direct supervision of the humans. The number of manual section vary from industry to industry depending on the concerning tasks in the industry. The employment of intuitive robots in such sections can improve the efficiency of cooperating humans. The intuitive robots should be capable to recognize the intention of cooperating human and should be able to act accordingly. These robots can perform the task better as compared to the less intelligent CoBots, requiring little human input. The CoBots require more focused human input as a tool to perform a task. The intuitive robots will work not as a simple tool, but like an intuitive coworker that can react according to the cooperating human. The robot must know the answers of the following questions to be intuitive with respect to the human requirements and thus effective during HRI. The questions are given below 1. When to do? 2. What to do? 3. Where to do? The question what to do corresponds to the robot actions in response to the human actions while interacting with the human. For this reason the robot needs to know the human intention. Knowing the human intention tells the robot when to do what, i.e., if the robot has recognized the human intention regarding a specific task. Then the robot must also know the cooperative actions in order to respond in an intuitive and cooperative way. That corresponds to the answer of second question that robot needs to know. The question three corresponds to a specific situation in which the selected robot action is to be taken. For example, if a human and a robot are cooperating in a HRI workspace. Two products are manufactured in the workspace. Manufacturing process is same for both the products except one operation. Thus the robot needs to take care what he needs to do where and when in order to be effective and intuitive. 1.2 Goals The goal of the research work is to propose a solution for the intuitive HRI by human intention recognition. The robot should be aware of the intention of the cooperating human for intuitive HRI. The following points are considered to make the HRI intuitive regarding the intention of cooperating human. A. Intuitive HRI by intention recognition B. Intention learning by scene observation C. Proactive intention estimation D. Interaction in unknown human intention scenario E. Rule-based intention generalization 20 1.2.1 Intuitive HRI by intention recognition The Goal A involves the proposition of a probabilistic framework for intuitive HRI by intention recognition. The apprehension of the human intention is based on the human actions along with the scene changes that occur due to the human actions. The given information corresponds to the human actions and the scene information of HRI workspace concerning the problem. The required is the recognition of human intention out of the already known human intentions. The robotic tasks involve picking and placing of an object according to the human intention. Experimentation with the proposed probabilistic system involve the following 1. Picking and placing of an object according to the human intention 2. Handing over the intended object to the human 3. Pile up and unpile of objects according to the human intention 4. Picking up an object and holding that object and placing the held object at a human intended place. 1.2.2 Intention learning by scene observation The input to the problem corresponds to the human actions, scene information, scene change information, and the human intentions in terms of scene information. The output corresponds to the modelling of a new human intention. The Goal B corresponds to the inference of the human intention from the actions performed by the human as well as from the scene changes occurred due to the human actions. Each newly learned human intention is modelled using a finite state machine. The inference of the human intention is performed based on the already known features. The expected experiments include the arrangements of the known objects with respect to a pattern according to the human intention. The robot responds by recognizing the newly learned human intention. 1.2.3 Proactive intention estimation The Goal C corresponds to quick recognition of a human intention. It includes the premature recognition of an intention in an ambiguous situation that may lead to two or more human intentions. The Goal C includes the proposition of probability-based approach that helps the system to adapt towards the human behaviour and to react proactively in the intermixed human intentions scenario. The system can either wait for disambiguation of the intention, requiring extra human actions or it can proactively react depending on its previous knowledge about the human behaviour. Proactive intention estimation task includes the proposition of the mechanism to update the intention recognition trigger states for the probabilistic finite state machines that model the human intentions. A state of a state machine is assigned as the trigger state. If the trigger state of a finite state machine is reached then the human intention concerning the finite state machine is recognized. The online trigger state update corresponds to the online selection of a state of a finite state machine as the trigger state. The experiments involve the arrangements of objects that represent different human intention but have similar portion too, e.g., the objects placed in a square pattern and the objects placed 21 along a line. There exists a pattern (placement along the line pattern) that is similar in both patterns. The objects placed in different patterns are shown in Figure 1.7. Figure 1.7: Left: Square pattern. Middle: Line pattern Right: Similarity in both the patterns 1.2.4 Interaction in unknown human intention scenario The Goal D corresponds to the solution of HRI in case if the robot does not know the human intention, i.e., by no means the robot can recognize the exact human intention. Based on the current actions and the history of the actions the robot tries to estimate the next most likely action. The solution corresponds to a reinforcement based probabilistic action selection for HRI. The HRI environment is already known to the robot. The sub tasks for the Goal D consist of the following 1. Action hypotheses generation based on the known actions 2. Prediction of the actions based on the previous action in the current task 3. Weighting of the predicted actions 4. Calculating the history support of the action hypotheses 5. Calculating the conditional probability (P(Action t | Action t-1 )) and the prior probability (P(Action t )) for the predicted actions 6. Related implementations The experiments involve the arrangement of known objects with unknown human intention. The task of the robot is to interact with the human according to the estimated human action. 1.2.5 Rule-based intention generalization The input to this problem corresponds to the rules inferred from the human actions. The required is the reduction of antecedents of the rules by HRI. The task in the Goal E is to enable the robot to generalize its HRI capabilities. The robot infers rules and generalizes them to extend its interaction capabilities with the cooperating human. The extension means that the robot performs the known actions that were not instructed to him to perform concerning a human intention. The rule-based intention generalization is divided into the following sub tasks 1. Rule generation 2. Rule application 3. Rule generalization Rule generation concerns the rule inference that describes an action performed on an object having certain known characteristics. During the rule generation, the system knows the 22 objects present in the scene, the change in the scene occurred due to the human action and different properties / characteristics of the objects present in the scene. Rule application corresponds to the selection of the objects on which the rule can be applied. Rule generalization corresponds to the elimination of maximum number of unnecessary antecedents from the inferred rule. The anticipated intention generalization experiments involve the following 1. Picking and placing speckled object into the container for the speckled object 2. Picking and placing broken object into the container for the broken object 3. Picking and placing non speckled object into the concerning container Generalizing the above defined operations on the other related (match with respect to property / characteristic) objects will enable the robot to perform a task that the robot has neither observed nor been instructed, e.g., the robot only knows to place a speckled object of a specific type into the speckled container. After the generalization, it can place all types of the speckled objects into the container for the speckled objects. The generalization enables the system to respond in an unknown situation (with known objects). Unknown means that system is not explicitly taught that how to react in case of a certain known object. 1.3 Demarcation HRI is a multiple domain research field. It contains the computer vision to monitor the HRI workspace for safety reasons concerning the avoidance of human robot collision. It contains the robot path planning, revising of the previously planned path, and collision avoidance for optimal movement from source to destination. It may also contain image reconstruction for scene monitoring. Along with human behaviour modelling, recognition of emotional states of the cooperating human and related fields can be part of the HRI. Similarly learning in HRI is also a complete subfield of HRI. The presented approach does not contribute to any of the above mentioned areas. The presented probabilistic approach to intention recognition for HRI is general and does not correspond to a specific environment. There is no strict connection between the presented approach and any specific HRI scenario. The presented approach does not propose an image-processing-based method for scene understanding. The process of scene understanding corresponds to the apprehension of scene. The approach also does not address the issue of apprehension of any performed human actions, operation on the objects in the scene, changing in the scene and related scene inferring parameters. The inferring parameters correspond to the known features for inferring the scene information. The recognition of human gestures is also not included in the focus of the presented approach. Moreover, the presented research work does not consider the issues concerning the resource sharing in the common HRI workspace. The proposed approach can be applied on humanoids and other robots for HRI. There is no robot specific operation proposed along with the given approach. There is also no sensor specification in the presented approach. Any kind of sensor can be used to monitor the HRI workspace. The selection of sensor depends on the current type of HRI workspace and the expected operations in the workspace. There is no specification about the respective robot actions in response to the human actions. Like the scene understanding the robotic action information depends on the current robot in HRI. 23 1.4 Overview The research work is organized as follows: Chapter 2 describes the already existing approaches for HRI. The discussed approaches correspond to the social issues concerning HRI, variable autonomy HRI, HRI approaches concerning robot as an assistant, and tactile HRI. At the end of Chapter 2, the differences are discussed between the existing approaches and the presented research work. In Chapter 3 the proposed approach for intention recognition is described in detail. The modelling of different human intentions using the finite state machines is described in this chapter. Chapter 3 also discusses the algorithm for the probabilistic intention selection. At the end of Chapter 3, the experiments concerning the intention recognition using the proposed approach are described. In Chapter 4 an online intention learning approach is introduced. The introduced approach is based on the intention recognition approach described in Chapter 3. Three types of intention learning methods are discussed. At the end of Chapter 4, the experiments are discussed that are performed for online intention learning. In Chapter 5 premature and proactive intention recognition is described. The described approach is based on the approaches discussed in Chapter 3 and 4. The described approach takes into account the HRI scenarios that are similar to an extent but lead to different human intentions. Additionally an algorithm is introduced for the finite state machines representing the human intentions. The algorithm enables the finite state machines to recognize the human intention as early as possible. At the end of Chapter 5, the experiments are discussed that illustrate the proactive and premature intention recognition. Chapter 6 discusses the HRI in a known environment with unknown human intention. The proposed algorithm hypothesizes the potential human actions and selects the most suitable action for HRI. The robot may be corrected by the human. The robot can reselect the next most suitable action for HRI depending on the interacting human. At the end of Chapter 6, the experiments are discussed, performed using the proposed approach. In Chapter 7, an approach concerning the generalization of human intention is discussed. The approach describes the rule based human intention generalization. This approach corresponds to the concept generalization. The rule-based generalization uses the approaches of Chapter 3 and 4 to implement the human intention generalization. The generalization procedure is performed by HRI. The generalization methods using HRI and the rule conflict resolutions are discussed in detail in the Chapter 7. At the end of Chapter 7, the performed experiments are discussed that demonstrate the generalization result obtained through the proposed approach. In the end, Chapter 8 summarizes the presented research work and provides an out look on future work. 24 25 Chapter 2 Related work In this chapter most of the discussed approaches relate to the HRI in which the human interacts with a robot in the vicinity of the robot. In Section 2.1 the overview of the existing approaches concerning HRI is given. The existing approaches are discussed with respect different aspects of HRI, i.e., social HRI, robot as an assistant, and tactile HRI. In Section 2.2, the approaches concerning the social issues of HRI are discussed. Section 2.3 corresponds to the HRI in which the robot acts as assistant to the human to complete the task. The discussed approaches correspond to robot as tour guide in museum, a harvester, assistant in rescue operation, etc. The third aspect in Section 2.4 discusses different types of approaches concerning sensors that are used for tactile HRI and the types of tactile HRI. The sensor based approaches correspond to interpretation of sensor data and the types of application of sensors in the tactile HRI. 2.1 Overview HRI is a mixture of many fields, e.g., psychology, cognitive science, social science, artificial intelligence, computer science, robots, engineering, and human-computer interaction [43]. The field of HRI corresponds to the research concerning understanding, designing, evaluation and the improvement of the robots that interact with the humans. One of the core issues in HRI is the effective communication between the interacting human and the robot. The motive of the HRI field is to consider all the possible communication channels and to improve them for better interaction. The HRI can be broadly classified into two classes [60], i.e., the teleoperation and the direct HRI. The class of teleoperation corresponds to the HRI in which the human and the interacting robot are separated. The separation concerns the temporal and / or spatial difference. In teleoperation the human and the robot are not required to exist at the same location. In direct HRI the human and the robot are present at a same location and physically interact with each other. The described classes can be further classified into sub classes taking into account the design issues, application fields, nature of information exchange, level of the autonomy required in the HRI, emotions based HRI, control issues, etc. A survey based on teleoperation is available in [132] and [69]. The survey in [132] discusses the teleoperation based on supervisory control and Human-machine interaction. A survey concerning the control theory of teleoperation is given in [69]. The space oriented teleoperation is surveyed by NASA given in [116]. The here presented literature focuses on the research work in the field of direct HRI. The direct HRI has two important aspects that may exist in almost all the categories of direct HRI, 32 resolution with respect to tactile sensation is quite low. The absence of skin can be handled with the installation of arrays of tactile sensors. The robots having the tactile sensors installed inside are mostly the industrial robot arms. In [62] the location and the tactile force of the human are sensed by the torque sensors installed at the joints of the light weight robot arm. There exist many approaches concerning the installation of tactile sensors on the body of the robot, e.g., entertainment robot SDR-4X II (QRIO) [86], dog robot [133], cat robot [134], the robotic creature [168]. In [86] the tactile sensors are used to detect the pinch operation at all the joints of the robot. In [133] the balloon pressure sensor is used to interact with the human. In [134] the piezoelectric force sensors are installed on different parts of the robot to detect hitting and touching on the cat robot. There exist 60 FSR sensors under the fur of the rabbit looking robotic creature to detect the human contact [168]. 2.4.2 Tactile HRI The physical HRI with respect to the existing tactile sensing approaches is divided into three categories [6]. In the first category the considered approaches correspond to the unexpected contact between the human and the robot. It means that either the human or the robot interfere with each other while operation. The tactile sensing corresponds to the safety involved in the HRI, in the first category. The tactile HRI in the second category corresponds to the expected contact between the human and the robot. The physical contact between the human and the robot is used as a communication channel to guide the robot to execute behaviour. In this category the human contact works as a trigger of behaviour of the robot. The third category corresponds to the human contact to the robot that is used to refine and build the behaviours in the robot. The human contact can also be used to correct the robot behaviour. A) Interfering interactions with the robots It is considered that unexpected human-robot contacts are unavoidable as the presence of robots in the human community increases day by day [6]. The existing approaches provide the reacting solutions in the result of a physical contact that can occur with a human. In [56] reactive control strategies are proposed. The proposed strategies use a bumper-based skin to detect the unexpected human contact. The redundant degrees of freedom present in light weight robotic arm are used for evasive motion of the robot in physical contact. During the evasive motion the orientation of the tool center point is maintained [62]. In [165] a robot arm of 8 degrees of freedom evades the human contact during the motion. The forces from the tactile sensors are measured in motion vectors and the resulting motion vectors are super imposed for the joint velocities. In [76] a predictive approach is described with respect to interfering interaction. The effects of the human-robot contacts are predicted and the concerning response are encoded into the robot behaviour. The collision tolerance in the endeffector control is implemented by modelling the compliance in the viscoelastic trunk of the robot [90]. In [90] no explicit tactile sensing is performed. B) Deliberated tactile interaction with the robot In this tactile HRI the robot expects the touch from the human. The human touch contributes to the robot behaviour. The contact is used as a medium of communication between the human and the robot. There can be two kinds of deliberate tactile HRI. In first case the human contact correspond to guide the robot. In this case the human contact is linked to the robot state. In the other case the human contact is used to convey the information about the human. 33 The case is linked to the human state. The context of the HRI is important in deliberated interaction with the robot concerning the robot state. There exist approaches that consider the tap sequence to select the robot behaviour. In [160] a tactile HRI is proposed that focus the industry robot and non-robot expert human user. The tactile interaction corresponds to the human contact at the end-effector of the robot. The human contacts are mapped to the known trajectory. Different human touches correspond to different trajectories. In [165] the tap sequence corresponds to different alphabets that are used to select the behaviour of robot. In [145] multi-finger touch is used to infer the alphabets for teleoperation and robot behaviour. The deliberate human contact is also used by the robot to interpret the human state. These human touches correspond to the contact that one human uses while interacting with the other humans. The other human estimates the state of first human from the contact. In [109] the robot classifies the five different human touches. The touch corresponds to slap, stroke, pat, scratch, and tickle. The approach proposed in [83] considers the contact-time, repetition, force, and contact area in order to interpret the human touch corresponding to hit, beat, and push. In [82] the humanoid interprets the human touch in different HRI scenarios, e.g., while executing a behaviour, co-execution, and reactive behaviour. In [100], the pose and position of human is estimated by the human touch. The estimated pose is used in reactive behaviour. A robotic bear [140] touches the human in response to the human touch. The robot orientates itself to the direction of human touch. The types of human touches are classified to estimate the human state. In [151] the tactile HRI corresponds to the interaction between the human and ballroom partner robot. In this HRI the human touches guide the robot behaviour and the robot also estimates the human state from the human contact in order to follow the human while dancing task. The contact with the human is used by the robot to predict the next dance step of the human. The force of the human contact is used to detect the human stride. C) Robot behaviour development by tactile HRI In this HRI the robot expect the human touch for the correction and development of robot behaviour. The human contact is used to communicate the intended human correction to the robot. The behaviour development is to produce the adaptive and compliant robots. The human contacts are expected while behaviour development but not at the time execution of the developed behaviour. The robot behaviour development by tactile HRI relates to the paradigm of “teaching by touching”. There exist different approaches for this paradigm. In [40] the behaviour of the robot is developed by human touch. The robot behaviour corresponds to the pose change of the robot according to the human touch. If the pose change is not according to the human then direct manipulation of robot pose is performed. A mapping is learned between the human touch and the directly manipulated robot pose. In [4] the translating finger touch is used to change the pose of the robot. The pose change is performed while the robot manipulates the objects. The tactile feedback is used to move an industrial robot arm for the demonstration of task. The task corresponds to the insertion of piston [62]. In [97] the idea of “motor development with physical help” is introduced. The experiments are performed with a child sized android CB2. In experiments the human provide physical help to the robot for going from prostrate state to the standing state. The robot minimizes the supporting force provided from the human and also learns the resulting motion. 34 2.5 Conclusion HRI is a vast field covering many aspects from the robot side and the human side. It is a multidisciplinary field involving human-computer interaction, artificial intelligence, robotics, natural language understanding, and social sciences. In the literature of HRI multiple aspect of the research are quite heavily explored. For example social human robot interaction, robot as an assistant, autonomy-based issues in HRI, tactile HRI, vision based safe HRI, HRI for teaching the robot, i.e., Programming by Demonstration (PbD), etc. The area of intuitive HRI specifically by intention recognition is not explored considerably. For intuitive HRI the robot needs to know the human intention. The human intention can be estimated by multiple ways, e.g., language understanding, monitoring, by guessing using the prior knowledge about the human, by combining the described aspects, etc. There exist different approaches for intention recognition in the literature. The existing approaches [75][101][50] focus on specialized solutions based on the problem at hand. There exist a couple of generalized approaches [139][149] for intention recognition but the inference in the proposed architecture requires a large numbers of prior and conditional probabilities [98]. The corresponding modelling required for the general intention recognition approach is quite large that there exist approaches to reduce the modelling [84]. There exists another concerning approach that corresponds to the intention recognition as an observer without letting the robot to actively take part in HRI [125]. The modelling structure used for the approach in [125] requires a relatively large state space [98]. A theoretical approach also exists that deals with intention recognition without taking into account the intuitive HRI [169]. There exist also another approach concerning intention recognition but the approach does not consider the existence of robot in the discussed idea. The described approach relates to the existing literature of plan recognition [98]. Similarly for proactive nature of HRI there exists multiple approaches but they do not strictly correspond to the direct HRI. Either they correspond to teleoperation or involve the mobilerobot navigating in an environment. A couple of approaches concerning direct proactive HRI exist that require that the specific number of intention estimates that should be given already [131][77]. In [77] the experiments do not involve any human rather a simulation is used and proactivity is achieved by the application of entropy. Furthermore, they are not extensible in the sense that they can be used online to add new intentions understanding to increase the interaction capability of the robot. Similarly the generalization of the human intention is also not available in these approaches. Moreover in the existing literature of HRI the intuitive HRI in an unknown human intention scenario is not explored considerably. In this research work we introduce a simple approach for intention recognition. The approach is also applied in the areas pointed out, i.e., online intention learning and generalization where the existing approaches do not provide an explicit solution. Additionally the research work also discusses an approach for HRI in a scenario if the human intention is not known to the robot. 35 Chapter 3 Intention recognition With the era of modern technologies, machines are becoming necessary part of the human life. More specifically, the presence of robots among the humans is increasing day by day [6]. The goal is to provide the services to the humans. The robots that are intuitive in providing the required services will be preferred to the machines that require considerable input for providing the required service. Intuitiveness is necessary for a robot to exist as a service provider, amongst the humans. Therefore, a robot needs to recognize the intention of an interacting human. Recognizing the human intention, the robot can smoothly cooperate with the human. There are many working scenarios, described in Chapter 1, where the intelligence of a human and the efficiency of a robot can be combined to provide a better output. Intention recognition of the interacting human is the key to intuitive HRI. It guides the robot by answering him the questions about what to do in a HRI workspace. For recognizing the human intention, different methods can be employed, e.g., the human may be directly asked about his intention, the intention can be presumed from the daily strict routines of the interacting human, the human actions along with HRI workspace can be monitored to estimate the human intention, etc. In this chapter we describe a novel approach [12] for intention recognition based on the human action and / or changes in the HRI workspace. This chapter is organized as follows: In Section 3.1, intention recognition is motivated with the examples of HRI and the problem discussed in Chapter 3 is defined. In Section 3.2, the literature review of the existing intention recognition approaches is provided. The description of the human intention modelling is given in Section 3.3. Each human intention is modelled using a Finite State Machine (FSM). The formal description of a FSM is given in Section 3.3. The approach for intention recognition is discussed in Section 3.4. The approach described in Section 3.4 uses the intention hypotheses to recognize the actual human intention. The experiments performed using the proposed approach, are described in Section 3.5. Section 3.6 summarizes the chapter. 3.1 Problem definition and Motivation The discussed problem corresponds to the recognition of a human intention. The robot is required to recognize the human intention by the information from the HRI workspace and the human actions A = {a 1 , a 2 , a 3 , …, a m }, m ∈ ℕ. The robot already knows the human intentions I = {i 1 , i 2 , i 3 , …, i n }, n ∈ ℕ. The robot can recognize the human intention by the commanding actions (gestures) performed by the human. The robot can also recognize the human intention by the human actions performed on the objects present in HRI workspace. The human is allowed to switch between his intentions without completing the actions 36 concerning an intention. The human is also allowed to perform unrelated actions while performing the actions concerning an intention. The input to the problem involves the human actions, scene information, the scene change information, and the human intentions. The output corresponds to the recognition of a human intention out of the already known human intentions. The effectiveness of intention recognition in HRI is motivated with the help of Figure 3.1. The interaction of a humanoid and a human is shown in Figure 3.1 left. The humanoid offers a tray of coffee cups to the human. An accident can occur if the human and robot do not understand each others intention. If the human does not intend to take the tray and the robot does not recognize the human intention. Then the tray may fall down. The interaction of an industrial robot and a human is shown in Figure 3.1 middle. The human piles up the objects. In order to interact intuitively the robot needs to recognize the human intention of pileup of objects. The interaction of an industrial robot and a human is shown in Figure 3.1 right. The human holds the object grasped by the robot. The robot needs to recognize if the human wants to take the object from the robot or wants to orientate it in a direction. If the human intends to orientate it and the robot releases the object. Then object will fall down as the robot does not interact intuitively. The robot can only assist the human if it can understand the human intention. Thus recognition of human intention is inevitable for effective HRI. Moreover, in industrial HRI, safety of the interacting human is an important issue. The human intention can be used to predict the future position of the human to improve the safety in HRI. The robot can use the human intention to plan his collision free trajectory. Figure 3.1: Left: Humanoid HRI [1], Middle: Laboratory HRI, Right: Industrial HRI [127] 3.2 Related work Youn and Oh presented an approach in [169] for intention recognition, using a graph representation. They used three layered approach for intention recognition. The three layers include action layer, proposition layer, and the goal & intention layer. The action layer has action nodes, the proposition layer has state nodes, and goal & intention layer has goal and intention nodes. The presented approach makes relationships amongst the action, state, and goal & intention nodes. The connected nodes represent an intention graph. Each state node represents a ground literal. A ground literal is an atomic formula. It is assumed that any condition not mentioned in the state is false. An action is represented by a set of preconditions and a set of effects. The preconditions correspond to the conjunction of literals that must be true for the concerning action to be executed. The set of effects correspond to the conjunction of literals concerning the state changes. The effects are generated in result of the executed action. A goal consists of the desired states and termed as goal descriptors. An intention 37 corresponds to goal conditions and related user profile. The nodes in action, states, goals, and intention are connected to each other with six different kinds of edges in an intention graph. The intention recognition process consists of two phase, i.e. goal recognition and intention recognition. It is a theoretical approach that deals with intention recognition without taking into account the intuitive HRI. There exist no experiments that are performed with this approach. In [149] Tahboub proposed cycle elimination in Dynamic Bayesian Networks (DBN) for intention recognition. The approach in [149] describes that the cycles are generated due to the feedback from the sensed states to the intention states and the actions states. The proposed solution for cycle elimination considers the feedback of sensed states from the previous time slice instead of the current time slice [149]. The inference in the proposed architecture requires a large number of prior and conditional probabilities [98]. The corresponding modelling required for the intention recognition is so large that there exist approaches to reduce the modelling [84]. Mao and Gratch [98] have proposed an intention recognition method based on expected utility [48]. The intentions of the agent are represented by the plans that an agent may have. The expected utilities of the plans are calculated and a plan with maximal expected utility represents the estimated intention of the agent. A plan is represented probabilistically. The actions concerning the plan may have conditional as well as non-deterministic effects. The utility values represent the desirability of action effects. The actions have success or failure probabilities. The actions may be primitive or abstract. A primitive action corresponds to an action that can be directly executed. An abstract action can be decomposed into further abstract actions or primitive actions. The presented approach emphasizes on the desirability of the outcome of the intended task. The outcome of a task corresponds to the utility value of that task. According to this approach the agent whose intention is to be recognized, tries to maximize the expected utility. Thus the results of the plan / intention recognition are influenced by the already defined utility values of the plan / intentions as the agent will try to maximize the utility. The approach concerns intention recognition but the approach does not consider the existence of a robot in the discussed idea. The described approach relates to the existing literature of plan recognition [98]. Richard proposed an approach in [125] for understanding the human intention. They used Hidden Markov Models (HMMs) to recognize the human intention. The experiments are performed with a mobile robot equipped with laser sensor and a camera. The performed experiments involved the human intentions including Follow, Meet, Pass by, Drop off, and Pick up. These intentions correspond to the intentions between two humans that may follow each other, meet each other, cross without meeting and dropping or picking some thing. For each intention a HMM is designed. These models are trained by the Baum-Welch algorithm. The described novelty in [125] corresponds to the models that focus on dynamic interacting properties of an agent, i.e., Meeting, Passing by, Dropping, and Picking up. The selected visible variable for HMM corresponds to the change in the position and angle of the interacting agents. The introduced approach has two parts, i.e., activity modelling and intent recognition. In activity modelling, the already designed HMMs are trained. To train the models concerning Following, Meeting, Passing by, Picking up an object, and Dropping off an object, the robot executes these activities with an interacting human. The transition probabilities concerning HMM are estimated using Baum-Welch algorithm while activity execution. In the intent recognition part, the robot acts as an observer and evaluates the intent 38 of different interacting humans using the already trained HMMs. In recognition part the variables corresponding to the observed states are calculated differently as compared to the activity modelling. The Forward algorithm is used for the calculation of most likely sequence of observation. The Viterbi algorithm is used to detect the most probable sequence of hidden states. The approach corresponds to the intention recognition as an observer without letting the robot to actively take part in HRI. The modelling structure used for the approach in [125] requires relatively large state space [98]. In [139] Schrempf and Hanebeck introduce a generic model based on Hybrid Dynamic Bayesian Network (HDBN) for the estimation of human intention in a HRI scenario. They have emphasized the importance of hybridity for the robots operating in the real world. The hybridity corresponds to the continuous-valued and discrete-valued states. The continuous states are described for the sensor measurements. The sensor measurements and the probabilities concerning the measurements are directly related to the continuous scales. The human aspect, e.g., human intentions is mostly described by discrete values. The proposed HDBN contains the intention variables that are represented by the discrete values and the sensor measurements that are represented by the continuous values. Once again the inference in the proposed architecture requires a large number of prior and conditional probabilities [98]. The corresponding modelling required for the intention recognition is so large that there exist approaches to reduce the modelling [84]. Our approach provides a novel frame work for intention recognition [12]. It considers the possible intentions as particle and provides a particle filter based intention recognition. The particles representing the intentions are modelled using FSMs. The presented approach is discussed in detail in Chapter 3.3. The presented approach [12] models the human intentions as discussed in [84]. 3.3 Finite State Machines (FSMs) It is fairly difficult to come up with a straight forward mathematical state prediction model that can predict the next human action or next state of human, i.e., next posture of the human body or part of the human body concerning the human intention while performing a task. For example if the human has a glass in his hand and he approaches toward the beverages then it can not be mathematically predicted that he will select cola, water, wine, juice, etc from the beverages. These are all hypotheses. If we consider these hypotheses as complete action sequences for performing different possible tasks then these sequences can be represented by different models that will represent different intentions of the human. The action sequences considered as strings will not be robust due to intolerant string matching, e.g., if ABCD is the target string and the experienced string is ABCDE then the result of comparison will be negative. The E may be due to false recognition or unintentionally performed action. If all the action sequences are considered as a FSM then the state transition will become very complex. The FSM may require multiple start and end states due to distinct starting and ending action sequences. A state transition problem may occur if the human changes its action sequence (intention) without completing it, e.g., if the human performs actions A and B corresponding to an intention but switches its intention and performs an action E. If there is no state transition at the state (reached after action B) corresponding to action E then no transition will occur. Thus the changed human action sequence will not be recognized. If there 39 is an action sequence that is completely irrelevant from other action sequences then this situation may result in unconnected states in the FSM. Therefore each action sequence corresponding to a human intention is modelled by a distinct FSM. Different FSMs are designed regarding different human intentions. Each FSM represents the flow of different human actions one after another concerning the human intention. The FSM models the human intention by considering the concerned action sequence. The performed actions concerning an action sequence give the estimate of current human intention. During the execution of actions of an action sequence concerning an intention, if the human reverses the last performed action. Then there may be different reasons. For example, he can take the same action again with possible correction, he can start performing the actions of a totally different action sequence concerning another intention, he can stop performing further actions, he can keep on reversing the actions, etc. The term reverse means that if the human performs an action A then he can reverse that action A by performing the action ¬A. If the human reverses an action then it is not taken care by the FSM model. The reason is that it is assumed that an action sequence corresponds to the concerning human intention. The action sequence must be performed in a sequence for the concerning intention to be recognized. Therefore if the human reverses an action but does not change his intention then that action will be taken again by the human. If one intention corresponds to different action sequences then different FSMs can be used to model the same human intention. It is assumed that an action sequence is attached to the concerning intention. Thus an intention is defined by an action sequence concerning the intention. A FSM modelling a human intention has a single start state. The start state corresponds to the start of the action sequence. The discussed probabilistic FSM model does not consider multiple start states. There are different disadvantages of having multiple start states. The disadvantages exist with respect to the human intention recognition and intuitive HRI. If we consider more than one FSMs having multiple start states then it may be the case that an action A existing as one of the initial actions of a FSM also belongs to the final action of another FSM. If the human intends a task that has the action A as one of initial action but as it exist as the final action in another FSM the false human intention will be recognized due to the multiple start states. Similarly if the human starts performing the actions beginning at the start state S 1 and in-between switches to another action sequence beginning at the start state S 2 of the same FSM then the concerned human intention will not be recognized if the state transition does not exist between the concerning states. During work the human workers may perform the actions that are not directly related to their working intention. For example a human worker can scratch somewhere on his body, drink, divert his intention from work, talk to some other human, perform some unintentional task due to anxiety, etc. A human working on a task can perform arbitrary actions that are not related to the current task. The arbitrary actions do not emphasize that the human intends to change his intention concerning the current task. The change of intention means that he is not interested to perform that task any more. There may be the case that the human wants to suspend the task for some time. Afterwards, the human worker may start performing the actions concerning the intention. Similarly a human can switch between two tasks corresponding to two different intentions. He may come back to the previous task and starts performing the actions corresponding to that intention. He may also continue with the switched task concerning another intention. There may be multiple reasons that a human can perform arbitrary actions while having the intention of performing a specific task. Thus it is 40 significant to take care of the arbitrary actions of the human while recognizing the human intention. This is taken care automatically by the presented FSM model. A FSM is shown in Figure 3.2. Each unique human intention is represented by a distinct FSM. A FSM models the action sequence corresponding to a unique human intention. Each FSM carries a probabilistic weight. The weight represents how closely the FSM represents the human intention. If the weight is high then the FSM closely relates to the currently estimated human intention and vice versa. Each action ji a has a probability value )|( iji SaP at a state i S, i = 1,…,n and j = {1,…,m} \ k with jkmk ≠ ∧ ∈ },...,1{ . The n ∈ ℕ represents the number of states in a FSM and m ∈ ℕ represents the number of transition conditions. For a state i Sthe j can have all the values from 1 to m except one value k, jkmk ≠ ∧ ∈ },...,1{ . The k is variable and it is not required to be the same for n states of a FSM. 22 a 3 a n S 2 S . . . ... ... 21 a 11 a 1 S 2 S n S 31 a 1j a 1m a 1k a 12 a 32 a 2j a 2 m a 2 k a . . . . . . 22 a 3 a n S 2 S 3 a n S 2 S . . . ... ... ... ... 21 a 11 a 1 S 2 S n S 31 a 1 j a 1 m a 1 k a 12 a 32 a 2j a 2 m a 2 k a . . . . . . . . . . . . Figure 3.2: A FSM with n states, i.e., S 1 ,…,S n . Each state S i i=1,…,n (n ∈ ℕ) has m transition conditions (actions / scene changes), i.e., a 1i ,…,a mi , m ∈ ℕ. For each state S i i=1,…,n, it is defined that j = {1,…,m}\ k with k ∈ {1,…,m} ∧ kj. If a ji is observed at state S i then no state transition occurs. The transition only occurs if a ki is observed at state S i . The k is variable and it is not required to be the same for n states of a FSM The probability value P(a xi | S i ) describes how likely an action a xi is for the state S i of a FSM and x = {1,…,m}. The action a ki represents an action that has highest probability P(a ki | S i ) for the state S i and the state transition only occurs if a ki is observed as shown in Figure 3.2. The action a ki is not required to be the same for the n states of a FSM. The formal description of a FSM is given below in Figure 3.3. { } { } [ ] { } n ikiiijii ijiikiji m kji kii m ax ixii m n SF Sq niSaSδandSaSδ QQδ SaPSaPthatholdsitaaS SaPthatholdsitQS ,...,a,a,aa ,...,S,S,SSQ δFqQFSM x = = ,...,1==),(=),( →∑×: )|(>)|(:∑∈∃:∀ 1=)|( ∈∀ =∑ = ,,,∑,= 10 1+ ≠,1= ∑∈ ,1= 321 321 0 ∀ ∑ Figure 3.3: A formal description of a FSM. It describes that a FSM is a tuple of five elements 41 Each FSM = <Q, Σ, q, F,δ> is a tuple that contains set Q = {S 1 , S 2 , S 3 ,…, S n } that represents the number of n states in a FSM. The set Σ = {a 1 , a 2 , a 3 ,…, a m } represents the possible actions for a state S i ∈ Q. The sum of probabilities of all the actions for a state S i ∈ Q adds up to 1, i.e. For each state S i ∈ Q there exists an action a ki such that the probability of the action P(a ki | S i ) is greater than all the other actions, i.e. 1 If the action a ji occurs at a state S i ∈ Q then the transition occurs to the same state, i.e.,  (S i , a ji ) = S i . If the action a ki occurs at a state S i ∈ Q then the transition occurs to the next state, i.e.,  (S i , a ki ) = S i+1 . The action a ki is not required to be the same for the n states of FSM. The start state and the final state of a FSM are represented by q 0 = S 1 and F = S n respectively. The general flow of the algorithm for probabilistic intention recognition using FSMs is shown in Figure 3.4. Initialization of finite state machines Observation Weighting of the finite state machines Observation Likelihood for a finite state machine Intention recognized Weight of the finite state machine yes State transition in the finite state machine Low High No Final state of a finite state machine reached Low High Observation Intention recognized yes Low High No Low High Initialization of finite state machines Observation Weighting of the finite state machines Observation Likelihood for a finite state machine Intention recognized Weight of the finite state machine yes State transition in the finite state machine Low High No Final state of a finite state machine reached Low High Observation Intention recognized yes Low High No Low High Figure 3.4: The flow diagram describes that initially all the FSMs are in their start state. On each new observation the weights of the FSMs are updated. A currently active state of a FSM is a state whose previous states are travesred and the next states are not traversed. If the observation corresponds to the action that has the highest probability value for the currently active state of a FSM. Then a state transition occurs in that FSM. The state transiton can occur in more than one FSMs if the obseraction corresponds to the highest probability value actions at the currently active states of the concerning FSMs. The intention of the human is considered recognized if the concerning FSM has the highest weight and that FSM reaches its end state 1 The model is well defined that an action that causes the transition from a state S i to the next state S i+1 has the highest probability as compared to the other actions at the state S i . )|( ∀ ≠,1= iji m kjj SaP ∑ ∑∈ ,1 )|( m ax ixi x SaP = ∑ ∑∈ ,1 1)|(∈∀ m ax ixii x SaPthatholdsitQS = = [ ] )|()|(:∑∈∃:∀ ∀ ≠,1 ijiikiji m kjj kii SaPSaPthatholdsitaaS > = 48 2. Passing intention of the human: The human has the intention of passing an object to the robot and performs the concerning action. The human gives an object to the robot by offering an object on his hand. 3. Placing intention of the human: The robot places the already picked up object at a specific place according to the human intention. The human points with his pointing finger to the desired location. The robot places the already picked object at that location. 4. Picking and holding intention of an object: The human intends that the robot picks up a specific object in the human-robot collaboration workspace. The human points to the specific object in the HRI workspace and performs the pick up gesture. 5. Taking a pointed object intention: The robot provides the human the intended object that exists in the human-robot collaboration workspace. The human points to an object in HRI workspace and performs the taking gesture. The above described intentions from 1 to 5 were tested with 3 persons. The results of the number of tested intentions and the number of successfully recognized intention for the explicitly communicated intentions are given in the Table 1. Table 1 : The result of explicitly communicated intention The implicitly communicated intentions are described as under 1. Pile up of the objects: Human comes into the scene and starts working without engaging the robot actively. The human starts to pile up the objects. The robot estimates the human intention by observing the human actions and the changes occurring in the HRI workspace. After understanding the human intention of pile up, the robot collaborates with the human by performing the operation of pile up of the objects. 2. Unpile of the objects: Human comes in the human-robot collaboration workspace and starts the operation of unpile of the objects without engaging actively the robot. The robot understands the human intention and unpiles the objects. The results of the number of tested intentions and the number of successfully recognized intention for the implicitly communicated intentions are given in the Table 2. The false results shown in the Tables 1 and 2 are due to the unrecognized human hand gesture, e.g., the pointing hand gesture shown in Fig 3.10 (left) is recognized as pointing hand Recognized intention Tested Intentions Int1 Int2 Int3 Int4 Int5 Experiments Int1 19 0 0 0 0 20 Int2 0 20 0 0 0 20 Int3 0 0 18 0 0 20 Int4 0 0 0 20 0 20 Int5 0 0 0 0 20 20 49 while the hand gesture in Figure 3.10 (right) is not recognized as the pointing hand. In case if no expected action sequence is observed then no intention is recognized. Table 2 : The result of implicitly communicated intentions Figure 3.10: Extracted outlines of pointing hand posture The presented approach recognizes the exact human intention using the probabilistic representation of action sequence, e.g., if the human performs a non-related action during explicit intention communication then the FSM concerning the non-related action gets low weight and the FSM concerning the actual human intention gets high weight due to the completely performed action sequence. The Figures 3.11, 3.12, 3.13, and 3.14 represent how the weights of the intentions represented by different FSMs change during the intention recognition process. In the start all the intentions have normalized equal weight as shown in the intention graphs in Figures 3.11, 3.12, 3.13, and 3.14. At Step 1 an action of human is observed. The FSMs for which currently active state has high probability for the current human action get high weight and the others get low weight. If the end state of a FSM is reached and the weight of that FSM is also high then the concerned intention is recognized as the human intention. The graph in Figure 3.11 describes the intention recognition of picking an object from one place and placing that object at another place. At the start all the intentions have equal probabilities as shown at Observation 0 in Figure 3.11. An observation can be an action performed by the human and / or scene change information. The first observation made is not directly related to any particular intention of the human. Therefore all the intentions get almost the same weight at Observation 1. The human makes a pointing action to an object that he wants to be operated by the robot. The performed human action has high observation probability for pickup, pickandplace and take intentions. Therefore the weights of these intentions go up and the weights of others go down at the Observation 2 as shown in Figure 3.11. The state transitions occur in the FSMs for which the observed human action is highly Recognized Intentions Tested Intention Int1 Int2 Experiments Int1 7 0 10 Int2 0 9 10 50 probable. Therefore state transitions occur in pickup, pickandplace and take FSMs. At the Observation 3 the perceived human action was unintentionally performed human action. It was due to the fact that the human unintentionally opened his hand while moving it to another location. The unintentional action stance may occur if a human changes his actions stance from one action to another. At Observation 3 the perceived human action has high probability for place intention. Therefore the intention weight for place goes up and weights for others goes down. The state transition only occurs in the place FSM. The human now points to a place where he wants the object to be placed by the robot. The performed human action has high probability for pickandplace and low probability for others at the Observation 4. Thus the state transition only occurs in the pickandplace FSM and the weight of the intention also increases. The final state of the pickandplace FSM is reached and it has also has the high weight as compared to others. Thus the intention of pickandplace is recognized. Figure 3.11: The graph represents the change in the weights of FSMs concerning take, give, pickandplace, place pickup, and unpile intention. The graph shows the recognition of pickandplace intention The graph in Figure 3.12 describes the recognition of pickup intention. The intention weights are equal at Observation 0. At Observation 1 once again the human action stance does not corresponds mainly to any intention. Therefore the intention weights of all the intentions remain almost the same. The human once again makes the pointing action to an object. At Observation 2 the performed human action has high probability for take, pickup and pickandplace intentions and low for others. At Observation 3 the perceived human action does not corresponds mainly to any intention. Therefore there is no significant change in the intention weights. The human makes a makes an upward motion of his open hand for picking up of the pointed object. At Observation 4 the performed human action corresponds mainly to the pickup intention. Therefore the intention weight increases for this intention as shown in Figure 3.12. The state transition occurs in the FSM relating to the pickup intention and the end state of the FSM is reached. The weight of pickup is the highest and the end state is reached. Thus the pickup intention is recognized. 51 Figure 3.12 : The graph represents the change in the weights of FSMs concerning take, give, pickandplace, place pickup, and unpile intention. The graph shows the recognition of pickup intention Figure 3.13 : The graph represents the change in the weights of FSMs concerning take, give, pickandplace, place pickup, and unpile intention. The graph shows the recognition of take intention The graph in Figure 3.13 describes the recognition of take intention. The human points to an object that he wants to be provided by the robot. At Observation 2 the intention weights of take, pickandplace and pickup increase and others decrease. The state transitions occur in the 52 corresponding FSMs. At Observation 3 and 4 the perceived human actions do not correspond mainly to any intention. Therefore there is no significant change in the intention weights is observed. The human opens his hand and keeps it in this position. At Observation 5 the performed human action mainly corresponds to the take intention. Therefore the weight of intention increases significantly and the state transition only occurs in the FSM relating to take intention. The end state of the FSM is reached and the weight is the highest. Thus the take intention is recognized. Figure 3.14 : The graph represents the change in the weights of FSMs concerning take, give, pickandplace, place pickup, and unpile intention. The graph shows the recognition of unpile intention The intention graph shown in Figure 3.14 describes the recognition of implicitly communicated intention of unpile of the objects. The human grabs an object that is placed on the pile of objects. At Observation 2 the recognized human action mainly corresponds to the unpile intention. Thus the weight of unpile intention increases and the weights of other intentions decrease. While considering an observation the scene information is also taken into account as the human actions correspond to an implicitly communicated intention. The human picks the object from the pile and places it on the surface of the table. At Observation 3 along with the human action, the scene information is also inspected to check the increase or decrease in the unpiled objects. The human action and the scene information relates significantly to the unpile intention. Therefore the state transition occurs in the concerning FSM and the intention weight increases significantly. The state transition at Observation 3 53 brings unpile FSM in the end state and the weight of the unpile intention is also the highest. Thus unpile intention is recognized. 3.6 Summary In this chapter we presented a probabilistic FSMs based intention recognition algorithm. A human intention is modelled by a FSM. A FSM corresponds to a human action sequence and / or the concerning changes in the HRI workspace. A specific human action sequence and / or the concerning changes in HRI workspace directly relates to a human intention. Each FSM carries a probabilistic value that is called the weight of the FSM. The weight of the FSM describes how closely the FSM represents the human intention. The FSM with highest weight corresponds to the best estimated human intention and vice versa. The weights of the FSMs are updated at each new observation in HRI workspace. The FSM that carries the highest weight and reaches its end state represents the recognized human intention. The suggested solution is applicable for both explicitly and implicitly communicated intention recognition. Explicit intention communication addresses to all the situations where the human commands the robot and implicit intention communication addresses to all the situations where human does not engage the robot but robot actively starts the cooperation by recognizing the intention through scene information and human actions. Addressing both explicitly and implicitly communicated intentions recognition make the human-robot collaboration intuitive. The approach presented in Chapter 3 has differences from Particle Filter and HMM. Particle Filter is mostly used in the problems in which the dynamics of problem can be mathematically modelled, e.g., in robot localization the motion model of the robot is used for prediction of next potential position of the robot. In the current problem the FSMs are used to model the human intention that is totally different from a motion model of the robot. In the normal Particle Filter the resampling is performed to generate new particles and eliminate old particles with less weight. In the presented approach no resampling is required. The normal particle filter applications hypothesize the possible solution that is similar to the approach discussed in Chapter 3. In the HMM each hidden state is considered to have different observation probabilities and the different sequences of observations correspond to different sequences of the hidden states. In the discussed approach in Chapter 3 the sequence of observations corresponds to a human intention. 54 55 Chapter 4 Intention Learning A human has his intentions depending on the scenario, the goal, and the tasks that he is to perform in the current situation and in the near future. A human has different intentions at different places. It is difficult to model all the possible human intentions as the total number of human intentions can be huge. In advance the robot can not anticipate all the services that may be required and should be provided by the robot. That is why Programming by Demonstration (PbD) is introduced and extensively explored. The approach of PbD is different from online intention learning. In PbD the human commands the robot based on the demonstration. In online intention learning the robot learns the new human intention that is used afterwards by the robot to interact with the human. Learning the new human intentions from the human actions and the scene information can help the robots to collaborate with the humans more intuitively. In this chapter an approach [14] concerning online intention learning is discussed. The chapter is organized as follows: In Section 4.1 the problem of online intention learning is motivated and defined. In Section 4.2 the approaches concerning learning are discussed. In Section 4.3 three intention learning cases are introduced and discussed in detail. In Section 4.4 experimental results of the three intention learning cases are presented. In Section 4.5 the conclusions of chapter on online intention learning are given. 4.1 Problem definition and Motivation The intention learning corresponds to the modelling of a new human intention that can be used by the robot for HRI. It is assumed that the human only performs the actions concerning a human intention during the intention learning. The human performs the concerning actions in a sequence. The human completes an action sequence concerning a human intention without switching to another action or action sequence. The robot knows the human actions and the information of HRI workspace. The modelling mechanism is provided the human actions and scene information from HRI workspace to model the human intention. The input to the problem is the human actions, scene information, scene change information, and the human intentions in terms of the scene information. The output corresponds to a FSM concerning the new human intention. For intuitive HRI all the concerning human intentions can not be known to the robot in advance. Thus the robot is required to possess the ability to learn the human intention online to increase its intuitive interaction capability. We discuss three example scenarios of HRI to motivate the intention learning problem. The examples are 56 1. (House hold example) Consider a robot collaborating with the human for washing the utensils in a house. The robot is provided with the common human intentions specific for the purpose. For example the robot knows what to do with the dirty and clean utensils according to the human intention but he does not know what to do if a utensil is broken, old, useless, etc. Therefore in order to increase the intuitive interaction capabilities of a robot the robot needs to learn the new human intentions online. 2. (Laboratory example) The scenario for HRI is shown in Figure 4.1. The human intentions correspond to the arrangements of the objects in a specific pattern. Figure 4.1: Organization of the objects in a specific pattern according to human intention. Left: Organization of objects in a square pattern. Middle: Organization of objects in a longitudinal pattern Right: Organization of objects in a diamond pattern The human intention of arranging the objects in Figure 4.1 left and middle are known to the robot. The robot can recognize the human intention and interact intuitively regarding the arrangements of the objects. If the human has the intention of arranging the objects shown in Figure 4.1 right. Then the robot can not respond intuitively. The robot is required to learn the new human intention for intuitive interaction in case of Figure 4.1 right. 3. (Industry example) For HRI in industrial workspace the robot is provided with the information of standard human intentions concerning the specific HRI workspace. If any unknown event occurs in the HRI workspace then the robot can not intuitively interact with the human according to his intention. In order to improve the robot’s intuitive interaction capability the robot is required to learn the new human intention online. We consider an HRI scenario in Figure 4.2. Figure 4.2: Exemplary industrial HRI [88] The robot knows the human intention about holding and releasing the object but if the human intends to orientate the object then the robot will not interact and release the 57 object. Thus the robot needs to have the capability to learn the new human intention online. 4.2 Related work In literature, there exist multiple solutions for gesture and action recognition. There exist HMM based gesture understanding solutions [89] [9] that require the Baum-Welch algorithm to train the Markov models. They deal with the recognition of actions and gestures but not with the intention of a human. The available approaches for action recognition consider the image-processing as a core issue. There exist many human action recognition methods based on feature tracking [49] [79], intensity or gradient, and silhouette [163]. The human actions can be recognized based on the pose primitives [153]. The approach [96] uses 2D poses for 3D human pose recovery to recognize the human actions. Similarly the approach in [106] uses local motion appearance features to recognize the human action. The action recognition using HMM corresponds to different phases or key poses of the human while performing an action. These key poses are considered as the hidden states of HMM [118]. The main focus of the above described approaches is the recognition of activity / action / gestures. The recognition of intention is not considered and it can be estimated from the activity / actions. Other related information can also be used, e.g., different characteristics of entities present in the concerning environment. The intention may correspond to more than one activity or an action sequence. There is another significant difference between intention recognition and action / gesture / activity recognition. The difference corresponds to the fact that activity / gesture / action can only be recognized if the concerning action / gesture / activity is completely performed [125]. On the other hand, the intention recognition can only be helpful if the intention is recognized before the concerning action sequence is completely performed [125]. A multitude of research work already exists in the field of PbD in the direction of intention recognition. But the research work does not directly relate to the human-robot collaboration; because it commands the robot an action based on the demonstrated program [5]. The solutions present in the area of PbD use reinforcement learning [126] [139] [80], neural networks [91] [17] and also HMM [166]. In the literature, there exist multiple approaches for intention recognition. The approach proposed in [150] uses DBN, in [139] uses HDBN. The approach proposed in [77] [169] [98] use Ontology, Graph and Utility based intention recognition. The approach [125] uses a novel formulation of HMM to recognize the human intention. The intention recognition approach introduced in [12] uses probabilistic state machines. The described approaches recognize the human intention if the human intentions are already known and modelled. In case if the new intention is to be recognized then that has to be modelled explicitly by the human. The proposed approach [14] describes how a new human intention can be added without the explicit modelling by the human. To the best of our knowledge, there exists no solution for online intention learning in the area of human-robot collaboration by intention recognition. The presented approach [14] has vital differences to gesture and action recognition. The approach does not focus on single action rather an action sequence concerning the human intention. Further the core issue is not imageprocessing concerning modelling of different human poses. It models the action sequence concerning the human intention. The sequence concerning an intention may also involve the environment information along with the actions. 64 performed experiments. The more the line is near to the value 1 at a point the more successful and vice versa. Figure 4.8: Learning phase for Case 1 The successful experiments are represented by a point at the value of 1 and 0 otherwise in the following graphs. A successful experiment means that the expected results are obtained. The expected result in case of teaching the system a human intention means the construction of the corresponding FSM. The expected result in case of testing a human intention means the recognition of the concerning human intention by the system. In case of testing, the robot is required to react according to the human intention. If the robot reacts according to the human intention then the experiment is considered successful and vice versa. Each point represents one result of experiments of a person. Fluctuations in the average line represent the success and the failure due to the variance of the action postures by different humans with respect to the same action task. The success rate is the ratio between the successful experiments with respect to the total numbers of experiments in one phase of a case. Figure 4.9: Testing phase for Case 1 65 The success rate is high as the average line remains at the value 1 and vice versa. The average success rate in the learning phases is 73 % and in the testing phase the average success rate is 87 % for the Case 1. The fluctuation of the average line describes that mostly for each point of experiment, two persons performed successfully, as shown in Figure 4.8. The graph in Figure 4.9 describes the experiment results in the testing phase of Case 1. The fluctuation in the average line of Figure 4.9 is less with respect to Figure 4.8. It is due to the fact that few actions are required to recognize the human intention and the robot reacts afterwards. Figure 4.10: Learning phase for Case 2 The reason for the difference in success rate in the testing and learning phases is due to the fact that the system has to perform more image processing in learning phase as compared to the testing phases. In the learning phase all the actions and the human intention are processed and in testing phase only the initial action sequence is processed. Using very simple imageprocessing (Fourier descriptor for contour recognition and skin detection), an action performed with unexpected human body part posture is less likely to be detected. For Case 2 and 3 the performed experiments involve the placing of objects in a human intended pattern as shown in Figure 4.11. Figure 4.11: Different human intentions regarding placement of objects in a specific pattern 66 The average line in Figure 4.10 represents that the success rate is almost equal to the success rate shown in Figure 4.8. The success rate of experiments is 69 % shown by the average line in Figure 4.10. The success rate of experiments shown by the average line in Figure 4.12 is 80 %. The difference between the success rates of experiments shown in Figure 4.10 and in Figure 4.12 is almost the same as in experiments shown in Figure 4.8 and in Figure 4.9, due to the same reasons discussed for Case 1. Figure 4.12: Testing phase for Case 2 The success rates of experiments shown by the average lines in Figure 4.13 and 4.14 are 100 % and 95 %. The reason of 100 % success rate is due to the fact that the action sequence was considered in terms of the scene changes performed by the human, i.e., it is observed the result of what the human has performed rather than how the human has performed the action. Figure 4.13: Learning phase for Case 3 67 Figure 4.14: Testing phase for Case 3 The ditches in the average line in Figure 4.14 describe the fact that the human has performed an action and in response to that scene change information the human intention was not recognized. Then the human made an appropriate amendment to his action and due to that correction the intention was recognized. This fact is represented by the dotted line part in Figure 4.14. 4.5 Summary In this chapter we have discussed three cases of intention learning. The cases discussed the mapping of human intention to the corresponding observation sequence. The mechanism used for intention recognition consists of the probabilistic FSMs, described in Section 3.4. For online intention recognition a FSM regarding to a specific intention is constructed online. The online intention learning contributes to the intuitive HRI capability of the robot. The experiments were performed for all the three cases of online intention learning. During the learning phase the intention is conveyed once by performing the concerned action sequence. It was observed that the Case 3 is more flexible for capturing the human actions and human intention and robust in results. The reason of comparative success for capturing human actions is the simple image processing, in the Case 3. The results in Case 1 and Case 2 are also acceptable. In Case 1 and Case 2 the recognition of the action sequences are performed by recognizing the human actions. For this purpose the image analysis corresponds to the processing of the different human gestures. The recognition of gestures focuses on different body parts of the human. The specific orientations of the human body parts are used to recognize the human gestures. For this purpose Skin detection [161] and Fourier descriptors [171] are used. A human can perform a same gesture with different orientation of the same body part. Moreover the structure of the body parts of the humans also varies, e.g., a person may have long hands and others may have wide hands etc. Therefore the recognition of the human actions based on the image processing of the body parts is difficult. In Case 3 the recognition of the human actions is performed based on the scene changes occurred due to the human actions. It is comparatively easy to recognize the human actions based on scene 68 change information as compared to the analysis of the body parts. For example if the human picks an object and places that object at some other place then it is easy to recognize the pick and place action by the scene change information occurred in the presence of the human as compared to the image processing of the concerned body parts. 69 Chapter 5 Proactive interaction Proactivity is an important aspect for effective cooperation. The proactivity is defined as the quick response of the robot during HRI. It means that the robot can recognize the human intention as early as possible to quickly start the interaction with the human. The humans working on a common task are required to intuitively collaborate with each other. They are required to be proactive towards each other for intuitive collaboration. Intuitive HRI requires the robot to attain a high level of understanding of the collaborating human. Therefore it is equally important in the HRI that the robot should be proactive according to the collaborating human, depending on the current situation. For being proactive, the robot needs to recognize the intention of collaborating human as early as possible. The collaborating robot is also required to adapt to the human in the ambiguous situations to be proactive. Being proactive is not an easy job even for the humans. The humans take into account multiple aspects for a proactive interaction. These aspects may correspond to the interaction situation, social indications, personal profile, etc. Using all these they may be wrong in their decision about the selection of proactive initiative. Thus they are also required to adapt in their proactive behaviours. The approach [13] introduced in this chapter describes the FSM based method for proactive interaction. The FSM based intention recognition [12] is discussed in Chapter 3. The proactiveness corresponds to the recognition of the intention as early as possible. The remainder of the chapter is organized as follows: In Section 5.1, the problem of proactive interaction is defined and motivated. The existing approaches concerning the proactivity of the robot are discussed in Section 5.2. The proactive intention recognition is discussed in Section 5.3. Section 5.4 describes the adaption capability of robot according to human intention for proactive HRI in almost similar scenarios. The experiments performed using the current approach, are discussed in Section 5.5. Section 5.6 summarizes the chapter. 5.1 Problem definition and Motivation The problem of proactive HRI corresponds to the fact that the robot can quickly start the interaction with the human. The robot can also decide in an ambiguous situation A that may lead to two or more different human intentions, i.e., A → {i 1 ,…,i m }, m ≥ 2, m ∈ ℕ . In case of quick HRI, the robot knows the human intentions I = {i 1 ,…,i n }, n ∈ ℕ and the robot is required to recognize the current human intention i current ∈ I quickly without confusing with the other known human intentions. In case of decision making, the robot is provided with an ambiguous situation in which the robot is required to choose a human intention for HRI. In 70 both the cases the robot is given the actions performed by the human, the scene information, the scene change information, and the human intentions. The output in both the cases corresponds to the selection of a human intention. Proactivity is an important aspect of intuitive interaction. Along with many other characteristics for intuitive interaction, the humans possess the capability of proactiveness. The capability of proactiveness helps the human to ease the interaction and causes the reduction of explicit commands for communication. The humans practice the capability of proactiveness by the different kinds of indications from each other [22]. The capability of proactiveness in the humans concerns the early recognition of the intention of the cooperating person with the help of available indications. We consider an example of interaction between the mother and the young children. Before the children start playing at a place, the mother cleans that place and removes all the harmful objects proactively. The removal of harmful objects will keep the children unharmed and safe. Similarly the example of a waiter serving drinks at a party. The waiter observes the guests in the party and proactively offers the drinks to the guests. The waiter takes proactive decisions depending on the social cues from the guests. The waiter adapts its proactive behaviour with respect to the responses from the guests. The significance of proactiveness in HRI is discussed in the following two examples, i.e. A. Safety in HRI B. Improvement of intuitiveness in HRI 5.1.1 Safety in HRI We consider the HRI in an industrial scenario shown in Figure 5.1. The robotic arm and the human interact to complete a common task. The robotic arm interacts intuitively with the human according to his intention. The robot plans his motion path to interact and perform the tasks according to the human intention. There may be a collision between the human and the robot. The chance of collision can be reduced if the robot can proactively anticipate the human intention and plan its motion path accordingly. Figure 5.1: [148] Human and robot working on a common task in a workspace. The robot can avoid any potential collision by anticipating the future human intentions proactively. The robot then plans its collision free trajectory based on the estimated future locations of the human For example the human is performing an operation concerning a task. The robot is given the information about the complete task, i.e., what is the sequence of different operations 71 concerning the task. The robot can also learn the interaction profile of the interacting human or it can be given to the robot. With this information, the robot can predict the future locations of the human taking into account the left over operations and the interaction profile. More parameters influencing the human location can be considered to predict the human locations for more accurate results. 5.1.2 Importance of proactiveness in intuitive HRI Proactivity is an important aspect for intuitive HRI. In intuitive HRI the proactivity of the robot can correspond to the recognition of human intention as early as possible even if the intention of the human is not obvious. In a motor mechanic’s workshop, the robot can proactively recognize the intention for repairing the specific problem and can operate on the concerning areas of the vehicle performing the necessary initial tasks. In the big kitchens, the robot can proactively perform the necessary operations concerning the tasks in the kitchen, e.g., if the cook intends to prepare a specific dish then the robot can proactively perform actions in order to ready the necessary ingredients. The robot interacting intuitively with the craftsman can prove to be the third hand of the craftsman. The proactivity of the robot may correspond to different robot operation concerning different tasks during HRI. For example the robot can grasp and orientate the surface of the object on which the craftsman intends to perform. Proactivity in the rescue robots may correspond to the necessary first aid actions that should be taken by the rescue robots depending on the situation and the condition of the victim. A man-machine interaction in a close contact to each other on the vehicle assembly line is shown in Figure 5.2. The humans have the task of installing different accessories into the vehicle. The accessories may include the doors, dash board, seats, steering wheel, wind screen, etc. The robot can help by the bringing the next objects that are to be assembled after the object that is currently being assembled. In an industrial HRI the proactivity of robot can increase the output of the assembly line workers, e.g., by providing the necessary articles by the early recognition of the human intention and the HRI workspace. Proactiveness eases the HRI and increases the intuitiveness in HRI. In case of correction from the human the robot can rectify its proactive behaviour according to the human intention. Figure 5.2: The assembly line workers are fixing the wind screen on a vehicle. The wind screen is carried by a device and guided to its exact location by the human workers [72]. In case of intuitive HRI the robot can create ease of navigation by slowly moving in the guided direction and proactively cooperating in the next tasks 72 5.2 Related work Proactiveness corresponds to the early anticipation of the future actions of the interacting person and their application. It is mostly applied in the business to anticipate the business trends and take the anticipated actions that are beneficial. In robotics, there exist also the approaches that apply the proactiveness in robotic tasks. The application mostly exists in mobile robots. In robot navigation the term of proactivity is used in the context of planning, e.g., in [105] the term proactivity is used for parallel planning and execution of navigation operations. In [103] proactivity is used for planning a safe trajectory for a mobile robot. The safety corresponds to the selection of the robot speed to avoid collisions. Armano, Cherchi, and Vargin proposed an agent planning in a dynamic environment [3]. The agent is created to act in the virtual world designed for a computer game. They described a layered approach for agent actions in the dynamic environment. Each layer consists of deliberative, proactive, and reactive modules. The theoretical description is not given in detail and no related experiments or simulation is performed. Finzi and Orlandini have proposed a control architecture for rescue robots using HRI. It shows the combination of decision processes with the functional process of the robot. HRI introduced in the approach [51] corresponds to the mixed initiative planning. The term mixed initiative means that along with the human the robot can also proactively react. The mixed initiative based approach described in [51] does not correspond to a HRI in which robot responds proactively understanding the human intention of the victims to be rescued. Dee proposed the use of internal states in order to design the proactive embodied agents [44]. The difference between the reactive and proactive embodied agents is the application of internal state. The proactive agents develop the internal states by integrating the sensorymotor information with respect to time. Afterwards the agent can use the internal state to apply the stored motor modulation information. The motor reaction is the function of sensory information of the internal state and the current environment state. The internal states are modelled using different variations of neural networks. The authors described that the understanding of internal states can help to develop better proactive agents. The simulating robot “Embodied Proactive Human Interface” named “PICO-2” described in [85] is an interactive interface. The idea corresponds to the two humans communicating over a telephone line. Instead of using the telephone, the robots communicate the information between the two persons. The robot communicates the message by performing the gestures demonstrating the intention of the remote person. The robot works as an Avatar of the remote person. To our understanding “PICO-2” is the demonstration of the recognized intention of the remote person. The proactiveness of the robot in the current scenario may correspond to the capability of the robot if the robot can anticipate the future response of the remote person and indicate the anticipation in current message. The message is conveyed by the corresponding robot gestures. Cesta proposed the proactive behavior of the robot by activity monitoring [28]. The proposed approach focuses on the elder care by the robot assistant. Two ways of interactions are described namely, On-Demand interaction and Proactive interaction. Proactive interaction corresponds to the activity monitoring and constraint-based proactive and warning giving response. An abstract algorithm is described for adding and removing the constraints while monitoring the activities. The elder care project is at its beginning as described by the authors. 73 The described behavior appears warning or reminding of some operation forgotten by the humans. Jeon, Kim, and Choi have presented Ontology based user intention recognition in [77]. Their focus is on the planning given the user intention. The user intention is recognized using the rule-based RuleML approach. The experiments are performed with a simulated robot. The system is implemented using DBN. To our understanding, the DBN is used to model the plan concerning the actions to be performed given a recognized intention using RuleML. The experiments in [77] do not involve any human rather a simulation is used and proactivity is achieved by the application of entropy. All the previously discussed approaches do not relate exactly to direct HRI. The proactive action selection proposed in [131] describes the proactive robot response concerning HRI. The robot selects actions concerning the known human intention. The proactive action selection is performed given the estimate of the human intention. The number of given intention estimates are represented by | Г |. An action tendency value is assigned to each action selected for proactive reaction. If | Г | = 0 then all the action get zero action tendency value. If the number of estimated intentions is greater than threshold ω then once again the action tendency is zero. The value of threshold ω is used 3. If | Г | = 1 then the actions related to the intention get the concerned action tendency values. These values are assigned by the human experts. Then an action is selected using Lorenz’s psycho-hydraulic model [95]. In case if 1 < | Г | < ω then action selection is performed using conditional entropy, expected success rate, valence value, safety requirement and most likely action sequence. The proactive reaction in [131] given the intention estimate is related in its concept to the research presented here [13]. However, there are vital differences between both approaches. The approach presented in [131] assumes that the intention estimates are given, uses a threshold number of intention for proactive reaction, focuses on the actions rather on the proactive action selection and considers all the intentions without considering them any relevance with respect to the current situation. The presented approach [131] also does not provide a confusion resolution if the intention estimates consists of conflicting intentions. All in all, the proactive reaction of robot is still novel when the human and robot are in strict cooperation with each other. The proactive response of the robot means that the robot can quickly recognize the human intention and react accordingly. The robot recognizes the intention implicitly by the human actions and the surrounding environment. Although the safety is an issue for autonomous HRI; that is why the proactiveness of robot is still in its beginning with respect to strict HRI especially in industrial robotics where big and power full robotic arms work at considerably high speed. 5.3 Trigger state determination Proactive and in-time reactions from the robot are important for intuitive HRI. The procedure given in Figure 5.3 describes the method to enable the robot to respond as quickly as possible by selecting the earliest possible trigger states of the FSM i , ni ,...,1 = . The trigger state is the end state which corresponds to the state that finishes the intention recognition process. The intention recognition process [12] is discussed in detail in Chapter 3. The FSM i , ni ,...,1 = represent the human intentions. The input to the procedure in Figure 5.3 consists of all the previous FSM i and FSM n+1 that may be added to a group of Finite State Machines (FSMs). 80 Now if a unique observation relating FSM 1 (Figure 5.7) at state S n occurs and intention regarding FSM 1 is recognized then the transition weight of a 2 in FSM 1 (Figure 5.7) is increased and intention weight of a 2 in FSM 2 (Figure 5.7) is decreased. The update is performed by the addition of 1/| FSMs|, i.e., the average value of the numbers of FSMs having the common transition condition. In the above described example there are two FSMs having one common transition condition. The transition weight is multiplied with the adaption factor to calculate the transition value as shown in Figure 5.8. The intention related to FSM 1 (Figure 5.7) is recognized thus the transition weight of a 2 in FSM 1 (Figure 5.7) is increased by 0.5. The adaption factor θ increases the transition weight to the extent that a common transition condition in a specific FSM is triggered as shown in the calculation, given in Figure 5.8. For that intention to be recognized the human produces the unique transition condition relating to the concerned FSM. The common transition condition causes the transition between the states for a specific FSM that represent the recognized human intention. If θ is selected as |FSM|/2 then the adaption rate for a common transition condition a ij (at i th state of j th FSM) of FSM j becomes 2 for |FSM| > 3. The adaption rate of 2 means that if an intention represented by FSM j is recognized 2 times consecutively with respect to other FSMs in a group having the common transition conditions. It is assumed that the transition weights are uniform. Then the transition weight of a ij of FSM j is increased and the weights of other related common transition conditions in the group FSMs are decreased. The two times consecutive increments of transition weight of a ij and the scaling performed with |FSMs|/2 causes the state transition due to a ij for |FSMs| > 3. If |FSMs| = 3 then three times consecutive increments in the transition weight of a ij is required to trigger the a ij state transition. Similarly, if θ = |FSMs|/3 then the specific increment in the transition weight requires 3 steps for |FSMs| > 7. In case if 5 ≤ |FSMs| ≤ 7 then 4 steps are required. The Table in 5.1 describes the number of steps required with respect to the |FSMs| and θ. θ No of Steps |FSM| | FSMs|/2 2 > 3 | FSMs|/2 3 = 3 | FSMs|/3 3 > 7 | FSMs|/3 4 ≥ 5 | FSMs|/3 5 ≥ 3 | FSMs|/4 4 > 11 | FSMs|/4 5 ≥ 8 | FSMs|/4 6 ≥ 6 Table 5.1: Description of θ with respect to |FSM| and no of steps The transition weights are calculated in terms of 1/|FSMs| as shown in Figure 5.9 and 5.10. The calculation is so performed that the transition weight m 1 is increased at each step by 1/n = 1/| FSMs|. At each step m 1 is updated (increased by 1/n) and then normalized. The six step ● ● ● ● ● ● ● ● ● 81 update and normalization is performed for m n transition weights in Figure 5.9 and 5.10. Thus m 1 increases and m 2...n decrease. Figure 5.9: Calculated Transition weights 133 ,..., 133 133 134 )1( 133 )1( )1( )1( 134 ionNormalizat )1( 1341 )1( 13 updationWeight :3Step )1( ,..., )1( , )1( 13 1 1 1 )1( )1( 13 ionNormalizat )1( 131 1 2 updationWeight :2Step 1 1 ,..., 1 1 , 1 1 , 1 2 11 )1( 2 ionNormalizat 211 updationWeight :1Step Initially 1 conditioncommonwithFSMofNumber .,...,3,2,1weighttransitionLocal 23 2 23 2 2 23 2 1 2 23 22 2 1 2 2 2 1 22 2 2 1 1 1 321 1 1 +++ = +++ = +++ ++ = + +++ = + ×−+ + ++ = + ++ =+ + + = + = + = + + = + = + ×−+ + + = + + =+ + = + = + = + = + = + =×−+= =+= = = ∑ ∑ ∑ = = = nnn n m nnn n m nnn nn m nn nnn n n n nn nn m nn nn nn n m n n m n n m n n m n n n n nn n m nn n nn m n m n m n m n m n n n n n m nnn m n m n nim n n ii n n ii n n ii i s i 82 Figure 5.10: Calculated Transition weights Therefore it can be easily checked by multiplying the θ with m 1 at different steps that how many consecutive steps (weight increments) are required for increment of m 1 such that m 1 can cause state transition, e.g., if we take θ = |FSM|/2 and |FSM|= 3 m 1 can cause state transition, results are shown in Figure 5.11. It is also mentioned above in the Table 5.1 row 2 that at Step 3 the transition weight (updated and normalized) multiplied by θ causes the state transition. That value multiplied with the transition probability (a j ) will not decrease the calculated transition value and will cause the state transition. Figure 5.11: Consecutive increment of a transition weight 10781.1 5.1 64 1936 2133 134 :3 193750.05.1 )13( 133 2)1( 13 :2 175.05.1 4 2 21 2 :1 3|FSMs| 23 2 1 22 1 1 >= × ++ =× +++ ++ = <=× + +× =× + + = <=×=× + = = = n nnn nn mStep n n n mStep n n mStep n • • • ++++++ = ++++++ +++++ = +++++ = +++++ ++++ = ++++ = ++++ +++ = 161520156 ,..., 161520156 161520157 :6Step 1510105 ... 1510105 1510106 :5Step 1464 ... 1464 1465 :4Step 23456 5 2 23456 2345 1 2345 4 2 2345 234 1 234 3 2 234 23 1 nnnnnn n mm nnnnnn nnnnn m nnnnn n mm nnnnn nnnn m nnnn n mm nnnn nnn m n n n 83 5.5 Experiments The experiments have been performed with a robotic arm of six degrees of freedom. The human and the robot interact in a HRI workspace shown in Figure 3.9. The work space consists of a table with objects and buttons on the table along with the robotic arm. The video data is captured with an overhead FireWire digital camera with the frame size of 640 x 480 pixels. The camera provides video data at the speed of 30 frames / sec. HRI and image analysis is implemented using Programming language C++. The robot reactions are realized using the robot Programming language V++ for the robotic arm. The robot is communicated the cooperative instructions using the TCP/IP connection for assigning different operation, e.g., pick, place and move to a certain location, etc. Common Skin detection, Edge detection algorithms and Fourier descriptors are used for the image analysis. The buttons on the table include Stop, Learn, Pause, Play, and Reset as shown in Figures 5.12, 5.13, and 5.14. These buttons are used by the human for communication with the robot during HRI. If the human wants to teach the robot about his intention then the human puts the hand on the Learn button. Afterwards the human performs the intended task. The Stop button is used by the human if the human wants to stop the robot from performing a task and undo the current robot action. The robot temporarily stops its activity if the Pause button is used. If the Play button is used then the robot starts recognizing the known intentions and after recognizing an intention the robot reacts accordingly. The Reset button is used to remove all the known intentions that are stored as FSMs. Figure 5.12: Intention for placing the boxes in a square pattern The perception of human intention is performed based on Case 3 discussed in Chapter 4, i.e., the human actions and intention is recognized from the scene changes occurred due to the human action. For performing the experiment regarding the arrangements of objects on the table, different human intentions are taught to the robot as discussed in Chapter 4. The two taught human intentions are shown in Figure 5.12 and 5.13. 84 Figure 5.13: Intention for placing the boxes in longitudinal pattern The Figure 5.14 shows the similarity of the situation (ellipse) for which the robot needs to decide for premature action selection. First the intentions relating to Figure 5.12 and 5.13 are taught to the robot. Then the robot is presented the situation shown in Figure 5.14. The robot can not decide how to react in the situation shown in Figure 5.14. The robot waits for the human to disambiguate the situation. Figure 5.14: Human robot interaction workspace Now, if the human performs the action regarding to one of the intentions as shown in Figure 5.12 and 5.13 then the transition weight of the common transition condition in concerned FSM is increased and for the other FSM is decreased. Initially, the ambiguous case as shown in Figure 5.14, if a task is disambiguated consecutively two times and third time the robot is 85 faced with the ambiguous situation then the robot reacts accordingly, i.e., the robot performs the most likely human intended task in that situation. Figure 5.15: Transition weights with out adaption factor The graph in Figure 5.15 corresponds to the transition weights in two FSMs with one common transition condition as shown in Figure 5.12 and 5.13. Initially, at Step 0 the transition weights are uniform, i.e., 0.5 for both the common transition conditions. The transition weight represented by red line represent the transition condition whose concerning intention is selected consecutively three times. Thus the red line rises and green line falls. In spite of rise in the red line, the transition weight (red line) is less than 1 at the Step 2 and 3 as shown in Figure 5.15. Figure 5.16: Transition weights with adaption factor The transition weight scaled with adaption factor 1.3 reaches the value 1 at Step 2, as shown in Figure 5.16 and causes the state transition. Form Step 3-7 the transition weight of transition condition (green line) is increased due to the consecutive selection of the concerning intention 86 as shown in Figure 5.15 and 5.16. At Step 7, the transition weight (green line) in combination with adaption factor can cause transition. Similarly in the case of trigger state determination and update the premature intention recognition is performed with the help of priors. The robot reacts according to the intention of highest prior FSM in the group. If the human intends an FSM with lower prior then the robot switches to the next intention (FSM) with the next highest prior. The priors of FSMs are updated such that the prior of intention (FSM) that is successfully applied is increased and the priors of the others are decreased. The priors of two FSMs in a group are shown in Figure 5.17. The graph in Figure 5.17 represents that for first 11 interactions an intention is selected consecutively and for the rest of 9 interactions the other intention is selected consecutively. Figure 5.17: Priors alternating due to the intention switch after 10 th interaction. 5.6 Summary In this chapter we presented a probabilistic proactive approach for the intuitive HRI in the ambiguous situation. Two cases were discussed for proactive robot response for intuitive HRI. For making the robot interactions as quick as possible, trigger state selection algorithm is discussed that describes how the trigger states are selected in case of similar state sequence of different FSMs. In this algorithm FSM n+1 is compared with all the already existing FSM i , i = 1,…, n. During the comparison the FSM n+1 can be added to the already existing group of FSMs. A new group can also be made concerning the FSM n+1 and already existing FSM i . A group of FSMs is only created if there is an already existing FSM i and FSM n+1 such that they exactly match with each other and one has higher number of states than the other. In case of groups the intention priors concerning the FSMs are created or updated. The increment and decrement of the intention priors is performed if an intention concerning a FSM that belong to a group is recognized. In this case the intention prior of the FSM concerning the recognized intention is increased by 1/ |FSMs|. The term |FSMs| corresponds to the number of FSMs in the group. After increment in the intention prior of the concerned FSM, all the intention priors are normalized. Thus the intention priors of other FSMs concerning other intentions in the group are decreased. 87 In the second case, the proactive nature of HRI is discussed at lower level, i.e., the ambiguous (leading to two or more different human intentions) human action performed by the interacting human is probabilistically handled for proactive HRI. The ambiguous human intention case is handled by the transition weights. The transition weights correspond to the weights assigned to the transition conditions in the FSMs. The transition conditions that are common in different FSMs are assigned the uniform transition weight. A common transition condition with uniform transition weight can not cause the state transition. Although the transition probability of the common transition condition is high but multiplied with the uniform transition weight the state transition does not occur. If the recognized intention corresponds to a FSM that has common transition condition then the transition weight for that common transition condition is increased and for other concerning common transition conditions the transition weights are decreased. An adaption factor θ is used to quickly increase the transition weight of a common transition condition. The adaption factor is multiplied by the transition weight to increase the value of the transition weight. If the robot has proactively responded according to the human intention then the human intention does not change. In case if the proactive response is not exactly according to the human intention then the human intention may change, e.g., if the human intended to drink cola but he was offered water then he may change his intention to drink water. If the robot’s proactive response is totally different from the human intention then the human intention may not change and the robot’s reaction can be rejected by the human. If the human has no specific intention then a proactive action by the robot may induce the intention (concerning the robot’s proactive action) in the human. 88 89 Chapter 6 Interaction in unknown scenarios A robot as a machine can not extend its interaction model to adapt to the changing human intention that is not already known to the robot. For a robot to be intuitive, it should possess the capability to interact with the human even if the intention of the human is not known. In this chapter an approach is introduced to HRI in a known scenario with unknown human intention. Initially, the robot reacts by copying the human action. Before each reaction, the robot hypothesizes its potential actions and selects one that is found most suitable. The robot may also use the HRI history to hypothesize the potential actions. Along with the history, the robot also considers the action randomness and action predictions to hypothesize the potential actions. As solution, a general Reinforcement Learning (RL) based algorithm is proposed that suggests learning of HRI in an unknown human intention scenario. A Particle Filter (PF) based algorithm is proposed to support the probabilistic action selection for HRI. The experiments for HRI are performed by a robotic arm involving the arrangement of known objects with unknown human intention. The task of the robot is to interact with the human according to the estimated human action. The remainder of this chapter is organized as follows: In Section 6.1 the problem of HRI with unknown human intention is defined and motivated. The approaches related to the discussed problem are described in Section 6.2. In Section 6.3 a general RL based HRI algorithm is proposed. In Section 6.4 the process of probabilistic action selection is explained in detail. Section 6.5 describes the experiments performed using the proposed approach. Finally, Section 6.6 summarizes the chapter. 6.1 Problem definition and Motivation The problem corresponds to HRI in an unknown human intention scenario. The human can has an intention i ∈ ,  = {i 1 ,…,i n }, n ∈ ℕ while working in the HRI workspace. The robot does not know . The robot also does not know how to react. The robot knows the HRI workspace and the actions  = {a 1 ,…,a m }, m ∈ ℕ that can be performed by the human in HRI workspace. The input to the problem corresponds to the scene information, the scene change information, and the understanding of human actions. The robot is required to select an action a k , k ∈ {1,…, m} for interaction with the human. It is assumed that during the HRI the human only performs the actions that are related to his current intention. The humans possess the capability of responding to the problem in an unknown or unseen situation. This is a significant difference between a machine and a man. A machine can only perform the task that it is made to perform. A machine can usually not perform in a new situation. On the contrary the human has built-in capability to interact with another human 96 with respect to newly generated hypotheses. The higher weight is due to the assumption that an action can be repeated while performing a task. 6.4.4 History-based actions prediction As descried earlier that a human intention consists of a sequence of actions. Each action can be characterized by a set of n features. It means that each action can be represented as a point in the n dimensional space. Thus each intention consisting of a sequence of action (represented as point in the n dimensional space) is represented as an ordered set of points. A complete action sequence concerning an intention represents an intention trajectory. Graphically an intention can be represented as a trajectory in the n dimensional space as shown in Figure 6.4. ( x 11 ,x 21 ,x 31 ,…,x n1 ) ( x 13 ,x 23 ,x 33 ,…,x n3 ) ( x 12 ,x 22 ,x 32 ,…,x n2 ) ( x 14 ,x 24 ,x 34 ,…,x n4 ) ( x 11 ,x 21 ,x 31 ,…,x n1 ) ( x 13 ,x 23 ,x 33 ,…,x n3 ) ( x 12 ,x 22 ,x 32 ,…,x n2 ) ( x 14 ,x 24 ,x 34 ,…,x n4 ) Figure 6.4: Action sequence trajectory concerning a human intention, each dot represents an action and a complete trajectory represents a task concerning a human intention Using the trajectories of the different intentions the similarities between different intentions can be found. The future action hypotheses can be evaluated with respect to the previous trajectories. It is explained with the help of following Figure 6.5. The green trajectories represent the already performed action sequences concerning the human intentions. The blue trajectory represents the current interaction action sequence. The predicted action hypotheses are placed as black dots with dotted lines. The hypothesis with significant historical support gets higher weight with respect to others. Figure 6.5: Hypothesis evaluation using previous intention trajectories 97 6.4.5 Combination of action aspects The final selection out of all the action hypotheses is made by considering the randomness, history support, and the weight of each hypothesis. The history support and randomness of each action hypothesis is weighted by the hypothesis weight. For each action hypothesis a value is calculated by adding the weighted history support value w it p(A it |H) and weighted action randomness w it p(A it |A t-1 ), i.e., w it p(A it |A t-1 ) + w it p(A it |H). The calculated actions are stored in descending order with respect to their action values. The top action in the action list is selected for the robotic reaction. The next lower value actions are selected if the human asks the robot to switch its reaction. The combination of the history support, randomness and the hypotheses weight is shown in Figure 6.6. Heuristic Hypotheses WeightHistory Randomness Action’s Tray Heuristic Hypotheses WeightHistory Randomness Action’s TrayAction’s Tray actions Heuristic Hypotheses WeightHistory Randomness Action’s Tray Heuristic Hypotheses WeightHistory Randomness Action’s TrayAction’s Tray actions Figure 6.6: Final action selection for HRI by the combination of history support and randomness of each action hypothesis with the weight of the hypothesis, resulting in a value, the action hypotheses are arranged in descending order with respect to the resulting value 6.5 Particle Filter based action selection The mechanism used in Particle Filter (PF) considers all the possible solutions as particles. PF is an iterative algorithm and operates in two phases, i.e., prediction and update. In the prediction phase each particle is modified according to the given prediction model. Each particle has weight that represents the significance of the particle. In the update phase the particle weights are updated based on the incoming sensory information. According to the weight the particles are re-sampled [45]. Using the particles the distribution of solution is estimated. PF is used to track and estimate the solution of a problem with respect to time. The current problem of the robotic reaction in the unknown human intention scenario also corresponds to the prediction and update of the current belief of the reacting agent about the unknown human intention. The robot uses its history knowledge as well as the immediate previous human action or the previous accepted action of the robot to predict the action for HRI. The update is performed on the basis of human response. If the human accepts the reaction then the accepted action is predicted with more likelihood (Figure 6.3 left). In case if the human responds by simply correcting the robot’s reaction. Then the corrected human action (new human action) is predicted with more likelihood (Figure 6.3 right). The difference between the application of PF algorithm for the current problem and problems where PF is usually applied is spatial. As PF is mostly applied in the robot localization and 98 usually the state space involves the two dimensional (2D) space in which the robot exist. For more accuracy the orientation of the robot is considered. In the current case the state space corresponds to the human actions. If the human actions are represented in a n-dimensional space. Then the future actions can not be predicted based on the location of the current action. There is no action model for human action prediction as the motion model for robots. Thus we have to assume all the possible actions as hypotheses and then the evaluation of the hypotheses can be performed on the basis of currently performed action and the history of performed action sequences concerning the intentions and the actions probabilities. Therefore the PF algorithm can not be applied directly to the current problem. for end dofor Output Input - 5 }0,1,A{-4 A(A from A Sample-3 ...1-2 -1 particlesaction of Set : actionshuman : tion()Initializa 1 ><∪= = = − nSS )|p ni S S k ttt tt k t t t φ Figure 6.7: Initialization of the action particles The algorithm is described in Figures 6.7, 6.8, 6.9, and 6.10. In the initialization phase, all the action particles are created with equal weights as shown in Figure 6.7. The elements of set S t (Line 4) correspond to a tuple of action particle, its weight and action value (discussed later in this section). In the probabilistic action selection described in Figure 6.8 the action values of all the existing action particles are calculated as shown in Line 3. The value of each action particle is calculated by multiplying the conditional probability of the action and historical support with the weight and adding them. In case if the conditional probability of a predicted action with respect to the previously performed action is not available then the prior of that action is used. At the Line 5 all the expected actions are sorted with respect to their values and stored. The highest value action is selected for reaction. The system loops from Line 6 to 20 until a suitable action is selected or all the actions are tried or the human performs a correction. If the robotic reaction is accepted, i.e., the robot performs a suitable action then the particles are generated with respect to the performed action with higher weight as compared to the previously existing particles (Lines 10-11), Figure 6.3 left (green lines). The particle corresponding to the accepted action gets higher weight than the newly generated particles. It is shown in Figure 6.3 left (blue). If the robotic reaction is not accepted then the human may ask the robot to change its reaction. The robot selects the next highest value action for reaction (Lines 12-13). The human may also correct the robotic reaction without asking the robot to change its reaction. If the human correction belongs to the set of the predicted action then the particles are created with respect to that action with higher weight as compared to the exiting action particles (Lines 15-16), Figure 6.3 left (green lines). The particle corresponding to the human correction gets higher weight than the newly generated particles, Figure 6.3 left (blue). 99 { } );A|| A ˆ || A! A ˆ (} -20 -19 ),ization(ARe_Initial -18 robot known tonot is correctionhuman // )(A-17 },A{-16 )A(A from A Sample-15 robot known toalready is correctionhuman // )(A-14 A).(.A ˆ -13 reaction thechange toasked isrobot // ))A ˆ ((-12 },A{-11 )A ˆ (A from A Sample-10 accepted isaction robot performed// ))A ˆ ((-9 reaction robotic // A ˆ -8 ueaction valhighest h action wit theg//selectinA).(. A ˆ -7 { -6 ueaction val respect to with actions thestoring//).(-5 -4 ncalculatio ueaction val//)H|A()A|A(-3 ,...,1int-2 00-1 )ilter(Particle_f Algorithm react robot to for theaction an ofSelection : feedbackhuman actions,human ,: ,,A,...,,,A,,,A _ _ _ high1 _11 _ high1 11 1 222111 correctionhumanoptimal tcorrectionhuman correctionhuman k ttt correctionhumant k t correctionhuman k ttt t k t t ttttti t t nnnt S wSS |p indexat Change wSS |p Accepted Execute indexat vSOrderDescending pwpwv ni index S S vwvwvwS φ η === Λ∉ ><∪= Λ∈ Λ= ><∪= Λ= =Λ ⋅+⋅= = == = + ++ ++ + ++ ++ − while if end thenif else thenif else thenif else thenif do for end dofor Output Input Figure 6.8: Probabilistic action selection for HRI In case if the human corrected action does not belong to the set of predicted actions (Line 17- 18, Figure 6.8) then re-initialization of the particles is performed, described in Figure 6.9. The human correction is represented by A t in Figure 6.9. The new action is added to the list of known actions and new action particles are created with respect to the newly added action. The new particles are created for the newly added action with respect to previously existing actions with high weight as compared to the previously exiting particles (Lines 2-5, Figure 6.9), Figure 6.3 (right) (red line along with black lines). The new action particles are also created using the previous actions with respect to the newly added action (Lines 6-7, Figure 6.9). The weight of these new action particles is higher than the previously created (Lines 2-5, Figure 6.9) new action particles. The newly created particles (Lines 6-7, Figure 6.9) correspond to the green lines in Figure 6.3 (right). The new action particle representing the repetition of newly added action Figure 6.3 (right) (red line among green lines) is given the highest weight with respect to the all newly created particles. 100 for end dofor for end dofor for end dofor Ouput Input -13 -12 // ...1-11 -10 -9 particlesaction ofnumber total theis // ...1-8 //}0,,A{-7 )A|(A fromA Sample-6 -5 //}0,,A{-4 A(A from A Sample-3 particlesaction existing previously the toscorrespond //...1-2 0-1 A therespect to with updated : n))(correctioaction human (new A,particles)action of(Set : ),ization(ARe_Initial 1221 11 _111 11 η ηη η ii i k ttt tt k t particlesprevious k ttt tt k t tt tt tt ww mnni w nmnni wwwSS p wwwSS )|p mmi S S S = >= += >= >><∪= >><∪= = = + ++ + −+ Figure 6.9: Creation and weighting of the new action particles Afterwards the particles weight is normalized (Lines 8-13, Figure 6.9). The high weighting of the latest actions biases the robotic reaction towards the currently performed action. The resampling of the particles is described in Figure 6.10. A threshold value τ is selected between 0 and 1 / (total number of the particles), including 1 / (total number of the particles). If the weight of a particle is less than τ then that particle is eliminated. The other particles are kept. Then the weights of the particles are normalized. for end dofor for end dofor for end if end thenif dofor Output Input -12 -11 ,...,1-10 -9 -8 ,...,1 -7 -6 -5 },,{S -4 ]n1]0, )( -3 ,...,1 -2 0 ,S -1 ,: : η ηη ττ ηφ ii i iii i ww mi w nmmi vwA w ni SSS S = = += ≤= ><∪ ∈> = == ⊆ Figure 6.10: Re-sampling of the action particles 101 6.6 Experiments The experiments are performed with a robotic arm with six degrees of freedom. The human and the robot interact in a HRI workspace shown in Figure 3.9. The HRI workspace consists of a table with the objects on the table. The video data is captured by an overhead FireWire digital camera with the standard frame size of 640 x 480 pixels at a frame rate of 30 frames / sec. HRI and image analysis are implemented using programming language C++. The robot reactions are realized using the robot programming language V++ for the robotic arm. The human actions are inferred from the scene changes occurred due to the human actions. The performed experiments involve actions that are characterized by two features, i.e., the distance between the objects and the orientation of the objects with respect to each other. The objects in the experiments involve the boxes on the table as shown in Figures 6.11 and 6.12. The performed experiments concern different arrangements of the objects according to the human intention. Each task representing a human intention is described by a trajectory (Section 6.4.4). In 2D case the trajectory is drawn in a plane having distance between the objects and orientation as axes. Each action is represented as a point in the plane. For a trajectory the angle concerning the slope of the line passing through the two immediate connected-points (Figure 6.4) is calculated. Thus for each trajectory, there exists a set of angles between the consecutive action points. For trajectory comparison the difference is calculated between the related sets of two concerning trajectories. The difference corresponds to different angle values in the two sets. The difference between the current (incomplete) HRI trajectory and the previous (complete task representing a human intention) trajectory is calculated. The trajectory, for which the difference is least, is used to evaluate the predicted action hypotheses. For spaces more than two dimensions, the direction vector between the two n-dimensional (n > 2) points of the intention trajectory can be calculated by subtracting second point from the first point, i.e., if ( n xxx 11211 ,...,, ) and ( n xxx 22221 ,...,, ) are two points then the direction vector will be calculated as [ ] nn xxxxxxv 1212221121 ,...,, − − − = . Now the comparison between the two direction vectors can simply be performed by calculating the angle between the two direction vector using the dot product of the vectors, i.e., 21 21 1 cos vv vv • = − θ First the anatomy and reasons for selecting a reaction in the experiments is explained. Afterwards the results of the performed experiments are discussed. The following experiments are considered for explaining the reaction selection mechanism. The unknown human intentions correspond to the arrangement of the objects. The considered unknown human intentions involve the arrangement of the objects in vertical and horizontal pattern, arrangements of the objects in a square pattern, and the arrangement of the objects in a diamond pattern. The arrangements of the objects according to the above described unknown human intentions are shown below in Figures 6.11 and 6.12. At the start the boxes are placed randomly on the table. The human picks and places the box at a point on the table as shown in Figure 6.13 by the box at (235, 150). Afterwards the human places another box vertically near the previously placed box as shown in Figure 6.13 by the 102 box at (235, 208). The system infers the human action as the distance and angle between the two vertically placed boxes, i.e., θ 1 and d 1 . A hypothesis is created based on the observed action, i.e., the place to put the next box, shown in Figure 6.13 by the green circle. The hypotheses weights (Figure 6.9), action probabilities (Section 6.4.1), and the history support (Section 6.4.4) are considered for the estimation of the action. Figure 6.11: Unknown human intentions for arranging the boxes Figure 6.12: Unknown human intentions for arranging the boxes For the very first reaction there exist no conditional probability value and historical support value. In such case the prior probability of that action is used. In case of absence of historical support of action sequence all the predicted actions for the current HRI are given uniform weight. Therefore the system has the only highest value available reaction, i.e., placement of the box in the vertical pattern at the next location at an angle θ 1 and distance d 1 . The weight of the hypothesis is represented by red cross at first interaction step in Figure 6.14. Now if the human intercepts and corrects the robot reaction then the system updates its possible actions by adding the corrected action if it is new and updates the conditional probability tables and the prior probability tables. The system also appends the corrected action in the current human robot interaction action sequence. The human accepts the robot reaction and the system updates its table without adding any new actions in its action table. Now the robot once again creates the hypothesis as shown in Figure 6.13 by the blue circle. Till now the robot has observed one action thus it only creates one hypothesis which is the next place in the vertical pattern. 103 Figure 6.13: Hypotheses graph for intention shown in Figure 6.11 left Figure 6.14: Weight graph for intention shown in Figure 6.11 left That robotic reaction is also accepted by the human as he has the similar intention of arranging the boxes in vertical pattern. After each complete interaction the system stores the action sequence separately. The sequence consists of the human action, human correction and accepted robot reactions. In the hypotheses weight graph if a hypothesis is accepted as a reaction then that hypothesis is removed by making its value zero as shown in Figure 6.14 at interaction Step 2, i.e., the red line goes to zero. In the next HRI experiment the human intends to arrange the boxes in a horizontal order. The boxes are once again placed randomly on the table. The human picks a box and places it at a point on the table as shown in Figure 6.15 by the box at (156, 193). Now the system creates a hypothesis based on the known action, i.e., θ 1 and d 1 represented as green circle in Figure 6.15 and the robot reacts by picking and placing another box at the angle θ 1 and distance d 1 . This time the human has the intention of placing the boxes horizontally. 104 Figure 6.15: Hypotheses graph for intention shown in Figure 6.11 right Figure 6.16: Weight graph for intention shown in Figure 6.11 right Therefore the human corrects the robotic reaction and picks and place the box (placed by the robot) at an angle θ 2 and distance d 2 as shown in Figure 6.15 by the box (238, 193). Now the system adds the new action to its action table and updates the conditional probability as well as prior probability tables. For the next reaction, the robot has three hypotheses based on the two actions (green circle and blue crosses Figure 6.15). The hypothesis instructing the robot to place the next box horizontally with respect to the previously placed box (238,193) gets highest weight according to the hypothesis weighting mechanism described earlier with respect to the other two hypotheses. The hypothesis (320, 193) has the highest weight at interaction Step 2, represented by blue star in Figure 6.16. The reaction value is calculated using the conditional probability or prior probability, historical support and the hypotheses weight. As there is no historical support for the currently predicted actions thus all the hypotheses based on the predicted actions get equal weight. There exists no conditional probability value for the currently predicted actions. Thus the prior probability is used instead of conditional probability. 105 As the prior probability of vertical box placing action is high therefore the hypotheses value for placing the box vertically gets higher values as compared to the horizontal placement of the box. Thus the robot reacts by placing the box vertically that is rejected by the human as the human intends to place the boxes horizontally. Now due to the rejection of the human the robot resorts to next highest value action that is once again placing the box vertically at another location which is once again rejected by the human. The robot resorts to next available action that is placing the box horizontally with respect to the lastly placed box at (320, 193). The reaction is accepted by the human. It is not selected for reaction for the first time due to the low prior of the concerning action (horizontal action). The robot creates the new hypotheses (represented as purple stars in Figure 6.15) for the next reaction for placing the fourth box. This time the robot reacts by placing the box horizontally with respect to the lastly placed box. This time placing the box horizontally has highest conditional probability of 1 and highest weight as compared to the priors for vertically located hypotheses with low weight. It is shown by the black box in Figure 6.16 at intersection Step 3. In the next HRI experiment the human intends to place the boxes in a squared pattern, Figure 6.12 left. Once again the human places the box and the robot generates the reaction hypotheses based on the previously observed actions, represented as green circles in Figure 6.17. The hypotheses get the same action value due to the same hypothesis weight, same history weight and same prior probability. The robot places the box (318, 156) in horizontal pattern and that is accepted by the human due to the similar intention. Next time the robot once again places the box on the horizontal pattern as the action has highest value due to the high hypothesis weight and history support for the action as the current pattern matches to more to the horizontal placement than vertical placement. Figure 6.17: Hypotheses graph for intention shown in Figure 6.12 left The reaction is rejected and the robot resorts to next highest value reaction, i.e., place the box orthogonal to previously placed box (236, 156) (according to the squared pattern intention, Figure 6.12 left) at (236, 207) which is accepted. Afterwards the robot reacts by placing the box on the fourth corner of the square (Figure 6.12 left) due to the high value for that action (blue triangle in Figure 6.17). That high action value is due to highest hypothesis weight and 112 few parameters or by the application of knowledge based problem solution. The retaining of a learned case is performed by storing successfully the adapted case to the target problem [21]. The CBR may use different feature representations, e.g., Rough Sets [138]. The Lazy Learning algorithms use different kinds of distance functions to calculate the similarity between cases [164]. Traditionally nearest neighbour approaches are used that compute similarity between the stored cases and the new cases, based on the features and the weights of cases [38]. The distance function in CBR is used for case comparison. The distance function does not consider the concept generalization and HRI. Lutz Formberger describes the learning of KT for generalizing the navigational capabilities of a mobile robot [57]. An agent-centered qualitative spatial representation is used for generalization and KT. The author claims that the learned strategies become robust using the described representation. The robustness corresponds to the ability of the robot to cope up with the environment noise and imprecise world knowledge. A simulated indoor robot is used for experimentation. The task of the robot is to learn a goal directed path finding strategy. The generalization is described by the abstract state space selection mechanism [57]. Association based rule learning in Data Mining [59], [136], [36] requires a data set of huge amount to generate rules with certain probabilistic measures. An association rule can be represented by the expression X ⇒ Y. The symbols X and Y represent sets of items. The rule describes that if the transaction of X occurs then the transaction of Y will also occur [66]. There exist several generalizations of the rule problem in Data Mining [66], e.g., [59] proposed generalized rule induction using probabilistic rules. The data mining approaches require huge amount of data and no HRI is involved. Learning from Demonstration (LfD) [19], [110] use the term generalization for the robot to learn comprehensively from many times performed demonstration in different conditions for a certain task. An approach proposed in [58] describes that correction based HRI gives better results in LfD. HRI is used to correct the task performed by the robot [4], for behaviour adaption [104], and to learn the environment dynamics [118]. The approach in [4] introduces a tactile policy correction algorithm. Initially, a policy is derived from LfD technique. Afterwards through the tactile interface, the human teacher indicates the relative refinement in the robot pose. The robot adjusts its poses according to the human tactile input. The corrected robot pose corresponds to the new training data for the policy generated by the tactile input. The tactile input from human helps the robot to refine its demonstrations. The experiments are performed with a humanoid. The HRI concerns the touching of human at the tactile interfaces of the robot to adjust the robot pose. The learning performed by robot during HRI corresponds to the robot’s pose. The focus of the HRI-based approach is correction and not generalization. Mitsunaga, Smith, and Kanda introduce the behaviour adaption based on policy gradient reinforcement learning [104]. They use HRI to adapt towards the human behaviour. The focus of the approach is the interaction-distance between the human and the robot. The robot monitors the repositioning count of the human in order to estimate the human discomfort during HRI. The robot also considers the gaze averting of the human. Both of the values are used as rewards that are minimized in the policy gradient reinforcement learning algorithm. The robot adapts towards the human behaviour during HRI. There exist generalization approaches for mobile robot, e.g., [20] describes navigational generalization based on evolutionary algorithms. The focus of the approach is general 113 behaviour of mobile robot for navigation, concerning obstacle avoidance. The approach [20] corresponds to a feed-forward neural network in combination with evolution strategies. The presented work focuses on Piagetian schemes [118]. The dynamics of environment are estimated. The model anticipates the step forward values of the chosen variables. The variables relates to the dynamics of environment. The anticipation is performed by the current state of sensors and taken actions. The model assumes that the world is deterministic. The HRI occurs in a simulated experiment. The human interacts by clicking on the cart target location on the interface. The interface displays the 2D physical system. The approach in [115] discussed Differential Equations based motor skill generalization. All the above described approaches do not consider generalization as concept generalization. The most related approach to concept generalization is described in [99]. It is also known as Version Space strategy. This approach can be successfully applied in classification. A version space can be represented by two sets. One set contains most specific consistent hypotheses and the other set contains the most general consistent hypotheses. The most specific consistent hypotheses correspond to the hypotheses that contain the smallest number of conditions that are necessary to select a positive training example. The most general consistent hypotheses contain the conditions such that no negative training example can be selected. The approach in [99] considers generalization as a search problem. The HRI based concept generalization can not be performed using [99]. Since [99] does not suggest what to do if a correction is performed by the human during HRI. Similarly, [99] also does not explicitly describe the rule conflict resolution. In the concept generalization, the generated generalized rules may face conflict with each other. There exist approaches for conflict resolution, e.g., [32] uses the classification frequencies of the rules (that cover the example to be classified) with respect to the classes to classify a conflicting example. For example, consider a class A and class B with two conflicting rules R 1 and R 2 . The samples X that belong to both the classes A and B are needed to be classified. For this purpose, the number of classified examples of a class with respect to all the conflicting rules are summed. This is performed for both the classes. The class that has higher number of examples is assigned to the samples X. The approach proposed in [37] uses the product of prior probability of the class with the product of conditional probabilities of the rule with respect to that class. The class with higher value is selected. The Rule Discovery System proposed in [37] applies the naive Bayes classifier to resolve the rule conflict. The posterior probability of a class with respect to all the conflicting rules is determined. Since the joint prior probability of the rules in conflict does not influence thus it is ignored. It is assumed that each conflicting rule is independent from each other. The posterior of each class is determined. The class with higher maximum value is assigned to the sample X that belongs to more than one class. In [130] each conflicting rule votes for its predicted class with a weight. The weights of all the classes are summed up and the class with the highest weight is selected for the samples X that belong to more than one class. The approach in [92] is the same as [37] if there is no training example in the intersection of the conflicting rules. If it exists then it uses the conditional probabilities of the intersecting rules with respect to the class. The double induction approach in [93] proposed the new induction of rules from the examples that are covered by the rules in conflict. The idea introduced in [93] suggested that there is higher chance to find rules that can classify the classes by concentrating on a small subspace of the example space in conflict. 114 The above described approaches discuss the resolution of conflict using probability and the frequency of the class and by inducing new rules. No approach tries to focus on the antecedents of the rule that influence classification. Along with concept based generalization, the conflict resolution of rules is also suggested based on the importance of individual antecedents of the rule. The presented approach [15] is the generalization concerning the reduction of a concept criterion by HRI. A conflict resolution is also proposed that is based on the importance of the antecedents of the generalized rule. The application of the generalized rules in the intuitive HRI improves the interaction capabilities of the robot. 7.3 Rule generalization We introduce an approach to human intention generalization based on the rule generalization. The rule generalization corresponds to the generalization of the transition conditions of FSMs discussed in detail in Chapters 3 and 4. The transition conditions correspond to the actions a ki for the FSMs, discussed in Section 3.2 of Chapter 3. Each transition condition corresponds to a rule that is generalized using HRI. After recognizing the human intention the robot also reacts according to the generalized rules. The rules are induced online during HRI. The rule generalization is performed by HRI, based on the idea of concept generalization. The robot generalizes the human intention by applying the known action on a group of related objects. The group of objects corresponds to those objects that are similar to the object in some respect on which the human has performed the operation. The process of intention generalization is performed according to the following steps A. Grouping of the objects B. Online rule induction C. Rule application D. Rule generalization E. Transition pool 7.3.1 Grouping of the objects The procedure describes the construction of the classes from the available objects such that the characteristics of all the objects are similar. For example, if we have a group of the following objects, i.e., jug, plate, bowl, book, notebook, shirt and trousers. Then the jug, plate and bowl will fall into one class, book and notebook will fall into second class and shirt and trousers will fall into third class. The reason is that the jug, plate and bowl have similar characteristics of being broken along with other characteristics. The book, note book, shirt and trousers do not have the characteristics of being broken. Similarly shirt and trousers can be dirty and book and note book can not be dirty. There may be more than one different characteristics present in the different classes. The procedure of classifying the objects based on their characteristics is described in Figure 7.2. Initially the set S of classes is empty. Each class C contains the elements having similar type of characteristics. 115 { } { } for end if end else thenif dofor Output Input − − − − ∪=− − >− ∪=− ∪=− ==− = 10 9 )(8 7 6 )),((5 1 4 3 2 1 ,...,1 : : 11 1 1 11111 j j j j i i OCLASSIFY OCC OOCOMPARE jOjectsall the ob CSS CsettoOobjectAddingOCC CS miCclassesallofSSet istics characterwith knowncts Oknown objen Figure 7.2: Objects classification based on their characteristics At start (Lines 2-3, Figure 7.2), an object is classified into a class C 1 . Other objects O are compared with the characteristics of the object in the existing class (Line 5, Figure 7.2). true return for end for end if end falsereturn thenif dofor dofor − − − − − − − − − 9 8 7 6 5 ),(4 3 2 ),(1 qp yq xp yx ChChnotSimilar OofChsticscharacteriall OofChsticscharacteriall OOCOMPARE Figure 7.3: Comparison of objects based on their characteristics If the object O j is similar to an object of class C i then it is added to the class C i (Line 6, Figure 7.2). The objects are compared with respect to their characteristics (feature) that are already known, shown in Figure 7.3. The types of characteristics of the objects O are compared with each other (Line 4, Figure 7.3), e.g., if an object O j has the characteristics of a class C i , i =1,…,m then for the other object O k it is checked if O k also has the same characteristic of class C i . If O k does not has the characteristics then it is classified into another class (Line 8, Figure 7.2). The object O k is checked against an object of all the exiting classes, (Lines 4-5, Figure 7.4). If it matches to an object of exiting class then it is added to that class (Line 6, Figure 7.4). The object O k is classified into a new class if it does not match with the objects of the existing classes (Line 11, Figure 7.4). The newly created class is added to the set S (Line 12, Figure 7.4). In case of the object classification, if the set S has only one element (Line 3, Figure 7.4) then a new class C N+1 for O k is created and added to the set S (Lines 14-16, Figure 7.4). 116 if end new else new for end if end exit thenif dofor thenif − ∪=− ∪=− =− − ∪=− ∪=− =− − − − ∪=− ∈− =− >− ==− − + ++ + + ++ + 17 16 O15 )(14 13 12 O11 )(10 9 8 7 O6 )),O((5 N,...,14 1)(N3 ,...,1||N2 )O(1 1 11 1 1 11 1 N NN N N NN N ii i i CSS CC ClassC CSS CC ClassC CC COCOMPARE i miCclassesofnumberS CLASSIFY Figure 7.4: Classification of objects into concerning classes 7.3.2 Online rule induction The procedure of online induction is given in Figure 7.5. The n objects present in the scene are known to robot. The objects may belong to m classes which are already known. The robot also understands the human actions and the changes in the scene occurred due to the human action. The characteristics of n objects present in the scene are known to the robot. If an action comprehensible by the robot is performed by the human on an object O j with characteristics Ch k then the robot induces a rule considering the characteristics Ch k of object O j as the antecedents and the performed action as the consequent of the rule (Lines 1-8, Figure 7.5). while end if end for end dofor thenif dotruewhile Output Input − − − =− − =− − − 8 7 6 .5 4 )(.3 ))((2 )(1 : : kj jk j j kj jk ChAntecedentRule OofChsticscharacteriall OActionConsequentRule OormedactionPerf ChsticscharacteriwithOObjectonactionandescribingRule actionhumanperformedA knownarescenetheinOobjectstheofChsticsCharacteri knownareactionhumantoduesceneinChanges Classesmtobelongingscenetheincts Oknown objen Figure 7.5: Online rule induction 117 7.3.3 Rule application For generalization based on HRI, the robot initiates by applying the induced rules on the suitable objects present in the scene. The suitable objects are determined using the procedure given in Figure 7.6. The input of the procedure includes the n objects along with their characteristics present in the scene. The number r of induced rules are also available from the previously performed step. The output of the procedure given in Figure 7.6 is a set S = {〈L, R, O〉 1 ,…, 〈L, R, O〉 h }. The set S consists of h hypotheses. Each hypothesis 〈L, R, O〉 i consists of a list L i , a rule R i , and object an O i . The List L i corresponds to a set of characteristics that are similar in the object O i present in the scene and the antecedents of rule R i , in the hypothesis i. The List L i is obtained by the intersection of the set of characteristics of the object O i and the set of antecedents of the rule R i (Figure 7.9). The hypotheses are constructed by comparing all the given rules and the known objects present in the scene. The object O o and the rule R r are matched and if found similar then a hypothesis is constructed (Lines 3-4, Figure 7.6). { } for end for end if end thenif dofor dofor Output Input − − − ∪=− − − − = = + 7 6 5 ,,4 )),,(( 3 2 1 ,...,1, ,,,...,,,: : 1 1 i rO o r i ii h ORLSS LROMATCH OObjectoall RRulerall hiihypothesisainRRuleand OObjectofChsticscharactericommonthecontainsL ORLORLShypothesesofSet RulesofrNumber Classesmtobelongingscenetheincts Oknown objen Figure 7.6: Selection of object for rule application The similarity is checked by comparing each antecedent A a of the Rule R with each characteristic Ch k . If a characteristic Ch k found similar with antecedent A a then it is added to the list L (Lines 2-8, Figure 7.7). Ω− − − − − ∪Ω=Ω− − − − return for end for end if end break then if dofor dofor 9 8 7 6 5 R4 ))L,R((3 L2 R1 )R L,( i ji equal ofsticscharacterijall ofsantecedentiall INTERSECT Figure 7.9: Intersection between the antecedents R and list L of characteristics 118 If the List L contains one or more than one element then True is returned (Lines 9-10, Figure 7.7) to construct a hypothesis (Line 4, Figure 7.6). If no similarity is found between the antecedents A a of rule R and the characteristics Ch k of object O then False is returned (Line 12, Figure 7.7) and no hypothesis is constructed. Falsereturn else Truereturn thenif for end for end if end thenif dofor dofor 12 11 10 )( 9 8 7 6 ),( 5 ),(4 3 2 ),,(1 − − − − − − − − − − − − LnotEmpty LChadd ChAmatch RRuleofAsAntecedentaall OObjectofChsticsCharacterikall LROMATCH k ka a k Figure 7.7: Matching the objects with respect to the rules 7.3.4 Rule generalization In the process of generalization the robot applies the rule R h to the object O h of the h hypothesis in set { } h ORLORLS ,,,...,,, 1 = . The robot expects the human’s feedback on the application of rule R h to the object O h . Generalization is performed based on the human action in response to robot’s rule application. The robot expects three kinds of responses from the human. The human can accept, reject or correct. The generalization algorithm is given in Figure 7.8. The input of the algorithm includes set S of hypotheses that is generated in the rule application step (Figure 7.6). The robot can recognize the n objects present in the scene and their related characteristics. The human feedback and the changes occurred in the scene due to the human actions are also known to the robot. The output of the algorithm is the possible generalization of the rules in the hypotheses set S . The algorithm in Figure 7.8 proceeds by applying the consequent part of rule R h on the object O h for each hypothesis 〈 L , R , O 〉 h in the set S (Lines 1-2, Figure 7.8). If the human accepts the robot’s action the rule R h is updated (Line 4, Figure 7.8) by replacing the antecedents of rule R h with the list L h (Lines 1-2, Figure 7.10). The list L h is produced by the intersection of set of characteristics of O h and the set of antecedents of R h (Figure 7.9). All the necessary antecedents concerning the consequent of R h exist in the list L h . Since the application of rule R h on the object O h is accepted thus the necessary antecedents are also present in the set of characteristics of object O h . If the robot’s action is rejected by the human (Line 6, Figure 7.8) then the difference between the rule R h and the L h is performed (Line 7, Figure 7.8). The rejection means that the necessary antecedent / antecedents for the application of the rule R h do not exist in set of characteristics of the object O h . Figure 7.10: Update of the antecedents of rule R by list L LAntecedentR RLUPDATERULE =− − .2 ),(_1 119 { } ( ) ( ) ( ) ( ) for end if end L L thenif if end L L thenif if end thenif dofor Output Input − − − =− +− − − =− − − − − − − = 14 13 ,,,12 ,11 )( 10 9 ,8 ,7 )( 6 5 ),(_4 )( 3 )),((2 1 : ,,,...,,, : 1 hCorrectionh hh h hh hh hh h OARDATERC_RULE_UP ORCERC_DIFFERE CorrectedRejectedReward RERULE_UPDAT LRER_DIFFEREC RejectedReward RLUPDATERULE AcceptedReward ORConsequentExectue Hypotheseshall rulesoftionGeneralizaPossible sectionnapplicatioruleingeneratedHypotheses ORLORLS scenetheinobjectstheofsticsCharacteri knownareactionhumantoduesceneinChanges scenetheinOobjectsknownn backHuman feed Figure 7.8: HRI based rule generalization The necessary antecedents exist in the rule R h . Thus a difference is performed between R h and L h in order to find out the necessary antecedents. The difference between R h and L h results in a list L that contains the elements that belong to R h but do not belong to L h (Lines 1-13, Figure 7.11). Ω− − − ∪Ω=Ω− − − − − =− − − =− − return for end if end then if for end if end break then if dofor dofor 13 12 11 R10 ADD)(9 8 7 6 False ADD5 ))L,R((4 L3 True ADD2 R1 )L R,(_ i ji equal ofsticscharacterijall ofsantecedentiall DIFFERENCER Figure 7.11: Relative complement of L with respect to R 120 The rule R h is updated (Line 8, Figure 7.8) by replacing the antecedents of rule R h with the list L (Lines 1-2, Figure 7.10). The list L is produced at Line 7, Figure 7.8. In case if the human not only rejects the robot action but also corrects the robot reaction (Line 10, Figure 7.8). Then once again the difference between the rule R h and the list O h is performed (Line 11, Figure 7.8). The rejection and correction corresponds to the fact that the antecedents in rule R h are not exactly related to the characteristics of the object O h . Therefore only those characteristics are considered as antecedents of the rule that exist in O h but do not exist in R h . The consequent of the rule is changed with the correction performed by the human. The difference between the R h and O h results in a list L containing the elements that belong to O h but do not belong to R h (Lines 1-13, Figure 7.12). Ω− − − ∪Ω=Ω− − − − − =− − − =− − return for end if end then if for end if end break then if dofor dofor 13 12 11 O10 ADD)(9 8 7 6 False ADD5 ))O,R((4 R3 True ADD2 O1 )O R,(_ i ij equal ofsantecedentjall ofsticscharacteriiall DIFFERENCERC Figure 7.12: Relative complement of R with respect to L The rule R h is updated (Line 12, Figure 7.8) by replacing the antecedents of rule R h with the list L (Lines 1-4, Figure 7.13). The list L is produced at Line 11, Figure 7.8. AConsequentIRRCAntecedentIRR AConcequentR LAntecedentR CARLUPDATERULERC ==− =− =− − ..,..4 .3 .2 ),,,(__1 Figure 7.13: Update of the antecedents and consequent of rule R and the Induced rule IR The consequent of rule R h is replaced by the human correction (Line 3, Figure 7.13). The Induced Rule (IR) for the newly constructed rule is also updated (Line 4, Figure 7.13). The IR corresponds to the rule induced from the human action or by the human correction. The rules that are generalized by the process of REJECT and REJECT plus CORRECT are tested before they are moved into the transition pool (Section 7.3.5). The intermediate generalized rules (IGRs) are the rules that are produced by the result of ACCEPT, REJECT or REJECT plus CORRECT, performed by the human during the process of generalization in Figure 7.8. Each IGR has its corresponding IR. 121 The IGRs generalized by REJECT may lead to false generalized rule. There can be two cases of false generalizations. In Case 1, if the IR is applied on an object of another class then the intermediate generalized rule (IGR) will be false generalization. For example, if IR and the characteristics of the object are as under IR : IF {Plate, Dirty, Intact} THEN W.B (Wash Basin) Object : {Shirt, Dirty, Good} Then the IGR due to REJECT (Lines 6-9, Figure 7.8) will be as under IGR : IF {Plate, Intact} THEN W.B The objects Plate and Shirt belong to two different classes. The IGR is a false generalization as the robot will put a Plate that is Intact into the W.B without taking into account if it is Dirty or not. In Case 2, if IR is applied on the object of the same class and if the IGR does not contain all the necessary antecedents then IGR will be a false generalization. For example, if IR and the characteristics of the object are as under IR : IF {A, B, C, D} THEN A Object : {A, B, D} Then the IGR due to REJECT (Lines 6-9, Figure 7.8) will be as under IGR : IF {C} THEN A If B and C are the necessary antecedents with respect to the action A then IGR is a false generalization. Similarly in case of REJECT plus CORRECT, there exist two cases. In Case 1, if IR is applied on an object of another class then the IGR will be a false generalization. For example, if IR and the characteristics of the applied object are as under IR : IF {Plate, Dirty, Intact} THEN W.B Object : {Shirt, Dirty, Good} Then the IGR due to REJECT plus CORRECT (Lines 10-13, Figure 7.8) will be as under IGR : IF {Shirt, Good} THEN W.M (Wash Machine) IR : IF {Shirt, Dirty, Good} THEN W.M The IGR is a false generalization as the robot will put a Shirt that is Good into the W.M without taking into account if it is Dirty or not. In Case 2, if the necessary antecedents are not considered then the IGR will be a false generalization. For example, if IR and the characteristics of the object are as under IR : IF {A, B, C, D} THEN A Object : {A, B, D, E} Then the IGR due to REJECT plus CORRECT (Lines 10-13, Figure 7.8) will be as under IGR : IF {E} THEN A IR : IF {A, B, D, E} THEN A If B and E are the necessary antecedents with respect to the action A then IGR is a false generalization. Therefore the IGRs are first tested with the procedure given in Figure 7.14 and then moved into the transition pool (Section 7.3.5). The input to the procedure given in Figure 7.14 is the set IGR. Each IGR i i=1,…, M has its corresponding IR. The output of the procedure in Figure 7.14 is the set of IGRs with corrected generalization problems. All the IGRs are tested for all the related objects (Line 1, 2 Figure 7.14) present in the scene. The 128 (i): Placement of speckled object in response (j): Placement of speckled object in response Figure 7.16: Intention generalization by HRI The graph in Figure 7.17 represents the generalization capability of the robot. The generalization axis represents the number of objects acted upon by the robot while reacting to the recognized human intention. The green bars represent the results with generalization and red bars represent without generalization. Figure 7.17: Graph for intention generalization by HRI The experiments were performed on the objects shown in Figures 7.16 (a-f). The experiments number 1 in Figure 7.17 corresponds to the placement of speckled objects into the speckled tray. The human picks and places a speckled square into the speckled tray. In a non generalized HRI, the robot picks and places another square that is speckled into the speckled tray and stops. In the generalized HRI, the robot picks and places all the speckled objects other than the objects that have hole in them. The objects that have hole in them are considered broken. The objects having speckle on them are considered dirty and without speckle are considered clean. The antecedent (characteristic) of broken has high importance factor as compared to the dirty. Therefore a dirty object that is also broken is not operated as the HRI corresponds to picking and placing the dirty objects into the speckled tray. The second experiment corresponds to the placement of intact objects, the third experiment corresponds to the placement of clean objects, and the fourth experiment corresponds to the placement of broken objects. 129 The graph in Figure 7.18 shows the rule conflict results. The RDS [37] , CN2 [32] and C5.0 [130] produce false results as they resolve using probability and do not consider the individual antecedents as importance factor does. The success axis in Figure 7.18 represents the binary scale, i.e., either all the expected objects are acted upon or a few are left. If all the expected objects according to the human intention are operated then the result is considered 1 and 0 otherwise. The first experiment shown in Figure 7.18 corresponds to the resolution of the conflict concerning the objects with the hole (broken) and the objects with the speckles (dirty) on them. In the generalized HRI, using the importance factor the robot takes into account the importance factor of individual characteristics of each object while applying the rule of placing the speckled object (dirty) into the container of speckled object (wash basin). The robot does not pick and place a dirty object that is broken. The importance factor of broken is greater than that of dirty. It means that a dirty object can be washed for reuse and is supposed to be placed in the wash basin. A dirty and broken object is not required to be placed in the wash basin because it is broken and thus useless. The RDS, CN2, and C5.0 use the probability without taking into account the significance of individual antecedents and thus produce false results. Similarly the second experiment corresponds to the conflict resolution between the dirty and different shaped objects, i.e., triangles, squares, and pentagons. The robot is supposed to pick and place all the dirty objects no matter of which shape into the wash basin. Using the importance factor, the robot picks and places all the dirty objects into the wash basin. Using the probabilistic conflict resolution approaches RDS, CN2, and C5.0 the robot does not pick and place all the dirty objects. Similarly the third experiment corresponds to the placement of no speckle object (clean) into the concerned container. Using the importance factor the robot picks and places all the clean objects and using the probabilistic approaches specific shaped objects are left. The probabilistic approaches RDS, CN2, and C5.0 do not perform well due to the fact that they decide probabilistically and do not consider individual antecedents of the rule with the concerning significance. Figure 7.18: Graph for conflict resolution in intention generalization by HRI 130 7.6 Summary In this chapter we have introduced a generalization approach for the human intention generalization. The focus of the approach is intuitive HRI by human intention generalization. The intention generalization corresponds to the understanding of the key concept of the human intention and to react accordingly to that concept. The approach describes the rule generalization by HRI. This rule is then embedded into the probabilistic FSM, discussed in detail in Chapter 3. That is used to recognize the general human intention and to react generally. The experiments performed with the robotic arm demonstrated the usefulness of generalization approach, i.e., the robot reacts generally according to the human intention. For example, if the human has intention of putting the speckled objects into the speckled container then the human picks and places a speckled square into the speckle container. The robot recognizes the generalized human intention of picking and placing the speckled objects into the speckled container. The robot places all the speckled objects into the speckle object container. The robot also takes into account the importance factor and does not places an object into speckle object container that is speckled and also broken. The approach enables the robot to react intuitively in a known scenario that was not explicitly instructed to the robot. The known scenario corresponds to the known objects present in the scenario and the known related actions. The generalization capability of robot increases the range of intuitive reactions. The false generalization problem is resolved by testing each IGR i . The solution provided for false generalization resolves the problem but it may not produce the optimal generalization. The optimality of the resolution of false generalization depends on the present objects. The objects are used to resolve the generalization problem. The optimality of generalization with respect to the objects is explained using two cases. In Case 1, the false generalization does not consider the necessary antecedents. For example, consider IGR i with concerned IR i and the available object O j that is used to resolve the false generalization. IR i : IF {A, B, C, D} THEN A IGR i : IF {C} THEN A O j : {A, B, G, F, C} After resolving the generalization problem (Line 6, Figure 7.14) IGR i becomes as under IGR i : IF {A, B, C} THEN A If B and C are the necessary antecedents then the generalization is solved. The antecedent A decreases the optimality of generalization as it is unnecessary antecedents for action A. Thus the optimality of the generalization depends on the object used for generalization. The IGR i may be optimized in the transition pool. If all the objects present in the scene are of diverse characteristics then the generalization can be optimal and vice versa. In Case 2 the false generalization (IGR i ), produced due to the different class object, is discussed. For example, consider the IGR i with concerned IR i and available object O j that is used to resolve the false generalization. IR i : IF {Shirt, Dirty, Good} THEN W.M IGR i : IF {Shirt, Good} THEN W.M O j : {Pants, Dirty, Good} The generalization problem IGR i is resolved by ACCEPT, given in Figure 7.14. IGR i : IF {Dirty, Good} THEN W.M The resolved IGR i is generalized differently if the O j is as under 131 O j : {Shirt, Dirty, Good} Then the IGR i resolved would be as under IGR i : IF {Shirt, Dirty, Good} THEN W.M These resolved IGRs are moved to the transition pool. There the IGRs are further generalized. Moreover, if the objects available for generalization are properly classified and one characteristic is significant for the concerned action then the algorithm in Figure 7.14 is not required. Furthermore, the robot needs to distinguish between the situations when he needs to react based on the generalization or based on specialization according to the human intention. 132 133 Chapter 8 Conclusions In this chapter, in Section 8.1, 8.2, 8.3, 8.4 and 8.5 the presented work is concluded. Section 8.6 provides an outlook with respect to the presented research work. In the presented research work, five contributions to the area of intuitive HRI are discussed. The discussed contributions mainly correspond to the intention of the cooperating human, i.e., how the robot can improve its intuitive interaction with the human based on different aspects of the human intention. The following intention aspects of the interacting human are considered to improve the intuitive HRI. A. Intuitive HRI by intention recognition B. Intention learning C. Proactive HRI D. Interaction in unknown scenarios E. Intention generalization 8.1 Intuitive HRI by intention recognition The recognition of the human intention plays a key role in human-human interaction. It is equally significant for HRI. An intention recognition approach based on probabilistic FSMs is proposed. A FSM represents a human intention. The FSM corresponds to a human action sequence and / or the concerning scene changes in the HRI workspace. Each FSM carries a probabilistic value that is called the weight of the FSM. The weight of the FSM describes how closely the FSM represents the human intention. The FSM with highest weight corresponds to the best estimated human intention and vice versa. The weights of the FSMs are updated at each new observation in HRI workspace. The suggested solution is applicable for both explicitly and implicitly communicated intentions. Explicit intention communication addresses to all the situations where the human commands the robot and implicit intention communication addresses to all the situations where the human does not engage the robot but robot actively starts the cooperation by recognizing the intention through scene information and human actions. 8.2 Intention learning It is quite difficult to anticipate all the real time situations a robot may encounter. Therefore the capability of extension is inevitable for a robot. Three different cases are discussed to learn the new human intentions. The cases discussed the mapping of human intention to the corresponding observation sequence. The discussed Case 1 corresponds to the mapping between the sequence of the known actions and the known human intention. The known 134 human intention corresponds to the scene information. The Case 2 corresponds to the situation in which the actions are known to the robot but the human intention is inferred using the learning parameters. In the Case 3 the human actions and the human intentions are not given. The robot infers the human actions from the scene changes and the human intention is also inferred from the scene information. The mechanism used for intention recognition consists of the probabilistic FSMs. For online intention recognition a FSM regarding to a specific intention is constructed online. The capability of learning the new intention can be made more intelligent. The intelligence corresponds to the fact that the robot can take a decision about the human intention. The decision corresponds to the fact that the human intention is already known to the robot or not. If not then the robot is supposed to learn the new human intention. The robot should also be intelligent enough to recognize the start and end of the sequence of action and / or scene changes concerning the new human intention. Moreover the robot must also be able to decide if the human actions correspond to an intention or they are just random actions. The random actions correspond to the actions that are performed unintentionally by the human. 8.3 Proactive HRI Proactivity is also an important aspect in intuitive HRI. For a robot to be proactive in HRI, he needs to quickly recognize the human intention. A probabilistic approach for the intuitive HRI in an ambiguous situation is presented. Two cases are discussed for quick robot response for intuitive HRI. In first case, trigger state selection algorithm is discussed that describes how the trigger states are selected in case of similar state sequence of different FSMs. In the second case the proactive nature of HRI is discussed at lower level, i.e., the robot is required to prematurely decide in an ambiguous situation that may lead to two or more different human intentions. The ambiguous human intentions are handled by the transition weights that correspond to the weights assigned to the transition conditions in the FSMs. The robot can extend its capability of proactiveness by taking into account the daily routine work and concerning intentions of the interacting human. The robot can consider which tasks are most probable, which tasks are least probable, etc. Similarly the robot can consider the habits of the interacting human and customize itself according to them for being proactive. The domain information about the HRI workplace can also improve the proactive behaviour of the robot. The domain information can help the robot in quick decision making. 8.4 Interaction in unknown scenarios In reality a human can encounter the situation while interacting with other human that he does not know the intention of the other human. The human can either intuitively interact depending on the previous experiences or he can simply ask about the unknown intention. The presence of this capability in robots is also important for HRI. A probabilistic approach for the robotic reaction in the known scenario with unknown human intention is presented. The approach corresponds to a RL-based interaction algorithm. In which the robot performs the most suitable action in order to cooperate with the human without knowing the human intention. If the action performed by the robot corresponds to the human intention then the robot action is accepted by the human. Otherwise the human rejects the robot action and expect from the robot to act differently. The human can either wait for the expected action from the robot or he can simply correct the robot according to his expected action. The most 135 suitable action selection is performed probabilistically. The robot considers the predicted action, weight of the predicted action, action probability, and the history support of the action. The value of all the action hypotheses is calculated using the considered aspects. The capability of the robot for interaction with the human without knowing his intention can be improved by providing the domain knowledge to the robot. The domain knowledge corresponds to the end results of the known tasks concerning the unknown human intention. Similarly the interaction can also be improved by the agreed upon procedures. The agreed upon procedures correspond to the gestures that are known the robot and the human. The human can guide and convey his message to the robot using the agreed upon procedures. 8.5 Intention generalization A generalization approach for the human intention is introduced. The intention generalization corresponds to the understanding of the key concept of the human intention and to react according to that concept. The approach describes the rule generalization by HRI. A rule is induced online and then that rule is generalized by removing the unnecessary antecedents according to the human intention during HRI. This rule is then embedded into the probabilistic FSM. This rule is used to recognize the general human intention and to react according to the general intention. The robot can extend the range of its intuitive interaction with the human by the intention generalization. For being intelligent partner of the human in HRI, the robot must distinguish the situations in which he should generalize and the situations in which he should specialize the human intention. The robot should also customize itself according to the interacting human with respect to its intention generalization capability. 8.6 Outlook The presented research work can be extended in multiple ways as described in Section 8.2, 8.3, 8.4 and 8.5. The extension mainly corresponds to the human intention with applications in the HRI workspace. The robot possessing the capabilities of intention recognition, proactivity, and intention generalization may interact more safely with the human in a HRI workspace. The safety in HRI can be improved based on the intuitive HRI. The robot can anticipate the current and future human intentions. The robot can predict the future locations of the human. The robot divides the HRI workspace into cells. The current human location along with the future possible locations can be considered as the occupied cells. The robot planes its motion trajectory by taking into account the occupied cells to avoid human robot collision. The robot can differentiate between the virtually occupied locations with respect to the probability of being occupied. The robot can also consider the cells with low occupancy probability for its path planning to be efficient in its motion. The robot can also ignore the cells with more than zero occupancy probability in order to improve the safety in HRI. 136 137 Bibliography [1] Asimo.http://asimo.honda.com/ASIMO_DCTM/News/images/highres/01ASIMOCarr yTray.jpg, 11-09-2012. [2] Asimo. http://world.honda.com/news/2007/c071211Enabling-Multiple-ASIMO-to- Work/photo/pages/01.html, 11-09-2012. [3] Armano. G, Cherchi. G, and Vargiu. E. An Agent Architecture for Planning in a Dynamic Environment. In Proceedings of 7 th Congress of the Italian Association for Artificial Intelligence on Advances in Artificial Intelligence (AI*IA 2001), 2001. [4] Argall. B, Sauser. E, and Billard. A. Tactile guidance for policy refinement and reuse. in Proc. 9th IEEE Int. Conf. on Development and Learning (ICDL), 2010, pp. 7–12. [5] Argall. B and Schall. S. Robot learning from demonstration. In Proceedings of 14th International Conference on Machine Learning (ICML), 1997. [6] Argall. B and Billard. A. A survey of tactile human–robot interactions. Robotics Autonomous Syst. 58, 1159–1176, 2010. [7] Armon-Jones. C. The social functions of emotions. in: R. Harré (Ed.), The Social Construction of Emotions, Basil Blackwell, Oxford, 1985. [8] Arkin. R.C. Behavior-Based Robotics. Cambridge, MA, USA: The MIT Press, 1998. [9] Atkeson. C and Schall. S. 1997. Robot learning from demonstration. In Proceedings of 14th International Conference on Machine Learning (ICML), 1997. [10] Adams. B, Breazeal. C, Brooks. R.A, and Scassellati. B. Humanoid robots: a new kind of tool. IEEE Intelligent Systems 15 (4) 25–31, 2000. [11] Akella. et al. Cobots for the automobile assembly line. IEEE International Conference on Robotics and Automation, 1999. [12] Awais. M and Henrich. D. Human-Robot Collaboration by Intention recognition using Probabilistic State Machines. In Proceedings of 19th IEEE International Workshop on Robotics in Alpe-Adria_Danube Region (RAAD), 2010. [13] Awais. M and Henrich. D. Proactive premature-intention estimation for intuitive human robot collaboration. In the Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS’2012. [14] Awais. M and Henrich. D. Online Intention learning for human-robot interaction by scene observation. ARSO 2012, IEEE International Workshop on Advanced Robotics and its Social Impacts, Munich, Germany, 2012. [15] Awais. M and Henrich. D. Rule based Intention Generalization through Human-Robot Interaction. Robotik 2012, 7 th German Conference on Robotics, May 21-22, Munich, Germany, 2012. [16] Bartneck. C and Okada. M. Robotic user interfaces, in: Proceedings of the Human and Computer Conference, 2001. [17] Billard. A and Hayes. A. DARMA. A connectionist architecture for control and learning. Autonomous robots, Adaptive. Behaviour. 7 (1), 35-64, 1999. [18] Billard. A. Robota: clever toy and educational tool. Robotics and Autonomous Systems 42, 259–269, 2003. 144 [120] Rani. P and Sarkar. N. Anxiety detecting robotic system Towards implicit human-robot collaboration. Robotica 22(1): 85-95, 2004. [121] Reeves. B and Nass. C. The Media Equation. CSLI Publications, Stanford, 1996. [122] Rogers. E, Murphy. R, and Burke. J. Nsf/darpa study on human-robot interaction. IEEE Transactions on Systems, Man and Cybernetics, special issue on Human-Robot Interaction. [123] Rescue robots. http://www.popsci.com/technology/article/2011-03/six-robots-could- shape-future-earthquake-search-and-rescue, 11-09-2012. [124] Rescue robots. http://robotsfuture.blogspot.com/2011/04/rescue-robots-are-here.html, 11-09-2012. [125] Richard et al. Understanding Human intentions via Hidden Markov Model in Autonomous Mobile Robots, in the Proceedings of ACM/IEEE international conference on Human-robot interactions, pages 367-374, Amsterdam, Netherlands, March 2008. [126] Rosenstein. M and Barto. A. Supervised actor-critic reinforcement learning. Learning and Approximate Dynamic Programming: Scaling Up to the Real World, John Wiley & Sons, (Chapter 14), 2004. [127] Roesel. W. http://www.iwb.tum.de/en/Research/Automation_and_Robotics-p- 955304/Robotics/JAHIR___Joint_Action_for_Humans_and_Industrial_Robots.html, 11-09-2012. [128] Roomba. http://www.irobot.com, 11-09-2012. [129] Robotic arms. http://www.lspautomotive.com/images/DSCN0421.JPG, 11-09-2012. [130] RuleQuest. Research See 5: an informal tutorial, 2003. [131] Schmid. A, Weede. O, and Worn. H. Proactive Robot Task Selection Given a Human Intention Estimate. Robot and Human interactive Communication, RO-MAN. 16 th IEEE International Symposium 726-731. [132] Sheridan. T. B. Telerobotics. Automatica, 25(4), 487–507, 1989. [133] Shibata. T, Yoshida. M, and Yamato. J. Artificial emotional creature for human machine interaction. In Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, SMC'97, 1997. [134] Shibata. T and Tanie. K. Physical and affective interaction between human and mental commit robot. In Proceedings of the IEEE International Conference on Robotics and Automation, ICRA'01, 2001. [135] Spiliotopoulos. D. et al. Human–robot interaction based on spoken natural language dialogue. In Proceedings of the European Workshop on Service and Humanoid Robots, 2001. [136] Spruit. M. Discovery of association rules between syntactic variables, In Proceedings of the 17th Meeting of Computational Linguistics in the Netherlands, 2007. [137] Smart. W and Kaelbling. L. Effective reinforcement learning for mobile robots. In IEEE Proc. International Conference on Robotics and Automation (ICRA), 3404- 3410, vol.4, 2002. [138] Salamo. M and Golobardes. E. Rough Sets Reduction Techniques for Case-Based Reasoning. In Proceedings of 4th International Conference on Case Based Reasoning, 2001. 145 [139] Schrempf. O and Hanebeck. U. A Generic Model for Estimating User-Intentions in Human-Robot Cooperation. In Proceedings of the 2nd International Conference on Informatics in Control, Automation and Robotics, ICINCO 05, 2005. [140] Stiehl. W.D, Lieberman. J, Breazeal. C, Basel. L, Lalla. L, and Wolf. M. The design of the Huggable: a therapeutic robotic companion for relational, affective touch. In: Proceedings of the AAAI Fall Symposium on Caring machines: AI in Eldercare, 2005. [141] Sutton. R and Barto. A. Reinforcement learning: An introduction, The MIT Press, Cambridge, MA, London, England, 1998. [142] Suzuki. K. et al. Intelligent agent system for human–robot interaction through artificial emotion. In Proceedings of the IEEE SMC, 1998. [143] Scholtz. J. Human Robot Interactions: Creating Synergistic Cyber Forces. In A. Schultz and L. Parker, eds., Multi-robot Systems: From Swarms to Intelligent Automata, Kluwer, 2002. [144] Sinder. L and Lee. C. Robots as laboratory hosts. Interactions 12(2):24-26, 2005. [145] Schmid. A, Hoffmann. M, and Worn. H. A tactile language for intuitive human_robot communication. In Proceedings of the Ninth International Conference on Multimodal Interfaces, ICMI'07, 2007. [146] Scholtz. J and B. Antonishek. A Comparison of Situation Awareness Techniques for Human-Robot Interaction in Urban Search and Rescue. CHI 2005 alt.chi, April 2-7, Portland, Oregon, USA, 2005. [147] Sheridan T. B and Verplank W. L. Human and Computer Control for Undersea Teleoperators. MIT Man-Machine Systems Laboratory, 1978. [148] Safe Human-robot interaction http://www.ai3.unibayreuth.de/press/pictures/simero/Si232.Planung_Abstandsbasiert_StatischesNetzwer k_schematisch_vomPoster.png,11-09-2012. Si232.Planung_Abstandsbasiert_StatischesNetzwerk_schematisch_vomPoster.png [149] Tahboub. K. Intelligent Human-Machine Interaction Based on Dynamic Bayesian Networks Probabilistic Intention Recognition, Journal of Intelligent Robotics Systems, vol. 45, no. 1, pp. 31–52, 2006. [150] Tahboub. K. Intention recognition of a human commanding a mobile robot. Proceedings of the IEEE, Conference on Cybernetics and Intelligent Systems, Singapore, pages 896:1-6, 1-3 December, 2004. [151] Takeda. T, Hirata. Y, Kosuge. K. Dance partner robot cooperative motion generation with adjustable length of dance step stride based on physical interaction. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 2007. [152] Tenbrink, T and Fischer. K. Spatial Strategies in Human-Robot Communication. Korrekturabzug Kuenstliche Intelligenz, Heft 4/02, pp 19-23, ISSN 0933-1875, arendtap Verla, Bemen, 2002. [153] Thurau. C and Hlavácˇ, V. Pose primitive based human action recognition in videos or still images. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR’08), Anchorage, AK, pp. 1–6, June 2008. [154] Thrun. S. et al. Probabilistic Algorithms and the Interactive Museum Tour-Guide Robot Minerva, International Journal of Robotics Research, Vol. 19, No. 11, pp. 972- 999, 2000. [155] Thrun. S. Towards a Framework for Human Robot Interaction, Human Computer Interaction, Vol. 19, pp. 9-24, 2004. 146 [156] Tsoukalas L. H and Bargiotas. D. T. Modeling instructible robots for waste disposal applications. Proceedings of the IEEE International Joint Symposia on Intelligence and Systems, Nov 4‐5, Rockville, MD, USA, Los Alamitos, CA, USA, 1996. [157] Tversky. B and Lee. P. Why do Speakers Mix Perspectives? Spatial Cognition Computing 1: 399- 412. [158] Universität of Bayreuth, Lehrstuhl für Angewandte Informatik III, Robotik und Eingebettete Systeme, robot Labor. [159] Universität of Bayreuth, Lehrstuhl für Angewandte Informatik III, Robotik und Eingebettete Systeme, Presse, http://www.ai3.uni-bayreuth.de/press/pictures/, 11-09- 2012. [160] Voyles. R and Kholsa. P. Tactile gestures for human/robot interaction. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS'95, 1995. [161] Vezhnevets. V, Sazonov. V and Andreeva. A. A survey on pixel-based skin color detection techniques. Proceedings of the International Conference on Computer Graphics, pp. 85–92, 2003. [162] Wada. K and Shibata. T. Living With Seal Robots Sociopsychological and Physiological Influences on the Elderly at a Care House, IEEE Transactions on Robotics, Vol. 23, No. 5, October 2007. [163] Wang. L and Sutter. D. Learning and matching of dynamic shape manifolds for human action recognition. IEEE Transactions On Image Processing (TIP) 16 (6) 1646–1661, 2007. [164] Wettschereck. D and Aha. D. A Review and Empirical Evaluation of Feature Weighting Methods for a Class of Lazy Learning Algorithms. Artificial Intelligence Review 11:273-314, Kluwer Academic Publishers, 1997. [165] Wosch. T and Feiten. W. Reactive motion control for human_robot tactile interaction. Proc of the IEEE International Conference on Robotics and Automation, 2002. [166] Yang. J and Xu. Y. Hidden Markov model approach to skill learning and its application in telerobotics. In IEEE Proc International Conference on Robotics and Automation (ICRA), 396-402, 1993. [167] Yanco. H. A and Drury. J. L. Beyond usability evaluation: Analysis of human-robot interaction at a major robotics competition. Human-Computer Interaction Human- Robot Interaction 19(1-2): 117-149. [168] Yohanan. S and MacLean. K. A tool to study affective touch: Goals & design of the haptic creature. Proceedings of ACM SIGCHI Conference, 2009. [169] Youn. S and Oh. K. Intention Recognition using a Graph Representation. World Academy of Science, Engineering and Technology, 25, 2007. [170] Zlatev. J. The Epigenesis of Meaning in Human Beings and Possibly in Robots, Lund University Cognitive Studies, vol. 79, Lund University, 1999. [171] Zhang. DS and Lu. G. Shape-based image retrieval using generic Fourier descriptor. Signal Process Image Community 17(10):825–848, 2002.