Toward a Systematic Development of Affective Intelligent Agents

DEPARTAMENTO DE SISTEMAS INFORM´ ATICOS Y COMPUTACI´ ON UNIVERSIDAD POLIT` ECNICA DE VAL` ENCIA P.O. Box: 22012 E-46071 Valencia (SPAIN) Technical Report Ref. No.: Pages: 18 Title: Toward a Systematic Development of Affective Intelligent Agents Author(s): Bexy Alfonso and Emilio Vivancos and Vicente Botti Date: April 11, 2016 Keywords: Emotions, Affect, Affective architecture, Affective representation, Agent language VoBo Vicente J. Botti Bexy Alfonso The representation of the knowledge that is used for the specification of affective processes in agents, is almost as diverse as number of approaches that have addressed this issue. This diversity is due, to a large extent, to the need of systematic guidelines and standards that support computer scientists on the creation of affective models and architectures. Our aim is to perform a further step towards the standardization of this process, in order to improve the creation and enhancement of affective agent languages, architectures, and models. We offer a method to build affective BDI (Beliefs, Desires, and Intentions) agents, adapted to the problem to solve, and specifically, adapted to the way affect influences the agent behavior. To this end we offer GenIA3, a General-purpose Intelligent Affective Agent Architecture, which can be committed with specific psychological theories to create the architecture of the final agent. We also offer general guidelines that allow to define the processes performed in the agent architecture. These guidelines allow to select and adapt a BDI agent platform in order to include the processes of the proposed agent architecture and adapt a BDI agent language to include the representation of the required affect-related attributes. 2 1 Introduction Emotions and affective characteristics influence human day-to-day decisions and behavior. Computer science and researchers are aware of this and, in the last years, the simulation of human behavior has received a special and increasing interest. New emotion models of intelligent agents are designed and deployed by the affective computing community, which has been mainly motivated by the ambitious purpose of creating intelligent agents that resemble, to the greatest extent, human behavior. In parallel the need of standards and systematic guidelines for that systems has also grown [3]. Nevertheless, only few standards and systematic guidelines have been proposed, and they often don’t go beyond a theoretical level that include high level abstract models, generics concepts or tasks, and very few proposals offer a standard agent affective architecture. Agent language developers and computer scientists often face the challenge of starting from scratch when including affect in the representation of an agent and its behavior. Scientists use their creativity to define the way emotions are appraised, experienced, and their effects on behavior. In fact “the direct implementation of an emotion theory as a computer program is usually not the best way to go” and “this is a cumbersome way to proceed” [28]. In this work we make a further step toward the above mentioned standardization process. We start from inspiring psychological theories, which have helped to identify the main affect-related processes in an agent, and the representation of the related knowledge in an agent language, in order to make possible to carry out those processes. Our goal is to offer a method to build affective BDI agents, since the BDI agent’s architecture is a suitable and widely exploited alternative to model intelligent agents. This method relies on general guidelines that cover the main stages of the development of an affective agent. These guidelines start from the problem to solve (where it is emphasized the way affect influences the agent behavior). We present GenIA3(a General-purpose Intelligent Affective Agent Architecture), which supports these guidelines, and contains the required modules to model affect and related processes in a BDI agent. We also propose a representation of the main affect-related concepts, which results in constructions that are independent of the agent programming language that is used. The identification of the main affect-related processes, and related representation to perform these processes will support the development of affective agent languages regardless of the specific ways those affective processes are performed and what theoretical models support them, i.e. what appraisal theories, emotion dynamic theories, or emotion-effect procedures are used. Same way agent languages and platforms could be compared and improved with the support of a common language and basic-theory elements. This report is organized as follows. Section 2 presents some related works. The main processes of a BDI agent architecture and the main affective processes are presented in Section 3. Section 3 also presents GenIA3(which includes those processes). Section 4 shows the representation of the knowledge useful to carry out those affective processes. Section 5 offers general guidelines to develop affective BDI agents, and Section 6 describes how the steps of the guidelines were followed to integrate the processes and representation proposed in the agent language Jason. Section 7 offers the final conclusions. 3 2 Background and Supporting Theories When scientists model computationally affect, they face two broad challenges: how to model affect and how to enrich artificial agents architectures and languages to include those affective models [28]. Several psychological theories provide almost complete support for affect-related processes (e.g., emotion generation, and emotions effects on cognition, expression, and behavior) [14]. Consequently, approaches for agent modeling are different considering what psychological theories support them [28]. For a review about computational approaches for modeling emotions see [20], and [28]. Nevertheless, the task of systematically recreating existing emotion theories, and to use the general strategy of building formal languages for this, may be cumbersome. Two more viable strategies can be used: “1) break up existing emotion theories into their component assumptions and 2) reformulate these assumptions in a common conceptual framework”[28]. In line with these strategies Hudlicka proposes some guidelines for designing computational models of emotion [13]. In this work the author deconstructs emotion modeling into two processes: emotion generation and emotion effects, and describes the computational tasks that are necessary for theses processes. In this article we also propose general guidelines, but our approach differs from [13] in that the processes we propose are built on top of a BDI architecture, with the aim of using BDI agent languages and platforms. Our approach also offers a representation of the knowledge required by these processes in a BDI agent language. 3 Processes of an Affective Agent Life Cycle In this section we identify the main cognitive processes that are part of GenIA3. These processes are in line with theories of motivation and action generation. According to [28] “nearly all current theories of motivation and action generation are variants of a singly basic theory, the belief-desire theory of action”. These psychological theories of motivation have inspired computational emotion models, which have been used in cognitive agent architectures. In particular the belief-desire-intention (BDI) architecture [5] is a practical and powerful conceptualization, widely accepted in agent’s community which has been the base of numerous computational approaches. The affective processes addressed in this work are in line with cognitivemotivational (or belief-desire) emotion theories, whose constituents or mental causes are both: beliefs and desires [28]. Thus, the motivational function of emotion is to guide goal’s priorities and/or to generate new goals. We start from the BDI model to identify the core cognitive processes of the life cycle on an affective agent, due to the dependence of emotions on beliefs and desires, and the constructs for building autonomous and goal-directed agents that BDI logics provide. 3.1 Processes in a BDI Agent The BDI architecture has its foundations on the philosophical theory of “practical reasoning”, where the actions to be performed are decided according to certain goals. According to the BDI theory, rational agents are committed to their intentions, intending to do always what they believe will lead to their desires [4]. Practical reasoning requires both: establishing what goals to achieve 4 (deliberate) and how to achieve them (also called means-end reasoning), what can be summarized in four main processes performed consecutively [36]: Belief revision function (brf). Determines new beliefs starting from a perceptual input and the agent’s current beliefs. Option generation function (options). Takes the agent’s current beliefs and intentions to determine its desires (options or courses of actions available), i.e. the means to achieve its intentions. Filter function (filter). Determines the agent’s intentions, i.e. what to do, through a deliberation process that uses previously-held intentions, and the agent’s current beliefs and desires. The new set of intentions will contain either newly adopted or previously-held intentions. Action selection function (execute). Returns the next action to be executed on the basis of current intentions. 3.2 Agent’s Affective Processes According to [14] the agent’s core affective processes can be modeled across four emotions’ modalities of biological agents. These emotion modalities are the behavioral/expressive (which results in action-oriented characteristics and behavior); the somatic/physiological (which has a neurophysiological orientation); the cognitive/interpretive emotions (which are manifested as the result of the agent’s evaluation according to its goals, preferences, and the current situation); and the experiential/subjective (which results in an idiosyncratic and conscious experience of emotions). We focus on the cognitive emotion modality, and hence, on the corresponding processes for emotion generation and effects. We also focus on the behavioral/expressive emotion modality, specifically on the processes for emotion effects related to action selection. There is a wide consensus in computer science, with a solid theoretical background, of the main affective processes that should be considered when building appraisal-based models for affective agents [1, 14, 20]. These processes include both: affect generation and affect effects. The set of processes we propose is inspired on the Open Affective Agent Architecture (O3A) [1]. These processes are: Appraisal. Is the process whereby a set of appraisal variables are derived as the result of a transformation of the agent’s current situation, concerns, and cognitive state. This process can be triggered for example if a change is produced on the environment, although this is not the only cause that triggers appraisal. It can also be part of a continuous appraisal-reappraisal process or it can be triggered by other internal cognitive events. Conceiving this process in such a way allows to implement different appraisal theories like Smith and Lazarus’ [32], Scherer’s [31], Roseman’s [29], or Ortony’s [26] appraisal theories. Affect generator. In this process the appraisal variables that result from the appraisal process are transformed into a representation of the agent’s affective state. In this work we consider that this affective state can be represented by one ore more emotion categories and their intensities, similar to the emotion categories of the OCC model [26] (e.g. joy, hope, anger). The affective state can also have a dimensional representation, where values in a multidimensional space determine a point for the affective state (e.g., the Pleasure-Arousal-Dominance model of Mehrabian and Russell [22]). 5 appraisal affect generator coping affective modulator of beliefs execute filter options brf Beliefs affect's temporal dynamic actions external events Concerns Internal events Personality Current options External events Affectively relevant events Affective state BDI Affective Intentions Figure 1: GenIA3: a General-purpose Intelligent Affective Agent Architecture that integrates BDI and affective processes. Sequence (solid line arrows), subprocess (dashed line arrows), and exchange of information (dotted line arrows). Affect regulator. Through this process, the possible emotional behaviors and coping responses are determined. Emotional behaviors may include body gestures or facial expressions for example. The coping responses may change the environment or the individual’s cognitive representation like plans, beliefs, or intentions [19]. Affective modulator of beliefs. Determines if and how the affective state biases the agent’s beliefs. This process contributes to the beliefs maintenance according to the affective state. It is known for example that a negative affective state makes us questioning our beliefs and makes us to be more prone to accept new information; also a positive affective state makes us to rely more on our current beliefs [27]. The influence of affective states on the maintenance of beliefs, has been computationally modeled in several ways [18, 27, 15]. Even though it is not represented in other works (e.g., the idealized computational appraisal architecture of Marsella et. all. [20]), we consider relevant to include in our approach a representation for modeling the impact of the affective state on the agent’s beliefs. Emotions influence the “content and the strength of an individual’s beliefs, and their resistance to modification” and this influence “has traditionally been considered to be one of the most important things to be said about emotions” [9]. Affect’s temporal dynamic. This process doesn’t depend of any other process and no other process depends on it. It determines the duration of the affective state’s components as well as how their intensities decay over time. We include this process as an independent process for modeling the affect’s temporal dynamic since theories of mood and affect emphasize this issue [11, 35, 34, 17]. We have found the implementation of the processes of sections 3.1 and 3.2 essential according to our view of what a rational and affective agent should be. But we are aware that such general specification admits many possible ways of modeling an agent. 3.3 Integration of affective and rational BDI processes In order to establish the relation and sequence of rational BDI and affective processes we propose GenIA3(General-purpose Intelligent Affective Agent Ar6 chitecture), which is shown in Figure 1. The sequence of the affective processes is independent from the sequence of the BDI processes. This means that, an agent that is built on top of this architecture may have, at least, one execution thread for each set of processes. Moreover there is an affective process which is “disconnected” from the others: the affect’s temporal dynamic, because its function of controlling variations on the affective state intensity doesn’t depend on other processes. The sequence of the affective processes takes place as follows. When internal or external events are triggered, they are evaluated in the appraisal process. Other parameters are also used in the appraisal process that are related to the cognitive information of the agent (e.g., its concerns, beliefs, or the agent’s personality). The resulting appraisal variables are used by the affect generator, which updates the current affective state of the agent. By varying the agent’s affective state, the coping process checks if some action is necessary in order to either taking back the affective state to a desired state, or to perform some reactive action. Optionally, the coping process may require updating the agent’s beliefs if this is one of the agent’s “coping strategies”[26]. This update is performed by the affective modulator of beliefs process. The appraisal process can be triggered again after the coping process, even when there isn’t an event to be processed. For example, if appraisal complies with Sherer’s theory (which is one of the more complex theories of appraisal), the appraisal is conceived as a multilevel sequential checking where a set of evaluation checks are performed in sequence [31]. The Gratch and Marsella’s EMA model [19] also fits in this structure, because, in this model, affect is derived from a continuous cycle of appraisal, coping and re-appraisal. In general most appraisal theories can be represented in this way. On the other hand, the BDI processes in the GenIA3architecture maintain their original functions, however some of them include new ones. After the agent perceives its environment it updates its beliefs in the brf process, which includes the affective modulator of beliefs process in order to evaluate how the current affective state influences the content and the strength of the beliefs. Then, the available options are selected in order to determine the new course of actions in the options process. The filter process determines what to do by selecting the next intention, including the agent’s affective state as a parameter. Finally the selected intention is executed in the execute process. In the GenIA3architecture the interaction between affective and BDI processes is produced by using and updating the information related to the agent’s cognition. These concepts become agent attributes including: the agent’s beliefs (i.e., the information the agent has about the environment, about himself, or about others); the agent’s concerns (the agent’s personal values, such as its interests, motivations, standards, norms, or ideals); internal events (events that take place during the agent execution and that may generate changes on the agent state, for example, in the agent’s affective state); personality (a representation of the agent’s personality); current options (the options of the agent for facing the current situation, which is in line with the “desires” concept of the BDI model); external events (events perceived from the environment); affectively relevant events (those events that have produced important changes on the affective state of the agent); affective state (a representation of the affective state of the agent). Some of these attributes may not vary along the whole life time of the agent, 7 such as the agent’s personality and its concerns. Other attributes, in turn, have a bigger frequency of variation, for example, the affectively relevant events. The rest of the agent attributes can vary in every reasoning cycle, such as the agent’s beliefs,internal events,current options,external events, and affective state. Table 1 shows the precise interaction of BDI and affective processes with these attributes. A more detailed description of these attributes can be found in Section 4. 4 Representation of the agent attributes for BDI and Affective Processes This section explains in more detail the agent attributes introduced in Section 3.3, (also shown in Table 1). It is also discussed why they are considered as inputs or outputs for the processes of GenIA3. The representation of the agent attributes that are related to the BDI processes has been a widely studied issue. Therefore, we begin this section by describing the representation of these concepts. 4.1 Representation for BDI processes Current beliefs are both inputs and outputs of the brf process, since its function is to revise beliefs. Likewise a representation of external events is created in this process, which are the events perceived from the environment. In order to determine the possible actions at a certain moment of time, the options process takes the current external events,internal events, and the agent’s beliefs. Then the current options are updated. The next action to be performed is selected from the current options by the filter process, and then, intentions are updated. This process is influenced by the current affective state. Different agents have different propensities to be influenced by their affective state when making decisions. Therefore, this aspect of the agent’s personality can also be an input for the filter process1. The next action to be performed is determined by the execute process, which uses the current intentions and generates internal events like the intention’s failure or success. 4.2 Representation for affective processes The appraisal process is included in the affective side of the agent. We understand it as part of a continuous appraisal-affect generation-coping cycle that may or may not process certain event. The goal of this configuration of the architecture is to make it flexible enough to represent most appraisal theories. Thus, those appraisal theories that argue for a relation event-appraisal variables [26, 29] fit in the appraisal process of the architecture, since it has both external and internal events as inputs. On the other hand, appraisal theories that use processes with several steps [32, 31], can also be represented in this architecture. In Lazaru’s theory [32], for example, primary and secondary appraisals are performed in two different stages2. During primary appraisal the relevance and congruence of an event is evaluated starting from 1Some tests like the Cognitive Reflexion Test (CRT) [8] are used in psychology to determine this issue. 2Lazaru’s theory also argues that primary and secondary appraisals are not necessarily sequential, but we limit it to be sequential in our architecture in order to have as much compatibility as possible. 8 Table 1: Inputs and outputs for the processes of GenIA3. Process Inputs Outputs Appraisal beliefs, concerns, internal events, external events, affectively relevant events, intentions, current options affectively relevant events Affect generator affective state affective state Coping personality (coping strategies), affective state current options, intentions Affective modulator of beliefs affective state, beliefs beliefs Affect’s temporal dynamic affective state, personality affective state brf beliefs external events, beliefs options beliefs, external events, internal events current options filter personality (e.g. rationality level), current options, affective state intentions execute intentions internal events the agents concerns,current options, and intentions. During secondary appraisal, the event is evaluated according to the agent’s capacities and resources. We argue that such issues (i.e. capabilities and resources), could be represented by using the probabilities of the beliefs, the probabilities that an action can be performed, and the probability that an agent performs an action3. For example, the agent’s capabilities and resources to perform an action, can be expressed as the probability that the agent can perform such action. Other issues like the memory of affectively relevant events may influence the appraisal process as well [6]. Our idea of memory is in line with the psychological concept of “autobiographic memory” in psychology [24], and it stores meaningful experiences of an individual. Optionally, and depending on the structure of the particular architecture that is used (based on GenIA3), affectively relevant events can be updated. For example, if some event causes a strong emotional impact, it can be saved as an event that has been significant from an affective point of view. The main outputs of the appraisal process are the appraisal variables4. These variables vary from one computational appraisal model to another but they have in common that these variable represent the agent’s judgments, and they produce changes in the affective state. The way these changes are produced is determined by the affect generator process which uses the appraisal variables. The coping process acts as an affect regulator, modifying the agent’s current options,beliefs, or intentions, i.e. altering “the nature or content of cognitive processes” [20] in order to take the current affective state to a desired state or to an “equilibrium state”. This process can also generate intentions oriented to perform reactive physical actions such as body gestures or facial expressions, for example, as a way to mitigate negative emotions. Each agent may have particular ways of reacting to changes of its af3We make a review of psychological theories in other work (to be published), where it is shown how these probabilities allow to represent a variety of inputs of appraisal theories. 4Although appraisal variables are an output of the appraisal process, they are not considered in Table 1 because this table only represents inputs and outputs that are persistent during the agent’s life cycle or that are used by two or more processes in the architecture. 9 representation is required when defining the affective BDI agent. Our proposal is not a formal model (although a formal model is needed). It is a way of avoiding designers of BDI affective agents to start from scratch, simplifying the creation of a computational affective model. This also avoids to make irrelevant commitments to implementation details. The terms that are used across computational approaches and their supporting theories, to characterize affect-related concepts and processes, is wide and diverse. We unified them in a common terminology that allows the comparison of existing computational models. Also the identification of implementation-independent constructions that should be included in an affective BDI agent language, offers a common language that allows to perform language-independent formalizations. We make a step toward the standardization of the development of affective agents. Due to the space limit we didn’t provide enough details of how the different processes addressed in this work can be implemented. We focused on offering a broad view of how the different stages of the design of the affective agent could be addressed, and, the links between these stages. It is a task of the designer to perform more specific commitments for each process involved. Nevertheless, as part of our future work, we will provide a default implementation for those processes. References [1] B. Alfonso, E. Vivancos, and V. J. Botti. An Open Architecture for Affective Traits in a BDI Agent. In Proceedings of the 6th ECTA 2014. Part of the 6th IJCCI 2014, pages 320–325, 2014. [2] R. H. Bordini, J. F. H¨ubner, and M. Wooldridge. Programming multiagent systems in AgentSpeak using Jason. Wiley, 2007. [3] T. Bosse, J. Broekens, J. Dias, and J. van der Zwaan. Emotion Modeling. Springer, 2014. [4] M. Bratman. Intention, plans, and practical reason. Harvard University Press, 1987. [5] M. Bratman. Intention, Plans, and Practical Reason: Center for the Study of Language and Information. The David Hume Series: Philosophy and Cognitive Science Reissues. Cambridge University Press, 1987. [6] A. R. Dam´asio. Descartes’ error: emotion, reason, and the human brain. Quill, 1994. [7] H. J. Eysenck. Biological dimensions of personality. In L. Pervin, editor, Handbook of Personality: Theory and Research, pages 244–276, New York: Guilford, 1990. [8] S. Frederick. Cognitive reflection and decision making. The Journal of Economic Perspectives, 19(4):25–42, 2005. [9] N. Frijda, A. Manstead, and S. Bem. Emotions and Beliefs: How Feelings Influence Thoughts. Studies in Emotion and Social Interaction. Cambridge University Press, 2000. [10] N. H. Frijda. The Emotions. Studies in Emotion and Social Interaction. Cambridge University Press, 1987. [11] N. H. Frijda. The laws of emotion. Lawrence Erlbaum Associates, Incorporated, 2007. 16 [12] P. Gebhard. ALMA: A Layered Model of Affect. In Proceedings of the 4th AAMAS, pages 29–36, NY, USA, 2005. ACM. [13] E. Hudlicka. Guidelines for Designing Computational Models of Emotions. Int. J. Synthetic Emotions, 2(1):26–79, Jan. 2011. [14] E. Hudlicka. From Habits to Standards: Towards Systematic Design of Emotion Models and Affective Architectures. In Emotion Modeling, pages 3–23. Springer International Publishing, 2014. [15] J. Ito, D. Pynadath, and S. Marsella. Modeling self-deception within a decision-theoretic framework. Autonomous Agents and Multi-Agent Systems, 20(1):3–13, 2010. [16] D. Kahneman and D. T. Miller. Norm theory: Comparing reality to its alternatives. Psychological review, 93(2):136, 1986. [17] P. Koval, E. A. Butler, T. Hollenstein, D. Lanteigne, and P. Kuppens. Emotion regulation and the temporal dynamics of emotions: Effects of cognitive reappraisal and expressive suppression on emotional inertia. Cognition and Emotion, pages 1–21, 2014. [18] S. Marsella and J. Gratch. Modeling coping behavior in virtual humans: don’t worry, be happy. In Proceedings of the 2nd AAMAS ’03, pages 313–320, New York, NY, USA, 2003. ACM. [19] S. C. Marsella and J. Gratch. EMA: A process model of appraisal dynamics. Cognitive Systems Research, 10(1):70–90, 2009. [20] S. C. Marsella, J. Gratch, and P. Petta. Computational models of emotion. In A Blueprint for Affective Computing: A Sourcebook and Manual, Affective Science, pages 21–46. OUP Oxford, 2010. [21] R. R. McCrae and O. P. John. An introduction to the five-factor model and its applications. Journal of personality, 60(2):175–215, 1992. [22] A. Mehrabian and J. A. Russell. An approach to environmental psychology. MIT Press, 1974. [23] I. B. Myers, M. H. McCaulley, and R. Most. Manual: A guide to the development and use of the Myers-Briggs Type Indicator. Consulting Psychologists Press Palo Alto, CA, 1985. [24] K. Nelson. The psychological and social origins of autobiographical memory. Psychological science, 4(1):7–14, 1993. [25] A. Ortony. On Making Believable Emotional Agents Believable. In R. P. Trapple, P. Petta, and S. Payer, editors, Emotions in Humans and Artifacts, chapter 6, pages 189–212. MIT Press, 2003. [26] A. Ortony, G. L. Clore, and A. Collins. The Cognitive Structure of Emotions. Cambridge University Press, July 1988. [27] C. Pimentel and M. Cravo. Affective Revision. In C. Bento, A. Cardoso, and G. Dias, editors, Progress in Artificial Intelligence, volume 3808 of Lecture Notes in Computer Science, pages 115–126. Springer Berlin Heidelberg, 2005. [28] R. Reisenzein, E. Hudlicka, M. Dastani, J. Gratch, K. Hindriks, E. Lorini, and J.-J. Meyer. Computational modeling of emotion: Toward improving the inter-and intradisciplinary exchange. Affective Computing, IEEE Transactions on, 4(3):246–266, 2013. 17 [29] I. J. Roseman. A Model of Appraisal in the Emotion System: Integrating Theory, Research, and Applications, pages 68–91. Oxford University Press, 2001. [30] R. Santos, G. Marreiros, C. Ramos, J. Neves, and J. Bulas-Cruz. Personality, Emotion, and Mood in Agent-Based Group Decision Making. Intelligent Systems, IEEE, 26(6):58–66, 2011. [31] K. R. Scherer. Appraisal considered as a process of multilevel sequential checking. Appraisal processes in emotion: Theory, methods, research, 92:120, 2001. [32] C. A. Smith and R. S. Lazarus. Emotion and Adaptation. In L. A. Pervin, editor, Handbook of Personality: Theory and Research, pages 609–637, 1990. [33] B. R. Steunebrink, M. Dastani, and J.-J. C. Meyer. A formal model of emotion triggers: an approach for BDI agents. Synthese, 185:83–129, 2012. [34] P. Verduyn, I. Van Mechelen, and F. Tuerlinckx. The relation between event processing and the duration of emotional experience. Emotion, 11(1):20, 2011. [35] P. Verduyn, I. Van Mechelen, F. Tuerlinckx, and K. Scherer. The relation between appraised mismatch and the duration of negative emotions: Evidence for universality. European Journal of Personality, 27(5):481–494, 2013. [36] G. Weiss. Multiagent Systems. Intelligent robotics and autonomous agents. MIT Press, 2013. 18