Swarms on Stage: Swarm Simulations for Dance Performance

12th Generative Art Conference GA2009 Page 105 Swarms on Stage Swarm Simulations for Dance Performance Daniel Bisig Institute for Computer Music and Sound Technology Zurich University of the Arts e-mail: [email protected] Tatsuo Unemi Department of Information Systems Science Soka University e-mail: [email protected]p Abstract Simulations of swarm behavior have been employed as a generative processes in many musical and artistic realizations. Despite this popularity, the application of swarm simulations as a visual and interactive component for stage performances seems to be very rare. This paper tries to show that swarm simulations can be employed as flexible and fascinating choreographic elements for dance performance. In particular, a swarm's strong spatial presence and it's behavioral flexibility provide a wide range of choreographic possibilities that accentuate or complement the human dancers' activities. This paper places a particular focus on the presentation of several examples of swarm based stage effects that have been realized as collaboration between the authors and choreographers. 1. Introduction Dance as an art form relies on the expressive qualities of the human body. The body's physical immediacy, it's spatial movement and it's participation and synchronization within groups form the constituents of choreographic design. Dance draws much of its fascination and aesthetics from the creation of antagonisms between naturalness and artificiality. The body's gestural capabilities and social phenomena of body language and group formation form the source material from which basic behavioral elements are extracted via a process of abstraction and formalization. A dance piece then constitutes an artificial re-synthesis of human behavior that results from a modification, exaggeration and recombination of these 12th Generative Art Conference GA2009 Page 106 basic elements. Thus, on a very general level, choreography and the sciences of Artificial Life and Artificial Intelligence share a similar synthetic approach of recreating an artificial system as an abstraction of a natural system. But there exists a much more striking and profound similarity between choreography and a particular type of simulation. Swarm simulations constitute multi-agent systems that model group formation and spatial movement. These simulations directly deal with some of the very fundamental constituents of dance. Both swarm simulations and choreography deal with relationship between local and global patterns. Both deal with the occupation of space via clustering and dispersion. Both deal with the synchronization of behaviors. And both blur the distinction between individuals and groups in that individuality appears both on the level of the single dancer/agent and on the level of the entire group. On the other hand, dance choreography and swarm simulations differ with regard to several fundamental aspect. The most obvious and important distinction concerns the manifestation of the presence of the actors. Human dancers naturally possess a very strong and immediate presence in the physical space whereas a simulated entity is inherently dissociated from the physical world. This dissociation can be partially overcome by rendering the virtual entity perceivable via one or several feedback modalities, but a certain level of in-corporality always remains. A further distinction concerns the differences in complexity on the level of the individual and the group. Each single dancer possesses an intricate and highly individual body that always stands out as a strong visual element. This complexity of the individual dancer contrasts with the comparatively simple patterns that groups of dancers can form. Any emphasis on group patterns in choreography are therefore hampered by the dancers own idiosyncrasies. The simulated entities in a virtual swarm on the other hand are usually rendered as simplistic and totally uniform shapes. In these simulations, the complexity clearly manifests on the level of the group and the appearance of the individual agents hardly distracts from these group phenomena. It is therefore relatively simple to perceive a simulated swarm as a sort of metaindividual whose morphological diversity far exceeds those of the human dancers. Furthermore, the relationship between space and actors differs considerably between human dancers and virtual agents. The spatial scale of the dancers and the stage is immutable. The movement and distribution of dancers is mostly restricted to the two dimensional plane of the stage surface. In a simulation on the other hand, the mapping of the topology and scale of the virtual world into the physical stage is very flexible and can change throughout a performance. This mapping creates a correspondence between the visual and physical properties of the swarm, the stage and the dancers. Any physical surface on the stage can serve as a window into the virtual world and / or represent physical barriers within the virtual world. Depending on the scale and continuity of this mapping, the stage can appear as a narrow cage for the simulated swarm or open up into a seemingly endless universe. Finally, the choreographic process usually represents a top-down approach of designing the local and global structures and dynamics of the performance. A swarm simulation on the other hand represents a complex system whose global properties emerge via bottom-up processes of self-organization. For this reason, swarm simulations can introduce elements of unpredictability and surprise into an otherwise entirely prespecified choreography. For all of these reasons, swarm simulations represent an attractive and alternative choreographic element that both supports and 12th Generative Art Conference GA2009 Page 107 complements the presence and activity of human dancers. The swarm allows to transform a stage into a responsive environment and at the same time it can act as an artificial dancer whose movements and morphological changes are unlike anything a human dancer would be capable of. Throughout the remainder of this paper, the authors present four exemplary applications of interactive swarm visualizations for dance performance. These works have been realized in collaboration with the two choreographers Jiri Kylian and Pablo Ventura and where shown in a variety of venues. 2. Dance Performances 2.1 Vanishing Twins The dance performance "Vanishing Twins" was choreographed by Jiri Kylián and premiered at the Lucent Dance Theatre in The Hague, Holland in 2008. In this performance, two swarm simulations were projected into the left and right half of the stage background. Each simulation consisted of 5000 agents that where depicted as triangles and that densely populated the entire two dimensional simulation space. The agents' exhibited the classical Boid's type of behaviors [1] as well as additional interaction dependent reactions [2]. These reactions were triggered via a camera based tracking system that captured the dancer's image from an inclined frontal position. The reactions included an evasion behavior away from the position of a dancer's movements and a color change behavior from white to black that was proportional to the amount of movement. The swarm simulation was running and projected throughout the entire performance but the interactive behaviors became active only at the very end of the piece. During the non-interactive part of the simulation, the agents would engage into a constant circular motion that was constrained by the outer boundaries of the stage background. The simulation switched to the interactive mode when only two dancers were left on the stage. Each of the dancers was positioned at the center of the left and right swarm projection, respectively. The dancers' behaviors alternated between stationary phases and short bursts of very active movements. During these short bursts, the dancers ripped a large black region in the shape of their bodies into the projected visualization that was otherwise densely covered by white triangles (Figure 1). As soon as the dancers stopped their movements, the agents reclaimed these black regions and the uniform coverage of the circularly moving white triangles was recreated. In "Vanishing Twins", the manifestation of the swarm simulation was dislocated from the physical position of the dancers. The dancers moved in the two-dimensional space of the stage surface, whereas the swarm visualization covered the vertical stage background. But at the same time, the dancers possessed a virtual presence within this vertical agent world. The horizontal position of the dancers on stage was mapped into a vertical position within the visualization of the simulation. Contrary to the dancers physical presence, their virtual presence in the simulation was constantly compromised by the swarm. The swarm assumed the role of a hostile environment that constantly threatened the existence of its inhabitants. Only via highly exaggerated movements could the dancers maintain their unharmed virtual presence 12th Generative Art Conference GA2009 Page 108 as a black silhouette that was devoid of any white triangles. As soon as the dancers' movements slowed down, their silhouettes started to distort due to the agent's circular movements and was soon swallowed by the crowd of white triangles. The dissolution of the dancers' virtual existence by the swarm's activity is a metaphorical reenactment of the biological phenomena that an unborn child fails to compete with its twin's need for resources and disappears within the mother's womb. This phenomena forms the inspiration for the piece "Vanishing Twins". Figure 1: Swarm projection for the choreography “Vanishing Twins” by Jiri Kylián. 2.2 Swarms The dance performance "Swarms" was choreographed by Pablo Ventura and premiered at the Schauspielhaus in Zürich, Switzerland in 2008. The performance comprised three scenes for each of which a specific swarm simulation had been created. Both the swarm simulation and the tracking system were based on libraries that had been developed as part of the “interactive swarm orchestra” project [3][4]. In each scene, the visualization of the agents consisted of black lines that traced the agents' movement trajectories and whose opacity gradually decreased to zero towards the older sections of the trajectories. The agents themselves possessed no visual shape. The visualization was projected on a vertical screen in the background of the stage. Throughout the first two scenes, the dancers were tracked via a camera that was positioned at the front of the audience on ground level. No camera tracking was required for the last scene since the corresponding swarm simulation didn't respond to interaction. During the interactive scenes, the dancers' bodies and movements generated several force fields within the two dimensional swarm 12th Generative Art Conference GA2009 Page 109 simulation. Each of these fields consisted of two a dimensional grid of two dimensional vectors. A distance force field comprised vectors that point from each grid point towards the closest point on the dancers' body contours. A tangential force field comprised vectors that point from each grid point along the tangent of the dancers body contours. A motion force field comprised vectors that pointed along the gradient of a motion history image of the dancers' movements. All these force fields were continuously updated based on tracking information. The simulated agents engaged not only into Boids type of activities but also possessed a set of behaviors that caused them to move along these force vectors. For this reason, the swarm simulation combined biological aspects of collective movement with purely physical phenomena of passive propulsion within force fields (Figure 2 left side). For the noninteractive scene, the swarm existed in a three-dimensional simulation space. The agents were subjected two distance and tangential force fields that had been precalculated based on a polygon model of a human figure. Accordingly, the agents trajectories extended from an initially random distribution towards and along the figure's surface, which thereby became increasingly recognizable (Figure 2 right side). The ambiguous connotations of the human body form an underlying topic in all of Pablo Ventura's choreographies. The human body forms an integral part of our subjectivity, at the same time, its functioning resembles that of a very intricate machinery, and finally, the human body constitutes a very iconic element in our culture. In his choreographies, Pablo Ventura experiments with these connotations by alternating the dancers' appearance and behaviors between biological and artificial, aestheticized and natural, human-like and robot-like. The behaviors and interactive reactions of the simulated swarms have been specifically adapted to this concept. The dancers' natural bodies act as a machinery for the generation of physical forces in space. The artificial swarms respond to these forces via behaviors that blend seemingly natural biological and physical phenomena. Accordingly, both the dancers and the simulation mix machinic and natural qualities. During the last scene, the dancers movements slow down until they finally lay down on the hardly illuminated stage. In this moment, when the dancers human shape is no longer perceivable, the swarm reforms into a depiction of a single large human body. By doing this, the swarm shifts from a simulated complex entity towards a mediation of the human body as a visual icon. 12th Generative Art Conference GA2009 Page 110 Figure 2: Two scenes from the choreography “Swarms” by Pablo Venura. 2.3 Gods and Dogs The dance performance "Gods and Dogs" was choreographed by Jiri Kylián and premiered at the Lucent Dance Theatre in The Hague, Holland in 2008. For this performance, a visualization of a two dimensional swarm simulation was projected directly on a single dancer's body (Figure 3). The dancer was tracked from a frontal and elevated camera. The tracking system extracted the dancers' body as a foreground object that formed a zone of attraction within the simulation world. The size of the simulated world was only slightly larger than the dancer's body size and its position with respect to the stage changed in correlation with the dancer's movements. The agents themselves became visible as short trails once the entered the zone of attraction. Accordingly, whenever the dancer moved quickly across the stage, the zone of attraction would shift away from the agents and the agents would fade away. Once the dancer stopped his fast movements, the agents would quickly move into the zone of attraction and thereby became once again visible on the dancer's body. This fast alternation between disappearance and reappearance of the agents was facilitated by the moving simulation world that forced the agents to stay in the vicinity of the attractive zone. In "Gods and Dogs", the swarm visualization is no longer dislocated from the physical position of the dancer. Accordingly, the dancer no longer appears to assume a remote presence within the virtual world of the swarm. Rather, the swarm manifests itself on the physical surface of the dancer's body and thereby partially looses its incorporal qualities. The dancer's body and the swarm visualization blend into a form a shared existence. The underlying subject of this choreography is the ill-defined and 12th Generative Art Conference GA2009 Page 111 shifting distinction between what is considered normal and non-normal by a society. The swarm that gradually covers the body surface changes the dancers appearance from a normal to a non-normal state. The swarm behaves like a parasite that cannot exist without its host, the dancer's body. Once the swarm has caught up with the dancer, it spreads like a disease from the dancer's body outline until it covers the entire body surface. The dancer can try to escape this threat to his normality but will eventually succumb to the swarm's parasitic behavior. In this situation, the swarm is in position of power and dominance and partially supersedes the physical reality of the dancer's body. Figure 3: Swarm projection on the dancer's body in the choreography “Gods and Dogs” by Jiri Kylián. 2.4 2047 The dance performance "2047" was choreographed by Pablo Ventura and premiered at the Tanzhaus Wasserwerk in Zürich, Switzerland in 2009. The most prominent element of the stage design for this performance consisted of a white surface that covered the back half of stage surface and the sloped upwards to the ceiling along the rear wall of the stage. This surface served as a screen for a frontal projection via two video beamers. The interaction between a single dancer and a swarm simulation played a prominent role in a particular scene. As for the “Swarms” choreography, the swarm simulation and the tracking system were based on libraries that had been developed as part of the “interactive swarm orchestra” project [3][4]. During this scene, the dancer was performing in the middle of the horizontal section of the surface. The dancer was tracked via two cameras, one positioned above the dancer and pointing straight down, the other was placed in a frontal position and horizontal orientation. Similar to the setup for the "Swarms" piece, the dancer's body contour and motion created virtual force fields to which the swarm responded. The force fields from the frontal tracking were mapped to a position on the vertical surface behind the dancer and the force fields from the vertical tracking were mapped to a 12th Generative Art Conference GA2009 Page 112 position exactly underneath the dancer. The swarm was projected as a single continuous visualization on the surface and the dancer's body (Figure 4). The swarm was continuously moving from the top of the vertical surface towards the front of the horizontal surface. When the swarm crossed the two force fields, the tangential and attractive forces of the dancer's contour narrowed it down to a very dense streak that followed the dancers silhouette whereas the motion field countered this vertical movement and caused the swarm to spread horizontally. The title of the performance refers to a subplot in the film 2047 by director Wong Kar-wai. In this subplot, two men traveling in a train are served by android waitresses. The notion of travel and the mixture of natural and artificial traits of the androids form the underlying conceptual and aesthetic aspects of the dance performance. The virtual swarm embodied the natural and artificial duality of the android. The unnatural beauty of the female androids was represented by combining the swarm visualization with its own horizontal mirror image and thereby creating a highly symmetrical appearance. The projection of the constantly moving swarm on both the white surface and the dancer's body blurred the visual difference between the stage environment and the dancer. This blurring accentuated the fact that the androids and the moving train form part of the same artificial system. At the end of the dance performance, the audience was allowed to enter the stage and interact with the swarm simulation. Through this participatory element, the swarm that had been an aspect of an entirely pre-specified choreography transformed the stage into an environment for explorative improvisation. This opportunity allowed the audience to gain an intuitive understanding for the connection between the dancer's choreographed activities and the spontaneous behavior of the swarm. It is via the interactive capabilities of the swarm that certain aspects of an otherwise highly abstract choreography became accessible to the subjective curiosity of the audience. 12th Generative Art Conference GA2009 Page 113 Figure 4: Swarm projection for the choreography “2047” by Pablo Venura. 3. Conclusions and Outlook The preceding description of four dance performances that employed swarm simulations as an important choreographic element hopefully illustrates some of the potential capabilities that these systems offer for contemporary dance. In all these examples, the experimentation and adaptation of the swarm simulations and their visualizations formed an important aspect of the rehearsal process. But in front of the audience, these performances followed an entirely pre-specified structure. The choreography "2047" is the only example that breached this choreographic tradition but merely as part of an installative situation that took place after the performance had finished. The authors believe that the audience could witness the swarms spontaneous and autonomous qualities in a more substantial way if the interaction between the dancers and the swarm comprised elements of improvisation. The authors look forward to expand their work into this direction.