Orbis

This work is licensed under a Creative Commons Attribution 4.0 International License. NIME ’25, June 24–27, 2025, Canberra, Australia © 2025 Copyright held by the owner/author(s). Orbis YUKIHIRO SUGAWARA, Keio University, Faculty of Environment and Information Studies, Japan KOTARO WATANABE, Keio University, Faculty of Environment and Information Studies, Japan SHINNOSUKE HIROSE, Keio University, Faculty of Policy Management, Japan MOE MIYAKE, Keio University, Faculty of Environment and Information Studies, Japan KENSHIRO TAIRA, Keio University, Faculty of Environment and Information Studies, Japan SAKURA TAKADA, Keio University, Faculty of Science and Technology, Japan RYOHO KOBAYASHI, College of Performing and Visual Arts, J. F. Oberlin University, Japan YUTA UOZUMI, Keio University, Graduate School of Media and Governance, Japan KEI FUJIWARA, Keio University, Faculty of Science and Technology, Japan SHINYA FUJII, Keio University, Faculty of Environment and Information Studies, Japan Program Notes When cells divide, they initiate cell division through ripple-like patterns called “Min waves”. In the field of artificial cell engineering, researchers have successfully generated these waves within artificially constructed cells. Although cell division itself has not yet been achieved, the Min waves observed in artificial cells exhibit diverse behaviors under various conditions. “Orbis” expresses the cycles and spatial movement of Min waves in artificial cells with sound and light, creating a space where the rhythm and energy flow of the artificial cells can be experienced intuitively through hearing and vision. We developed a system to detect the movements of Min waves generated in artificial cells through image analysis and utilized their unique behavior as an oscillator. For image analysis, Blob Tracking TOP, an OpenCV-based detection model in TouchDesigner, is used to track the positional coordinates of Min waves in the observation footage. The Min wave coordinate data from TouchDesigner is sent to Max via OSC (Open Sound Control) and is reflected in the output of several sound speakers and light bulbs in a circular arrangement. “Orbis” is an interface that enables the unique behavior of this quasi-life form to be converted into sound and light. By expressing the vitality that gradually emerges from matter through auditory and visual information, it encourages the audience to question “What is life?” NIME '25 June 24-27, 2025, Canberra, Australia Yukihiro et al. Fig. 1. Orbis Project Description This system tracks the movement of Min waves occurring within artificial cells and reconstructs their behavior through sound and light based on the acquired data. This chapter provides explanations of each component of the system. We created a procedure to process the observation video of artificial cells in order to track the movement of Min waves generated within them. First, images of the interior of the artificial cells are captured using a fluorescent microscope at 5second intervals. The images are put together to make a video then processed using TouchDesigner and OpenCV. To identify the Min waves in the video, thresholding and edge detection techniques are used to remove noise while extracting the positions of the waves. Subsequently, the movement of the Min waves are analyzed, and the position of each wave is recorded as coordinate data for each frame. The extracted Min wave coordinate data is transmitted in real-time to Max through Open Sound Control (OSC) for sound and lighting control. In the sound system, audio is mapped spatially based on the Min wave coordinate data. Six speakers are arranged in a circle, and the volume of each speaker is dynamically adjusted according to the position of the Min wave. The lighting system generates changes in light based on the Min wave's coordinate data. Twelve light bulbs are arranged in a circle, and their brightness is adjusted in realtime to correspond with the position of the Min wave. The sound and lighting parameters are unified to achieve consistent expression in real-time. Dynamic changes in the space are achieved through the synchronization of the position of sound and light with the movement of the Min waves. Orbis NIME '25 June 24-27, 2025, Canberra, Australia Fig. 2. Layout diagram Fig. 3. System diagram Artificial Cell Artificial cells are a general term for artificial objects imitating the structure of living cells. In biology, artificial cells indicate cell-sized spaces covered with membrane-like structures which encapsulate biological molecules such as proteins and DNA. Because artificial cells can reproduce biological phenomena in cell-size spaces, they are attracting attention as an experimental tool for precisely analyzing the dynamics of molecular reactions within cells. NIME '25 June 24-27, 2025, Canberra, Australia Yukihiro et al. Artificial cells have the following characteristics: • Micrometer-sized closed space: This characteristic is beneficial to investigate the molecular placement mechanism within cell-size spaces and analyze how cell-size space affects biochemical reactions. • Controllability of experimental conditions: It is possible to precisely adjust experimental conditions such as elements composition, temperature, and energy supply. Owing to these characteristics, artificial cells have been widely used to study biological systems [1]. By utilizing artificial cell systems, researchers can reproduce and analyze molecular behaviors under controlled experimental conditions, enabling detailed investigations into self-organization dynamics. Min Wave In cells, self-organization of molecules regulate various biological phenomena. Especially, reactiondiffusion wave generated by coupling of biochemical reactions and molecular diffusions serves a key role in the regulation. Min system, which determines the cell division plane of bacteria, is a wellknown reaction-diffusion wave. The Min system consists of three proteins (MinC, MinD, and MinE) and form a reaction-diffusion wave (Min wave) where the high concentration regions of Min proteins change spatiotemporally [2]. Min waves belong to dissipative structures and maintain their patterns by utilizing energy molecules like ATP. Min wave is generated by following reaction cycles: 1. Membrane binding of MinD by ATP-dependent manner: MinD in the cytosol binds to the membrane after forming the dimer structure by binding to an energy molecule ATP. MinD on membrane recruits another MinD dimer, which forms a high-concentration region of MinD. 2. ATP hydrolyzation by MinDE: MinE binds to MinD on the membrane and induces the activity of ATP hydrolyzation of MinD, which causes the dissociation of MinD from the membrane. 3. Molecular diffusion of MinD into the cytosol and repetition of the reactions: After dissociating from the membrane, MinD binds to ATP in the cytosol and binds to the membrane again. These reactions are coupled with the differences in diffusion rates of molecules in the cytosol and on the membrane, resulting in the generation of reaction-diffusion waves oscillating between cell poles. MinC, a cell division inhibitor, co-localizes with ATP-bound MinD, and oscillation of the Min wave makes MinC concentration gradient with the minima at mid-cell. Consequently, cell division initiates only at the center position of cells. Researchers have successfully reconstituted Min waves in artificial cells, making them the only biological reaction-diffusion system reconstituted in artificial cells using defined factors. By using the artificial cell system, it has been revealed that Min waves behaviors in terms of their dynamic modes and velocity can be controlled by experimental conditions [3][4][5][6]. For example, not only the oscillation of Min waves but also traveling waves and the pulsing mode appear in artificial cells (Fig. 4), and the differences in the generation conditions of each mode have been revealed. Characteristics of each wave mode are described as follows. Orbis NIME '25 June 24-27, 2025, Canberra, Australia • Oscillation: Most of Min waves in living cells show this mode. Although it also appears in artificial cells, the conditions for its generation are limited compared to living cells. It is generated by repeating the formation of MinD domain on the membrane at one cell pole, membrane dissociation of MinD induced by MinE binding to the surrounding of the MinD domain, and the domain formation of MinD at another cell pole. This mode determines the cell division plane by inhibiting cell division at cell poles. • Traveling wave: It propagates on the membrane continuously and appears mainly in artificial cells while it appears in living cells under only limited conditions. It cannot determine the cell division plane because the timeaveraged concentration of MinC becomes homogeneously and MinC inhibits cell division throughout the cell. It is generated by MinD dissociating from only one side of its high-concentration region and rebinding on the MinD region. Due to the difference in the relative position of MinE to the highconcentration region of MinD between the oscillation and the traveling wave, the two wave modes are regulated by the balance of MinD and MinE. Although this mode is not physiologically important in bacteria, this mode has a role in various physiological events in the case of other reaction-diffusion waves in eukaryotes. • Pulsing: It is a state in which MinD oscillates between the homogenous localization in the cytosol and on the membrane. Unlike the oscillation and the traveling wave, it has no spatial asymmetry. It appears mainly in the early stage of Min wave generation and transits to the oscillation and the traveling wave. It seems that this mode shows no apparent physiological role. Fig. 4. Various dynamics of Min waves reconstituted in artificial cells Time-lapse images of sfGFP-MinD (green) and MinE-mCherry (red) in artificial cells encapsulating these Min proteins, ATP, and BSA (left). Schematic images of three types of dynamics of Min proteins are shown (right). Scale bar: 5 µm. Min Wave Detection As a pre-processing step for tracking, TouchDesigner is used to extract the movement of Min waves from the observation video of the artificial cells. The specific method involves converting the footage to grayscale, adjusting the brightness to emphasize the Min wave regions, and using the difference in brightness to isolate only the Min waves (Fig. 5). NIME '25 June 24-27, 2025, Canberra, Australia Yukihiro et al. Next, the monochrome video of the Min waves obtained from the steps above is then tracked in realtime using the Blob Track TOP model, which uses OpenCV within TouchDesigner. By quantifying the coordinate data on the screen, the dynamics of the Min waves are detected. Specifically, the difference of pixels between monochrome video and background are detected, and the detected areas are enclosed in rectangles. The center of mass of the X and Y coordinates of these rectangles are then calculated and output as the Min wave coordinates (Fig. 6). The Min wave coordinate data obtained from the video was scaled to a range of -1 to 1 and sent to Max via OSC. This enables the real-time tracking and quantification of the Min wave movement in the observation video, allowing the dynamics of the Min waves to be utilized for music generation. Fig. 5. Preparation for Min Wave Tracking Fig. 6. Tracking process of Min Waves generated in artificial cells Sound System The six speakers are placed at six equally spaced positions along the circumference of a circle with a radius of 3.5 meters from the central display (Fig. 7). The figure on the right shows only the speakers, depicted from a top-down view. The center coordinates were defined as (X, Y) = (0, 0), and the coordinates of the speakers were defined within the range of +1 and -1. Orbis NIME '25 June 24-27, 2025, Canberra, Australia Fig. 7. Speaker layout diagram Then using TouchDesigner, the arctangent function is applied to the Min wave’s XY coordinate data to calculate the angle θ between the line segment connecting the origin and the Min wave coordinates and the line y = 0 (Fig. 8). The volume of each speaker is set to increase as the angle θ approaches the position of each speaker. Fig. 8. Angle detection schematic diagram This enabled the Min wave movement in artificial cells to be reconstructed through the audio output from speakers arranged along the circumference. Lighting System The twelve light bulbs are placed at twelve equally spaced positions along the circumference of a circle with a radius of 3.5 meters from the central display (Fig. 9). The figure on the right shows only the light bulbs, depicted from a top-down view. The center coordinates were defined as (X, Y) = (0, 0), and the coordinates of the light bulbs were set within the range of +1 and -1. NIME '25 June 24-27, 2025, Canberra, Australia Yukihiro et al. Fig. 9. Lighting layout diagram Using TouchDesigner, the arctangent function is applied to the Min wave’s XY coordinate data to calculate the angle θ between the line segment connecting the origin to the Min wave coordinates and the line y = 0 (Fig. 9). The brightness of each light bulb was set to increase as the angle θ approaches the position of each speaker. By using this angle-based calculation, we were able to create an algorithm that maps the movement of Min waves to the light bulbs placed along the circumference, which remained independent of the artificial cell’s size as long as the center was aligned. This enabled the Min wave movement in artificial cells to be reconstructed through the illumination of light bulbs arranged along the circumference. Table 1. Simulation Configuration Active Speaker ×6 Audio Interface ×1 Speaker Stand ×12 Display ×1 Display Enclosure ×1 (The enclosure contains a laptop computer, a dimmer pack, and a multichannel audio interface in its interior.) Laptop ×2 (One is used for system monitoring.) Dimmer Unit ×3 Light Socket ×12 Incandescent Bulb ×12 DMX Cable ×3 Extension Cable ×20 TRS-XLR Cable ×6 LAN Cable ×1 Wifi ×1 Note: The necessary equipment will primarily be transported from Japan. However, if any equipment is available for rent from the organizers (e.g., TRS to XLR cables), they would be greatly appreciated. Access to any on-site storage space (if available) for storing equipment would also be appreciated. For further details regarding equipment and logistics, please refer to the supplementary material. Orbis NIME '25 June 24-27, 2025, Canberra, Australia Fig. 10. Display enclosure / Machine box An LCD (with protective acrylic cover) is placed on top and serves as a display surface. The enclosure contains a laptop computer, a dimmer pack, and a multichannel audio interface in its interior. Fig. 11. Speaker stand / Enclosure Program Notes Space Requirements The ideal space for this work measures 7m by 7m. Visitors can experience the installation from both inside and outside of the circle of speakers and lights that surround the display in the center. To ensure smooth audience flow, cable lengths and wiring will be managed safely, with enough space between the speakers and lights to allow for passage. A ceiling height of approximately 3m is optimal. However, the layout and the number of speakers / light bulbs can be adjusted according to the exhibition venue. The feasibility of this work as a deployable installation has been demonstrated through our debut exhibition at the ZOU-NO-HANA FUTURESCAPE PROJECT 2024 held in December 2024. For in-person exhibitions, we have the following preferences: 1. A private room for the installation. 2. A location with minimal acoustic interference. 3. A space that can be darkened.