The Walkable Instrument: Modular Patches as Entangled Environments in OpenSoundLab

This work is licensed under a Creative Commons Attribution 4.0 International License. NIME ’25, June 24–27, 2025, Canberra, Australia © 2025 Copyright held by the owner/author(s). The Walkable Instrument: Modular Patches as Entangled Environments in OpenSoundLab LUDWIG ZELLER, Basel Academy of Art and Design FHNW ANSELM BAUER, Independent Artist Program Notes What happens if a modular synthesizer patch is no longer constrained to a flat, rigid configuration, but can instead be copied infinitely (and at no expense) and arranged in midair all around the performer? The performance showcases two modular patch designs assembled in OpenSoundLab, a mixed-reality patching system for Meta Quest that was developed in previous research [1, 2]. This platform allows users to freely position basic sound modules (oscillators, samplers, sequencers, effects, etc.) throughout real-world environments. Figure 1: A screenshot of a mixed-reality view within OpenSoundLab showing a person while performing on a spatial modular patch in a public park in Cologne, Germany. A long-standing challenge in electronic-music performance—whether you’re working with hardware synths, modular rigs, software patches, or full DAW setups—is that the patch architecture and live tweaks remain opaque, even to expert listeners. The system is often too small, too cluttered by patch cables, or simply too complex and hidden in menus, sub-patches, or binary code to be grasped visually. When a musician plays a traditional keyboard or a self-built NIME-style instrument, the audience can readily perceive timing, effort, and skill. NIME '25 June 24-27, 2025, Canberra, Australia. Zeller / Bauer Figure 2: A screenshot of a mixed-reality view within OpenSoundLab showing a modular patch aligned on a plane with 90-degree shaped cable connections against the wall of an underground carpark. In classic synth-tweaking shows, however, such clear audiovisual cues are largely absent. OpenSoundLab resolves this by letting performers position a modular patch at any scale and in any arrangement within the stage space. Freed from rigging and gravity, a modular setup can expose its inner sonic architecture in three dimensions, with less-critical modulation oscillators tucked behind larger, more prominent elements. Multichannel components become visibly entangled around the performers, spreading out in front of—or even encircling—them to map the patch’s full sonic possibility space. During the performance, Anselm Bauer will present two patches that illustrate this concept. Both pieces inhabit the realm of abstract glitch techno. The first features tightly defined sound structures laid out as a clearly readable tree of audio channels and modulations (see figure 2), while the second explores generative, self-modulating processes, resulting in an even more entangled, rhizomatic patch topology (see figure 1). Project Description Beginning in the 1990s and 2000s, digital innovations spread widely across design, art, and music, leading to a strong wave of dematerialization in experimental fields [cf., 3]. Within sonic media art circles, some creators have viewed the shift toward purely software-based processes as a loss of the tactile, physical dimensions of music-making. This concern is especially apparent in plugin-focused DAW workflows and visual programming environments such as Pure Data [4], Max [cf. 5], and VCV Rack [6]. While such audio software has made much hardware equipment redundant (and drastically diminished the financial footprint required for music production), they typically constrain user interaction to a 2D interface on a computer screen. This has diminished the immersive, spatial engagement once integral to working in analog studios or dedicated audio workspaces. In the 2010s, the popularity of modular synthesizers has seen a significant surge in interest, prompting a variety of emerging companies to craft specialized boutique hardware modules [7]. Many Eurorack enthusiasts see these tangible systems as a way to break away from digital instruments, seeking instead a more physically satisfying approach. Despite the high-fidelity algorithms now available to softwarebased musicians, digital environments still grapple with issues rooted in dematerialization: a typical The Walkable Instrument NIME '25 June 24-27, 2025, Canberra, Australia setup within Max or VCV Rack may hold dozens (if not hundreds) of interface components, all confined within a relatively small visual area. Interaction usually occurs via a mouse, which provides minimal haptic feedback while isolating controls within a two-dimensional screen. Consequently, even though software theoretically unlocks vast sonic potential, the primary mode of interaction can feel archaic and unintuitive. Musicians may resort to MIDI controllers or more bespoke input devices, but these often involve additional abstract layers of mapping and interpretation that can weaken the direct, hands-on bond associated with hardware. For extensive patches, maintaining an intuitive one-to-one correspondence between all components becomes impractical. Against this backdrop, NIME (New Interfaces for Musical Expression) has long pushed beyond screen-and-mouse or conventional MIDI paradigms to conceive new performance interfaces. Commonly, the patch is performed via custom sensors or actuators, but the actual process of patching or coding the audio architecture tends to unfold on a standard computer display. Our research explores how digital patching itself can be reimagined as a spatial, immersive, and dynamic act—before and during the performative act—thereby recapturing one of the most significant appeals of modular systems. The electronic musician Richard Scott suggests in regards to the modular synthesizer’s non-symbolic interactions, physical dimensions, and tactility that “thingness” entails more than just material hardware [8]. We ask whether such qualities must remain confined to physical devices or whether they can be transitioned into virtual and mixed reality contexts. Recent developments in extended reality (XR) technologies indicate that it may be possible to reproduce—or sometimes even surpass—the fascination associated with physical modular rigs, despite the absence of literal materiality. Prior studies on VR-based sonic interaction span multiple areas, including the enhancement of musical engagement [e.g. 9, 10 , 11 ], usage of virtual and mixed reality in performance [e.g. 12], and investigations of immersive audio-visual experiences [e.g. 13, 14]. Yet, modular synthesis specifically has received limited scholarly attention in VR, with Michael Palumbo’s “Mischmasch XR” project [15, 16] standing as a notable exception. Emerging XR technology adds gesture detection, motion tracking, and spatial arrangement—offering the potential for three-dimensional sound labs tailored to modular patching. Our performance project aims to demonstrate how these new setups enable immersive, location-based patching of modular synthesizer networks, transcending what is feasible with either physical modular hardware or conventional screen-based software. Performance notes Ideally, the performance should be experienced on-site with a Meta Quest 3 headset, enabling both immersive viewing and listening. Because our modular patching application supports multiple concurrent users, performers and audience members can share the same multiplayer instance in a colocated mixed-reality session. But unfortunately, we cannot travel to NIME in person this year and provide the according hardware setup, so we will provide a live video stream of the performance instead. Using OBS, we can multiplex several camera angles into a single feed, which can be broadcast via Twitch or any streaming platform NIME designates. The two spatial patch concepts will be presented in short segments—approximately 5–7 minutes each—yielding a total performance window of 10–15 minutes. The stream should be projected on a suitably large screen and played back through full-range stereo speakers. Our team is based in Central Europe, so an ideal performance slot would be between 6:00 p.m. and 11:00 p.m. Australian time. We will supply a pre-recorded version of the performance that can be shown when connectivity issues arise. Media Link(s) Available as supplementary material. NIME '25 June 24-27, 2025, Canberra, Australia. Zeller / Bauer Ethical Standards No people or animals were involved in our research and work. References [1] Ludwig Zeller and Hannes Barfuss (2022). “OpenSoundLab – A virtual sound laboratory for the arts”. In: Proceedings of the 17th International Audio Mostly Conference (AM ’22). New York: Association for Computing Machinery. 159–162. [2] The source code of OpenSoundLab is available at http://opensoundlab.org. [3] Matthew Ward. 2015. Rapid Prototyping Politics: Design and the De-material Turn. In: Transformation Design. Perspectives on a New Design Attitude. Edited by Wolfgang Jonas, Sarah Zerwas and Kristof von Anshelm. Basel: Birkhäuser. 227–245. [4] Miller Steve Puckette. 1997. Pure Data. In: Proceedings of the International Computer Music Conference. [5] Joe Snape and Georgina Born. 2022. Max, music software and the mutual mediation of aesthetics and digital technologies. In: Music and Digital Media. A Planetary Anthropology. Edited by Georgina Born. London: UCL Press. 220–266. [6] Andrew Belt. 2017. VCV Rack. Retrieved from https://vcvrack.com, last accessed February 5th 2025. [7] Beat Rossmy and Alexander Wiethoff. 2019. The Modular Backward Evolution. Why to Use Outdated Technologies. In: Proceedings of the International Conference on New Instruments for Music Expressions 2019. [8] Richard Scott. 2016. Back to the Future: On Misunderstanding Modular Synthesizers. In: EContact! Journal of the Canadian Electroacoustic Community 17 (4). [9] Florent Berthaut. 2021. Musical Exploration of Volumetric Textures in Mixed and Virtual Reality. In: Proceedings of the International Conference on New Interfaces for Musical Expression 2021. [10] Anıl Çamcı, Matias Vilaplana, and Lusi Wang. 2020. Exploring the affordances of VR for musical interaction design with VIMEs. In: Proceedings of the International Conference on New Interfaces for Musical Expression 2020. 121–126. [11] Chase Mitchusson. 2020. Indeterminate Sample Sequencing in Virtual Reality. In: Proceedings of the International Conference on New Interfaces for Musical Expression 2020. [12] Jack Atherton and Ge Wang. 2020. Curating Perspectives: Incorporating Virtual Reality into Laptop Orchestra Performance. In: Proceedings of the International Conference on New Interfaces for Musical Expression 2020. [13] Jonathan Weinel. 2021. Cyberdream: An Interactive Rave Music Visualization in Virtual Reality. In: SIGGRAPH Asia 2021 XR. New York: Association for Computing Machinery. [14] Fede Camara Halac and Shadrick Addy. 2020. PathoSonic: Performing Sound in Virtual Reality Feature Space. In: Proceedings of the International Conference on New Interfaces for Musical Expression 2020. 520–522. [15] Michael Palumbo, Alexander Zonta and Graham Wakefield. 2020. Modular Reality: Analogues of Patching in Immersive Space. In: Journal of New Music Research 49 (1). 8–23. [16] Graham Wakefield, Michael Palumbo, and Alexander Zonta. 2020. Affordances and Constraints of Modular Synthesis in Virtual Reality. In: Proceedings of the International Conference on New Interfaces for Musical Expression. 653–657.