Distributed Listening

Title: Distributed Listening FRANCESCO CORVI #1, Institute of Sonology, Netherlands DANIEL GORELICK #2, Indipendent, US Additional Key Words and Phrases: distributed agency, machine listening, feedback systems, prepared cello, live coding ACM Reference Format: 1 PROGRAM NOTES Combining Daniel Gorelick’s work on feedback systems on a prepared cello with Francesco Corvi’s research on adaptive processes and machine listening in SuperCollider [ 3 , 5 ], “Distributed Listening” is a performance that questions the notion of listening as a centralized activity and instead explores how it can be distributed across digital and electroacustic musical instruments. This is achieved by sharing data in the form of audio signals and descriptors between the instruments employed in the performance, creating a web of interconnections that the performers play and explore together. A specific focus is given to how such interconnections shape the agency and the sound capabilities of each instrument, continuously redefining its role and functionality in relation to the whole, and fostering adaptive behaviors in response to the changes in the data it receives. The performance highlights the distinctive performative idiom that emerges among the performers through this approach, developing a narrative around the fluid sonic presence of each element on stage and the ability of each action to influence the overall behavior of the system. 2 PROJECT DESCRIPTION “Distributed Listening” is an artistic project dealing with the concepts of distributed agency[ 9 ], machine listening [ 1 , 2 , 6 ] and adaptive sound processing[ 7 , 8 ]. The focus of the project is the exploration of a miscellaneous technological apparatus, where digital and electroacoustic instruments are situated in the same environment[ 4 ], sharing data in the form of audio signals and descriptors. Particular relevance is given to how this shapes the functioning of each instrument: as a mediator between performers, as a semi-autonomous agent able to exhibit adaptive behaviors and react to the incoming data, and on its ability to influence the rest of the system. The project is presented through a performance featuring two musicians: The first performer plays a prepared cello equipped with the following components: a transducer that can make the wooden body of the instrument resonate, a piezo-electric pickup to capture the cello’s direct signal, and a microphone to capture sounds from the surrounding environment. This setup creates a feedback system influenced by the acoustic characteristic of the cello and by the space where the performance happens. In addition to being played as an acoustic Licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0). Copyright remains with the author(s). Francesco Corvi #1 and Daniel Gorelick #2. 2024. Title: Distributed Listening. 1, 1 (September 2024), 5 pages. Music Proceedings of the International Conference on New Interfaces for Musical Expression NIME’24, 4–6 September, 2024, Utrecht, The Netherlands 2 Corvi and Gorelick Fig. 1. The feedback system on a prepared cello with (A) body mounted transducer, (B) piezo-electric pickup, (C) cello signal output, and (D) input signal for the transducer mixing SuperCollider, room mic, and cello input. instrument, the cello serves as a resonant body capable of entering self-oscillation through feedback with internally and externally positioned microphones. The cello also functions as a speaker for sounds generated by the computer of the second performer. The second performer uses a live coding environment developed in SuperCollider that employs machine listening to control synthetic sounds. Through a flexible and reconfigurable routing system, the performer has the ability to amplify the synthetic sounds through the cello’s wooden body as well as from the venue’s sound system. Simultaneously they can use the sound captured by the microphone positioned in the cello and in the space as an input for the machine listening system. The live coding environment serves as: a generator of adaptive synthetic sounds controlled by descriptors extracted from the cello’s sound, as a spatialization engine for the sounds captured by the microphones, and as a mixer controlling the amount of signal amplified through the cello’s body and by the speakers in the room. In this setup, each component is strongly interconnected with the others and fulfills various functions depending on the overall system configuration. Both performers essentially play the cello: one physically playing the instrument, while the other using it as a resonant body to amplify the computer generated sounds. Furthermore both performers also contribute to the creation of the computer generated sounds, as the sounds produced by the cello drive the generation of synthetic sounds generated by the second performer’s computer. 3 PERFORMANCE NOTES The two performers are available to come to Utrecht to perform the proposed work in person. We would like to propose the Wave Field Synthesis at HKU as our venue of choice for this performance. The choice is motivated by the following reasons: • The performance is best suited to an intimate setting where performers and audience share the same space. Ideally, the performers would be positioned centrally within the space with the audience surrounding them. This configuration matches well the setup of the wave field, also taking advantage of its wide sweet spot. • The interaction between the acoustic sounds generated by the cello creates a unique feeling when they are combined with sophisticated spatialization systems such as those provided by the Wave Field Synthesis. This is Title: Distributed Listening 3 Fig. 2. Stage setup for two performers and (A) prepared cello with transducer, (B) SuperCollider machine listening system, (C) a room microphone, and (D) DAW to mix and attenuate all input signals. particularly useful for the performance, as it accentuates the blend of the synthetic sounds with those of the cello, which is one of the aspects on which the performance narrative focuses. • The spatialization capabilities of the Wave Field Synthesis in addition to enhancing the spatial quality of synthetic sounds, create interesting interactions when combined with the feedback from the two microphones connected to the cello, and in conjunction with the machine listening system. This develops an interesting context for utilizing Wave Field Synthesis, where changes in spatialization can influence the sonic outcomes on multiple levels. • Given its improvisational and exploratory nature, the performance is well suited to be performed multiple times. • Last but not least, the performers already have experience with the WFS collider software for past projects. Although they have not yet tested this particular performance in the Wave Field Synthesis, they have experimented with other multichannel systems and obtained very interesting results. This would therefore be a great opportunity and motivation for us to make a version of the performance for the Wave Field Synthesis. STAGE PLAN The performance requires enough space to set up one station with a table containing a laptop stand, an audio interface, and a MIDI controller, and a second station with a cello stand, a stool, a second audio interface, and a laptop. The performers shall be positioned close to each other in the center of the room with the audience all around. If that is not possible the performer will be positioned on stage facing the audience. In both scenarios there should be enough space to allow the performers to move and to be able to take and put back the cello on its stand. LIGHTNING The performance requires a semi-dark room with the possibility of having a spotlight focused on the cello and a second 4 Corvi and Gorelick diffused light to illuminate the two performers. Simple instructions on how to operate the lights will be provided before the performance. If a projector is available, the artists will consider projecting visuals so that they encompass the cello and the performers. SOUND SYSTEM As previously mentioned the ideal sound system for the performance is the Wave Field Synthesis at HKU. Alternatively, the performance can also be executed on different types of audio systems; however, a quadraphonic or octophonic configuration is preferable over stereo. TECHNICAL REQUIREMENTS •2 Laptop stands; •2 sheet music stands with adjustable height; •1 stool for the cello player; •2 power strips with 3+ plugs. VIDEO The performers plan to create a video for the online version of the conference, alternating footage of various performances with voice-overs and concise textual explanations outlining the conceptual framework, the technical realization and the artistic process behind the work. The feasibility of this will be discussed with the conference chairs if the proposal is accepted. Alternatively, we are also prepared to produce a video focusing only on a performance. 4 MEDIA LINKS •Video: https://www.youtube.com/watch?v=9CwJnnz18a8 ETHICAL STANDARDS The development and execution of "Distributed Listening" adheres to ethical principles concerning research, artistic expression, and human subjects’ involvement. This includes: informed consent by performers, collaborators and audience; respect for all participants involved in the project; and integrity in the research methodology by citing all sources appropriately and accurately representing the work of others. REFERENCES [1] John Bowers and Owen Green. 2018. All the Noises: Hijacking Listening Machines for Performative Research. In Proceedings of the International Conference on New Interfaces for Musical Expression. Zenodo, 114–119. https://doi.org/10.5281/zenodo.1302699 [2] Nick Collins. 2015. Live Coding and Machine Listening. In Proceedings of the First International Conference on Live Coding. ICSRiM, University of Leeds, 4–11. https://doi.org/10.5281/zenodo.18747 [3] Francesco Corvi. 2022. Live coding with Adapt, International Conference on Functional Programming (ICFP 2022). [4] Alice Eldridge and Chris Kiefer. 2023. The self-resonating feedback cello: interfacing gestural and generative processes in improvised performance. (6 2023). https://sussex.figshare.com/articles/conference_contribution/The_self-resonating_feedback_cello_interfacing_gestural_and_generative_ processes_in_improvised_performance/23444609 [5] Francesco Corvi et al. 2023. Commuta: A Cross Adaptive Laptop Ensemble, International Conference of Live Coding (ICLC),. [6] Chris Kiefer and Thor Magnusson. 2020. Live Coding Machine Learning and Machine Listening: A Survey on the Design of Languages and Environments for Live Coding. In Proceedings of the Fourth International Conference on Live Coding. Medialab Prado / Madrid Destino, 353. https: //doi.org/10.5281/zenodo.3946188 [7] Øyvind Brandtsegg Marije Baalman, Simon Emmerson. 2018. Instrumentality, perception and listening in crossadaptive performance.. In Proceedings of the 2018 Conference on Live Interfaces (Porto). Title: Distributed Listening 5 [8] Joshua D. Reiss and Øyvind Brandtsegg. 2018. Applications of Cross-Adaptive Audio Effects: Automatic Mixing, Live Performance and Everything in Between. Frontiers in Digital Humanities 5 (2018). https://doi.org/10.3389/fdigh.2018.00017 [9] D. Sanfilippo. 2020. Complex Musical Behaviours Via Time-variant Audio Feedback Networks and Distributed Adaptation: A Study of Autopoietic Infrastructures for Real-time Performance Systems. University of Edinburgh. https://books.google.de/books?id=PlDjzQEACAAJ