MSN/AV: Maximum Silence to Noise/Audio-Visual

This work is licensed under a Creative Commons Attribution 4.0 International License. NIME ’25, June 24–27, 2025, Canberra, Australia © 2025 Copyright held by the owner/author(s). MSN/AV: Maximum Silence to Noise/Audio-Visual JOHN R. FERGUSON, Queensland Conservatorium Griffith University ANDREW R. BROWN, Queensland College of Art and Design Fig. 1. MSN/AV: Maximum Silence to Noise/Audio-Visual. 1 Project Description/Abstract MSN/AV (Maximum Silence to Noise/Audio-Visual) is an interactive audiovisual system performed by two musicians. The sound world is created through ring modulation synthesis controlled by multidimensional touch gestures. This approach provides a rich diversity of sonic potential whilst maintaining clear remnants of physical gestures. Each musician uses a ROLI Lightpad Block and an iPad running TouchOSC to control bespoke software written in Pure data. Interaction data from each performer is passed to a Touch Designer network which generates and/or manipulates visual materials. MSN/AV celebrates physical interplay with gestural interfaces and situates live improvisation within a responsive audiovisual environment. Sensor data is used to generate sound and graphics in real-time, aiming for an audiovisual entanglement that provides sonic and visual connections ranging from the gentle steering of musical improvisation to more autonomous and potentially disruptive behaviour. At the heart of MSN/AV is a novel implementation of ring modulation that features two identical oscillators. These oscillators morph from silence to white noise, passing through sine and square waveforms along the way. This morphing capability enables a smooth transition from simple to complex audio spectra. 2 MSN Synthesis The term ‘maximum silence’ refers to the absence of sound in a ring modulator when either modulating oscillator is silent. To maintain modulation independent of the amplitude envelope, a DC offset NIME ’25, June 24–27, 2025, Canberra, Australia Ferguson and Brown ‘silence’ signal is introduced, ensuring consistent gain as the modulating oscillator's amplitude decreases. A ROLI Block serves as the primary gestural controller for MSN/AV. Its touchpad maps silence-tonoise transitions on the x-axis and oscillator pitch on the y-axis, while pressure controls the amplitude envelope. Two oscillators, each controlled by a finger on the multi-touch pad, are ring modulated, offering six dimensions of control for expressive sound shaping. An iPad running TouchOSC provides on-screen sliders and buttons to control audio effects and, when applicable, visual parameters. These include volume, panning automation, delay, and reverb. An MSN implementation in Pure Data can be downloaded at https://github.com/algomusic/MaxSilence-Noise. 3 Research Question(s) The goal is to explore gestural control of sound and vision while creating new artistic work that engages with rich audio spectra and the entanglement within audiovisual practices. This expressiveness in gesturally controlled ring modulation synthesis builds on a history of ‘noisy’ interactive electronic instruments like the Crackle Box1, BoardWeevil2, and Atari Punk Console3, which use ring modulation or similar techniques to produce dynamic inharmonic spectra. While ‘live visuals’ are now common in contemporary performance, the role of the musical score is evolving. Yet, aside from a few notable exceptions (Louise Harris, Myriam Bleau, Vicki Bennett, Nonotak Studio), there is little artistic research on improvising musicians in responsive audiovisual environments—an area MSN/AV seeks to explore. Our primary research questions are: ● To what extent can improvised performance with interactive technology be successfully situated in a responsive audio-visual environment? ● What creative opportunities emerge when ‘musical improvisation and ‘live visuals’ combine in an environment where the line between cause and effect is blurred? 4 Audiovisual Context A useful starting point is visual music: Oskar Fischinger’s (1938) An Optical Poem is an early example that William Moritz (1986) has used to suggest that ‘[s]ince ancient times, artists have longed to create with moving lights a music for the eye comparable to the effects of sound for the ear.’ [4] However, many early exemplars work with fixed musical and visual resources, whereas MSN/AV focuses on improvised live performance in interactive environments. The real genesis might therefore be a work like John Cage’s Variations V (1965), which was created in collaboration with the Merce Cunningham Dance Company and made use of shadows cast on walls 1 https://www.eam.se/kraakdoos/ 2 https://www.bugbrand.co.uk/product/weevils-2005-2015/ 3 https://en.wikipedia.org/wiki/Forrest_Mims MSN/AV: Maximum Silence to Noise/Audio-Visual NIME ’25, June 24–27, 2025, Canberra, Australia to trigger sound via light sensors. This resonates with what Simon Waters (2013) has termed ‘Touching at a Distance’ [5] and might also be considered via John Whitney’s (1994) notion of ‘audio-visual complementarity’. [6] These are all useful historical ideas that have helped develop this project. More recently, Cat Hope and Lindsay Vickery (2010) have furthered discussion of ‘The Aesthetics of the Screen-Score’ [3] and it is clear their Decibel ensemble is a leader in this field. Similarly, Louise Harris (2021) has examined the nature of audio-visual experience [2] with a particular emphasis on media hierarchy; our work builds upon her 2016 article ‘Audiovisual Coherence and Physical Presence’ [1], which emphasises the relationship between audio-visual media and physical (human) presence. 5 Performance / Technical notes This is envisaged as a live performance with both performers physically present @ NIME 2025. ● All of performers' gear is on-stage; ● Audio output on 2 x TRS jacks so will need adapters for 1/4” to XLR if that’s necessary for house system); ● Vision output on HDMI to house projector. Performers will supply: ● 1 laptop running Touch Designer and Pure data; ● 1 Audio interfaces with ¼” outputs; ● 2x iPad and associated peripherals; ● 2 x ROLI Blocks; Needs from house: ● Table (ideally at standing height) suitable for two humans to stand side-by-side (approximately 600 x 1800mm); ● Projector + wall projection area or projection screen; ● 3 x power outlet (power board required) 6 Media Link(s) Video: https://vimeo.com/1054018899/a050c16da0?share=copy 7 Program Notes MSN/AV (Maximum Silence to Noise/Audio-Visual) is an interactive audiovisual system performed by two musicians. The sound world is created via ring modulation synthesis controlled by multidimensional touch gestures, which provides a rich diversity of sonic potential whilst maintaining clear remnants of physical gesture. Each musician uses ROLI Lightpad Blocks and an iPad running TouchOSC in combination with bespoke software written in Pure data. Data from each performer’s interactions with their instruments is passed to a Touch Designer network which generates and/or manipulates visual materials. MSN/AV celebrates physical interplay with gestural interfaces and situates live improvisation with interactive technology within a responsive audiovisual environment. Sensor data is used to generate sound and graphics in real-time, the overall goal is an audiovisual entanglement that provides sonic NIME ’25, June 24–27, 2025, Canberra, Australia Ferguson and Brown and visual catalysts ranging from the gentle steering of musical improvisation to more autonomous and potentially disruptive behaviour. 8 Acknowledgements We acknowledge the support from various Griffith University elements; The Queensland Conservatorium, Creative Arts Research Institute, Queensland College of Art, and the Interactive Media Lab. 9 Ethical Standards This research raises no ethical concerns. References [1] Harris, L. (2016). Audiovisual Coherence and Physical Presence: I am there, therefore I am [?]. eContact! Canadian Electroacoustic Community (CEC). 18.2. Retrieved March 24 2022 from https://econtact.ca/18_2/harris_audiovisualcoherence.html [2] Harris, L. (2021). Composing Audiovisually: Perspectives on audiovisual practices and relationships. Routledge. [3] Hope, C. and Vickery, L. (2010). The Aesthetics of the Screen-Score. Retrieved March 24 2022 from https://www.lindsayvickery.com/uploads/1/7/0/8/17081762/2010hopevicaestheticsscreenscore .pdf [4] Moritz, W. (1986). Towards an Aesthetics of Visual Music. Retrieved August 20, 2019, from http://www.centerforvisualmusic.org/TAVM.htm [5] Waters, S. (2013). Touching at a Distance: Resistance, tactility, proxemics and the development of a hybrid virtual/physical performance system. Contemporary Music Review 32/2–3 (2013) ‘Resistant Materials in Musical Creativity,’ pp. 119–134. [6] Whitney, J. (1994). To Paint on Water: The Audiovisual Duet of Complementarity. Computer Music Journal, 18(3), 45-52. doi:10.2307/3681184