Glass and Fans: Spatial Infra-Instrument Framework

Glass and Fans: Spatial Infra-instrument Framework MEL HUANG BUNTINE, University of Melbourne, Australia DR ANTHONY LYONS, University of Melbourne, Australia DR HEATHER GAUNT, University of Melbourne, Australia EUGENE UGHETTI, Speak Percussion, Australia Additional Key Words and Phrases: Instrument, Infra-instruments, Spatial Interaction, Augmented Instruments, Co-creativity, HumanComputer Interaction, Installation, Interactive Music Systems Fig. 1. (left) Fans and Glass, (top right) glass object/s, (bottom right) 12V computer fan 1 Program Notes Glass and Fans are two participatory spatial sound installations created with the Spatial Infra-Instrument Framework (SIF), investigating how interactive interfaces like SIF can expand the scope of music-making to include collaborative, non-linear processes that can evolve with their artists, audiences, and the public over time. SIF utilises open-source tools such as openFrameworks and Kinect v1 to provide a low-cost and flexible framework that supports and encourages the education and artistic practice of building New Interfaces for Musical Expression (NIMEs). Each SIF iteration is a stand-alone structure and has two performance modes: •Machine mode, instrument performs as the composer has specified •Human mode, instrument becomes interactive for audience Glass takes a series of fragile, hand-blown glass percussion and provides access to their tonal and textural qualities. Machine Mode plays audio and visual samples as programmed by the composer, Human Mode invites audiences to compose within the same ecosystem through rear projection and set of spatial triggers. Authors’ Contact Information: Mel Huang Buntine, , University of Melbourne, Melbourne, VIC, Australia; Dr Anthony Lyons, , University of Melbourne, Melbourne, VIC, Australia; Dr Heather Gaunt, , University of Melbourne, Melbourne, VIC, Australia; Eugene Ughetti, , Speak Percussion, Melbourne, VIC, Australia. This work is licensed under a Creative Commons Attribution 4.0 International License. NIME ’25, June 24–27, 2025, Canberra, Australia ©2025 Copyright held by the owner/author(s). NIME ’25, June 24–27, 2025, Canberra, Australia Buntine et al. Fans translates the everyday material of computer fans into an infra-instrument. Machine mode performs with the “tuned” fans at set speeds and intervals. Human Mode allows audiences to compose with the electromagnetic field (EMF) microphones by changing their distance and position. 2 Project Description This installation explores the entangled relationships between technology, creativity, accessibility, and participation through Glass and Fans, two interactive spatial sound installations developed using the Spatial Infra-Instrument Framework (SIF). Rooted in Percy Grainger’s vision to “blur the distinctions between performance, composition, and listening”[ 5 ], SIF transforms everyday materials—such as glass and fans—into versatile instruments, enabling participatory engagement for composers, performers, and the public, regardless of prior musical or technical experience. As described by Hugill, Grainger “foresaw that bricolage, often using fairly cheap and readily available technologies, would become central to ... a ‘democratic’ approach to music-making” [ 5 ] in comparison to observations from Roser et al. that “emerging technologies are often expensive and, therefore, initially limited to society’s richest. A key part of technological progress is making... these innovations affordable for everyone”[9]. SIF embodies the principles of: • Flexibility; to bend and mold to individual artistic practices by utilising high performing, open-source toolkits such as openFrameworks to interpret real-time spatial data to endless amounts of actions • Adaptability; we didn’t want to limit ourselves to the body as the input (e.g. hand-tracking, skeleton-tracking) so we decided on depth tracking which allows for multiple performers and/or objects as input • Affordability; we selected the Kinect v1(USD 10 in 2025[ 4 ]) over cameras such as Intel Realsense (USD 272-499 in 2025[ 6 ]) for cost-efficiency to allow for multi-camera installations to run concurrently. In addition, the cost affords a freedom for "hardware hacking"[3] should the urge arise. •Accessibility; to both composer, regardless of technical ability and performer/s, regardless of musical knowledge SIF is an effort to support the multilayered creative and technical contexts involved with building new instruments by aligning with an infra-instrument tenet of the building with the “deployment of few sensors”[ 1 ]. However, the framework is not solely focused on the technology itself, but fostering artistic and educational opportunities and communities for creating New Interfaces for Musical Expression (NIMEs). Its dual performance modes — Machine Mode, in which the instruments autonomously perform preprogrammed compositions, and Human Mode, where audience interaction shapes the sonic output—reflect the evolving and reciprocal interactions between users, instruments, and environments. Glass reimagines hand-blown glass percussion objects, drawing on Grainger’s experiments with “musical glasses”(Fig. 3, left). The sampling of a unique and historically important glass instrument collection and incorporation within the Spatial Instrument Framework enables new forms of engagement. For composers/performers or lay audience alike, movements trigger sonic explorations of the unique tonal and textural qualities of these fragile instruments, overcoming their inherent fragility through innovative sampling and interaction. Fans repurposes computer fans into a matrix of sonic possibilities. Audience engagement with the fans generates and modulates electromagnetic frequencies, creating an immersive interplay of acoustic, electro-acoustic, and virtual sound layers. Together, these installations highlight the entangled dynamics of interaction, materiality, and sound, embodying a creative dialogue that spans technology, heritage, and experimentation. By providing accessible frameworks for participatory engagement, SIF bridges the gap between professional and public music-making, fostering deeper connections between musicians, audiences, and the evolving possibilities of new technology. 2.1 The system A Kinect v1 camera is mounted from above to measure the depth at one or more chosen 3D (x,y,z) point/s within the active zone (Section 4, 00:08 in video): x-axis runs left to right, y-axis measures depth and distance from surface and z-axis runs top to bottom measuring the distance from Kinect. Each designated point (see Figure 2, left) works in a loop: (1) Measure depth (y) at coordinates (x,z). (2) If desired depth is true, go to 3. If desired depth is false, return to 1. (3) Disable point and run action. Go to 4. (4) Wait nmilliseconds then enable point. Return to 1. The logic for actions can be simple, i.e. using a depth point to trigger a MIDI note, or as complex, i.e. controlling the voltage depending on the triangulation calculations of multiple points and objects in 3D space, as the composer/s require. Glass and Fans: Spatial Infra-instrument Framework NIME ’25, June 24–27, 2025, Canberra, Australia Fig. 2. Screenshots from demo video (see Section 4): (00:08) Active zone and structure, (00:11) Spatial co-ordinate system, (00:18) Glass and Fans installation and features, (00:30) Metal marimba demos, (00:33) Past performance and system inspiration, (00:47) (x,y,z) trigger random sample, (01:02) (x,y,z) trigger a vertical marimba note sample, (01:18) Theremin demo; distance from Kinect (z) triggers pitch shift, (01:27) Glass progress demo, (01:29) Past performance and inspiration, (01:37) (x,y,z) unmutes/mutes audio and visual sample, (01:48) unmute/mute in swiping motion, (01:56) improvisation test, (03:39) Fans progress demo, (03:40) Past performance and inspiration, (03:51) Motor sound test and construction test, (04:34) Motor drivers; composing with speed, (05:09) EMF microphones; testing latent space between fans 2.2 Metal Marimba; a demo becomes a framework SIF extends upon Living Instruments[ 8 ] which ‘involves mapping real instruments (by sampling their sound) to new tactile digital interfaces that anyone can use’[ 8 ] (see Section 4, 00:33 in video for demo performance[ 7 ]). The instruments were sampled from the Grainger Museums’s, collection as a way to capture, hear, catalogue and preserve fragile and rare objects. As a first example, SIF created its own metal marimba demo (Section 4, 01:02 in video) which uncovered spatial data’s innate quality of being ‘free of the limitations of the human hand’[ 5 ]. We began to explore other instruments like the theremin (Section 4, 01:18 in video), principles from Grainger’s Free Music machines, graphical scores and Grainger’s sonic vision for ’tonal glides and curves’[ 5 ]. As a result, SIF expanded from being a one-time spatial interactive metal marimba into a larger framework to support educational and artistic pursuits. 2.3 Glass; for education The vertical orientation of the active zone means that some instruments are more suited than others to adapt to the framework. The Glass Percussion Project was inspired by Percy Grainger’s number of works using tuned and untuned hand-held glass objects [Fig. 3, left]. Being that the original glass objects were too fragile to sample, over 1500 new and unique custom-made glass percussion objects were commissioned and performed (Section 4, 01:27 in video for performance clip) as part of the project with 550 acquired by the Grainger Museum in 2020. The unique qualities of these glass instruments, including their materiality (high fragility), aethereal visual qualities, and enormously diverse sonic opportunities through percussive techniques (sticking, stroking, scraping, etc.), will be sampled in April 2025 as part of a course subject at the The University of Melbourne. This involves sampling a subset of the glass instruments with a range of percussive techniques safely in a studio environment. The audio and visual samples will then be loaded into Glass providing access to these fragile objects for composition, performance and listening. 2.4 Fans; for artistic practice Although visual cues are useful for orientation, the SIF framework is not dependent on a screen (see drawn lines - Section 4, 01:18 in video) nor a single action, i.e. to trigger A/V via MIDI. As part of Speak Percussion’s Transducer, which placed the microphone at the center of the creative investigation exploring its potential as an expressive musical object, Eugene Ughetti was experimenting with inaudible electromagnetic field (EMF) frequencies and computer fans (see Section 4, 03:40 in video). Fans is a multi-layered acoustic, electro-acoustic and virtual infra-instrument created using SIF for installation at an upcoming September 2025 exhibition. It consists of a matrix of 90mm computer fans. These “tuned” fans omit a score through motor drivers which change the speed of individual fans in time with one another. These affect their frequencies which are being read through EMF microphones which audiences can move to augment the sonic composition. We are currently developing the spatial component of the work to explore the manipulation of the analog sound sonically and/or physically. We are also exploring other augmentations such as physical interventions, for example, NIME ’25, June 24–27, 2025, Canberra, Australia Buntine et al. Fig. 3. (left) Percy Grainger’s annotated ”Out of tune” glass [2], (right) Glass Percussion rods inserting/brushing wooden skewers on the surface of the fan, the inclusion of magnets to manipulate the EM field, and/or the inclusion of Grainger’s idea of the “controller, which may be mapped onto a given musical parameter”[ 5 ] akin to self-playing pianola music rolls. 3 Technical Notes The two infra-instruments, Glass and Fans will stand side by side to demonstrate various translations of SIF. The installation will require a minimum of a 3 x 3 x 2.3m (W x L x H) in a semi-dark space to allow for projection. For cable management, a corner is preferable to avoid trip hazards. The installation may include a small number of original glass percussion objects (Fig. 4, G5). Fig. 4. Technical specifications of Glass [G0] and Fans [F0]. For key information, see Table 1 for labels 4 Media Links A video containing demonstrations and more information is available: https://doi.org/10.5281/zenodo.15293072 Glass and Fans: Spatial Infra-instrument Framework NIME ’25, June 24–27, 2025, Canberra, Australia Key Equipment name Artist to provide NIME to provide* G1 Kinect v1 ✓ G2 Projector (1080x1920px, portrait) and stand (min. height 120cm) ✓ G3 2 x Speakers ✓ G4 Mac Mini ✓ G5 Glass object display (TBC) G6 Table and light F1 Kinect v1 ✓ F2 Audio interface (min. 6 mic input, 2 output) ✓ F3 2 x Speakers ✓ F4 Mac Mini ✓ F5 24 x 12V computer fans, 4 x Arduinos and components ✓ F6 6 x Electromagnetic Field Microphones ✓ - Floor plinths for speakers and tech equipment (optional) ✓ Table 1. List of Equipment. See Figure 4 for visual key. * Where NIME cannot provide the specified equipment, alternative arrangements may be possible in consultation with author/s 5 Ethical Standards All audio-visual samples presented as part of Glass will be created by students enrolled in a class for credit or external volunteers. The authors do not believe there are any potential conflicts of interest, whether financial or non-financial. Our work does not have any implications that would require a research ethics review (e.g. the research does not involve the participation of animal subjects, human or otherwise). This project does not involve any form of personal data collection. Acknowledgments The authors would like to thank Grainger Museum, Elaine Miles, Speak Percussion and Monica Lim for their wealth of knowledge and appetite for supporting and piloting experimental and educational outcomes. This work was supported by the School of Computing and Information Systems, Faculty of Engineering and IT and Museums and Collections at The University of Melbourne. This work is further supported by the students and teaching team from Interactive Composition, Melbourne Conservatorium of Music, Faculty of Fine Arts and Music, The University of Melbourne, Australia. References [1] John Bowers and Phil Archer. 2005. Not Hyper, Not Meta, Not Cyber but Infra-Instruments. In Proceedings of the International Conference on New Interfaces for Musical Expression. https://doi.org/10.5281/zenodo.1176713 [2] Grainger Museum collection. 1930s. Musical glass created by Percy Grainger, c. 1930s. Retrieved January 26, 2025 from https://omeka.cloud.unimelb. edu.au/grainger/items/show/398 [3] Nic Collins. 2006. Handmade Electronic Music: The Art of Hardware Hacking. Routledge, New York. [4] eBay. 2025. Sold prices for Kinect v1. Retrieved Feb 1, 2025 from https://www.ebay.com.au/itm/196570797248?_skw=xbox+360+kinect+v1& itmmeta=01JJZSGY5Y6P68PQR45JZM9R4G&hash=item2dc48854c0:g:HwwAAOSwIpJmYm6v&itmprp=enc%3AAQAJAAAA8HoV3kP08IDx% 2BKZ9MfhVJKmOb8oEJltMKGJjZ2j0LMzlsX5BlP29ovkwjsbp7OFmgbzyKMz1JY4lniIhkuuEW%2B44BNlWe4N78js5OfZA7YO2v2QZ9Iwm0o% 2FPpgpObyYvqvdLHQAnrLIl0PdGp1QwaU6KLicTfkw1Ztp0T4sPdTbDMmpKkNs%2Fawo9UnHTYTF7utK2vWI5aEGZ1uyCfWFuhtYh% 2BjdubVaMkknddWSEMx2xoUah5TV1EudHSTwyBLQswyFqZSGFk0EcpV6szjcvPjL5vejyj5Fv0ZXJnKi346nVyD5dSKh95L3K%2BsRsTRpybA%3D% 3D%7Ctkp%3ABk9SR4rjw_mXZQ [5] Andrew Hugill. 2015. Percy Grainger: A Pioneer of Electronic Music. In Grainger the Modernist, Suzanne Robinson Kay Dreyfus (Ed.). Routledge, Chapter 14. https://doi.org/10.4324/9781315585772 [6] Intel. 2025. Inter Realsense Online Store. Retrieved Feb 1, 2025 from https://store.intelrealsense.com/?_ga=2.99203166.1553259649.1738381439477182553.1737598881 [7] University of Melbourne. 2021. Living Instruments audio demonstration by Reuben Cumming. Retrieved January 26, 2025 from https://vimeo.com/ 542602220 [8] University of Melbourne. 2021. The Living Instruments Project. Retrieved January 15, 2025 from https://livinginstruments.grainger.unimelb.edu.au/ [9] Max Roser, Hannah Ritchie, and Edouard Mathieu. 2023. Technological Change. Retrieved January 26, 2025 from https://ourworldindata.org/ technological-change