VS : Improvising with Vibrating Virtual Entities

VS : Improvising with Vibrating Virtual Entities SEBASTIEN BEAUMONT, Muzzix, France IVANN CRUZ, Muzzix, France ARTHUR PATÉ, IEMN UMR 8520 CNRS, Junia/ISEN, France, France FLORENT BERTHAUT, CRIStAL UMR 9189 CNRS, Univ. Lille, France, France ACM Reference Format: 1 PROGRAM NOTES In this performance, two guitar players improvise using electro-acoustic guitars equipped with actuators that can hit each of the strings separately. By moving through virtual shapes placed around them with their guitars and bodies, the guitar players can control the actuators. By using minimal modifications of the instrument and subtly extending existing playing techniques, the setup aims at preserving the technical and cultural heritage of the acoustic instrument [1]. The improvisation is rooted in two different mappings between the position of the guitar’s headstock within the virtual shapes and the actuator’s behaviour. First mapping (guitarist on the left) associates the forward-backward axis with the frequency, or tempo, of string hits: a cylindrical shape is divided into three equal-sized parts of approximately 10 cm, thus allowing for a discrete control of the frequency, at 60, 90, and 120 bpm respectively. The vertical position within the shape controls which string is hit, i.e. by discretely changing from low E (6th string) through treble E (1st string) when the headstock goes from the bottom to the top of the shape. Meanwhile, the guitarist on the right plays with a mapping that combines a vertical string selection with a continuous control of the actuation frequency on the backward-forward axis. Simultaneously using these two mappings/playing modes allows for the realisation of a musical idea that only existed in the guitarists’ thought-experiments: unique overlays of an infallibly stable rhythm (left guitar) and a high liberty and accuracy in controlling the tempo (right guitar). Later in the performance the guitarists switch from one mapping to the other by entering one of two shapes with their right hands (see figure 1). The guitar players combine the elements of traditional playing and the new rhythmic features in a counterpoint that integrates with the virtual shapes. The result is a rhythmic complexity that would be impossible to obtain with traditional playing techniques on two guitars, or even with effects such as loopers that are not as responsive and flexible as the present device. With this new rhythmic vocabulary and control interface, the improvisation then develops as the musicians get inspired and involve the virtual shapes in their playing... Licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0). Copyright remains with the author(s). Sebastien Beaumont, Ivann Cruz, Arthur Paté, and Florent Berthaut. 2020. VS : Improvising with Vibrating Virtual Entities. 1, 1 (July 2020), 4 pages. Music Proceedings of the International Conference on New Interfaces for Musical Expression NIME’22, May 31–June 2, 2023, Mexico City, Mexico 2 Anonymous Author, et al. Fig. 1. Capture from the performance. Here the musician on the right is selecting a shape-to-actuator mapping by intersecting a sphere with his right hand. The other musician is combining a traditional playing technique with 3D interaction with the virtual shape that he intersects with the guitar headstock. 2 PROJECT DESCRIPTION A first iteration of design of this project was presented by Arlsan et al. [ 1 ]. The presented version involves two electro-acoustic guitars on which a series of electrodynamic actuators are attached and respond to sawtooth waveforms controlled in frequency and amplitude. The actuators drive the out-of-plane movement of wooden slats with screws that finally hit the strings. More in details, 12-string guitars are used, so that only the lowest of each string pair is hit (see figure 2). This allows the other string to sound through sympathetic resonance, and therefore to mitigate the “dryness” of the repetitive attacks by adding a little natural reverberation. The actuators are selected and controlled in frequency by intersecting virtual shapes placed around the seated musicians using their bodies or parts of the instruments (the headstock most of the time, as it turned out to be an easy and natural gesture). These shapes are displayed using a Spatial AR display that projects the intersections back onto the bodies and instruments, as seen in Figure 1. The project therefore relies on existing instruments, with minimal transformation (a few added lightweight actuators, miniature amplifiers and a nano-computer, keeping the device’s price to a very reasonable amount) and an augmented reality (AR) display that preserves the traditional instrumental space and gestures and leaves the focus on human actions (hand and instrument movements). Contrary to other AR technologies, Spatial AR displays provide an immersive experience with affordable, simple and shared equipment, here a single projector and depth camera. The device used in VS ended up being a bootstrap for musical exploration. Since the beginning of the project in 2018, the team of researchers and musicians decided to work together at every step of the process. A method based on regular confrontations allowed us to continuously evaluate the potential for creation of the system. With each technical improvement, the musicians were able to confront it to a creation context. Being professional improvisers, the musicians used improvisation phases during each work session to test the limits and potential inadequacies to guitar playing. VS : Improvising with Vibrating Virtual Entities 3 Fig. 2. Left panel: global view of a guitar with the excitation system mounted. Right panel: Closeup of the excitation system. Five electrodynamic transducers are glued onto a wooden board attached to the soundboard; one slat is glued onto the membrane of each transducer; a screw is attached to each slat and hits one string of a pair. To this day, the project continues being developed by 1) improving the usability and robustness of the attached device, 2) adding vibrotactile feedback to help guitarists explore the intangible shapes, 3) studying the appropriation by other musicians with an anthropo-musicological approach. 3 PERFORMANCE NOTES The performers will not be able to travel to Mexico for the conference, therefore the performance will be presented as a ≈9 minutes long video. 4 MEDIA LINKS •Video: https://nextcloud.univ-lille.fr/index.php/s/NJPBHyZGnmnfYSz •Audio: https://nextcloud.univ-lille.fr/index.php/s/kCmSzLoJHQtTJeB ACKNOWLEDGMENTS The authors would like to thank the Fédération de Recherche Sciences et Cultures du Visuel, the IRCICA, the Université de Lille and Junia/ISEN for funding and/or hosting the project. 4 Anonymous Author, et al. REFERENCES [1] Cagan Arslan, Florent Berthaut, Anthony Beuchey, Paul Cambourian, and Arthur Paté. 2022. Vibrating shapes : Design and evolution of a spatial augmented reality interface for actuated instruments. In New Interfaces for Musical Expression. Auckland, New Zealand. https://doi.org/10.21428/ 92fbeb44.c28dd323