ALÉAS: La connaissance ruisselée des solides / Streamed knowledge of solids, for hyper-bass-flute, live electronics, interactive video and recited poetry

ALÉAS: La connaissance ruisselée des solides / Streamed knowledge of solids, for Hyper-Bass-Flute, live electronics, interactive video and recited poetry CLÉO PALACIO-QUINTIN, Centre for Interdisciplinary Research in Music Media and Technology (CIRMMT), Canada Additional Key Words and Phrases: hyper-instrument, hyper-flute, live electronics, interactive video, poetry ACM Reference Format: 1 PROGRAM NOTES ALÉAS is a cycle of 12 transdisciplinary creations, which combines my music (acoustic and electroacoustic) and my visual creations (drawings, collages, objects, photographs and videos) with the poetry of Thierry Dimanche. Stemming from cosmological reflections, these works question the conception that we have of the universe that surrounds us, through poetic journeys exploring nature and matter (including the five fundamental elements: water, air, earth, fire, and cosmos). This creative work is inspired by the concept of diagonal sciences of Roger Caillois, a form of interdisciplinarity of knowledge, in which the invisible, the underground relations must be exhumed in order to allow the emergence of a new and more complex image of the universe [ 1 ]. Thus, in these compositions, the intertwining of sounds, images, and poetry triggers multiple perceptions in synesthesia, blurring the boundaries between reality, illusion, and imagination. The composition Streamed knowledge of solids is inspired by the materials that form the earth. Although they seem at first sight to be inert substances, rocks carry within them their whole history. A multitude of elements from the past are inscribed in their geological constitution, shaped over millennia by melting, sedimentation and water runoff. The structure and shape of each stone tells us where it came from and how it came to end up in that particular and unique shape. Matter can thus reveal profound knowledge. 2 PROJECT DESCRIPTION The composition ALÉAS: La connaissance ruisselée des solides / Streamed knowledge of solids is for solo Hyper-Bass-Flute: an augmented bass flute on which multiple sensors attached to the instrument capture the subtle tactility and movements of the performer. Thus, the performer interacts with live electronic processing of the flute’s acoustic sound, as well as with the photo and video images that are mixed in real time and vary constantly. All electroacoustic sounds of the piece are generated in real time from the sounds of the live performed bass flute. Only the voice of the poet was recorded to be added to this performance. Licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0). Copyright remains with the author(s). Cléo Palacio-Quintin. 2024. ALÉAS: La connaissance ruisselée des solides / Streamed knowledge of solids, for Hyper-Bass-Flute, live electronics, interactive video and recited poetry. 1, 1 (September 2024), 4 pages. Music Proceedings of the International Conference on New Interfaces for Musical Expression NIME’24, 4–6 September, 2024, Utrecht, The Netherlands 2 Palacio-Quintin The tactility of the performer’s hands on the flute is at the core of the Hyper-Bass-Flute performance. This highly interactive instrument gives to the performer the ability to sculpt the sounds as well as the images during the performance. Thus, there is a direct connection between the tactility of the instrument in its hybrid multimedia environment. 2.1 The Hyper-Bass-Flute Since 1999, Cléo Palacio-Quintin has been performing on the Hyper-Flute [ 3 ]. Interfaced to a computer by means of electronic sensors, a Microlab and a MIDI interface, this extended flute enables a close gestural relationship between the performer and the computer. To maintain the playing expertise developed over the years on the Hyper-Flute, we tried to build a similar system on the bass flute (in 2008-2009), but the design needed to be adapted in relation to the physicality of the playing of the bass flute [4]. The bass flute has the advantage of being much bigger than the standard flute, so there is more space to attach sensors to it. Nevertheless, the weight and size of the instrument changes its holding technique, and limits the capacity of the thumbs to reach different sensors while playing, as is done on the Hyper-Flute. The bass flute has a very rich sound palette, in some ways quite different from that of the standard flute. The different registers of the bass flute have distinct tone colors, while the standard flute has a much more homogeneous tone color over all of its registers. The much bigger pipe and the large key mechanisms produce more noise artifacts while playing, so the ratio of noise versus sound has a wider range of possibilities on the bass flute. Extended techniques using percussive or noisy sounds can be created more efficiently due to the size of the instrument. From a performer’s perspective, there his great interest in adding sensors to the bass flute, so that it gives the player the possibility to interact with digital sound processing, and thus extend its sound palette even further. Much more space is available on the instrument, so it enabled the installation of the entire sensor system on the flute itself (without using any external interface). Upon conceiving the Hyper-Bass-Flute, the idea was to make a system comparable to the Hyper-Flute but using a small Arduino interface that can be secured to the instrument. In this way, only a single USB cable connects it to the computer. Experience working with the Hyper-Flutes showed that sensors that do not disturb fingering technique while performing are very useful to control different sound processing parameters (i.e. sensors capturing ancillary gestures). An ultrasonic rangefinder and an two-axis accelerometer are thus installed on the Hyper-Bass-Flute to give multidimensional data about overall movements and position. These ancillary gestures are also visible by the audience, helping them to understand the interaction of the computer with the physical activity of the performer [5]. 3 PERFORMANCE NOTES Generally, two different types of interaction using the sensors were developed to compose music for these augmented instruments: the instrumentalist either directly and consciously controls the computer’s sound processing parameters while performing, or gestural data are captured and used to activate various processing algorithms. In the latter situation, the performer does not consciously “control" the computer’s processing, but instead “feeds" it with data which is used to create the electronic processing or synthesis that accompanies the instrumental performance. Otherwise, the computer can also play with aleatory algorithms that are not related to the sensors data. The performer has thus absolutely no control on the output and must react to the propositions given by the machine. For example, in some parts of the composition ALÉAS: La connaissance ruisselée des solides / Streamed knowledge of solids , the text recited consists of randomly chosen words from the written poems. For this performance, the recited poetry was recorded (with the voice of the author, Thierry Dimanche). Some parts are played back as recited (as written), ALÉAS: La connaissance ruisselée des solides / Streamed knowledge of solids, for Hyper-Bass-Flute, live electronics, interactive video and recited poetry 3 Fig. 1. Image from the video projection of ALÉAS: La connaissance ruisselée des solides / Streamed knowledge of solids but for the improvised sections, the computer is choosing what word to play, when, how loud, and where in the stereo sound image. The flutist, also improvising, must then adapt her playing to these unpredictable outputs. The same happens with sound processing. Some parameters are directly controlled by the performer through specific rehearsed gestures, while others are chosen by the algorithms, and vary at each performance. The performer must then adapt her playing techniques, both sound and gesture, according to the reactions of the computer at any moment. The processing of images also uses combinations of different types of interaction. The overall form of the composition is fixed with four main sections displaying different videos. Photographic images are mixed in real time with these videos. The photos are chosen randomly within several predefined banks, but the gestural data influence when those images change, and the moving mixing of those images with the video. Different combinations of interaction approaches have been used to create this interactive composition, in which both the performer and the computer play an active role. As mentioned by Joel Chadabe, in his fundamental overview about Interactive composing:“The computer responds to the performer and the performer reacts to the computer, and the music takes its form through that mutually influencial, interactive relationship" [2]. REFERENCES [1] Roger Caillois. 2008. Œuvres. Gallimard, Paris. [2] Joel Chadabe. 1989. Interactive composing: An overview. In The Music Machine: selected readings from Computer Music Journal, C. Roads (Ed.). MIT Press, Cambridge-London, 143–148. [3] Cléo Palacio-Quintin. 2003. The Hyper-Flute. In Proceedings of the Conference on New Interfaces for Musical Expression. 206–207. [4] Cléo Palacio-Quintin. 2008. Eight Years of Practice on the Hyper-Flute: Technological and Musical Perspectives. In Proceedings of the Conference on New Interfaces for Musical Expression. 4 Palacio-Quintin [5] Cléo Palacio-Quintin. 2017. 2008: Eight Years of Practice on the Hyper-Flute: Technological and Musical Perspectives. A NIME Reader: Fifteen Years of New Interfaces for Musical Expression (2017), 335–351.