Kamer/Ton – a system for analog audio-video feedback

This work is licensed under a Creative Commons Attribution 4.0 International License. NIME ’25, June 24–27, 2025, Canberra, Australia © 2025 Copyright held by the owner/author(s). Kamer/Ton – a system for analog audio-video feedback KRZYSZTOF CYBULSKI, Feliks Nowowiejski Academy of Music in Bydgoszcz Program Notes Kamer/Ton is an experimental audio-visual performance tool, based on audio and video feedback phenomena combined into a single feedback loop. It consists of a couple of devices: two custom-made cameras are directed at a CRT monitor - the picture is transformed into sound, which is consequently sent to a speaker; a microphone directed towards the speaker is in turn connected to a custom-made video synthesizer, producing images displayed on the monitor. The resulting audio-visual feedback can be interacted with by modifying camera and microphone positions and parameters, which influences simultaneously generated, perfectly synchronized sonic and visual structures. Fig. 1. Kamer/Ton in action; Project Description Introduction Audio feedback (Larsen effect) and video feedback are well known phenomena that have been explored for artistic purposes since at least the 1960s by artists such as Steve Reich, Alvin Lucier, Jimi Hendrix, and Woody Vasulka. Beyond the aesthetics, the complexity of video feedback was also a subject of scientific analysis [1]. Audio feedback currently experiences increased interest as a means of creating semi-autonomous, responsive musical instruments [2]. Similarly, video feedback also experiences a modest revival, e.g. in the work of polish artists and scholars such as Mila Nowacka and Jan Dybała. One of the reasons for this might be the key element of many feedback systems, which is its intrinsic interactivity and responsiveness to minor adjustments. As observed by Magnusson et al, „(…) feedback musicianship is not about exercising control but rather about giving it away: about playing »with an instrument« and not »playing on an instrument«.” [3] Video from audio and vice versa Over the last couple of decades there have been a number of attempts at sound and music visualization, such as XIX and early-XX century mechanical light organ (with colors arbitrarily assigned to certain musical notes)[4], through consumer-oriented solutions such as simple electronic light organ, more sophisticated timeand frequency-domain based commercially available sound visualizers, to digital sound visualizers such as the Atari Video Music [5], LAVA! Player [6] or Windows Media Player. For artistic purposes, since the early days of video art, there have been a number of custom solutions, NIME ’25, June 24–27, 2025, Canberra, Australia Krzysztof Cybulski such as video colorizers / manipulators / synthesizer that could take up audio as a modulation signal [7] as well as recent attempts at combining vintage technology with modern outlook, such as diy audiovisual synths [8], digitally-aided oscilloscope music [9] or sound-controlled mechanical manipulation of video feedback [10]. Direct use of an audio input on analog video display is also possible and would create simple, time-domain based visualizations consisting of alternating horizontal black and white stripes; this technique requires the audio signal to be connected to a video input of a device such as video mixer for generating a proper composite video signal (including sync pulses) in order for the video to be displayed on a standard monitor/projector. The opposite process - image sonification - has an almost equally long history. Although a number of technical solutions exist, one of the prevailing methods is the use of a photocell, directly translating the changing brightness of a light source into alternating electric current, which is then amplified and sent to a speaker. This approach has been used for soundtrack recording on a film, but has been explored artistically to great lengths by filmmakers such as Oskar Fischinger [11] and Norman McLaren [12], or in a form of musical instruments, such as the ANS Synthesizer [13] and commercially available Optigan [14]. The optical sound generating technique had been also incorporated in recent projects by Ei Wada [15][16]. The analog composite video signal can be used directly as sound signal (amplified and sent to a speaker), although the results are rather monotonous and uninteresting. To the best of my knowledge, the combination of both approaches - a direct audio-to-video-to-audio feedback loop - hasn’t been practically explored up to this point, although some recent works are getting close to combining the two processes. One notable example was the Anatomie einer Waage project by Robert Schalinski [17], seemingly combining audio and video feedback by placing a camera with a microphone in front of a CRT monitor with a built-in speaker, although in practice the audio and video feedback was occurring independently. A seemingly similar process takes place in two works by Ei Wada – Telelele [18] and CRT TV Drums [19], where a performer controls a video synthesizer generating striped pattern on a CRT monitor, which is then sonified. In both cases, however, the sonification relies on a magnetic field pickup, rather than a light sensor. Technical description Fig. 2. Basic setup and signal flow of Kamer/Ton Kamer/Ton NIME ’25, June 24–27, 2025, Canberra, Australia Kamer/Ton consists of a couple of regular, unmodified devices, such as a Sony PVM CRT video monitor, Shure SM57 dynamic microphone and an active speaker. However, two main elements of the system had to be conceived and built from scratch – the camera and video generator. Eventually two slightly different cameras were built, together with a simple, passive audio mixer/switcher for combining signals from both cameras. From my previous experience with both audio and video feedback setups, I assumed that a simplest possible way of generating audio from video and video from audio should be beneficial to the responsiveness of the resulting feedback loop, thus, while designing working principles for the camera and video generator, I opted for as straightforward audio-video-audio translation as possible. Hence the means of operation of each individual device is rather simplistic, nevertheless the combination of these devices in a feedback loop creates sufficient complexity. As already mentioned, in a composite video format, audio and video signals are interchangeable to some extent – video signal connected to an audio input will produce audible buzz, the timbre of which changes in relation to the video content. However, in order to connect the audio signal to a video output, the signal has to be mixed with sync pulses, which can be provided by a video mixer or other type of composite video signal generator. In the case of Kamer/Ton the latter strategy was chosen - a custom video generator has been built, based on Arduino Uno board, running the TVout library [20]. The library has been chosen as it outputs video and sync signals on two separate Arduino pins. An example sketch has been modified such that the video output produces stationary white rectangle; the signal from video pin is fed through a couple of transistors, which allow the audio signal from a microphone to modulate the video signal from Arduino. A slide switch, modifying the configuration of transistors, allows to choose whether the sound wave from a microphone produces positive (white stripes on a black background) or negative image. The negative setting has an advantage that silence results in a white image, which in turn immediately produces sound of maximum volume in the cameras, hence avoiding “dead spots” (dying out of the feedback). Fig. 2. General view and interior of the basic camera The basic camera consists simply of a C-mount camera lens, a single LDR wired with a potentiometer to create a voltage divider, a single-board microphone preamp and a 9V battery. The components are mounted inside a narrow aluminum enclosure, with the lens screwed into a 3d-printed cap on one end, and an on-off switch and jack output attached to another 3d-printed cap on the other end. The camera output can be connected directly to a line input of an active speaker and the movement of the speaker cone corresponds to the changes in the intensity of light, observed with the lens. A light source that flickers with a frequency greater than 20 Hz produces audible pitch. When directed at a video monitor, NIME ’25, June 24–27, 2025, Canberra, Australia Krzysztof Cybulski the camera produces sound with a frequency corresponding to the refresh rate of the monitor, which in case of the NTSC system is 60 Hz. The camera can be held in performer’s hand or secured on a modified desktop tripod. A second camera has an additional element – a rotating shutter with vertical slits. The shutter is attached to a DC motor with variable speed, controlled with an Arduino Nano board with an encoder for coarse and potentiometer for fine speed control. The shutter introduces amplitude modulation, thus an audible frequency can be created even from a non-flickering light source; in case of flicker, (i.e. video monitor) the shutter modulates the refresh rate frequency, introducing additional pitch or bringing up certain harmonics of the original frequency, depending on the shutter rotation speed. As the orientation of the shutter slits in relation to the vertical lines on the video monitor considerably influences the audio/video feedback, the camera has an integrated stand, which allows for a 360-degree rotation of the camera around the lens axis, facilitating the angle manipulation during performance. Fig. 3. General view and interior of the shutter camera The outputs of both cameras are combined by a purpose-built passive audio mixer with a volume control and both momentary and toggle switches for each input; additional global hi-pass filter allows further modification of the audio/video feedback. For performance purposes, the setup is expanded by the use of two stompboxes – compressor and graphic equalizer (Behringer CS400 and BEQ700, respectively) – plugged between the microphone and video generator. The stompboxes are used to bring up certain harmonics or to even out the dynamics – the visual results of the audio compression are similar to using the contrast knob on a video monitor. Fig. 2. Expanded setup and signal flow of Kamer/Ton Kamer/Ton NIME ’25, June 24–27, 2025, Canberra, Australia Interaction and performance The basic means of interaction is the physical manipulation of camera and microphone placement. Different camera angles and lens/iris settings, in combination with microphone-to-speaker placement variations, create various feedback artifacts, such as suppressing or bringing up certain harmonics, introducing distortion and clipping (resulting in a change of contrast of the video image). The position of black and white patterns on the video monitor follows the movements of the camera to some extent, which contributes to interesting audiovisual performative gestures. For the performance, Kamer/Ton is accompanied by a shaker and tuning forks, tuned to specific frequencies creating certain visual patterns on the video monitor, which serve as a starting point for the performance. The sound of the acoustic percussion instruments is being gradually processed via the audio-visual feedback loop, which eventually is used for the majority of performance as a sole audio-video source. Fig. 3. Actual performance setup, with additional elements (two stompboxes and percussion instruments; second microphone for amplification purposes); Performance Notes Below follows a list of equipment / facilities that need to be provided by the venue: • 9’’ – 14’’ CRT monitor with NTSC composite video input (BNC or chinch); preferably Sony PVM; • video projector with NTSC composite video input; • large standard-height table; • 4 x 230 v electrical socket; • single mono ¼’’ jack TRS cable for amplification; • dynamic microphone on a stand for additional percussion instrument amplification; There’s no need for audio monitoring, as a small speaker is an integral part of the Kamer/Ton system. The setup requires about 30-60 minutes, the soundcheck itself shouldn’t take more than 30 minutes. The ideal performance duration for the NIME performance would be 10 minutes. NIME ’25, June 24–27, 2025, Canberra, Australia Krzysztof Cybulski Media Links • Video: https://vimeo.com/1080203114/ab3ec0f584 • Additional videos and photos: https://krzysztofcybulski.com/kamerton.php Acknowledgements The project has been created under a scholarship from the Ministry of Culture and National Heritage of the Republic of Poland. Ethical Standards This research didn’t involve any human or animal participants. References [1] J. P. Crutchfield, Space-time dynamics in video feedback. In Physica D: Nonlinear Phenomena, 10(1-2), 229-245, 1984 [2] Feedback Musicianship Network. https://feedback-musicianship.pubpub.org/ Retrieved 30 Apr 2025. [3] T. Magnusson, C. Kiefer and H. Ulfarsson, Reflexions upon Feedback, in Proceedings of the International Conference on New Interfaces for Musical Expression, 2022. [4] M. McDonnell, Visual Music. http://homepage.tinet.ie/~musima/visualmusic/visualmusic.htm Retrieved 30 Apr 2025. [5] Atari Video Music. https://en.wikipedia.org/wiki/Atari_Video_Music Retrieved 30 Apr 2025. [6] Lava Player. https://web.archive.org/web/20000619224731/http://lava.com/ Retrieved 30 Apr 2025. [7] P. Weibel, W. Vasulka, S. Vasulka, D. Dunn, Eigenwelt Der Apparate-Welt, Pioneers of Electronic Art. Ars Electronica 1992 [8] G. Gieskes, Oscillatoscope. https://gieskes.nl/visual-equipment/?file=oscillatoscope2 Retrieved 30 Apr 2025. [9] Oscilloscope Music. https://oscilloscopemusic.com/ Retrieved 30 Apr 2025. [10] G. Gieskes, Feedback Video Log. https://gieskes.nl/visual-equipment/?file=feedback_video_log Retrieved 30 Apr 2025. [11] Center for Visual Music, Oskar Fischinger. http://www.centerforvisualmusic.org/Fischinger/ Retrieved 30 Apr 2025. [12] National Film Board of Canada, Normal McLaren. https://www.nfb.ca/directors/norman-mclaren/ Retrieved 30 Apr 2025. [13] S. Kreichi, The ANS synthesizer: Composing on a photoelectronic instrument. Leonardo (1995): 59-62. [14] Optigan. https://en.wikipedia.org/wiki/Optigan Retrieved 30 Apr 2025. [15] E. Wada, Electric Fan Harp. https://eiwada.com/projects#/electric-fan-harp/ Retrieved 30 Apr 2025. [16] E. Wada, Barcoder. https://eiwada.com/projects#/barcoder/ Retrieved 30 Apr 2025. [17] Column One, Anatomie einer Waage. https://columnone.bandcamp.com/track/anatomie-einer-waage Retrieved 30 Apr 2025. [18] E. Wada, Telelele. https://eiwada.com/telelele Retrieved 30 Apr 2025. [19] E. Wada, CRT TV Drums. https://eiwada.com/projects#/crt-tv-drums/ Retrieved 30 Apr 2025. [20] Jovinobunny, Arduino-tvout. https://github.com/Avamander/arduino-tvout Retrieved 30 Apr 2025.