Absence — Pe o mance o Audio–Visual Conca ena i e Syn hesis
DIEMO SCHWARZ, STMS I cam–SU–CNRS, F ance
Addi ional Key Wo ds and Ph ases: audio- isual mul i-modal pe o mance audio– isual pe o mance in e -modal analogies c oss-
modal pe cep ion conca ena i e syn hesis in e ac i e isuals elec o-acous ic music sound–image symbiosis
ACM Re e ence Fo ma :
1 PROGRAM NOTES
The piece Absence e okes se e al no ions o absence: Absence o ligh , absence o lo e, absence o humani y, when
socie ies and all human a e ac s will be des uc u ed o g adually disappea wi hin he ma e ials and ex u es o he
na u al wo ld, which, by hemsel es, will e en ually dissol e back in o p imo dial cosmic dus .
The piece s a s wi h he images o c epuscula ci ies, he buildings me e shadows, be o e e ol ing owa ds images
o ci ies in uins, wa ning o impeding and e y possible ca as ophies o humankind, o which sounds a e played ha
e oke emp iness and desola ion. These elemen s g adually dissol e in o he basic ex u es o na u al ma e ials o s one,
sand, wa e , wood, wi h he sounds explo ing di e en sensa ions o ma e iali y h ough oughness and ex u es.
This pa h h ough di e en sen imen s o absence is e oked in sound and images by a jou ney h ough in e wined
co po a o sounds and images, linked by hei c oss-modal aes he ic p ope ies o colou , ex u e, imb e, played ia
dynamic ges u al con ol.
The spa ial p ojec ion o sound is an impo an pa o he a is ic esea ch o his piece, un olding ac oss di e en
le els: One laye o he elec onic sound is based on ield eco dings, which a e econ ex ualized, ans o med, and
p ojec ed li e in o space by ges u al con ol. Ano he laye o sound consis s o a p e-composed elec o-acous ic
sound ack ha c ea es he wo k’s basic ambience and imme sion by dynamic su ound sound spa ialisa ion.
The na a i e o he piece e lec s i sel in he unde lying digi al ab ic o he wo k inso a as he ma e ial na u e o
he physical wo ld is ep esen ed in he o m o acous ic and isual agmen s (sound g ains and s ill images) ha ac as
a conglome a e o pa icles o o m he shape o he whole. This symbiosis o di e en modes and le els o pe cep ion
is nou ished by he desi e o subjec i i y in o de o emphasize he ela i i y o all expe ience.
2 PROJECT DESCRIPTION
The piece Absence explo es he symbiosis o audio– isual c ea ion by ins alling a pe cep ual link be ween he wo
senso y modali ies. These wo nou ish each o he by he linked ges u al na iga ion h ough spaces o audio and image
cha ac e is ics. The pe o mance hus p obes human mul i-modal pe cep ion by c ea ing in e -modal associa ions
in ges u al na iga ion in join audio and image co po a. I augmen s imb e-based sonic explo a ion by colou and
Licensed unde a C ea i e Commons A ibu ion 4.0 In e na ional License (CC BY 4.0). Copy igh emains wi h he au ho (s).
Diemo Schwa z . 2023. Absence — Pe o mance o Audio–Visual Conca ena i e Syn hesis. 1, 1 (May 2023), 2 pages.
Music P oceedings o he In e na ional Con e ence on New In e aces o Musical Exp ession
NIME’20, July 21-25, 2020, Bi mingham, Uni ed Kingdom
2
ex u e-based image explo a ion, some imes wi h cong uen in e -modal associa ions, ollowing uni e sal pe cep i e
expec a ions, some imes wi h con adic o y ones.
The wo k also p oposes a new ake on digi al a by asking whe he his pe o mance should be seen as a digi al
a wo k. A i s sigh his seems ob ious — he whole sys em is ea ing digi al sound, image, and mo emen ep e-
sen a ions algo i hmically. Howe e , looking a he con en , his is no so clea cu anymo e. The piece uses audio
ield eco dings and pho og aphic images aken in he eal wo ld o i s sonic co po a. Wha ’s mo e, he na a i e and
dynamic s uc u e o he piece is c ea ed om ges u e inpu , which means he physical body mo men o he a is in
he pe o mance space. And inally, he ou pu may well be p ojec ed ia a digi al p ojec o and ende ed om a digi al
audio in e ace, bu he sigh s and sounds a i e a he audience h ough analog ligh and sound wa es, c ossing and
mingling wi h he physical space o he pe o mance. In he end, wha is gi en o lis en and see is a join audio– isual
econ ex ualisa ion o imp essions o he eal wo ld, a symbiosis o he physical and digi al.
Ano he elling cha ac e is ic o he pe o mance is ha i is squa ely opposed o all ends in mode n compu e
g aphics and sound syn hesis, whe e e e y hing is luid, h ee-dimensional, pa ame ic, and gene a i e. Ins ead, jus
sho sound g ains and s ill images a e used, he appea ance o which is no gi en by pa ame e s o a gene a i e model,
bu by speci ying he desi ed isual desc ip o s, modeled a e he human senso y pe cep ion. And las , images a e
no gene a ed by an a i icial in elligence, bu chosen by he human sensibili y o he musician/AV pe o me . As such
i o e s an al e na i e app oach o anima ion and na a i e, c ea ing e olu ion om ins an s, and mo emen ou o
s illness.
3 PERFORMANCE NOTES
The pe o mance’s du a ion is a ound 10 minu es. I equi es HD ideo p ojec ion, ideally in an imme si e se ing
wi h p ojec ion on 2 o be e 3 sides su ounding he audience, and a leas quad ophonic su ound sound, ideally
oc ophonic. The elec onic musician should ideally be placed in he middle o he p ojec ion space. I would also be
possible o p ojec on o a plane a ium dome, i such a enue is by mi acle a ailable.
4 MEDIA LINKS
This ideo ease documen s he i s pe o mance o audio– isual conca ena i e syn hesis wi h iolin a he Ma seille
plane a ium a he end o Diemo Schwa z’s IMéRA esidency. This s udy was he basis o he piece Absence p oposed
he e, which has been composed wi h a cohe en audio– isual na a i e and a anged o solo elec onics+images.
•Video: Pe o mance o Audio–Visual Conca ena i e Syn hesis & Violin: h ps://you u.be/EFAN9 Oo d0
ACKNOWLEDGMENTS
This wo k was suppo ed by he A s, Sciences, Socie ies esidency p og am 2021–2022 o he IMéRA Ins i u e o
Ad anced S udy, Aix–Ma seille Uni e si é.
The au ho would like o hank IMéRA, Obse a oi e de Ma seille, Associa ion And omèede, Lionel Ruiz, Je emie
B ugidou, Labo a oi e Pe cep ion, Rep ésen a ions, Image, Son, Musique (PRISM), CNRS, Aix-Ma seille Uni e si é,
ISMM eam, STMS, I cam, So bonne Uni e si és, CNRS, Minis è e de la Cul u e.
