EVxRAVE: Inco po a ing Neu al Syn hesis in
an Augmen ed S ing Ins umen Pla o m
B ian Lindg en
Composi ion and Compu e Technologies (CCT)
Depa men o Music, College o A s and Sciences
Uni e si y o Vi ginia
How can neu al audio models ex end he sonic po en ial o an augmen ed
s ing ins umen in eal ime?
The EV’s in eg a ion wi h RAVE
(Real ime Audio Va ia ional
au oEncode ) e ames he
ins umen ’s own sound as a me a-
esona o : a sys em ha lis ens o
i sel h ough he lens o a neu al
model. Ins ead o p ocessing sound
as a ixed e ec , he model becomes
a pa allel pa hway back in o he
ins umen ’s imb al iden i y,
accessed h ough a di e en causal
ou e. The la en space ac s as a
landscape o possibili ies—each
dimension a po en ial esonance
be ween ges u e, syn hesis, and he
acous ic body.
he EV
The EV is an augmen ed iola ha me ges acous ic pe o mance wi h eal- ime digi al signal p ocessing. A
quad in a ed pickup cap u es each s ing’s mo ion independen ly, eeding he acous ic signal in o a Bela
single-boa d compu e ha uns C++ code o ansmi pe o mance da a o six ins ances o Pu e Da a (Pd)
on a Mac lap op, whe e he main p ocessing and syn hesis occu . The Pd en i onmen is o ganized in o
mul iple ins ances: one dedica ed o each s ing, one managing cen al command and da a ou ing, and
one p o iding he g aphical in e ace o pe o mance con ol.
RAVE in eg a ion
EV signal low
RAVE is a neu al audio
syn hesis model de eloped
by An oine Caillon and
Philippe Esling a IRCAM*.
Designed o high-quali y,
eal- ime pe o mance, i
lea ns a compac la en
ep esen a ion o sound
*Caillon, A., & Esling, P. (2021). RAVE: A a ia ional au oencode o as and high-quali y neu al audio syn hesis. IRCAM – So bonne Uni e si é.
Building he model
h ough a wo-s age aining p ocess: i s
as an au oencode cap u ing pe cep ually
meaning ul spec al ea u es, hen h ough
ad e sa ial ine- uning ha enhances he
na u alness o he gene a ed signal. T ained
on eco dings o he EV, RAVE enables eal-
ime syn hesis in which la en dimensions
can be mapped o musical pa ame e s such
as pi ch, ampli ude, o spec al ea u es.
Pe o ming he model
<<lis en o he
EV on YouTube
