P oceedings o he 6 h Con e ence on AI Music C ea i i y (AIMC 2025),
B ussels, Belgium, Sep embe 10 h-12 h
Con e ence on AI Music C ea i i y (AIMC) 2025
Things AIn’ Wha They Used To Be (composi ion, ~15 minu es)
Ma k Whi lam
Depa men o Music
Uni e si y o B is ol
ma k.whi lam.2019@b is ol.ac.uk
Abs ac
This pe o mance wo k— i led in e e ence o Duke Elling on’s big band jazz classic, eleased
o e six y yea s ago—o e s a gen le p o oca ion, con as ing adi ional app oaches o jazz
imp o isa ion wi h eme ging pa adigms in human–AI in e ac ion. Combining eal- ime machine
lea ning and deep lea ning ools, he piece s ages a li e collabo a ion be ween imp o ising human
musicians and gene a i e AI agen s. Cen al o he wo k is a sub e sion o he es ablished
echnique o he con a ac , whe eby new melodies a e composed o e p e-exis ing cho d
p og essions. He e, he p ocess is in e ed: AI agen s a e asked wi h eha monising composed
melodic lines, he eby dis up ing he expec ed ha monic amewo k. This inde e minacy bo h
encou ages and challenges he pe o me s o ind new musical esponses.
Le e aging echnologies such as Somax2, RAVE, Mosaïque, and Google MediaPipe wi hin
MaxMSP, he sys em enables algo i hmic agen s o ac as bo h collabo a i e and dis up i e
pa ne s in he pe o mance loop. These agen s gene a e unexpec ed musical ges u es and o e
no el, in e ac i e modali ies ha s imula e and p o oke he pe o me s. The esul is an e ol ing
musical language ha eme ges om he en angled dynamics o his ex ended ne wo k o human
and machine imp o ise s.
1. The Ac o -ne wo k o imp o ising musicians and AI agen s
Building on he widely used app oach o co pus-based conca ena i e syn hesis in elec oacous ic
composi ion (Hackba h e al., 2013; S u m, 2006), his composi ion uses a a ie y o AI agen s,
lis ening bo h o hei human co-pe o me s and o one ano he , c ea ing a complex ne wo k o
ac an s (La ou , 1996). This p ac ice-based esea ch looks h ough he lens o Ac o -Ne wo k
Theo y and 4E Cogni ion o gene a e meaning ul insigh s om he li ed-expe iences o imp o ising
musicians o in o m u u e composi ional app oaches wi h gene a i e AI. I explo es he complex
dynamics o a co-imp o ising ex ended human-compu e ne wo k and how in e ac ing wi h AI
agen s may s imula e new musical language om human pe o me s and how his in o ms no el
composi ional pa adigms. This 15-minu e pe o mance ea u es wo sec ions o a longe piece
o iginally w i en o eno saxophone wi h li e elec onics and d um ki /pe cussion wi h li e
elec onics, and a ious AI agen s lis ening o bo h playe s and spa ialised in Dolby A mos 9.1.4.
Fo his pe o mance a AIMC, he saxophone is subs i u ed o umpe and he audio p ocessing is
ende ed in s e eo.
2
2. AI agen s and hei ole in he human-machine ne wo k
D umme and compose Ma k Whi lam is joined by umpe e Celes e Can o -S ephens o his
pe o mance, bo h o whom ha e been in e na ionally pe o ming musicians o e he pas wo
decades. They d aw on his ich li ed expe ience o na iga e he o en unp edic able in e jec ions
om hei machine co-pe o me s, bu a e also challenged o ind new musical esponses o he AI
and one ano he . The pe o mance uses a MaxMSP o combine mul iple ins ances o IRCAM’s
Somax2 p o iding musical ges u es in esponse o bo h umpe and pe cussion inpu s. Somax2
d aws upon a ious co po a o he compose ’s own language and ha o Bill E ans’ jazz s anda ds
Ve y Ea ly and Na dis o p o ide melodic and ha monic esponses o he human pe o me s.
Mul iple ins ances o he AI agen allows o sel - e e en ial in e ac ion wi h he a ious audio and
MIDI co po a, adding g ea e emphasis on he agency o he a i icial in elligence sys em in he
li e ensemble. Each ime he piece is pe o med, new co po a o he p e ious pe o me s’ playing
a e inco po a ed in o he sys em’s ‘memo y’, esul ing in an e e -e ol ing, haun ological musical
pale e om which he AI d aws and esponds.
3. Embodied and enac ed in e ac ions
Nabi e al. (2024) explo ed embodied in e ac ion wi h he imb al ans e a ia ional au oencode
RAVE (Caillon & Esling, 2022) h ough in e disciplina y esea ch in ol ing dance s equipped
wi h wi eless gy oscopic mo ion senso s. In con as , he compose in his p ojec uses MediaPipe
as an in e locu o , wi h hand and body landma k ecogni ion models anscoding pe o me s’
mo emen s in o con ol da a o wo dis inc AI agen s. The i s is RAVE, whe e dynamic
na iga ion o he la en space gene a es no el imb es ha s imula e u he imp o isa o y
esponses om he pe o me s. The second agen , Mosaïque (Thibaul , 2024), employs co pus-
based conca ena i e syn hesis o a ange sonic agmen s ac oss a h ee-dimensional isual ‘map’.
A p ojec ed image o his p o ides he audience wi h isual cues abou he eal- ime in e ac ions
wi hin he human-compu e ne wo k, illus a ing how he musicians’ ges u es na iga e and shape
he sonic collage.
A is Biog aphies:
Ma k Whi lam is a compose , d umme /pe cussionis , and educa o . He has eco ded wi h and
pe o med in e na ionally wi h lumina ies o he UK jazz scene, including ECM eco ding a is s
Andy Sheppa d, Iain Ballamy, and Jason Rebello (S ing), as well as c oss-gen e p ojec s wi h
Po ishead gui a is Ad ian U ley and Gold app’s Will G ego y. Ma k’s composi ions and
pe o mances ha e ecei ed wide ai play on BBC 2, 3, 6, and Jazz FM, and he has ecei ed
commissions o an HBO ele ision minise ies. He is cu en ly unde aking a PhD in composi ion
a he Uni e si y o B is ol, explo ing pos -spec al language and he use o AI-based in e ac i e
agen s wi hin human, li e, imp o isa ion-led musical c ea ion.
Celes e Can o -S ephens is a highly accomplished imp o ise , compose , w i e , and eache , and
is ac i e on he music scenes bo h in he UK and New Yo k. He e sa ile umpe emb aces he
c ea i e and explo a o y, spanning a ange o p ac ices and adi ions: om ee imp o isa ion o
klezme , ia jazz, dub- eggae, classical, and mo e. She has collabo a ed wi h a di e se ange o
a is s, om pe cussionis Billy Ma in and Soundpain ing pionee Wal e Thompson o he la e
dub maes o Lee ‘Sc a ch’ Pe y.
3
Bo h Ma k and Celes e each unde g adua e and pos g adua e music s uden s in he UK a Ba h
Spa Uni e si y and BIMM Uni e si y, B is ol.
The compose would like o hank:
The Uni e si y o B is ol o unding owa ds his con e ence pe o mance. Paul Hamesse a HD
S udio, B ussels o d umki hi e. Musicians who ha e g aciously con ibu ed imp o ised
pe o mances o he RAVE and Mosaïque co po a: Lucy Anne-Allen ( iolin), Hen ik Jensen and
Ashley John Long (double bass), Jake McMu chie (saxophone), Joseph Vissaux (pe cussion),
Nicholas Malcolm ( umpe ).
Re e ences
Caillon, A., & Esling, P. (2022). RAVE: A a ia ional au oencode o as and high-quali y neu al
audio syn hesis. In P oceedings o he 25 h In e na ional Con e ence on Digi al Audio E ec s (DAFx-
22), Vienna, Aus ia, Sep embe 6–10, 2022.
h ps://da x2020.mdw.ac.a /p oceedings/pape s/DAFx20in22_pape _49.pd
Hackba h, B., Schnell, N., Esling, P., & Schwa z, D. (2013). Composing mo phology: Conca ena i e
syn hesis as an in ui i e medium o p esc ibing sound in ime. Con empo a y Music Re iew, 32(1),
49–59. h ps://doi.o g/10.1080/07494467.2013.774513
La ou , B. (1996). On ac o -ne wo k heo y: A ew cla i ica ions. Soziale Wel , 47(4), 369–381.
h p://www.js o .o g/s able/40878163
Nabi, S., Esling, P., Pee e s, G., & Be ilacqua, F. (2024). Embodied explo a ion o deep la en spaces
in in e ac i e dance-music pe o mance. In P oceedings o he 9 h In e na ional Con e ence on
Mo emen and Compu ing (MOCO ’24), U ech , Ne he lands, May 2024.
h ps://doi.o g/10.1145/3658852.3659072
S u m, B. L. (2006). Adap i e conca ena i e sound syn hesis and i s applica ion o mic omon age
composi ion. Compu e Music Jou nal, 30(4), 46–66. h ps://www.js o .o g/s able/4617983
Thibaul , D. (2024, Augus 24). Mosaïque – Conca ena i e syn hesis ins umen o he p ac icing
musicians [Wo kshop p esen a ion]. AIMC 2024. h ps://aimc2024.pubpub.o g/pub/buh7kcah/ elease/1
