B oken Fo ecas s: Feedbacking La en Gene a o s o
Sonic Ins abili y
Bła˙
zej Ko owski∗
Music Technology G oup
Uni e si a Pompeu Fab a
Ba celona, Spain
[email p o ec ed]
Abs ac
B oken Fo ecas s is a li e sound pe o mance buil a ound a cus om gene a i e
sys em ha in e p e s machine lea ning unce ain y as a design ma e ial. The sys-
em combines au o eg essi e p edic ion wi h a delayed eedback pa h, whe e pas
la en s a e ans o med and ed back in o gene a ion, p oducing an uns able, pe -
o mable ajec o y ha d i s om lea ned pa e ns. A neu al syn hesize decodes
hese la en ajec o ies in o audio in eal ime. This eedback dis up s lea ned
hy hmic and s uc u al pa e ns, gi ing ise o complex, gli ch-in lec ed ex u es
and unp edic able sonic e olu ions. Ra he han seeking cohe ence o con ol, he
pe o mance o eg ounds ecu si e ins abili y as a sou ce o aes he ic possibili y.
By engaging wi h he eedback dynamics o au o eg essi e sequence p edic ion,
B oken Fo ecas s p oposes an al e na i e mode o in e ac ion wi h gene a i e mod-
els, ea ing hem no as ools o p edic ion, bu as unce ain, pe o mable a i ac s
wi hin a si ua ed sonic p ac ice.
1 In oduc ion
B oken Fo ecas s is a li e sound pe o mance ha ea s Machine Lea ning (ML) as an uns able and
pe o mable ma e ial.
Ra he han seeking con ol o cohe ence, i explo es eedback wi hin a gene a i e model as a
mechanism unco e ing i s quasi-ma e ial
2
quali ies. The sys em e ou es delayed and modula ed
la en codes back in o i s own inpu , c ea ing a ecu si e loop ha des abilizes gene a ion. This
con olled de ia ion pushes he sys em beyond i s aining dis ibu ion, aligning wi h he Ac i e
Di e gence pa adigm (B oad e al., 2021).
This wo k ea s machine lea ning no as a p edic i e ool, bu as a malleable design ma e ial shaped
by a o dances such as s ochas ici y, ins abili y, and opaci y. I explo es non-seman ic o ms o
con ol, whe e he pe o me loosely nudges ajec o ies wi hou speci ying conc e e symbolic in en .
The use o a small-scale a chi ec u e and cu a ed da ase s esponds o e hical and exp essi e conce ns,
esis ing he ex ac i ism o la ge AI sys ems. The pe o mance asks: wha kinds o lis ening and
con ol eme ge when we a une o gene a i e models as ecu si e, misaligned p ocesses a he han
p edic able ools?
∗h ps://blazejko owski.com
2
The e m sugges s ha algo i hms and compu a ional sys ems can be app oached as i hey possess ma e ial
quali ies ha o e esis ances, a o dances, and exp essi e possibili ies. Aspec s such as in e ace beha io ,
pa ame e esponsi eness, and da a-induced dynamics a e no s ic ly physical, ye unc ion as ma e ial cons ain s
ha shape in e ac ion and c ea i e p ac ice (Ge lek and Weydne -Volkmann, 2025).
P oceedings o he 6 h Con e ence on AI Music C ea i i y (AIMC 2025),
B ussels, Belgium, Sep embe 10 h-12 h
2 Rela ed Wo k
This wo k aligns wi h a is ic p ac ices ha sub e machine lea ning sys ems, aming ailu e,
inde e minacy, and eedback as aes he ic s a egies.
The 2019 ISEA panel on “Machine Flaws in Gene a i e A ” showed how gli ches and b eakdowns
in AI models become ma e ial o exp ession (Boyé, 2019). Cascone’s aming o he pos -digi al
aes he ic simila ly posi ions gli ch and ailu e as co e o digi al sound a (Cascone, 2000).
Böhlen ea s classi ica ion slippage as a si e o disco e y, whe e sys em e o s become oppo uni ies
o c i ique and meaning-making. (Böhlen, 2021).
G ba coins he e m “ ac ical AI a ,” whe e a is s wo k wi h he assump ions and biases o AI
sys ems o expose, challenge, o ans o m hem (G ba, 2022). Ra he han me ely deploying sys ems
o s ylis ic e ec , such p ac ices expe imen wi h how models beha e unde p essu e, in un amilia
condi ions, o when delibe a ely pushed in o b eakdown.
Benjamin e al. concep ualize machine lea ning unce ain y as a de ining ma e ial p ope y o AI
sys ems, in oducing he no ion o " hingly unce ain y", as a o m o inde e mina e ela ion be ween
ML ou pu s and he wo ld ha lends i sel o open-ended in e p e a ion and aes he ic explo a ion
(Benjamin e al., 2021).
Musically, he piece d aws on p ac ices o empo al ecu sion. In "I Am Si ing in a (La en ) Room",
a Va ia ional Au oencode (VAE) is used o ecu si ely encode and decode audio in eal ime, echoing
Al in Lucie ’s acous ic eedback piece (Shaheed and Wang, 2024). The sys em enables eal- ime
manipula ion o la en pa ame e s, mi o ing a p ac ice commonly employed in musical eedback
sys ems: p ocessing he signal be o e i is ed back o he sys em.
Toge he , hese wo ks posi ion B oken Fo ecas s wi hin a body o expe imen al AI in es iga ion
ha emb aces ins abili y and ea s machine lea ning no as a ep oduc i e agen bu as a malleable
quasi-ma e ial a i ac wi h i s idiosync a ic quali ies, which a o d c ea i e possibili y.
3 Sys em A chi ec u e
The sys em is buil on he ounda ion o wo ML models, as illus a ed in Figu e 1:
1.
Neu al Decode : A decode ained o syn hesize audio om la en ep esen a ions de i ed
om aining in andem wi h i s encode coun e pa , in a VAE se ing (Kingma and Welling,
2019). The decode ea u es a DDSP a chi ec u e, bu unlike ypical DDSP implemen a ions,
which use pi ch and loudness o con ol, his sys em condi ions gene a ion solely on he
low-dimensional la en ec o (Engel e al., 2020). As a esul o VAE aining, he la en
space e ains a smoo h in e nal s uc u e, which suppo s he in e pola ion o ea u es.
2.
La en Sequence Model: The model d aws on au o eg essi e sequence modeling app oaches
such as GPT-s yle ans o me s (Vaswani e al., 2017). An encode -only ans o me is
ained on sequences o la en ec o s o model empo al un olding o he audio. The
ans o me p edic s a ixed-leng h block o la en codes a a ime, gi en a con ex bu e
o p e iously gene a ed codes. C ucially, okeniza ion is a oided, main aining he la en s
con inuous, allowing o smoo h manipula ion o he la en s eam by he pe o me .
In pe o mance, he model ope a es as a closed gene a i e loop. A each s ep, he La en Sequence
Model p edic s a new la en ec o block based on a con ex d awn om p io gene a ions. Howe e ,
a he han simply eeding i s ou pu s back in o he model unchanged—as in s anda d au o eg essi e
sys ems—B oken Fo ecas s in oduces a second empo al pa h. La en s om ea lie in he con ex
window a e i s selec ed a a delay o se (
τ
) and ans o med by he Feedback Modula o , which
applies ope a ions such as o se ing, scaling, o in e sion. These modula ed la en s a e hen added o
he newly p edic ed ones, and he esul ing composi e is appended o he con ex bu e , in oducing
ecu si e ins abili y in o he gene a i e p ocess. Li e pe o me inpu modula es bo h he eedback
gain and delay pa ame e s, s ee ing he ins abili y’s in ensi y and di ec ion in eal ime.
In pa allel, he esul ing la en ec o s a e passed h ough he La en Codes Modula o be o e syn hesis.
This module o e s he same ope a ions—o se , scaling, in e sion—bu applies hem ansien ly
du ing decoding a he han eeding hem back in o he gene a i e loop. These modula ions shape
2
Figu e 1: Sys em a chi ec u e o B oken Fo ecas s. The sys em consis s o he Tempo al Dynamics
Module ( op) and he Syn hesis Pa hway (bo om). The La en Sequence Model p edic s new la en
codes om a olling Con ex Bu e . In pa allel, ea lie codes delayed by
τ
a e p ocessed h ough
he Feedback Modula o , applying ope a ions such as o se , scaling, o in e sion. These modula ed
delayed codes a e hen combined wi h he cu en p edic ions and appended o he bu e , in oducing
ecu si e ins abili y in o he gene a ion p ocess. In he syn hesis pa hway, he La en Codes Modula o
applies simila ans o ma ions o he la en codes in eal ime, shaping he decoded audio wi hou
a ec ing he gene a i e loop. Pe o me inpu con ols bo h modula ion s ages, enabling dynamic
in luence o e bo h s uc u al e olu ion and imb al exp ession.
he sound’s imb e wi hou a ec ing empo al s uc u e o sys em’s memo y. The modula ed la en s
a e hen decoded by he Neu al Decode , p oducing audio shaped by bo h sys em ins abili y and
pe o me inpu .
4 A is ic Ou comes
In pe o mance, he sys em suppo ing B oken Fo ecas s beha es like a shape-shi ing ins umen . A
low eedback le els, he model p oduces cohe en ges u es e lec i e o he aining da a. Highe le el
o eedback in oduces spec al anomalies, hy hmic i egula i ies, o imb al d i , depending on he
eedback delay ime and speci ic amoun . The pe o me guides his p ocess, occasionally ese ing o
seeding new inpu s, bu mos ly shaping he un olding ins abili y by adjus men o eedback pa ame e s
and modula ion o la en ep esen a ions.
B oken Fo ecas s p oposes an app oach o AI music pe o mance cen e ed on ecu sion, ailu e, and
eedback. I ea s machine lea ning models as uns able, quasi-ma e ial a i ac s whose b eakdowns
a e no e o s, bu composi ional oppo uni ies. By wo king wi h small da ase s, emb acing non-
seman ic con ol, and looping p edic ions back in o hemsel es, he piece opens new spaces o sonic
explo a ion and in e p e abili y.
Ra he han op imizing o mas e ing AI, his wo k engages i ac ically, exposing i s limi s and
lis ening o i s de ia ions. In doing so, i con ibu es o an eme gen ield o c i ical, pe o ma i e AI
a ha o eg ounds i s ins abili y, unce ain y, and ma e iali y.
3
Re e ences
Benjamin, J. J., Be ge , A., Me ill, N., and Pie ce, J. (2021). Machine Lea ning Unce ain y as a
Design Ma e ial: A Pos -Phenomenological Inqui y. In P oceedings o he 2021 CHI Con e ence
on Human Fac o s in Compu ing Sys ems, CHI ’21, pages 1–14, New Yo k, NY, USA. Associa ion
o Compu ing Machine y.
Böhlen, M. (2021). Classi ica ion, Slippage, Failu e and Disco e y. In P oceedings o he 9 h
Con e ence on Compu a ion, Communica ion, Aes he ics & X, pages 15–29.
Boyé, P. (2019). Machine Flaws in Gene a i e A . In P oceedings o he 25 h In e na ional
Symposium on Elec onic A , pages 713–716.
B oad, T., Be ns, S., Col on, S., and G ie son, M. (2021). Ac i e di e gence wi h gene a i e deep
lea ning - A su ey and axonomy. In de Sil a Ga za, A. G., Veale, T., Aguila , W., and y Pé ez,
R. P., edi o s, P oceedings o he Twel h In e na ional Con e ence on Compu a ional C ea i i y,
ICCC 2021, México Ci y, México (Vi ual), Sep embe 14-18, 2021, pages 227–236. Associa ion
o Compu a ional C ea i i y (ACC).
Cascone, K. (2000). The Aes he ics o Failu e: "Pos -Digi al" Tendencies in Con empo a y Compu e
Music. Compu e Music Jou nal, 24(4):12–18.
Engel, J. H., Han akul, L., Gu, C., and Robe s, A. (2020). DDSP: di e en iable digi al signal
p ocessing. In 8 h In e na ional Con e ence on Lea ning Rep esen a ions, ICLR 2020, Addis
Ababa, E hiopia, Ap il 26-30, 2020. OpenRe iew.ne .
Ge lek, S. and Weydne -Volkmann, S. (2025). Ma e iali y and machinic embodimen : A pos -
phenomenological inqui y in o Cha GPT’s ac i e use in e ace. Jou nal o Human-Technology
Rela ions, 3:1–15.
G ba, D. (2022). Lu es o engagemen : An ou look on ac ical ai a . In P oceedings o he 10 h
Con e ence on Compu a ion, Communica ion, Aes he ics & X, pages 58–74.
Kingma, D. P. and Welling, M. (2019). An in oduc ion o a ia ional au oencode s. Found. T ends
Mach. Lea n., 12(4):307–392.
Shaheed, N. and Wang, G. (2024). I Am Si ing in a (La en ) Room. In P oceedings o he
In e na ional Con e ence on New In e aces o Musical Exp ession, pages 333–338.
Vaswani, A., Shazee , N., Pa ma , N., Uszko ei , J., Jones, L., Gomez, A. N., Kaise , L., and
Polosukhin, I. (2017). A en ion is all you need. In Guyon, I., on Luxbu g, U., Bengio, S.,
Wallach, H. M., Fe gus, R., Vishwana han, S. V. N., and Ga ne , R., edi o s, Ad ances in Neu al
In o ma ion P ocessing Sys ems 30: Annual Con e ence on Neu al In o ma ion P ocessing Sys ems
2017, Decembe 4-9, 2017, Long Beach, CA, USA, pages 5998–6008.
4
