G amma ical Syn hesis o Sonic Fo ms: A
Symbolic-Acous ic F amewo k o C ea i e
Recomposi ion in Rio Spa A eal Ni Asian Tu u
I an Eiji Simu a∗
Ins i u o de A es-IA/Núcleo In e disciplina de Comunicação Sono a-NICS
Uni e sidade Es adual de Campinas - UNICAMP
Cidade Uni e si á ia “Ze e ino Vaz”, Ba ão Ge aldo, Campinas, São Paulo, B asil
[email p o ec ed]
Abs ac
This pape p esen s a symbolic-acous ic sys em o musical ecomposi ion ha
b idges audio desc ip o analysis, machine lis ening, and symbolic AI h ough gen-
e a i e g amma s. The sys em in eg a es spec al- empo al clus e ing and o mal
g amma induc ion echniques: N-G ams, Sequi u , and L-Sys ems, o cons uc
ule-based sonic ecomposi ion om eal-wo ld audio. Unlike co pus- ained
neu al models, he sys em emphasizes local sa y, in e p e abili y, and s uc u al
pellucidi y, enabling composi ional con ol a bo h he mic o and mac o le els. The
esul ing composi ions, c ea ed h ough p obabilis ic ules and dynamic ans o ma-
ions, demons a e a hyb id app oach o musical c ea i i y: one ha is algo i hmic
ye pe cep ually g ounded, symbolic ye acous ic. The wo k con ibu es o con em-
po a y discussions on AI music c ea i i y by o e ing a lexible, explainable, and
a is ically guided composi ional me hod based on sound analysis, symbolic logic,
and gene a i e syn hesis. F om his p ojec , we designed he 2025 elec o-acous ic
music composi ion called Rio Spa A eal Ni Asian Tu u which is he i s c ea i e
music expe imen om his sys em.
1 In oduc ion: Towa d a Hyb id In elligence in Composi ion
Recen ad ances in a i icial in elligence ha e ans o med how compose s engage wi h ma e ials,
s uc u e, and sound. In pa icula , he in e sec ion be ween symbolic modeling and pe cep ually
g ounded audio analysis has become e ile g ound o c ea i e explo a ion. While many con empo-
a y AI music sys ems ely hea ily on la ge symbolic da ase s and neu al a chi ec u es Yang e al.
(2017); Huang e al. (2018), his wo k p oposes an al e na i e, localized hyb id app oach, whe e
symbolic g amma s eme ge om machine lis ening applied o eal audio samples.
The sys em desc ibed in his pape , de eloped in Py hon and applied in a gene a i e composi ion
p esen ed a AIMC
2025
, enables sonic ecomposi ion h ough a h ee-s age p ocess: (1) audio
desc ip o analysis and clus e ing, (2) symbolic sequence modeling h ough g amma ical o malisms,
and (3) ule-based sample manipula ion and syn hesis. Each s ep is guided by he compose ’s con ol
and algo i hmic easoning, ensu ing ha he sys em is no a black box, bu a he a hin pa ne in
c ea i e decision-making. The i le o he composi ion, Rio Spa A eal Ni Asian Tu u, is de i ed
om a ecu si e linguis ic pe mu a ion p ocess, e lec ing he composi ional s a egies employed in
he sys em i sel . I e okes a play ul, gene a i e logic o eo de ing and eme gence, much like he
symbolic g amma s and algo i hmic ans o ma ions used in he piece’s c ea ion. This name unc ions
as bo h a poe ic agmen and a concep ual mi o o he unde lying amewo k. The a ailable audio
ile om he music composi ion can be downloaded a h ps://sho u l.a / G aZ.
∗
Use oo no e o p o iding u he in o ma ion abou au ho (webpage, al e na i e add ess)—no o
acknowledging unding agencies.
P oceedings o he 6 h Con e ence on AI Music C ea i i y (AIMC 2025),
B ussels, Belgium, Sep embe 10 h-12 h
2 Me hods: F om Sound o Symbol and Back Again
2.1 Audio Desc ip o Analysis and Clus e ing
The p ocess begins wi h he ex ac ion o spec al and empo al desc ip o s using he lib osa Py hon
lib a y. Audio ea u es lis includes spec al cen oid, bandwid h, oll-o , la ness, RMS, ze o-
c ossing a e, MFCCs, and ch oma ec o s. These desc ip o s o m a mul idimensional ea u e
space, cap u ing essen ial quali ies o he inpu sounds—such as b igh ness, densi y, ha monici y, and
dynamic en elope.
Clus e ing is applied o his space using unsupe ised lea ning echniques such as KMeans, Gaussian
Mix u e Models (GMM), and Agglome a i e Clus e ing. As discussed by He emans e al. He emans
e al. (2017), clus e ing o e s a obus s a egy o pa i ioning pe cep ually simila sonic segmen s,
enabling subsequen symbolic labeling. Each esul ing clus e is assigned a symbol (e.g., ‘A’, ‘B’,
‘C’), e ec i ely ansla ing pe cep ual ea u es in o disc e e composi ional uni s.
This mo e esona es wi h p io musicological and compu a ional pa adigms. Roads Roads (2001), o
example, emphasizes he impo ance o spec al mo phology in mic osound composi ion, whe e sound
objec s a e ca ego ized and ecombined based on ea u es a he han no es. Simila ly, Wisha Wisha
(1994) explo es he no ion o ‘acousma ic g amma ’ o o ganize sonic ma e ials by audi o y a ibu es,
a concep o malized he e h ough clus e ing.
2.2 G amma Induc ion: Sequi u , N-G ams, and L-Sys ems
Once symbolic sequences a e ob ained, he sys em applies mul iple o mal g amma induc ion
echniques o model pa e ns and s uc u es. N-G am analysis Shannon (1948) iden i ies equen
symbolic sequences and ansi ion p obabili ies, o ming he s a is ical basis o pa e n ecogni ion
and s ylis ic con inui y. Sequi u , as in oduced by Clea y and Wi en Clea y and Wi en (1984),
de ec s ecu si e s uc u es and hie a chical comp essions. I con e s symbol sequences in o g am-
ma s ha e eal bo h epe i ion and s uc u al dep h. This aligns wi h Cope’s Cope (1996) ea ly
symbolic AI wo k in Expe imen s in Musical In elligence, whe e s ylis ic g amma was induced
om symbolic co po a. Lindenmaye Sys ems, a.k.a., L-Sys ems, o iginally o mula ed o modeling
biological g ow h Lindenmaye (1968), a e adap ed he e o sonic g ow h and ans o ma ion. By
applying pa allel ule subs i u ions, L-Sys ems expand sho symbolic ‘axioms’ in o e ol ing musical
sequences, yielding eme gen complexi y and o mal de elopmen .
This mul i-g amma a chi ec u e b idges symbolic AI and algo i hmic composi ion, si ua ing he
sys em wi hin he adi ion o Nie haus Nie haus (2009) and Kelle and Capasso Kelle and Capasso
(2006), while ex ending hese ideas owa d eal- ime audio manipula ion.
2.3 Symbol- o-Sound Recomposi ion and Sonic Rende ing
Each symbol in he g amma co esponds o an audio sample, e ie ed ei he om speci ic local
sound da abase o he eesound.o g online eposi o y. Fo ende ing, he sys em applies a ange o
ans o ma ions and e ec s:
• Pi ch shi ing and ime-s e ching ia lib osa e ec s;
• Segmen slicing wi h andom o e lapping;
• P obabilis ic e e sals, dynamic gain a ia ion, and spa ial s e eo panning;
• Con olu ional e e b using scipy.signal o simula e acous ic en i onmen s.
This syn hesis me hod pa allels Roads’ mic osound p inciples Roads (2001), whe e audio is con-
s uc ed om g anula and segmen al uni s. I also echoes Di Scipio’s wo k on sound-based composi-
ion sys ems ha use pe cep ual pa ame e s as con ol in e aces Di Scipio (2000).
The inal ou pu is a collage o ans o med samples, o de ed by g amma and shaped by he in e play
be ween analysis and ule-d i en syn hesis. The ollowing Figu e 1 depic s he i s page om he
applica ion g aphical use in e ac ion.
2
Figu e 1: Fi s page window om he GUI in e ace.
3 A is ic Mo i a ion and Con ibu ion
This p ojec was bo n om an in e es in making musical s uc u e eme ge om sound i sel . Ra he
han composing wi h abs ac pi ch-du a ion alues, he sys em lis ens o sound, ca ego izes i , and
gene a es i s own g amma . The compose becomes an ‘a chi ec ’ o se ings, na iga ing be ween
algo i hmic depu a ion and a is ic in en ion.
This app oach con as s wi h deep gene a i e models, which o en obscu e inne wo kings and ely on
massi e da a. By con as , he sys em he e is plain, small-da a-o ien ed, and in e p e able. I p opels
he use in o he p ocess—whe he a is , esea che , o lis ene — o co-compose wi h algo i hmic
logic.
In doing so, i con ibu es o AI Music C ea i i y by p oposing:
• A desc ip o - o-symbol pipeline ha links pe cep ual quali ies o composi ional s uc u es;
• A lexible g amma engine ha is modula and musically exp essi e;
•
A ende ing p ocess ha p io i izes imb al ichness and ans o ma ion o e symbolic
ideli y.
Conclusion
This companion wo k speaks di ec ly o he AIMC 2025 heme o c ea i i y wi h and h ough AI. I
p oposes a middle pa h be ween machine lea ning and symbolic AI, whe e compu a ional c ea i i y is
s uc u ed, explainable, and musically g ounded. In doing so, we designed he elec o-acous ic music
called Rio Spa A eal Ni Asian Tu u which is he i s c ea i e music expe imen om his sys em.
Ra he han ea ing AI as a gene a o o su p ises, his p ojec ea s i as a pa ne in s uc u ing ma-
e ial, d awing om ideas o eme gen o m, ecu si e ules, and in e ac i e lis ening. I embodies AI
as musical in elligence—no only p oducing ma e ial, bu unde s anding, shaping, and ans o ming
i .
Acknowledgmen s and Disclosu e o Funding
Use unnumbe ed i s le el headings o he acknowledgmen s. All acknowledgmen s go a he end
o he pape be o e he lis o e e ences.
3
Do no include acknowledgmen s in he anonymized submission, only in he inal pape . You can use
he
ack
en i onmen p o ided in he s yle ile o au oma ically hide his sec ion in he anonymized
submission.
Re e ences
Clea y, J. G. and Wi en, I. H. (1984). Da a comp ession using adap i e coding and pa ial s ing
ma ching. IEEE T ansac ions on Communica ions, 32(4):396–402.
Cope, D. (1996). Expe imen s in Musical In elligence. A-R Edi ions.
Di Scipio, A. (2000). Sound is he in e ace: F om in e ac i e o ecosys emic signal p ocessing.
O ganised Sound, 5(3):203–221.
He emans, D., Chuan, C.-H., and Chew, E. (2017). A unc ional axonomy o music gene a ion
sys ems. ACM Compu ing Su eys, 50(5):1–30.
Huang, C.-Z. A., Vaswani, A., Uszko ei , J., e al. (2018). Music ans o me : Gene a ing music wi h
long- e m s uc u e. ICLR.
Kelle , D. and Capasso, A. (2006). New concep s and echniques in algo i hmic composi ion.
Leona do Music Jou nal, 16:61–65.
Lindenmaye , A. (1968). Ma hema ical models o cellula in e ac ion in de elopmen . Jou nal o
Theo e ical Biology, 18(3):280–315.
Nie haus, G. (2009). Algo i hmic Composi ion. Sp inge .
Roads, C. (2001). Mic osound. MIT P ess.
Shannon, C. E. (1948). A ma hema ical heo y o communica ion. Bell Sys em Technical Jou nal,
27(3):379–423.
Wisha , T. (1994). Audible Design. O pheus he Pan omime L d.
Yang, L.-C., Chou, S.-Y., and Yang, Y.-H. (2017). Midine : A con olu ional gene a i e ad e sa ial
ne wo k o symbolic-domain music gene a ion. ISMIR.
4
