In o ma ion S o age and Re ie al: A Composi ion
abou Digi al Looms, Playe Pianos, and Ea ly
Compu e s
Molly Jones
Depa men o Pe o ming A s Technology
School o Music, Thea e, and Dance
Uni e si y o Michigan
Ann A bo , MI 48109
USA
[email p o ec ed]
Abs ac
In o ma ion S o age and Re ie al is an audio isual composi ion inspi ed by h ee
classes o machines: digi al looms, playe pianos, and ea ly compu e s. These ma-
chines ha e in e wined his o ies, sha ed con ol mechanisms, and common abs ac
da a s uc u es. All h ee classes o machines a e ully digi al and ully physical, all
mechanize wo k p e iously done by human hands, and all u n in o ma ion s o ed
on punched pape in o ma e ial a i ac s. Pianis s, p og amme s, and wea e s use
punched pape o s o e da a as g ids o holes, a da a s uc u e ha mi o s g idlike
wea ing pa e ns, s a music no a ion, and ma ices o numbe s. This composi ion
celeb a es he commonali ies o hese machines. The piece ea u es ield eco d-
ings o looms; MIDI sequences gene a ed by ans o me models ained on he
compose ’s o iginal ma e ial; p ojec ed digi al ideo collages o wea ing pa e ns;
and a pe o me con olling lap op, p ojec o , and p epa ed playe piano li e.
1 P ojec Desc ip ion
In o ma ion S o age and Re ie al is a en-minu e audio isual composi ion inspi ed by he in e wined
his o ies, sha ed con ol mechanisms, and common abs ac da a s uc u es o h ee classes o
machines: digi al looms, playe pianos, and ea ly compu e s (Wahl, 2018; King, 2019; Fuegi and
F ancis, 2003). A pe o me igge s and p ocesses ixed s e eo audio acks, MIDI clips played by
a p epa ed playe piano, and digi al ideo collage. The pe o me sends MIDI da a o Max 8/Ji e ,
Able on Li e, and a playe piano using a No a ion Launchpad X.
1.1 His o ical Con ex
Ea ly compu e s we e con olled wi h punched pape ca ds, a echnology di ec ly inspi ed by 18 h
cen u y digi al Jacqua d and Bouchon looms (Fuegi and F ancis, 2003; Wahl, 2018; Kau e al., 2014;
Pos el, 2020; Mano ich, 2002). Lady Ada Lo elace p oposed punched ca ds o Cha les Babbage as
a mechanism by which a human migh exp ess hei wishes o compu ing machines (Wahl, 2018),
w i ing “The Analy ical Engine wea es algeb aical pa e ns jus as he Jacqua d loom wea es lowe s
and lea es,” (Mano ich, 2002). In an 1843 le e , Lo elace p oposed he musical po en ial o he
Analy ical Engine: “[T]he Engine migh compose elabo a e and scien i ic pieces o music o any
deg ee o complexi y o ex en ...” (Fuegi and F ancis, 2003), p esaging he adop ion o compu e s in
music making. A punched pape con ol mechanism o iginally de ised by Basile Bouchon, a Lyon
o gan make , had e olu ionized wea ing, which in u n had sugges ed an in e ace o compu e s,
and now his in e ace’s in en o p oposed hese compu ing machines as ools o c ea e music, closing
a ci cle o mu ual in luence. Pianis s, p og amme s, and wea e s use punched pape o s o e da a
P oceedings o he 6 h Con e ence on AI Music C ea i i y (AIMC 2025),
B ussels, Belgium, Sep embe 10 h-12 h
Figu e 1: Playe piano p epa a ions. The numbe o magne s depic ed on a gi en no e a e s acked
and placed on he s ings.
as g ids o holes, a da a s uc u e ha mi o s g idlike wea ing pa e ns, s a music no a ion, and
he ma ices o numbe s cen al o many machine lea ning algo i hms. The commonali ies o hese
machines and hei da a s uc u es inspi ed me o c ea e a composi ion highligh ing he sounds,
isuals, and capabili ies o looms, playe pianos, and compu e s.
1.2 P epa ed Playe Piano wi h T ans o me Models
I employed a playe piano and ans o me models o c ea e a hyb id human-machine audio- isual
pe o mance ha d aws on he s a is ical dis ibu ions and pa e ns inhe en in my o iginal composed
ma e ial.
An elec onic playe piano plays sequences o MIDI da a ha I igge using he Launchpad X. A
Bösendo e g and piano wi h CEUS ep oducing sys em plays ph ases o ma e ial gene a ed by
bo h mysel and ans o me models ained on my composed ma e ial. The piano is p epa ed wi h
neodymium magne s and alliga o clips. Fo a desc ip ion and images o he piano p epa a ions,
please see Figu e 1.
I ained wo X-T ans o me s decode -only models (Chang, 2024; Vaswani e al., 2017). I ained
one on he MAESTRO da ase (Haw ho ne e al., 2019) and one only on my own o iginal composed
MIDI sequences speci ic o In o ma ion S o age and Re ie al. Bo h ained models we e p omp ed
wi h he 83 o iginal sequences I w o e speci ically o he piece. The in en ion o his s a egy was o
p omp he models o ex end he sequences in a s yle esembling my own composi ional decisions
and aes he ic. The expe imen was o de e mine whe he he MAESTRO- ained model o he model
ained only on my own ma e ial would gene a e sequences close o my musical p e e ences. The
inding was ha bo h models quickly ended o ep oduce exac ly he sequences I w o e wi h li le o
no elabo a ion. Expe imen s wi h model simpli ica ion o emedy his memo iza ion p oblem a e
ongoing.
1.3 Field Reco dings
In o ma ion S o age and Re ie al ea u es my ield eco dings o digi al looms and he oices o
wea e s a he Chicago Wea ing School and G een ield Village Wea ing Shop in Dea bo n, MI. The
piece also includes ield eco dings o powe looms om The Wea ing Mill in Chicago, IL, eco ded
by Je Milam. The ield eco dings ha e been sequenced and p ocessed using Able on Li e and
FluCoMa’s audio slicing, PCA dimensionali y educ ion, and K-means clus e ing algo i hms.
2
1.4 P ojec ed Visuals
The p ojec ed isuals consis o ideos o a 19 h cen u y machine shop and a Jacqua d loom a
G een ield Village in Dea bo n, MI; black and whi e wea ing pa e ns; and pho og aphs om The
Chicago Wea ing School and G een ield Village. These isual ma e ials a e collaged using Ke in
K ippe ’s VSyn h Max package and Ji e .
2 P og am No es
In o ma ion S o age and Re ie al is an audio isual composi ion inspi ed by h ee classes o machines:
digi al looms, playe pianos, and ea ly compu e s. These machines ha e in e wined his o ies,
sha ed con ol mechanisms, and common abs ac da a s uc u es. All h ee classes o machines a e
ully digi al and ully physical, all mechanize wo k p e iously done by human hands, and all u n
in o ma ion s o ed on punched pape in o ma e ial a i ac s. Pianis s, p og amme s, and wea e s use
hese punched pape a i ac s o s o e da a as g ids o holes, a da a s uc u e ha mi o s g idlike
wea ing pa e ns, s a music no a ion, and ma ices o numbe s. This composi ion celeb a es he
commonali ies o hese machines. The piece ea u es ield eco dings o looms; MIDI sequences
gene a ed by ans o me models ained on my o iginal composed ma e ial; p ojec ed digi al ideo
collages o wea ing pa e ns; and a pe o me con olling lap op, p ojec o , and p epa ed playe
piano.
3 Media
Fo an example o one o he anima ions c ea ed om wea ing pa e ns, please see Wea ing Pa e n
Anima ion.
Fo a ough eco ding o he piece, please see In o ma ion S o age and Re ie al.
Fo mo e ho ough his o ical con ex , please see his ex ended pape .
Acknowledgmen s and Disclosu e o Funding
Thank you o Je Milam and Emily Win e o gene ously dona ing ield eco dings om The
Wea ing Mill. Thanks o Na alie Boye o he Chicago Wea ing School and wea e s Deb Ade ,
Cindy Ande son, Ma shall Caal, Jess Hoppe , Te y Las o icka, Lydia Shepa d, Caleb She idan, Deb
Siddall, K is en Wa son o he in e iews and loom sounds. Thanks o docen Mindy a he G een ield
Village Wea ing Shop o he Jacqua d loom demons a ion and con e sa ion. Many hanks o P o s.
Zeynep Özcan, He man Dong, and Michael Gu e ich o hei eedback and encou agemen .
Re e ences
Chang, W.-C. (2024). X- ans o me s. h ps://gi hub.com/lucid ains/x- ans o me s.
Fuegi, J. and F ancis, J. (2003). Lo elace & babbage and he c ea ion o he 1843 ’no es’. IEEE
Annals o he His o y o Compu ing, 25(4):16–26.
Haw ho ne, C., S asyuk, A., Robe s, A., Simon, I., Huang, C.-Z. A., Dieleman, S., Elsen, E.,
Engel, J., and Eck, D. (2019). Enabling ac o ized piano music modeling and gene a ion wi h he
MAESTRO da ase , e sion 3.0.0. In In e na ional Con e ence on Lea ning Rep esen a ions.
Kau , R., Kuma , P., and Singh, R. P. (2014). A jou ney o digi al s o age om punch ca ds o cloud.
IOSR Jou nal o Enginee ing, 4(3):36–41.
King, E. (2019). Musica Uni e salis - The Pu sui o Pu e Abs ac ion. M.F.A., Uni e si y o
Missou i - Columbia, Uni ed S a es – Missou i. ISBN: 9781392465905.
Mano ich, L. (2002). The language o new media. MIT p ess.
Pos el, V. (2020). The ab ic o ci iliza ion: How ex iles made he wo ld. Basic Books.
3
Vaswani, A., Shazee , N., Pa ma , N., Uszko ei , J., Jones, L., Gomez, A. N., Kaise , Ł., and
Polosukhin, I. (2017). A en ion is all you need. Ad ances in neu al in o ma ion p ocessing
sys ems, 30.
Wahl, R. S. (2018). The His o y o Punched Ca ds: Using Pape o S o e In o ma ion. In The
Rou ledge Companion o Media Technology and Obsolescence, pages 27–45. Rou ledge.
4
