Information Storage and Retrieval: A Composition about Digital Looms, Player Pianos, and Early Computers

Information Storage and Retrieval: A Composition about Digital Looms, Player Pianos, and Early Computers Molly Jones Department of Performing Arts Technology School of Music, Theatre, and Dance University of Michigan Ann Arbor, MI 48109 USA [email protected] Abstract Information Storage and Retrieval is an audiovisual composition inspired by three classes of machines: digital looms, player pianos, and early computers. These machines have intertwined histories, shared control mechanisms, and common abstract data structures. All three classes of machines are fully digital and fully physical, all mechanize work previously done by human hands, and all turn information stored on punched paper into material artifacts. Pianists, programmers, and weavers use punched paper to store data as grids of holes, a data structure that mirrors gridlike weaving patterns, staff music notation, and matrices of numbers. This composition celebrates the commonalities of these machines. The piece features field recordings of looms; MIDI sequences generated by transformer models trained on the composer’s original material; projected digital video collages of weaving patterns; and a performer controlling laptop, projector, and prepared player piano live. 1 Project Description Information Storage and Retrieval is a ten-minute audiovisual composition inspired by the intertwined histories, shared control mechanisms, and common abstract data structures of three classes of machines: digital looms, player pianos, and early computers (Wahl, 2018; King, 2019; Fuegi and Francis, 2003). A performer triggers and processes fixed stereo audio tracks, MIDI clips played by a prepared player piano, and digital video collage. The performer sends MIDI data to Max 8/Jitter, Ableton Live, and a player piano using a Novation Launchpad X. 1.1 Historical Context Early computers were controlled with punched paper cards, a technology directly inspired by 18th century digital Jacquard and Bouchon looms (Fuegi and Francis, 2003; Wahl, 2018; Kaur et al., 2014; Postrel, 2020; Manovich, 2002). Lady Ada Lovelace proposed punched cards to Charles Babbage as a mechanism by which a human might express their wishes to computing machines (Wahl, 2018), writing “The Analytical Engine weaves algebraical patterns just as the Jacquard loom weaves flowers and leaves,” (Manovich, 2002). In an 1843 letter, Lovelace proposed the musical potential of the Analytical Engine: “[T]he Engine might compose elaborate and scientific pieces of music of any degree of complexity or extent...” (Fuegi and Francis, 2003), presaging the adoption of computers in music making. A punched paper control mechanism originally devised by Basile Bouchon, a Lyon organ maker, had revolutionized weaving, which in turn had suggested an interface for computers, and now this interface’s inventor proposed these computing machines as tools to create music, closing a circle of mutual influence. Pianists, programmers, and weavers use punched paper to store data Proceedings of the 6th Conference on AI Music Creativity (AIMC 2025), Brussels, Belgium, September 10th-12th Figure 1: Player piano preparations. The number of magnets depicted on a given note are stacked and placed on the strings. as grids of holes, a data structure that mirrors gridlike weaving patterns, staff music notation, and the matrices of numbers central to many machine learning algorithms. The commonalities of these machines and their data structures inspired me to create a composition highlighting the sounds, visuals, and capabilities of looms, player pianos, and computers. 1.2 Prepared Player Piano with Transformer Models I employed a player piano and transformer models to create a hybrid human-machine audio-visual performance that draws on the statistical distributions and patterns inherent in my original composed material. An electronic player piano plays sequences of MIDI data that I trigger using the Launchpad X. A Bösendorfer grand piano with CEUS reproducing system plays phrases of material generated by both myself and transformer models trained on my composed material. The piano is prepared with neodymium magnets and alligator clips. For a description and images of the piano preparations, please see Figure 1. I trained two X-Transformers decoder-only models (Chang, 2024; Vaswani et al., 2017). I trained one on the MAESTRO dataset (Hawthorne et al., 2019) and one only on my own original composed MIDI sequences specific to Information Storage and Retrieval. Both trained models were prompted with the 83 original sequences I wrote specifically for the piece. The intention of this strategy was to prompt the models to extend the sequences in a style resembling my own compositional decisions and aesthetic. The experiment was to determine whether the MAESTRO-trained model or the model trained only on my own material would generate sequences closer to my musical preferences. The finding was that both models quickly tended to reproduce exactly the sequences I wrote with little or no elaboration. Experiments with model simplification to remedy this memorization problem are ongoing. 1.3 Field Recordings Information Storage and Retrieval features my field recordings of digital looms and the voices of weavers at the Chicago Weaving School and Greenfield Village Weaving Shop in Dearborn, MI. The piece also includes field recordings of power looms from The Weaving Mill in Chicago, IL, recorded by Jeff Milam. The field recordings have been sequenced and processed using Ableton Live and FluCoMa’s audio slicing, PCA dimensionality reduction, and K-means clustering algorithms. 2 1.4 Projected Visuals The projected visuals consist of videos of a 19th century machine shop and a Jacquard loom at Greenfield Village in Dearborn, MI; black and white weaving patterns; and photographs from The Chicago Weaving School and Greenfield Village. These visual materials are collaged using Kevin Kripper’s VSynth Max package and Jitter. 2 Program Notes Information Storage and Retrieval is an audiovisual composition inspired by three classes of machines: digital looms, player pianos, and early computers. These machines have intertwined histories, shared control mechanisms, and common abstract data structures. All three classes of machines are fully digital and fully physical, all mechanize work previously done by human hands, and all turn information stored on punched paper into material artifacts. Pianists, programmers, and weavers use these punched paper artifacts to store data as grids of holes, a data structure that mirrors gridlike weaving patterns, staff music notation, and matrices of numbers. This composition celebrates the commonalities of these machines. The piece features field recordings of looms; MIDI sequences generated by transformer models trained on my original composed material; projected digital video collages of weaving patterns; and a performer controlling laptop, projector, and prepared player piano. 3 Media For an example of one of the animations created from weaving patterns, please see Weaving Pattern Animation. For a rough recording of the piece, please see Information Storage and Retrieval. For more thorough historical context, please see this extended paper. Acknowledgments and Disclosure of Funding Thank you to Jeff Milam and Emily Winter for generously donating field recordings from The Weaving Mill. Thanks to Natalie Boyett of the Chicago Weaving School and weavers Deb Ader, Cindy Anderson, Marshall Caal, Jess Hopper, Terry Lastovicka, Lydia Shepard, Caleb Sheridan, Deb Siddall, Kristen Watson for the interviews and loom sounds. Thanks to docent Mindy at the Greenfield Village Weaving Shop for the Jacquard loom demonstration and conversation. Many thanks to Profs. Zeynep Özcan, Herman Dong, and Michael Gurevich for their feedback and encouragement. References Chang, W.-C. (2024). X-transformers. https://github.com/lucidrains/x-transformers. Fuegi, J. and Francis, J. (2003). Lovelace & babbage and the creation of the 1843 ’notes’. IEEE Annals of the History of Computing, 25(4):16–26. Hawthorne, C., Stasyuk, A., Roberts, A., Simon, I., Huang, C.-Z. A., Dieleman, S., Elsen, E., Engel, J., and Eck, D. (2019). Enabling factorized piano music modeling and generation with the MAESTRO dataset, version 3.0.0. In International Conference on Learning Representations. Kaur, R., Kumar, P., and Singh, R. P. (2014). A journey of digital storage from punch cards to cloud. IOSR Journal of Engineering, 4(3):36–41. King, E. (2019). Musica Universalis - The Pursuit of Pure Abstraction. M.F.A., University of Missouri - Columbia, United States – Missouri. ISBN: 9781392465905. Manovich, L. (2002). The language of new media. MIT press. Postrel, V. (2020). The fabric of civilization: How textiles made the world. Basic Books. 3 Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30. Wahl, R. S. (2018). The History of Punched Cards: Using Paper to Store Information. In The Routledge Companion to Media Technology and Obsolescence, pages 27–45. Routledge. 4