Legato: una herramienta de música adaptativa para videojuegos inspirada en técnicas cinematográficas

Lega o: una he amien a de música adap a i a pa a ideojuegos inspi ada en écnicas cinema og á icas Lega o: an adap a i e music oolse o ideo games based on cinema ic echniques T abajo de Fin de G ado Cu so 2023–2024 Au o es Emile de Kad Pablo E ayo Rod íguez Di ec o es Jaime Sánchez He nández Miguel Gómez-Zamalloa Gil G ado en Desa ollo de Videojuegos Facul ad de In o má ica Uni e sidad Complu ense de Mad id Lega o: una he amien a de música adap a i a pa a ideojuegos inspi ada en écnicas cinema og á icas Lega o: an adap a i e music oolse o ideo games based on cinema ic echniques T abajo de Fin de G ado en Desa ollo de Videojuegos Au o es Emile de Kad Pablo E ayo Rod íguez Di ec o es Jaime Sánchez He nández Miguel Gómez-Zamalloa Gil Con oca o ia: Junio 2024 G ado en Desa ollo de Videojuegos Facul ad de In o má ica Uni e sidad Complu ense de Mad id 27 de mayo de 2024 Dedica o ia A Jaime, po dis u a an o de lo que enseña y a Miguel, po pone le los pies en el suelo cuando oca. Ag adecimien os A odo ASCII, po man ene mis ganas de i i es os cinco años. -Emile A mi amilia, po anima me a es o za me du an e es os años de ca e a. -Pablo Bueno, sí, a mi amilia ambién, cla o es á. -Emile ii Abs ac Lega o: an adap a i e music oolse o ideo games based on cinema ic echniques This p ojec p o ides ideo game de elope s wi h a se o ools o c ea e adap i e sound acks o hei games, which eac o game e en s, using echniques inspi ed by cinema sco es. The echniques ha inspi ed his p ojec (and he ones i he e o e acili a es) e ol e mainly a ound he cons uc ion o a “musical iden i y” o di e en aspec s o a scene o na a i e. A ce ain in-game loca ion may ha e an associa ed ins umen wi hin he sound ack; a e i ying illain migh cue an ominous lei mo i ; he game’s music may slow down d as ically du ing key playe decisions. Wi h he ools p o ided by his p ojec , elemen s wi hin he game can easily be assigned a musical iden i y, a ec ing he sound ack in a dynamic manne . Many sec ions o his documen assume a basic amilia i y wi h he Uni y game engine on behal o he eade . Keywo ds Adap i e music, ideo game engines, Uni y, Reape , musical composi ion, ideo game music, OSC. ix Cap´ ı ulo 1 In oduc ion “I you alk o any di ec o , hey’ll say music is i y pe cen o he mo ie.” — Hans Zimme 1.1. Mo i a ion E en in hei e y i s p ojec ions, he Lumiè e b o he s elied on a pianis o accompany hei sho ilms (Tudo , 2017). P eda ing e en he in oduc ion o dialogue o he medium, he sound ack has always been an indispensable pa o cinema. I is ha d o pic u e he g ea p o agonis s o cinema wi hou e oking he music ha he alded hei p esence. Don Co leone’s agic iolin segmen ; he hunde ous Impe ial Ma ch o he Galac ic Empi e; he lu e, oca ina and choi ha de ine he eponymous “The Good, he Bad and he Ugly” despi e playing he same une (Classic FM, 2014); he ull cas and s o y o he Spide - e se, made ha much mo e ib an by he a ay o echniques, ins umen s and wa e owl ha Daniel Pembe on spun in o hese ilms (Sony Sound acks, 2023). The iden i y o hese cha ac e s is jus as audible as i is isual. Many o hese amous sounds a e lei mo i s, melodic igu es o ph ases associa ed wi h he epea ed appea ance o a cha ac e , idea o si ua ion. O he s, like “The Good, he Bad and he Ugly”, use ins umen s o iden i y hei cha ac e s. This concep o “musical iden i y” ascina ed us. As game de elopmen s uden s, we wan ed a way o employ hese echniques in ou own games. The sound ack o a ideogame has an allu e compa able o ha o a ilm; howe e , sa e o ce ain excep ions like hy hm games, he sequence o e en s ha occu in a game is no de e minis ic. The decisions o one o mo e playe s, as well as o he andom e en s, make i impossible o compose songs ha pe ec ly esona e wi h each momen o he ac ion. Adap i e music as an a - o m ose as a way o pa ch his issue: a se ies o edi ing and sequencing echniques ha al e a game’s music in eal ime o be e ollow he e en s on sc een. The music may be al e ed by changing he ins umen a ion, adjus ing he empo ( he speed a which he music is played) o in oducing new 1 2Capí ulo 1. In oduc ion melodies in o he cu en song. We had a e y clea men al image (o a he , men al sound) o wha we wan ed: a ideogame sco e ha uses musical language o desc ibe he ac ion on sc een a any gi en ime, by combining signa u e cha ac e lei mo i s and ins umen s in ha mony wi h each o he . As game de elope s, we looked o middlewa e ( ha is, so wa e ha wo ks unde nea h he game’s code bu abo e he ope a ing sys em) ha would allow us o c ea e music o ou games s aigh om he game engine using hese echniques. Un o una ely, we could no ind any hing ha qui e ma ched ou speci ica ions. Compose s and game de elope s use such ools as DAWs and acke s ( wo di e- en ypes o applica ions used o p oducing music) o c ea e and in eg a e sound- acks in o games. Howe e , hese applica ions s ill equi e human in e en ion and can’ necessa ily be con olled om he game engine. This is impo an , since a ga- me’s logic is p og ammed wi hin he engine; i would be unwieldy o ha e o edi he sou ce iles o he sound ack e e y ime a change in gameplay a ec s how and when he music will be played. Wi hou a specialised ool, he unc ionali y we wan would become e y messy o implemen . E e y possible combina ion o lei mo i and ins umen would need o be ende ed indi idually in e e y empo. E en in specialised en i onmen , such as a DAW, his ask would become edious wi hou au oma ion. Addi ionally, synch onising a sound ack made up o so many sca e ed pieces would be i ’s own s uggle wi hou a cen alised handle o he music. These conside a ions helped us na ow down he equi emen s o he wo k needed o c ea e such a ool ou sel es: an adap i e music sys em ha bo ows musical echniques om cinema, allowing o complica ed, high-cos ins umen s and e ec s wi hou demanding la ge amoun s o p ocessing powe om he playe ’s compu e . 1.2. Key de ini ions Ho izon al e-sequencing: se o adap i e music echniques in which di e- en audio clips a e played one a e he o he , changing he o de o choice o clips in esponse o e en s wi hin he game. Ve ical e-o ches a ion: se o adap i e music echniques in which di - e en pa s o he sound ack ha a e played simul aneously a e al e ed in esponse o in-game ac ions, esul ing in a di e en inal mix. MIDI (Musical Ins umen Digi al In e ace): digi al encoding o ma o music used o communica ion be ween applica ions o de ices. MIDI i ems on hei own s o e no in o ma ion abou he music’s sound; only he abou he no e ha is played, as well as addi ional da a like he iming and way in which he no e is played. DAW (Digi al Audio Wo kspace): de ice o so wa e used o eco ding, edi ing and p oducing sound audio. Among o he hings, a DAW allows o 1.3. Goals 3 he ende ing o sound iles om an encoding o ma (such as MIDI) and lis o e ec s ha combine o de e mine wha sounds a e p oduced by each no e. Vi ual ins umen : so wa e o gene a ing sound om a symbolic ep esen- a ion (wi hin he scope o his p ojec , gene ally MIDI). These ins umen s se e as plug-ins wi hin a DAW. OSC (Open Sound Con ol): ne wo k p o ocol o mul imedia communi- ca ion be ween sound de ices. C oss ade: o change om one sound o ano he ia a g adual ading ou o one clip as ano he is aded in. Quan isa ion: wi hin he con ex o adap i e music, he p ocess o co ec- ing and synch onising di e en sou ces o music so ha hey sound pe ec ly aligned wi h hei empo. BPM (Bea s Pe Minu e): uni o measu emen o he music’s empo. Posi ional audio: in ideogames, any sound ha simula es coming om a speci ic poin wi hin he game space. 1.3. Goals The aim o his p ojec is o c ea e an ex ension o Uni y ha would include ools o he inclusion o adap i e music in ideogames using he abo e men ioned echniques. Uni y is a game engine wi h a e y wide use base. I was mainly chosen due o ou amilia i y wi h i , hough i has o he ad an ages, as explo ed in chap e 2. Since he ool is buil wi hin and o use wi h Uni y, i is also a ge ed a Uni y de elope s. The ool should simpli y he p ocess o building a musical iden i y o di e en cha ac e s, hemes and loca ions in he game’s na a i e, by al e ing he ambien sound ack whene e hose na a i e elemen s a e in a scene. While he ac ual com- posi ion o he music should s ill occu wi hin a DAW, a de elope should be able o make any changes o he logic o how he sound ack is used wi hou lea ing he game engine. This oolse should: Allow o he modula inclusion o lei mo i s in o a game’s sound ack. Allow pa s o he sound ack o ha e hei ins umen a ion changed in un i- me. Allow o he empo o he sound ack o be changed in un ime. Place a low demand o esou ces on he game engine. No equi e compose s o unde s and any p og amming. 4Capí ulo 1. In oduc ion No equi e p og amme s o unde s and music gene a ion so wa e. Place as ew es ic ions as possible on he music and e ec s ha he compose can c ea e o he game. Be easy o use. 1.4. Wo k plan In o de o ul il hese objec i es, he wo k was spli in o he ollowing pa s, some o which we e de eloped in pa allel: Resea ch o exis ing echnologies. Ra he han w i e all o he code om sc a ch, di e en applica ions we e s udied o be used as a ounda ion upon which o build he ool. This would also make he ool mo e accessible, by lin- king i wi h exis ing echnologies ha po en ial use s would al eady be amilia wi h. Au oma ing he ende ing o sound iles. Since he ool will use p e- ende ed sound asse s, hese will need o be c ea ed au oma ically based on use -es ablished pa ame e s. C ea ion o he use in e ace o he ool’s con igu a ion. The de elope should ha e access o a con ol panel wi hin he game engine i sel . C ea ion o game asse s ha could ep esen he dynamic elemen s o he sound ack. To make he ool mo e isual, he adap i e lei mo i s, ins umen s and empos will be ep esen ed as asse s wi hin he game engine. These asse s will need o be egene a ed e e y ime new audio samples a e ende ed. P og amming he dynamic playback logic. The adap i e playback sys em is he co e o he ool. O he han o de ing he en ance and exi o he dynamic elemen s, he de elope should no ha e o manually con ol he synch onisa- ion, quan isa ion o playback o he music. Cap´ ı ulo 2 S a e o Play Lega o is a oolse o c ea ing adap i e music o ideogames. As such, i would be poin less o s udy o he simila ools (i s “compe i ion”, as i we e) wi hou i s examining a ew illus a i e examples o adap i e music in popula ideogames. 2.1. Adap i e music in ideogames While he ollowing examples do no communica e he ull capabili ies and li- mi a ions o he ools used o c ea e hem, hey do pain a pic u e o he gene al equi emen s ha any simila ool mus ul il. This o cou se canno be conside ed an exhaus i e analysis o all he echniques used in he medium, bu i highligh s a ai ly ep esen a i e selec ion. Doom E e nal (id So wa e, 2020) is a high-ac ion, i s -pe son shoo e game wi h a hea y me al sound ack. I s sco e employs a a ie y o echniques o cap u e he game’s ene gy o an ic, non-s op ac ion: •A e ical e-o ches a ion sys em ha uses di e en acks o sco e di - e en le els o comba in ensi y. •A ho izon al e-sequencing sys em ha ansi ions be ween di e en songs when he playe mo es be ween scenes. These ansi ions a e occluded by addi ional diege ic and non-diege ic sound e ec s ha play o e hem. •When he playe ’s heal h is low, a low-pass il e is applied o all o he game’s music. This is a common e ec ac oss he ac ion game gen e. (Po kb ain, 2020) Ma io Ka 8 (Nin endo, 2014) is a mul iplaye acing game wi h a unique soun ack o e e y ace ack. The ack Moun Wa io makes no able use o ho izon al echniques; he le el uses 4 di e en songs ha play du ing hei espec i e pa s o he ack. Each ime a playe eaches a new zone, he cu en ly playing song con inues un il he end o i s nex measu e. Then, i ansi ions di ec ly in o a pe cussion-only b idge ha leads in o he nex song (8-bi Music Theo y, 2019). 5 6Capí ulo 2. S a e o Play Dead Space (Mo i e S udio, 2023) is a su i al/ho o game wi h hi d-pe son shoo e mechanics. The sound ack makes use o “alea o ic” echniques in he pe o mance o he music. This on i s own is no adap i e music, since he andomness is baked in o he eco ding, no gene a ed du ing gameplay (Pepia , 2023). Lu ause s (Vlambee , 2014) is a 2D a cade-s yle game in which he playe con ols a cus omisable igh e plane, dodging a acks and shoo ing down o he a ge s. The plane has h ee pa s ha can be cus omised. Each o hese pa s has a co esponding laye o he sound ack, and each selec able op ion has an associa ed ack. This way, each o he 125 possible combina ions o pa s c ea es a unique plane wi h a unique sound ack (Ligman, 2014). The Legend o Zelda: Skywa d Swo d (Nin endo, 2011) is a medie al an asy ad en u e game in which he playe mus explo e he map and i s a ious dungeons. Some o hese dungeons employ a e ical e-o ches a ion echnique: hey ha e a single ambien ack h oughou he whole dungeon ha has speci ic mo i s laye ed on op based on he cu en oom he playe is is (Game Make ’s Toolki , 2014). Wii Play (Nin endo, 2006) is a game wi h 9 mini-games, each one ocused on highligh ing ce ain capabili ies o he Wii Remo e. The las o hese mini- games, Tanks!, allows one o wo playe s o pa ake in oy ank ba les. The sound ack o his mini-game employs a combina ion o e ical echni- ques: a single, sho loop is played o e se e al laye s o music. Each enemy wi hin he game de e mines wha ack plays on laye . The music is a ec- ed no only by which ypes o enemies a e p esen , bu also by how many he e a e and whe he ha is he s onges enemy emaining (Sc u y, 2022; isa enewspape s8890 y BigKi y1011, 2022). 2.2. Exis ing echnology I is o en di icul o iden i y which ools we e used o p og am he adap i e music o a game; mo eso, ansi ions be ween s a es o he music a e equen ly p og ammed di ec ly om he game’s code, wi hou he use o specialised ools (middlewa e). 2.2.1. Sequence s Sequence s we e widely used in he 80’s, as a e y e icien way o compiling a game’s sound ack. In a ime we e e e y kiloby e was p ecious, s o ing an en i e sound ile in a game ca idge was oo cos ly. Ins ead, music was gene a ed wi h a soundchip and a lis o no es, simila o how a music box unc ions. Since he music 2.2. Exis ing echnology 7 was in e p e ed in eal- ime, his made i e y easy and e icien o change he speed o ins umen a ion o he music. Howe e , sequence s don’ scale well: he old games ha used hem (as well as newe games looking o emula e he e o" eel) used e y simple syn hesize s o play he game’s music. By con as , mode n i ual ins umen s can use la ge amoun s o memo y. I Lega o is o be compa ible wi h any sound he use wishes o use, i has o use p e- ende ed audio. Wi h his in mind, i is wo h exposing he capabili ies o h ee ools wi h pa - icula ele ance o his p ojec : 2.2.2. Reape Reape is a DAW ha allows a g ea amoun o eedom o sequencing music using i ual ins umen s and o he e ec s. These ins umen s can include any VSTi plugin, which is he comme cial s anda d o i ual ins umen s. The p og am can be managed wi h a wide a ay o ac ions, uni a y ins uc ions each wi h a e y speci ic unc ionali y. These ac ions do no ecei e a gumen s, hough in some cases hey deploy pop-up widows o he use o allow he use o manually con igu e ce ain pa ame e s. Some o hese ac ions a e ied o keyboa d sho cu s o hei o dina y use. Fo example, he ac ion “File: Sa e p ojec ” is associa ed o he keyboa d sho cu “C l + S” (CockOS Wiki, 2013). In addi ion o i s ac ions, Reape has i s own sc ip ing language called ReaSc ip . ReaSc ip can be w i en in EEL2, Lua o Py hon. We op ed o use Lua, since i is he mos easily accessible om wi hin Reape i sel and due o ou amilia i y wi h he language. ReaSc ip is e y poo ly documen ed; he as majo i y o i s unc ions ha e only a hand ul o wo ds o desc ibe hem, and e y ew o hem o e any explana ion abou hei pa ame e s. A ideo by Julio Ca eño was he mos use ul esou ce o lea ning how o use ReaSc ip in any capaci y (Ca eño, 2023). Reape has he abili y o ecei e ins uc ions ia OSC (Open Sound Con ol). This is designed p ima ily o acili a e communica ion wi h elec onic o digi al ins- umen s. Howe e , o his p ojec , i also p o ed a use ul means o c oss-applica ion communica ion. One o he p ojec ’s goals is o in e e e as li le as possible wi h he c ea i e oppo uni ies a ailable o he use s o he oolse . 2.2.3. FMOD S udio As a sound e ec s engine, FMOD no only has a e y accessible paymen model, bu i is one o he mos powe ul and widely used ools on he ma ke . These s eng hs a e no limi ed o adap i e music, as FMOD is use ul o mos all audio- ela ed asks in game de elopmen . In he ea ly s ages o he p ojec , FMOD was conside ed as a g ea po en ial ounda ion upon which o build he ool. Howe e , i e en ually p o ed un easo- nable, as he addi ional laye s o au oma ion equi ed o es ablish communica ion wi h FMOD s udio, as well as c ea e FMOD p ojec s ou o he samples ende ed in 8Capí ulo 2. S a e o Play Reape , would ha e equi ed mo e wo k han simply ec ea ing he main unc iona- li y ha FMOD would ha e p o ided o his pa icula ool (namely quan isa ion). A he same ime, using FMOD would ha e placed addi ional es ic ions upon he end use , o cing hem o use FMOD o he whole p ojec (since FMOD’s Uni y plugin is incompa ible wi h Uni y’s na i e sound engine). This on i s own would no ha e been a majo d awback hough, due o FMOD’s exis ing popula i y. 2.2.4. Uni y Uni y is a componen -based game engine wi h a isual edi o o building game scenes. Is is used ex ensi ely ac oss he indus y o i s amilia i y, popula i y, clea documen a ions and wide communi y suppo . Uni y’s audio engine, like Uni y i sel , is no he mos powe ul op ion, bu i is e sa ile and easy o use. The main pa s o Uni y’s audio engine (as conce n his p ojec ) a e he ollowing: The audio clock. This in e nal clock acks ime much mo e p ecisely han he es o he engine, since i is based on he numbe o audio samples ha a e p ocessed. The AudioSou ce componen . This componen plays a single audio clip. I has con igu able olume and posi- ionali y alues as well as public unc ions o pause, play, s op and change he clip. One no ewo hy unc ion is he PlayScheduled me hod, which allows he use o choose an exac u u e momen in ime (based on he audio clock) o he clip o begin playing. Howe e , his unc ion has a weakness: since he com- ponen can only s o e one clip a a ime, he componen canno simul aneously play one clip while ano he is scheduled o begin playing. The AudioLis ene componen . This componen de e mines he ela i e posi ion om which all posi ional sounds in he scene a e hea d. In o de o unc ion co ec ly, he e can be no mo e han one in he same scene. Audio Mixe s. Uni y’s mixe s a e a ype o asse ha allows o di e en audio sou ces o be mixed and ha e e ec s applied o hem. An audio mixe can ecei e inpu om mul iple AudioSou ce componen s as well as o he AudioMixe s. A wide a ie y o ools can be ound in he Uni y Asse S o e when explo ing he audio sec ion. The e a e ools o analysing songs o ex ac in o ma ion abou hem (HelloMeow, 2024), ools o c ea ing mo e ealis ic 3D sound (Techmo Voice Technologies, 2019), o hy hm isualiza ion (Legend, 2020)... bu he e a e no exis ing ools ela ed o he inco po a ion o lei mo i s. Cap´ ı ulo 3 The Toolse Lega o is a package ha can be downloaded om gi hub (de Kad y E ayo, 2024). I allows o he in eg a ion o an adap i e music sys em in o a p ojec ’s scenes. Wi h he addi ion o a single Uni y componen , a game objec can be g an ed one o mo e musical elemen s, which can be cued in o he sound ack wi h a single public me hod call. All o he sc ip s p o ided in he package (wi h he excep ion o he ones ha a e used exclusi ely in he demo scenes) a e con ained wi hin he namespace Lega o. The ool consis s o wo main pa s: one o acili a e he ende ing o he equi ed samples in Reape and ano he o allow he con igu a ion and execu ion o he ool in Uni y. Chap e s 3 and 4 assume a backg ound in Uni y o a ull unde s anding o hei con en . 3.1. Au oma ed ende ing in Reape In o de o unc ion, he ool equi es audio esou ces wi h a speci ic o ma and name s uc u e. Included as pa o he package is a cus om Reape ac ion ha ecei es a se o speci ica ions om he Uni y p ojec and ende s all he necessa y audio iles. Fo his ac ion o wo k p ope ly, he Reape p ojec mus be se up in he ollowing o ma : The i s acks o appea mus co espond one- o-one wi h he di e en ins- umen s ha will be used in he game’s adap i e sound ack. The ins umen acks mus con ain no MIDI o sound elemen s, only i ual ins umen s and o he e ec s. The e is no addi ional limi a ion o he sounds ha can be used as ins umen s: any combina ion o e ec s ha Reape is able o p ocess is alid. The emaining acks mus co espond one- o-one wi h he lei mo i s ha will be used in he sound ack. These acks can con ain e ec s, bu hese will no appea anywhe e in he inal ende s. Bo h he ins umen acks and he mo i acks mus appea in he same o de 9 16 Capí ulo 3. The Toolse when he e a e no addi ional mo i s queued om game e en s. He e i is im- po an o de ine he dis inc ion be ween a agmen and a mo i wi hin he con ex o he channel: a mo i is an objec o he F agmen class ha is queued in o he emi e by an ex e nal objec . Mo i s a e s o ed in he channel in an in e nal dynamic p io i y queue and always ake p io i y o e he es o he agmen s. All mo i s a e agmen s, no all agmen s a e mo i s. Algo i hm The selec ion algo i hm ha is used o a ange he agmen s om he p e ious a ay. The e a e h ee op ions o choose om a d op-down: •Ci cleQueue: he agmen s a e played in he o de hey appea in he a ay, looping back o he s a a e he end. •Random: each successi e agmen is chosen a andom om ou o he whole a ay. •Shu le: he agmen s a e played in a andom o de ; once all agmen s ha e been played, hey a e compiled in o a new andom o de and played again. Volume Con ols he channel’s olume. Ins umen De e mines he ins umen ha his channel will be played in a he s a . Tempo De e mines he empo ha his channel will be played in a he s a . Bea sPe Measu e De e mines he numbe o bea s pe measu e. This doesn’ al e he sound p oduced by he indi idual agmen s, i only a ec s he quan isa ion used o synch onising channels and changing be ween agmen s a he igh ime. PlayOnS a Boolean alue ha de e mines whe he he channel will begin playing a he s a o he scene, o emain inac i e un il i is gi en he command o s a . 3.3.3. Public in e ace o he Emi e Aside om he isual in e ace in he Uni y edi o , he emi e has public me hods ha allow o he ollowing unc ionali y: Change he olume o he emi e , o o a speci ic channel in he ac i e sound scene. Change he ac i e sound scene o ano he one in he SoundScenes a ay. Change he spa ialisa ion alues o sound scenes. 3.3. Emi e 17 Ac i a e a mo i in a speci ic channel. Cancel a mo i ha is queued bu no playing ye , o in e up one ha is al eady playing. Pause, esume o s op one o all channels in he cu en sound scene. S opping a channel means ha when i is esumed, i will pick up a he s a o he nex measu e, playing i s nex agmen . A paused channel will esume exac ly whe e i le o , and may all ou o sync wi h o he channels when esumed. Change he ins umen o a channel in he ac i e sound scene. Change he empo o a channel in he ac i e sound scene. Failed calls (such as a emp ing o change a channel o an ins umen ha he cu en sample is no ende ed in) will p oduce clea e o codes in he Uni y log, speci ying he ende ed asse ha was no ound. 3.3.3.1. Dynamic p io i y o mo i s The goal o his p ocess is o help make mo i s play as close as possible o when hey a e needed, in si ua ions whe e he game may be igge ing a la ge numbe o mo i e en s in a sho span o ime. Mainly, i aids in wo speci ic cases: When i is impo an o a mo i o sound close o he momen i was igge ed, because i loses meaning i i sounds oo la e. This would call o a g adual inc ease in p io i y. I a mo i is only wo h playing close o he momen i was igge ed, and becomes less ele an o e ime. This would call o a g adual dec ease in p io i y. When a mo i is queued by a game e en , i can be assigned a p io i y alue. A he s a o he nex measu e o a he end o he cu en agmen (de e mined by he use ) he mo i wi h he highes p io i y alue in he queue (i any) will play. In addi ion o i s ini ial p io i y alue he mo i can be gi en an inc ease o dec ease in p io i y o e ime. Bo h o hese alues ( he ini ial p io i y and he inc ease/dec ease p io i y) mus be gi en a he momen he mo i is queued. This alue is in e p e ed in uni s pe second. This means ha i a mo i is gi en an ini ial p io i y alue o 5 and a p io i y o e ime alue o -1 will ha e a p io i y o 2 a he poin in ime 3 seconds a e i was queued. A mo i ha e e eaches a p io i y equal o less han 0 will be disca ded om he queue. 18 Capí ulo 3. The Toolse 3.3.3.2. Ins umen c oss ading Any change in a channel’s ins umen can be con igu ed o occu suddenly o g adually, ia a c oss ade. In nei he case will he change happen he ins an he e en is igge ed, as i is necessa y o i s load he clip using he new ins umen ( his p ocess ypically akes a ac ion o a second). An ins umen change can be p og ammed o begin as soon as possible o a he s a o he nex agmen . In he la e case, he nex agmen will begin playing wi h he new ins umen , wi hou c oss ading o any addi ional loading ime. I addi ional ins umen changes a e igge ed be o e a p e ious one inishes, he las o he igge ed changes will be s o ed and ake e ec when he cu en change is inished. Figu e 3.8 illus a es how he p og am handles mul iple successi e ins umen changes on he same channel. Each ins umen is ep esen ed by a colou , wi h he e ical dashed lines ep esen ing he momen in ime ha an ins umen change is eques ed. The cu ed lines ep esen he c oss ade i sel , as one ins umen ades ou and ano he ades in. In igu e 3.8, ins umen A (blue) is playing in he channel a he s a . In o de : 1. The channel ecei es a eques o c oss ade o ins umen B (o ange). 2. A e a b ie loading ime, he c oss ade begins. 3. Du ing he c oss ade, he channel ecei es ano he eques o change ins u- men o ins umen C ( ed). Because he channel is al eady in he middle o a c oss ade, he change o C is pos poned un il he end o he cu en c oss ade. 4. A e his, bu s ill be o e he end o he i s c oss ade, he channel ecei es ye ano he eques o change ins umen , his ime o ins umen D (pu ple). Jus like wi h C, he change o D is pos poned un il he end o he cu en c oss ade. Howe e , his change o e w i es he change o ins umen C. 5. Once he A ->B c oss ade is inished, he channel de ec s ha he e is s ill ano he c oss ade queued ( o ins umen D). So, once again, i p epa es o c oss ade by loading he app op ia e clip o D. 6. Du ing his ime, he channel plays using only B, un il he B ->D c oss ade begins. 3.3.3.3. Tempo changes Tempo changes a e no as con igu able as ins umen changes. When he p og am igge s a empo change in he cu en scene, o in a speci ic channel, his will ake place a he s a o he nex measu e. I mul iple changes in empo a e igge ed wi hin he same measu e, he las one igge ed will be he one ha akes place, exac ly as i all he p e ious ones had no been igge ed a all. 3.4. Asse s 19 Fig. 3.8: G aphical ep esen a ion o mul iple consecu i e ins umen changes. 3.4. Asse s The dynamic elemen s o he sound ack ( agmen s, ins umen s and empos) a e all ep esen ed wi hin he Uni y p ojec as asse s. These classes canno be c ea ed manually, only using he Rende and Regene a e Asse s bu ons in he con igu a ion menu. These asse s make i easy o assign objec s om he edi o . Clicking on he Ins umen ield o a channel opens a isual menu ( igu e 3.9) ha allows he use o selec one o he ende ed ins umen s. Ano he op ion is o d ag an ins umen om he Asse s/Resou ces/Lega o/Ins umen s o he desi ed ield. The same p ocess applies o agmen s and empos. Fig. 3.9: Ins umen selec ion menu. 20 Capí ulo 3. The Toolse When e e encing one o hese objec s om code ( o example, o eques an ins umen change), he use has he op ion o pass he asse di ec ly, o o pass i s name as a s ing (o in , in he case o empos). The la e me hod is less e icien hough. 3.5. Auxilia y componen s To complemen he emi e , he ool also includes i e componen s ha sim- pli y he p ocess o ac i a ing e en s. These a e Lega o_Ins umen E en ,Lega- o_Mo i E en ,Lega o_TempoE en ,Lega o_S opE en and Lega o_ResumeE en . These componen s ha e a single public i ual me hod T igge , which se es o ac i- a e he associa ed ins umen , mo i o empo o o s op o esume speci ic channels, espec i ely. Fig. 3.10: Edi o iew o he i e classes ha inhe i om Lega o_E en . 3.6. Demo scenes 21 These i e classes inhe i om he abs ac class Lega o_E en , allowing hem o be used polymo phically. Each componen has addi ional pa ame e s ( igu e 3.10) ha a e isible om he edi o . Lega o_ResumeE en has a pa ame e Only Resume P e iously Playing which, i ma ked along wi h Apply To All, makes he e en only esume acks ha we e playing be o e a call was made o s op all channels. 3.6. Demo scenes The package includes wo demo scenes, ound in Asse s/Lega o/Examples/Scenes. 3.6.1. Lega o_Pad This scene con ains an emi e and a se o bu ons ha modi y he music played by he emi e ( igu e 3.11). The emi e has a single sound scene wi h a single channel. The playe can change he olume o he emi e o he channel, change he ac i e ins umen and empo and queue a mo i . The ChangeCu en oggle allows he playe o decide whe he he mo i will in e up he cu en agmen and whe he he ins umen change will play a he s a o he nex agmen o begin immedia ely by c oss ading. Fig. 3.11: Lega o_Pad demo scene. 22 Capí ulo 3. The Toolse 3.6.2. Lega o_Sample This scene, seen in he igu e 3.12, is mo e simila o a demo o a game. The use can con ol a cha ac e ep esen ed by a blue sphe e o mo e h ough he scene. Mo ing o a ce ain loca ion may igge an e en : When he playe en e s he long co ido , he empo will inc emen . Once ou o i , he empo will se o i s ini ial alue. When he playe app oaches he g een cha ac e (a he bo om- igh co ne o he scene), a mo i associa ed wi h his cha ac e will s a playing. When he playe en e s he ed oom he ins umen s playing will change using a c oss ade. Once he playe is back o he whi e oom, he ins umen will change back ( his ime wi h no c oss ade). Fig. 3.12: Lega o_Sample demo scene. Cap´ ı ulo 4 De elopmen The de elopmen o his p ojec is spli in o h ee main pa s: au oma ing he ende ing in Reape , es ablishing he communica ion be ween Uni y and Reape and building he adap i e music logic and in e ace in Uni y. 4.1. Au oma ed ende ing Reape is he applica ion chosen o ende ing he mo i samples om he com- pose ’s MIDI i ems. To quickly ecap he ea u es o Reape ha makes i a g ea i o he p ojec : Reape has a wide ange o ac ions, om simple play and s op ac ions o he selec ion o indi idual acks o c ea ion o ime selec ions. This lis can be u he expanded using Cus om Ac ions and Reasc ip . Cus om Ac ions a e compound ac ions c ea ed by combining exis ing ac ions. Reasc ip is a sc ip ing language ha allows us o c ea e ac ions om code (o he ac ions can be in oked om his code). I has suppo o he p og am- ming languages EEL2, Lua and Py hon. The goal o his pa o he p ojec is o allow he compose o c ea e a Reape p ojec (i is impo an ha he compose does no need o use any hing bu he DAW) whe e hey could c ea e hei ins umen s and melodies wi hou ha ing o go h ough he edium o manually ende ing each necessa y combina ion. To au oma e he ende ing p ocess, a new ac ion (Lega o/ ende _mo i .lua) has been c ea ed using Reasc ip . This sc ip is w i en in Lua, making use o he ReaS- c ip API (CockOS, 2024). I also makes use o many de aul Reape ac ions, epli- ca ing he s ep-by-s ep p ocess ha a use would o he wise ha e o ollow in o de o pe o m he job manually. As men ioned in he p e ious chap e , in o de o his sc ip o unc ion co ec ly, he Reape p ojec mus obey he ollowing s uc u e: Fi s , all o he acks con aining he ins umen s and e ec s. Then, he acks con aining he lei mo i s as MIDI i ems. Each ack can ha e mul iple MIDI i ems, which will be ende ed oge he as one mo i . 23 24 Capí ulo 4. De elopmen The Reape ac ions used by he cus om ac ion canno manage ack numbe s abo e 100, which is why he limi o he numbe o ins umen s and mo i s canno su pass 100 when added oge he . Reape ac ions can only ecei e pa ame e s by making use o pop-up windows wi h addi ional ields. Since he goal is o au oma e he en i e p ocedu e, he c ea ed ac ion canno ask he use o any pa ame e s. Ins ead, an auxilia y plain ex ile is used o communica e he speci ica ions be ween Uni y and Reape . This ile is w i en au oma ically by Uni y whene e a change o he con igu a ion window is sa ed. The ile has he ollowing s uc u e: <numbe o ins umen s> <lis o ins umen names, one pe line> <numbe o mo i s> <lis o mo i names, one pe line> <numbe o empos> <lis o empos, one pe line> <ins umen ende ing ec o o mo i 0> < empo ende ing ec o s o mo i 0> <ins umen ende ing ec o o mo i 1> < empo ende ing ec o s o mo i 1> . . . The ins umen and empo ende ing ec o s a e a ow o 1s and 0s indica ing, o each mo i , whe he i should be ende ed in each ins umen and empo. The cus om ac ion s a s by eading his ile. Then, o each mo i -ins umen - empo combina ion ha needs o be ende ed, he ac ion pe o ms he ollowing s eps: 1. Copy all he MIDI i ems om he ack co esponding o he mo i and pas es hem in he ack co esponding o he ins umen . 2. Silence all he acks excep he one co esponding o he ins umen (using Reape ’s solo unc ion). 3. Adjus he p ojec BPM o ma ch he desi ed empo. 4. C ea e a ime selec ion ha spans all he pas ed MIDI i ems, hen ex end his ime selec ion by an addi ional wo bea s. This allows mos e e b, echo and elease e ec s o inish wi hou being cu o by he end o he sample. 5. C ea e a egion ha spans he same ime as he ime selec ion. 4.2. C oss-applica ion communica ion 25 6. Queue a ende o he p ojec in he c ea ed egion and ime selec ion. 7. Dele e he copies o he MIDI i ems, as well as he egion and he ime selec ion and un-solo he ins umen ack. Once all he combina ions ha e been p ocessed, he whole ende queue is en- de ed. The iles gene a ed a e le in he Asse s/Resou ces/Lega o/Rende edSamples di ec o y in he Uni y p ojec . 4.2. C oss-applica ion communica ion The auxilia y ile explained in he p e ious sec ion is w i en by he con igu a ion window in Uni y each ime changes a e sa ed. This window is made using he Uni y’s Edi o Window API. Once he auxilia y ile is w i en, all ha emains is o no i y Reape o un he cus om ac ion. This is done using OSC, a ne wo k p o ocol o sound syn hesize s, which has na i e suppo in Reape . Reape ac ions can be in oked ia an OSC message using he ac ion’s ID. This, howe e , p esen s a p oblem: cus om ac ions c ea ed by Reasc ip ha e an ID gene a ed when Reape is opened. This means ha he same cus om ac ion will ha e a di e en ID on each compu e i is downloaded on. To ci cum en his issue, an ID can be manually assigned o non-na i e ac ions in he ile eape -kb.ini (loca ed wi hin Reape ’s esou ces olde ). A e sion o his ile, edi ed o include a ixed ID o he cus om ac ion, is included alongside he ac ion i sel and he OSC lis ening de ice in he ile Reape Con igZip, included in he Uni y package. This simpli ies he p ocess o con igu ing Reape o wo k wi h Lega o; he use only needs o impo he Reape Con igZip con igu a ion ile om Reape . Uni y does no ha e na i e suppo o sending OSC messages. The package ex OSC (Sigalkin, 2016), om he Uni y Asse S o e, is used o add his unc io- nali y. ex OSC p o ides, among o he hings, a pai o componen s o sending and ecei ing OSC messages om Uni y. By making use o his package, he con igu- a ion window only needs o c ea e a T ansmi e componen , using he ac ion ID de ined in eape -kb.ini and he IP add ess and des ina ion socke de ined in he ool’s code. 4.3. Asse c ea ion The h ee cus om asse ypes used in his ool (Ins umen ,Tempo and F agmen ) inhe i om Uni y’s Sc ip ableObjec class. This class p i ides he memo y-e icien s o age o da a ha will be used o e e enced many imes in a p ojec . I also g an s a be e use expe ience, as explained in he p e ious chap e . A p og ess ba displays he pe cen age o samples ha ha e been ende ed. A e ende ing all o he audio samples (which is de ec ed by coun ing he numbe o iles wi hin he Asse s/Resou ces/Lega o/Rende edSamples olde ), he window o ces Uni y o e esh i s asse da abase. This ensu es ha Uni y gene a es he co ec 32 Capí ulo 5. Conclusions and Fu u e Wo k Using his ool, only he compose o a game needs o in e ac wi h he music gene a ion so wa e. The es o he designe s and de elope s can wo k wi h he in e media y asse s ha he ool gene a es o Uni y. Howe e , he compose does need o a leas ha e access o he Uni y edi o . This is due o he dependency ha he au oma ed ende ing p ocess has upon he con igu a ion window, loca ed in Uni y. Place as ew es ic ions as possible on he music and e ec s ha he compose can c ea e o he game. Many o he c ea i e oppo uni ies ha Lega o does no di ec ly suppo can be acili a ed h ough he use o Uni y’s audio mixe s, which he ool is ully compa ible wi h. E en so, he e emain some no able ex ensions ha we e no implemen ed ( hese a e explo ed u he in he Fu u e Wo k sec ion). Be easy o use. While his is a subjec i e poin , he use in e ace is kep clean and somewha minimalis , bo h in he isual edi o and in he code’s public unc ions. A use doesn no need o know how he ool wo ks in e nally in o de o make use o i . The ool also does no in ude hea ily on he de elopmen o a game. The se -up o a scene’s music is con ined o a single Emi e and i s auxilia y componen s. This ees he es o he game’s code om being ull o e e ences o he sound ack. Fu u e Wo k While he ool is comple e and unc ional, he e is s ill plen y o oom o ex- pansion. This sec ion explo es se e al ea u es ha could be implemen ed upon he cu en e sion o Lega o. Some o hese we e sc apped ideas om ea ly in he p ojec ’s de elopmen , while o he s a ose oo la e o be easibly included o he deadline. 1. Cu en ly, changes in empo always ake e ec a he s a o he nex measu e. The op ion could be added o allow he use o delay hese changes un il he s a o he nex mo i (as is he case wi h ins umen and mo i changes). 2. Also conce ning empos, i was ini ially concei ed ha he p ojec could play music a in e media e empos be ween he ones ende ed. This would equi e he use o pi ch modi ica ion and in e pola ing be ween wo ende ed empos o allow o gen le ades be ween empos and a much g ea e deg ee o eedom wi h how hese empos a e used. 3. All o he music ha Lega o plays o igina es om a single Emi e . Making use o Uni y’s 3D audio ea u es, an in iguing possibili y in ol es allowing di e en pa s o he sound ack o ha e di e en posi ional se ings. This would allow o games whe e many di e en cha ac e s ha e hei own music ha ollows hem a ound, all o i quan ised in sync. 33 4. Rega ding he Reape p ojec , he simples imp o emen s would be allowing he use o choose wha audio o ma hey would like o wo k wi h (cu en ly, only .wa iles a e suppo ed) and how long o a ail o add o he end o each ende ed sample. This ail is he emp y s e ch o MIDI da a ha is added on he end o each sample, allowing long echoes, e e b e ec s and he like o end wi hou being cu o . Cu en ly, his is locked o always be equal o wo bea s o he sample’s empo. 5. Allow Uni y o be closed while he samples a e ende ed ( o sa e esou ces and speed up he p ocess) and opened once he p ocess has inished. 6. Making use o he loadS a e p ope y o Uni y’s AudioSou ce class o ci cum- en he need o a loading bu e . This would also allow he ool o wo k mo e smoo hly on ex emely slow de ices, while being mo e esponsi e on as e de ices. 7. De ec changes in he Reape p ojec and use his in o ma ion o a oid en- de ing samples ha ha e no changed since he las ende . 8. The use expe ience could be imp o ed by an au oma ed c oss- e e encing sys- em ha would de ec con adic ions be ween he con igu a ion menu and he Reape p ojec , such as a misma ched numbe o ins umen s and ins umen channels. In he case o such a con adic ion, he use could be no i ied o he issue, a he han p oceeding wi h a ende ha is al eady doomed o ail. Finally, he ollowing expansion was p oposed a he e y s a o he p ojec , bu was immedia ely disca ded due o amoun o wo k i would ha e equi ed (i could po en ially be a esea ch p ojec on i s own). This would be he inclusion o ad anced algo i hms o a i icial in elligence o analyse he lei mo i s p o ided by he compose and de e mine when hey would sound bes wi hin he sound ack. This way, a mo i ’s p io i y could be ecalcula ed in e e y single measu e o ensu e ha i plays a he bes -sounding momen , c ea ing he bes possible ha mony wi h he es o he music. In oducción In oducción a la ma e ia. Es e capí ulo con iene la aducción del capí ulo 1. “Si hablas con cualquie di ec o , e di á que la música es el cincuen a po cien o de la película.” — Hans Zimme Mo i ación Los he manos Lumiè e con aban, ya en sus p ime as p oyecciones, con un pia- nis a que acompañaba sus co as películas (Tudo , 2017). An e io incluso a la in- oducción del diálogo, la banda sono a ha sido siemp e un elemen o imp escindible del cine. Es di ícil pensa en los p o agonis as más céleb es del cine sin eco da la música que anunciaba sus p esencias. Don Co leone, con su ágica melodía de iolín; la e umban e ma cha mili a del Impe io Galác ico; las es ins umen os que de inen a los epónimos p o agonis as de “El Bueno, el Feo y el Malo”, a pesa de oca odas las mismas dos no as(Classic FM, 2014); el elenco del Spide - e so, do ados de ida po la mul i ud de écnicas, ins umen os y a es de es anque que en edó Daniel Pembe on pa a esas películas (Sony Sound acks, 2023). La iden idad de es os pe sonajes es casi an musical como isual. Muchos de es os amosos sonidos son lei mo i s, igu as o ases melódicas aso- ciadas a la apa ición epe ida de un pe sonaje, idea o si uación. O os, como “El Bueno, el Feo y el Malo”, usan ins umen os pa a iden i ica a sus pe sonajes. Es e concep o de “iden idad musical” nos ascinaba. Como es udian es de desa- ollo de ideojuegos, que íamos una o ma de emplea es as écnicas en nues os juegos. La banda sono a de un ideojuego iene simila pode cau i ado que la de una película; sin emba go, excep uando los juegos de i mo, la secuencia de e en os que ocu en en la mayo ía de juegos no es de e minis a, pues se en a ec adas po e en os alea o ios y las decisiones de uno o a ios jugado es. Po es e mo i o, es imposible compone canciones que esuenen pe ec amen e con cada momen o de la acción. La música adap a i a su gió pa a emenda es a debilidad: una se ie de écnicas de edición y ep oducción que al e an la música del juego en el momen o pa a ajus a se mejo a las acciones en pan alla. La música puede e se al e ada con 35 36 Capí ulo 5. Conclusions and Fu u e Wo k cambios de ins umen ación, ajus es de iempo ( elocidad a la que se ep oduce la música) o in oducción de nue as melodías en la canción ac ual. Teníamos una imagen men al muy cla a (o, más bien, un sonido men al) de lo que que íamos: una banda sono a de juego que desc ibe median e lenguaje musical la acción que es á ocu iendo en pan alla, combinando lei mo i s e ins umen os iden i ica i os de los pe sonajes en escena. Como desa ollado es de ideojuegos, buscamos middlewa e (es deci , so wa e que unciona en una capa in e media en e el código del juego y el sis ema ope a- i o) que nos pe mi ie a c ea música pa a ideojuegos usando es as écnicas. Po desg acia, no pudimos encon a una he amien a que pe mi iese hace jus o es o. Los composi o es y desa ollado es usan he amien as como DAWs y acke s (dos ipos de aplicaciones usadas pa a p oduci música) pa a c ea e in eg a las bandas sono as de sus juegos. Sin emba go, es as aplicaciones siguen equi iendo in e ención humana y no necesa iamen e pueden se con olados desde el mo o de juego. Es a es una pega impo an e, pues la lógica del juego se p og ama desde el en- o no del mo o ; se ía poco manejable ene que edi a los a chi os uen e de música cada ez que un cambio en el juego a ec e cómo y cuándo es os se ep oduzcan. Sin una he amien a especializada, se ía di ícil implemen a la uncionalidad que que emos. Cada combinación posible de lei mo i e ins umen o necesi a ía se en- de izada indi idualmen e en cada empo. Incluso en un en o no especializado, como una DAW (una aplicación usada pa a p oduci música), es a se ía una a ea ediosa si no se usa ninguna au oma ización. Además, sinc oniza una banda sono a o mada po a ias piezas dispe sas se ía una a ea di ícil sin c ea un sis ema cen alizado que lo maneje. Todas es as conside aciones nos ayuda on a ija los obje i os del abajo: c ea un sis ema de música adap a i a que u iliza écnicas musicales p opias del cine, dando la posibilidad de u iliza ins umen os y e ec os complicados o de al o cos e sin eque i un al o cos e de p ocesado. De iniciones cla e Re-secuenciación ho izon al: conjun o de écnicas de música adap a i a en la que se ep oducen di e en es agmen os de audio uno as o o, cambiando el o den o la selección de agmen os en espues a a los e en os del juego. Re-o ques ación e ical: conjun o de écnicas de música adap a i a que consis en en cambia las pa es indi iduales de una banda sono a que suenan de o ma simul ánea, al e ando así la mezcla inal. MIDI (Musical Ins umen Digi al In e ace): o ma o de codi icación digi al de música pa a comunicación en e aplicaciones o disposi i os. Los ob- je os MIDI po sí solos no almacenan in o mación del imb e del sonido; an solo in o mación de qué no a se oca, cuándo se pulsa y se suel a y algunos da os más sob e la o ma en la que se pulsó. DAW (Digi al Audio Wo kspace): disposi i o elec ónico o aplicación so - 37 wa e usado pa a g aba , edi a y p oduci audio. En e o as cosas, pe mi e ende iza mues as de audio a pa i de una codi icación (como MIDI) y una lis a de e ec os que combinan pa a de e mina qué sonidos se p oducen a pa i de cada no a. Ins umen o i ual: so wa e pa a gene a sonido a pa i de una ep e- sen ación simbólica (en el ámbi o de es e abajo, gene almen e en es ánda MIDI). Es os ins umen os uncionan como plug-ins en una DAW. OSC (Open Sound Con ol): p o ocolo pa a la comunicación ed mul ime- dia en e disposi i os de audio. C oss ade: cambia de un sonido a o o median e un undido g adual, en an- do uno a la ez que sale el an e io . Cuan ización: en el con ex o de la música adap a i a, p oceso de co ección y sinc onización de di e en es uen es de música pa a que suenen pe ec amen e en su empo adecuado. BPM (Bea s Pe Minu e): unidad pa a medi el empo en música, equi ale al núme o de pulsaciones en un minu o. Audio posicional: en los ideojuegos, aquellos sonidos que simulan p o eni de una ubicación conc e a en el espacio. Obje i os El obje i o de es e p oyec o es c ea una ex ensión pa a Uni y que incluya he a- mien as pa a la inclusión de música adap a i a en ideojuegos, usando las écnicas mencionadas an e io men e. Uni y es un mo o de ideojuegos muy usado en el sec o . Fue elegido pa a es e p oyec o p incipalmen e po nues a amilia idad con él, pe o cuen a con o as en ajas. Dado que la he amien a es á cons uida con y pa a Uni y, es á ambién o ien ada a los desa ollado es de Uni y. La he amien a debe simpli ica el p oceso de c ea una iden idad musical pa a los di e en es pe sonajes, luga es y emas den o de la na a i a del juego, al e ando la música ambien e cuando esos elemen os na a i os en en en escena. Aunque la p opia composición oda ía debe ocu i den o de una DAW, los desa ollado es debe ían pode hace cualquie cambio a la lógica de cómo se ep oduce la banda sono a sin ene que sali del mo o de ideojuegos. Es as he amien as deben: Facili a la inclusión modula de lei mo i s en la banda sono a de un juego. Pe mi i cambia en iempo de ejecución la ins umen ación de pa es indi i- duales de la banda sono a. Pe mi i cambia en iempo de ejecución el empo de la banda sono a. 38 Capí ulo 5. Conclusions and Fu u e Wo k Eje ce poca demanda de ecu sos al o denado del jugado . No exigi conocimien os de p og amación a los composi o es. No exigi conocimien os de so wa e musical a los p og amado es. Impone la mínima can idad de es icciones sob e la libe ad c ea i a del composi o . Se ácil de usa . Plan de abajo Pa a log a es os obje i os, el abajo ue di idido en los siguien es bloques, algunos de los cuales se desa olla on en pa alelo: In es igación de ecnologías exis en es sob e las que cons ui la he amien a. Pa a no esc ibi odo el p og ama desde ce o, se es udia on di e en es aplica- ciones pa a e alua su uso como base sob e la cual cons ui la he amien a. Es o ambién ayuda a enlaza la con un en o no con el que los usua ios ya es a ían amilia izados, lo cual mejo a la accesibilidad. Au oma ización del ende izado de a chi os de audio. Dado que la he amien a usa mues as p e- ende izadas, es as end án que se c eadas au omá icamen e en unción de cie os pa áme os de inidos po el usua io. C eación de la in e az de usua io pa a la con igu ación de la he amien a. El desa ollado debe ene acceso a un panel de con ol den o del mo o de ideojuegos. C eación de los ecu sos de juego que ep esen an los elemen os dinámicos de la banda sono a. Pa a hace que la he amien a sea más isual e in ui i a, los lei mo i s, ins umen os y empos dinámicos se án ep esen ados como ecu sos den o del mo o de juego. Es os ecu sos end án que se egene ados cada ez que se ende icen nue as mues as de audio. P og amación de la lógica de ep oducción dinámica. El sis ema de ep oduc- ción adap a i a es el co azón de la he amien a. Más allá de p og ama las en adas y salidas de los elemen os dinámicos, los usua ios de la he amien a no ienes que ges iona la sinc onización, la cuan ización ni la ep oducción de la música. Conclusiones y T abajo Fu u o Es e capí ulo con iene la aducción del capí ulo 5. Conclusiones El esul ado de es e abajo es una ex ensión de middlewa e pa a Uni y, un mo o de ideojuegos con uso muy ex enso, que acili a de c eación de bandas sono as adap a i as. Es o cumple con el obje i o p incipal. En cuan o al es o de pau as de inidas en la in oducción: Facili a la inclusión modula de lei mo i s en la banda sono a de un juego. Pe mi i cambia en iempo de ejecución la ins umen ación de pa es indi i- duales de la banda sono a. Pe mi i cambia en iempo de ejecución el empo de la banda sono a. La inclusión de lei mo i s en la banda sono a es la p incipal uncionalidad de la he amien a. Con an solo una llamada se puede inse a un lei mo i en iempo de juego y es os suenan sinc onizados en e sí. Cambia la ins umen ación o el empo de es os lei mo i s es igual de sencillo, pues la he amien a los plan ea como es aspec os independien es de la música; se puede cambia una cosa sin a ec a las o as dos. Eje ce poca demanda de ecu sos al o denado del jugado . U iliza mues as de audio p e- ende izadas implica un meno cos e en memo ia que ende iza en iempo de juego. En es e aspec o la he amien a es compa able a al e na i as como la que o ece FMOD. Po o o lado, deja al usua io elegi qué combinaciones de mo i /ins umen o y mo i / empo ende iza da opción a educi el cos e de almacenamien o ya que el usua io puede decidi gua da solo las mues as que a a u iliza . No exigi conocimien os de p og amación a los composi o es. No exigi conocimien os de so wa e musical a los p og amado es. 39 40 Capí ulo 5. Conclusions and Fu u e Wo k Usando es a he amien a, an solo el composi o de un juego iene po qué oca so wa e de audio. El es o de diseñado es y desa ollado es pueden abaja con los ecu sos de Uni y que la he amien a gene a. No obs an e, el composi o necesi a al menos ene acceso al edi o de Uni y. Es o se debe a la dependencia que hay en e el ende izado au omá ico y la en ana de con igu ación, que se encuen a en Uni y. Impone la mínima can idad de es icciones sob e la libe ad c ea i a del com- posi o . Algunas de las posibilidades c ea i as que Lega o no p opo ciona pueden se eme- diadas con el uso de los mezclado es de Uni y. Aún así, hay algunas uncionalidades que no se han p opo cionado (que se explo a án en la sección de T abajo Fu u o). Se ácil de usa . Es o es un pun o subje i o, sin emba go sí que se ha cuidado la in e az pública pa a que sea in ui i a, an o la pa e accesible desde la is a de edi o como las llamadas po código. El usua io no iene que en ende cómo unciona la he amien a po den o pa a pode hace uso de ella. La he amien a es ambién bas an e poco in usi a sob e el desa ollo de un juego. La con igu ación de la música es á muy encapsulada en el Emi e y sus componen es auxilia es. El código del juego no es a á sob eca gado con llamadas a unciones e e en es a la música. T abajo Fu u o Si bien el abajo ealizado da luga a una he amien a e minada, es a se puede expandi de di e en es o mas. A con inuación se expond án ideas de uncionalida- des que pueden comple a Lega o. Algunas de es as ideas se plan ea on al comenza la in es igación de es e abajo mien as que o as ue on su giendo en la ase de desa ollo. 1. Los cambios de empo se ealizan siemp e ins an áneman e, pe o se pod ía da la opción al usua io de e asa los has a que cambie de mo i o (como se pe mi e en la ep oducción de mo i os o en los cambios de ins umen o). 2. En elación ambién con los empos, se plan eó en un inicio pe mi i la e- p oducción de empos in e medios a los de inidos al ende iza . Hace uso de a ibu os como pi ch pa a modi ica el empo de un mo i o en ejecución y da o al libe ad en lo que a empos se e ie e. 3. La ep oducción de música u ilizando Lega o es á plan eada pa a que haya un único Emi e en la escena de Uni y del que p oceda el sonido. Al expone la uncionalidad de sonido 3D p opo cionada po los AudioSou ce de Uni y, se ía in e esan e un eplan eamien o donde se pe mi ie an a ios emiso es y 41 un mas e , de o ma que el sonido inie a de dis in os pun os de la escena 3D. Es o pod ía da luga a juegos donde, po ejemplo, los mo i os de dos pe sonajes es én sonando a la ez pe o cada uno desde un pun o de la escena dis in o, o donde haya una lis a de mo i os elacionados con la escena sonando en 2D mien as suena un mo i o 3D ocalizado en un pun o. 4. Po pa e del p oyec o de Reape y la au oma ización del ende izado, la p i- me a idea (y la más simple) se ía la de pe mi i dis in os o ma os pa a los a chi os de audio (ac ualmen e solo se ende iza en .wa ). Además, se pod ía pe mi i al usua io de ini el iempo que se in oduce al inal de los obje os MIDI pa a no co a los e ec os. De es a o ma, aún in oduciendo e ec os que ala guen mucho las no as (un eco muy p olongado en el iempo, po ejemplo) no se ap ecia ían co es. 5. Pe mi i que la en ana de Uni y se cie e du an e el p oceso de ende izado de agmen os (pa a aho a ecu sos y acele a dicho p oceso) y se uel a a ab i una ez odas las mues as hubie an sido ende izadas. 6. Hace uso de la p opiedad loadS a e de la clase AudioSou ce pa a elimina la necesidad de un iempo ijo pa a ca ga las mues as de audio. Es o pe mi i ía pone menos p esión a disposi i os muy len os a la ez que pe mi i ía exp imi más el endimien o de los disposi i os de al a gama. 7. De ec a cambios en el p oyec o de Reape y usa es a in o mación pa a e i a ol e a ende iza mues as que no han sido modi icadas desde el úl imo ende izado. 8. La expe iencia de usua io se e ía mejo ada si hubie a una comp obación de conco dancia en e los da os in oducidos en Uni y (y asladados al iche o in e medio) y el a chi o de Reape con sus pis as. Si el núme o de ins umen os o de mo i os (o de pis as en gene al) no coincidie a con los alo es in oduci- dos en Uni y, se in o ma ía al usua io en luga de in en a ende iza algo que, ine i ablemen e, a a da luga a e o . Po úli mo, es a siguien e idea se plan eó al de ini el abajo, pe o se desca ó inmedia amen e po su ele ado cos e de implemen ación (ya que pod ía supone po sí misma un TFG). Se ía el hace uso de algo i mos que analicen los mo i os pa a en ende qué mo i os a monizan bien en e ellos. De es a o ma, el lanzamien o de mo i os en ejecución no end ía po qué hace se en el siguien e compás o al inal del mo i o ac ual, sino en el momen o más opo uno pa a man ene la a monía y que el esul ado sea el mejo posible. 48 Apéndice A. Lega o Use Manual The emaining acks mus co espond one- o-one wi h he lei mo i s ha will be used in he sound ack. These acks can con ain e ec s, bu hese will no appea anywhe e in he inal ende s. Bo h he ins umen acks and he mo i acks mus appea in he same o de ha he ins umen s and mo i s do in he con igu a ion menu in Uni y. When he p ojec is ende ed, he MIDI i ems in he lei mo i acks abo e will be combined wi h he indi idual ins umen acks o c ea e he inal audio clips. Some addi ional poin s: Calling he ac ion di ec ly om Reape can cause p oblems, as any p e ious e sions o he iles c ea ed by he ac ion will no be dele ed. To a oid his issue, he ac ion should only be called om he con igu a ion menu in Uni y. The bea s pe minu e and bea s pe measu e o he Reape p ojec will be o e w i en. Reape uses a ende queue o s o e all indi idual ende s be o e p oducing hem all. Since his ac ion lushes he queue, any p e ious ende s ha may s ill be in he queue will also be p ocessed. The ac ion sa es he Reape p ojec be o e doing any hing else, bu does no sa e i a e i inishes. This means ha Reape will de ec he p ojec as di y (con aining unsa ed changes) e en i he e a e no ne changes ela i e o he p e ious s a e o he p ojec . This also means ha i he use wishes o eco e any hing ha was o e w i en by he ac ion (such as he p ojec BPM) hey can simply close Reape and disca d he unsa ed changes. Selec ing a ack name be o e he cus om ac ion is called can cause issues du ing ende ing. To a oid his, be o e ende ing, click anywhe e in he a ea o he igh , highligh ed in g een in he ollowing sc eensho : A.3. Lega o Con igu a ion 49 A.3. Lega o Con igu a ion Lega o is con igu ed om Uni y, in he Lega oCon igu a ion pop-up ha can be ound in Window->Lega o wi hin he edi o . The i s ields o he window a e o lis ing and naming he ins umen s, lei - mo i s and empos ha will be used in he sound adk. The numbe o ins umen s and lei mo i s se in his window mus be he same as he numbe o acks in he Reape p ojec . The nex sec ion con ains wo ma ices o choose he combina ions o mo i /ins- umen and mo i / empo o ende . A he bo om o he window he e a e bu ons o sa e changes, ende and egene a e asse s. This las bu on should only be clicked i any issue ocu ed du ing he c ea ion o he Uni y asse s (a e ende ing) o i he use decided o manually c ea e hei audio iles (no using his ool). In case he use c ea ed hei iles, hese mus be loca ed in Asse s/Resou ce- s/Lega o/Rende edSamples, and hei names mus be o ma ed as: mo i .<mo i name>.<ins umen name>.< empo>.wa (all o his be o e clicking he egene a e asse s bu on). Fo example, he clip o he mo i “cho ds A” wi h he ins umen “piano” and he empo 120 should be called mo i .cho ds A.piano.120.wa A.4. Lega o_Emi e Desc ip ion: The Lega o_Emi e class manages he audio playback o di - e en sound scenes, allowing o de ailed con ol o e olume, spa ial blend, and mo e. Lega o_Emi e is he main Uni y componen o he oolse ; he sou ce o he adap- i e sound ack. I can be ound in he Asse s/Lega o/Componen s di ec o y. The emi e is a single on wi h a publicly accessible ins ance. The use can add he componen o a game objec in he scene o ins an ia e he p e ab ound in Asse s/Lega o/P e abs. Se ialized P ope ies: bool debug Ac i a es in o ma i e message logs abou he music and e en s being played du ing un ime. loa olume Con ols he global olume o he music ha is played. SoundScene[] soundScenes • loa spa ialBlend How a ec ed by 3D spa ialisa ion he scene is. 50 Apéndice A. Lega o Use Manual •SoundSe ings3D soundSe ings3D Con olle o he AudioRollo Mode, minDis ance and maxDis ance a - ibu es o a 3D scene. •Channel[] channels ◦F agmen [] agmen s A ay o musical agmen s ha will be played by de aul . ◦So ingAlgo i hms algo i hm The algo i hm ha is used o a ange he agmen s. ◦ loa olume Con ols he channel’s olume. ◦Ins umen ins umen The ins umen ha he channel will be played in a he s a . ◦Tempo empo The empo ha he channel will be played in a he s a . ◦in bea sPe Measu e Numbe o bea s pe measu e, used o synch onising channels and changing be ween agmen s a he igh ime. ◦bool playOnS a Whe he he channel will begin playing a he s a o no . Public P ope ies: AudioMixe G oup ou pu Se whe he he audio should play h ough an Audio Mixe i s o di ec ly h ough he Audio Lis ene . cons loa loadingBu e = 0.4 Time (in seconds) ha is allowed o audio clips o load be ween measu es. Public Me hods: public s a ic Lega o_Emi e Ge Ins ance() Re u ns he single on ins ance o he Lega o_Emi e . public oid Se Volume(in c, loa ) Se s he olume o a speci ic channel o he cu en scene. •in c: The channel o se he olume o. • loa : The olume o he channel (0.0 o 1.0). public oid Se GlobalVolume( loa ) Se s he global olume o he emi e . • loa : The olume o he emi e (0.0 o 1.0). public oid Se Scene(in s) Se s he cu en scene. A.4. Lega o_Emi e 51 •in s: The scene o se as cu en . public oid Se Spa ialBlend( loa spa ialBlend) Changes he spa ial blend a ibu e o he cu en scene. • loa spa ialBlend: The alue o spa ial blend, whe e 0.0 is ully 2D and 1.0 is ully 3D. public oid Se Spa ialBlend(in s, loa spa ialBlend) Changes he spa ial blend a ibu e o he speci ied scene. •in s: The scene o change he spa ial blend o. • loa spa ialBlend: The alue o spa ial blend, whe e 0.0 is ully 2D and 1.0 is ully 3D. public oid Se 3DRollo Mode(AudioRollo Mode ollo Mode) Se s he ollo mode o he cu en scene’s 3D audio sou ce. •AudioRollo Mode ollo Mode: The new ollo mode. public oid Se 3DMinDis ance( loa minDis ance) Se s he min dis ance o cu en scene’s 3D audio sou ce. • loa minDis ance: The new alue o minDis ance. public oid Se 3DMaxDis ance( loa maxDis ance) Se s he max dis ance o cu en scene’s 3D audio sou ce. • loa maxDis ance: The new alue o maxDis ance. public oid PlayMo i (in c, s ing mo i , bool in e up = alse, loa p io i y = 1 , loa p io i yO e Time = 0 ) Queue a mo i o be played in a speci ic channel. •in c: The channel o queue he mo i in. •s ing mo i : The name o he mo i o add o he queue. •bool in e up : (Op ional) I ue, in e up s any cu en ly playing mo- i in he channel. • loa p io i y: (Op ional) The ini ial p io i y o he mo i . • loa p io i yO e Time: (Op ional) The a e a which he mo i ’s p io- i y inc eases o dec eases o e ime. public oid PlayMo i (in c, F agmen mo i , bool in e up = alse, loa p io i y = 1 , loa p io i yO e Time = 0 ) Queue a mo i o be played in a speci ic channel. •in c: The channel o queue he mo i in. •F agmen mo i : The mo i o add o he queue. 52 Apéndice A. Lega o Use Manual •bool in e up : (Op ional) I ue, in e up s any cu en ly playing mo- i in he channel. • loa p io i y: (Op ional) The ini ial p io i y o he mo i . • loa p io i yO e Time: (Op ional) The a e a which he mo i ’s p io- i y inc eases o dec eases o e ime. public oid CancelMo i (in c, s ing mo i ) Remo e all ins ances o a gi en mo i o a speci ic channel mo i queue. •in c: The channel o look o he mo i . •s ing mo i : The name o he mo i o cancel. public oid CancelMo i (in c, F agmen mo i ) Remo e all ins ances o a gi en mo i o a speci ic channel mo i queue. •in c: The channel o look o he mo i . •F agmen mo i : The mo i o cancel. public oid Resume(in c) Resume a speci ic channel o he cu en scene. •in c: The channel o esume. public oid S op(in c) S op a speci ic channel o he cu en scene. •in c: The channel o s op. public oid S opAll() S op all he channels o he cu en scene. public oid ResumeAll(bool onlyP e iouslyPlaying = ue) Resumes play on all he channels o he cu en scene. •bool onlyP e iouslyPlaying: (Op ional) I ue, only channels ha we e playing be o e S opAll was called will be esumed. public oid Se Ins umen (in c, s ing ins umen , bool changeCu en = alse, loa adeDu a ion = 0 ) Se he ins umen o a speci ic channel. •in c: The channel o se he ins umen o. •s ing ins umen : The name o he ins umen o se . •bool changeCu en : (Op ional) Change he cu en agmen o no , whe e ue changes he cu en agmen ’s ins umen and alse wai s un il he nex agmen . • loa adeDu a ion: (Op ional) The leng h o he c oss ade in seconds when changing he ins umen . A.5. E en igge s 53 public oid Se Ins umen (in c, Ins umen ins umen , bool changeCu en = alse, loa adeDu a ion = 0 ) Se he ins umen o a speci ic channel. •in c: The channel o se he ins umen o. •Ins umen ins umen : The ins umen o se . •bool changeCu en : (Op ional) Change he cu en agmen o no , whe e ue changes he cu en agmen ’s ins umen and alse wai s un il he nex agmen . • loa adeDu a ion: (Op ional) The leng h o he c oss ade in seconds when changing he ins umen . public oid Se Tempo(in c, in empo) Se he empo o a speci ic channel o he cu en scene. •in c: The channel o se he empo o. •in empo: The empo in bpm o se . public oid Se Tempo(in c, Tempo empo) Se he empo o a speci ic channel o he cu en scene. •in c: The channel o se he empo o. •Tempo empo: The empo o se . public oid Se Tempo(in empo) Se he empo in all channels o he cu en scene. •in empo: The empo in bpm o se . public oid Se Tempo(Tempo empo) Se he empo in all channels o he cu en scene. •Tempo empo: The empo o se . A.5. E en igge s To complemen he emi e , he package also includes i e componen s ha simpli y he p ocess o ac i a ing e en s. These a e Lega o_Ins umen E en , Lega o_Mo i E en ,Lega o_TempoE en ,Lega o_S opE en and Lega- o_ResumeE en . These componen s ha e a single public i ual me hod T ig- ge , which se es o ac i a e he associa ed ins umen , mo i o empo o o s op o esume speci ic channels, espec i ely. These i e classes inhe i om he abs ac class Lega o_E en , allowing hem o be used polymo phically. 54 Apéndice A. Lega o Use Manual A.5.1. Lega o_E en Desc ip ion: The Lega o_E en class ep esen s an abs ac e en in he Lega- o sys em ha can be igge ed o pe o m speci ic ac ions ela ed o audio playback. P o ec ed P ope ies: Lega o_Emi e emi e The Lega o emi e ins ance. in channel The audio channel ha he e en will igge on. Public Me hods: public abs ac oid T igge () Abs ac me hod o igge he e en . Subclasses mus implemen his me hod. A.5.2. Lega o_ResumeE en Desc ip ion: The Lega o_ResumeE en class ep esen s an e en ha ig- ge s he esuming o audio playback in he Lega o sys em. I inhe i s om he Lega o_E en class. Public P ope ies: bool applyToAll I ue, igge ing his e en will esume all channels a once. bool onlyResumeP e iouslyPlaying I ue, only channels ha we e playing be o e S opAll was called will be esumed. Public Me hods: public o e ide oid T igge () T igge s he e en . I applyToAll is ue, esumes all channels. O he wise, esumes he speci ic channel. A.5.3. Lega o_S opE en Desc ip ion: The Lega o_S opE en class ep esen s an e en ha igge s he s opping o audio playback in he Lega o sys em. I inhe i s om he Lega o_E en class. Public P ope ies: bool applyToAll I ue, igge ing his e en will s op all channels a once. A.5. E en igge s 55 Public Me hods: public o e ide oid T igge () T igge s he e en . I applyToAll is ue, s ops all channels. O he wise, s ops he speci ic channel. A.5.4. Lega o_Mo i E en Desc ip ion: The Lega o_Mo i E en class ep esen s an e en ha igge s he playing o a mo i in he Lega o sys em. I inhe i s om he Lega o_E en class. Public P ope ies: F agmen mo i The mo i ha will play on igge ing he e en . bool in e up I ue, he mo i will be played in he selec ed channel a he nex possible ba . I alse, i will be added o he dynamic p io i y queue. loa p io i y T igge ed mo i s a e added o a queue. When he cu en agmen ends, he highes p io i y mo i will play nex . loa p io i yO e Time Ra e ha he mo i ’s queued p io i y changes o e ime (in uni s/second). Can be posi i e o nega i e. I he mo i ’s p io i y is below 0, he mo i is emo ed om he queue. Public Me hods: public o e ide oid T igge () T igge s he e en , playing he mo i on he speci ied channel wi h op ional in e up ion and p io i y se ings. A.5.5. Lega o_Ins umen E en Desc ip ion: The Lega o_Ins umen E en class ep esen s an e en ha ig- ge s he se ing o a new ins umen in he Lega o sys em. I inhe i s om he Lega o_E en class. Public P ope ies: Ins umen ins umen The ins umen ha will be se on igge ing he e en . 56 Apéndice A. Lega o Use Manual bool changeCu en I ue, any music cu en ly playing will be c oss aded o he ins umen on e en igge . I alse, he cu en agmen will inish playing wi h i s cu en ins umen and he nex one in he channel will use he new ins umen . loa adeDu a ion I g ea e han 0, igge ing he e en will cause he channel o c oss ade i s ins umen o e adeDu a ion seconds. Public Me hods: public o e ide oid T igge () T igge s he e en , se ing he ins umen on he speci ied channel wi h op io- nal c oss ade. A.5.6. Lega o_TempoE en Desc ip ion: The Lega o_TempoE en class ep esen s an e en ha igge s he se ing o a new empo in he Lega o sys em. I inhe i s om he Lega o_E en class. Public P ope ies: Tempo empo The empo ha will be se on igge ing he e en . bool applyToAll I ue, changes in empo will apply o all channels a once, ega dless o selec ed channel. Public Me hods: public o e ide oid T igge () T igge s he e en , se ing he empo on he speci ied channel o globally wi h op ional applica ion o all channels.