The scenographic space(s) of augmented reality

327 In oduc ion The e has been a ecen wa e o esea ch a en ion ocusing on imme si e and mixed eali y echnologies ac oss a ange o a o ms, including hea e and pe o mance. Accompanying his is a pe ading sense ha he c ea i e po en ial o hese echnologies has no ye been ealised. Somewha su p isingly, he e has been e y li le wo k in his a ea wi hin a speci ically scenog aphic con ex . This pape will ocus speci ically on augmen ed eali y (AR), d awing on amewo ks wi hin scenog aphy o a gue ha his con ex p o ides a way o hink h ough and expe imen wi h he a ec i e po en ial o hese echnologies. AR is no necessa ily unde s ood he e as associa ed wi h a pa icula echnology. Ra he , I de ine i as any handheld o wea able echnology ha c ea es an augmen ed expe ience o space. Con a y o mos exis ing esea ch in AR, I esis hinking in e ms o a bina y dis inc ion be ween i ual objec s o in o ma ion, and ‘ eali y’. I ins ead build on he wo k a small mino i y o schola s who a gue ha AR c ea es new media ed spaces o eali ies. I ake his u he , p oposing ha AR, like he ela ed bu dis inc i ual eali y, can be discussed in e ms o i s spa iali y. I conside his spa iali y in he con ex o a p ac ice- esea ch pe o mance ins alla ion E nes THE SCENOGRAPHIC SPACE(S) OF AUGMENTED REALITY LUCY THORNETT 328 Remains, s aged a he Uni e si y o Leeds in 2019. In his ins alla ion, audience membe s explo ed a mul i- oomed space using a handheld able . I analyse he audience expe ience o his wo k h ough a discussion o audience esponses p o ided du ing pos -show in e iews and e lec ion on insigh s ha eme ged du ing my own expe iences o he ins alla ion. D awing on exis ing heo e ical amewo ks wi hin scenog aphy, I con end ha his scenog aphic expe imen e eals he way in which embodied ela ions wi h AR p oduce a ec i e, specula i e space(s). Augmen ed Reali y Imme si e echnologies, such as augmen ed eali y, a e no new, bu ecen de elopmen s in echnology making hem mo e accessible ha e spa ked enewed in e es in hei po en ial by esea che s and a is s. In he case o augmen ed eali y, he ubiqui y o sma phone and hei p og essi ely mo e echnologically sophis ica ed AR capabili ies means ha AR echnologies a e inc easingly becoming embedded wi hin he e e yday ( hink, o example, o ace il e s on came a apps o GPS loca ion da a). AR is o en associa ed wi h pa icula echnologies ha use he came as o handheld de ices such as sma phones o Head Moun ed Displays o o e lay i ual in o ma ion o objec s on o he ‘ eal’ wo ld. I should be no ed, howe e , ha e ms associa ed wi h mixed eali y echnologies a e unde going apid change, o en d i en by ech indus y ma ke ing. Academic esea ch in his a ea ends o de ine i mo e b oadly. Acco ding o Ronald Azuma, he h ee key cha ac e is ics o AR a e ha i ‘combines eal and i ual’; ‘is in e ac i e in eal ime’; and ‘is egis e ed in h ee dimensions’ (1997). Mos esea ch in AR ollows Azuma’s lead in de ining i acco ding o i s abili y o blend i ual and physical, a he han in ela ion o a pa icula echnology. Milg am and Kishino’s ‘Reali y-Vi uali y Con inuum’ is also in luen ial in de ining Augmen ed and Mixed Reali y (1994). They place AR on a Mixed Reali y scale ha places he ‘ eal en i onmen ’ a one end and he ‘ i ual en i onmen ’ on LUCY THORNETT 329 he o he . Howe e , his con inuum has been c i iqued o i s linea i y and he bina y i cons uc s be ween he eal and he i ual (see, o example, Ben o d and Giannachi, 2011). In an a is ic con ex , esea che s ha e ended o de ine AR qui e b oadly – a numbe ha e ci ed Jane Ca di ’s audio walks as pa adigma ic examples (e.g. Mano ich, 2006; W igh , 2018). An ea ly Ca di wo k, The Missing Voice (1999) guided audiences h ough an a ea o London using sound eco ded in he same loca ion ( ela ing o a ic ional na a i e), played back h ough headphones. La e examples such as Ci y o Fo king Pa hs (2014) ha e used sound combined wi h ideos displayed on a sma phone. These wo ks, unde s ood as AR, demons a e he b ead h o possibili ies o o e laying i ual space on o a physical loca ion, unmoo ed om a speci ic echnology. I p opose ha AR di e s om o he mixed eali y echnologies – such as, o example, p ojec ion – in ha i augmen s space om he posi ion o he body, h ough ei he head- wo n o handheld echnology1. Resea ch in AR c osses a wide ange o disciplines and applica ions, bu has been less well explo ed wi hin a is ic con ex s. Schola ly p ac ice in AR a o da e has b oadly ocused on su eying he ield (Ge oimenko, 2018) o ou lining i s poli ical po en ial as a ool o ac i ism (e.g. Skwa ek, 2018; Thiel, 2018)2. Howe e , o ele ance he e is a small bu g owing body o esea ch discussing he embodied ela ions p oduced by AR a . Pa ick Lich y discusses embodied encoun e s wi h bo h handheld and head-moun ed AR a wo ks. He a gues ha AR p oduces ‘a line o a en ion/ ligh be ween he in e ac o and he supe imposed media o e laid on he gi en en i onmen ’ (2018, p.137), which he calls a ‘pe o ma i e ges u al gaze’ (p.134). Lich y’s wo k is emblema ic o much esea ch in AR in ha i akes li e ally AR’s os ensible p emise o augmen ing 1 This di e s somewha om o he b oad de ini ions pa icula ly wi hin a hea e and pe o mance con ex . Fo example, Ben o d and Giannachi (2011) use he e m ‘mixed eali y’ while Vincs e al (2018) use augmen ed eali y – bo h include p ojec ion echnologies wi hin hei de ini ions. 2 These examples con ain an in e es ing discussion on AR’s abili y o in e ene in o spaces o powe due o i s abili y o hide hings in plain sigh . THE SCENOGRAPHIC SPACE(S) OF AUGMENTED REALITY 330 ‘ eali y’ wi h i ual objec s o in o ma ion. This, acco ding o he dominan pa adigm in AR, di e en ia es i om VR – as a he han being imme sed in a i ual space, he use ’s p ima y expe ience is o he eal physical space. This again unde sco es he way in which much schola ship in VR and AR se s up a p oblema ic dicho omy be ween i uali y and eali y. I con end ha his bina y ails o accoun o AR’s abili y o enac specula i e wo lds, in a manne simila , bu dis inc om VR. A ew esea che s ha e discussed he way ha AR migh p oduce an al e na e space o eali y. Bo ko Fu h a gues ha AR ‘modi ies he spa ial con igu a ion o eali y’ (2011, p.61) and p oduces a ‘new en i onmen ’ (p.48). Rewa W igh also con ends ha AR posi s a ‘pa allel eali y’ ha a ises ou o he ela ionali y o body and a wo k (2014), hough o he his eali y is dis inc om, bu connec ed o, physical space. Fu h ’s and W igh s amings poin o mul iple ‘ eali ies’ o o de s o space, a he han a bina y dis inc ion be ween eali y and i uali y. Following his, we can begin o open AR o a discussion o spa iali y – expanding i s a endan ques ions o embodimen o a discussion o bodily imme sion ac oss mul iple egis e s o space. I p opose ha his o e s a dis inc ly scenog aphic egis e wi h which o expe imen wi h AR. Exis ing esea ch in scenog aphy p o ides some amewo ks o hink h ough scenog aphy’s ole in c a ing embodied ela ions in and wi h space(s). Joslin McKinney and Sco Palme de ine scenog aphy as ‘a mode o encoun e ounded on spa ial and ma e ial ela ions be ween bodies, objec s and en i onmen s’ (2017, p.2). This no ion o scenog aphy as a ela ional p ac ice is also echoed by o he esea che s (e.g. A onson, 2017; Hann, 2019) McKinney and Palme iden i y ela ionali y as a key concep o unde s anding how scenog aphy ope a es in and beyond pe o mance. Taking AR echnology as an in e ace be ween bodies and a ious i ual and physical egis e s o space, i can also be seen o unc ion ela ionally. This highligh s one way we can hink abou he scenog aphic and spa ial po en ial o AR. Howe e , in p oposing ha AR enac s specula i e wo lds, I con end ha we can go u he in discussing how a LUCY THORNETT 331 scenog aphic aming migh b ing o ligh AR’s spa iali y, and in doing so, i s a ec i i y. E nes Remains E nes Remains was a p ac ice-led pe o mance ins alla ion s aged a he Uni e si y o Leeds in 2019. I was designed as sho expe ience o app oxima ely wen y minu es, o one audience membe a a ime. Audiences we e gi en a se o headphones and a handheld able and en e ed a mul i- oomed ins alla ion ha u ilised he en i e space o he hea e, including d essing oom and audi o ium. Foo age o a igu e mo ing a ound he space o he ins alla ion appea ed on- sc een pe iodically, guiding he audience h ough he en i onmen . In be ween hese in e ludes, he able could be held o e objec s in he space o igge sound and ideo cues as well as i ual objec s appea ing on he sc een. The con en o he piece was au obiog aphical and ela ed o my g and a he , E nes Tho ne , who was a de ec i e ic ion au ho and code-b eake o B i ish in elligence du ing he second wo ld wa . The piece d ew on ‘d awing- oom mys e y’ opes and cas he audience membe as de ec i e, using he able o disco e clues. The space was illed wi h a chi al ma e ial ela ing o E nes , such as le e s and pho og aphs, as well as con empo a y anach onis ic ma e ial and objec s such as p in -ou s o emails and online cha s, digi al sc eens, and loo plans o he ins alla ion space. Thema ically he wo k explo ed he digi al and ma e ial aces bodies lea e behind, and ouched on con empo a y and his o ical no ions o su eillance. In doing so hese hemes in en ionally o e lapped wi h he esea ch aims o in es iga ing he ela ions be ween bodies, ma e , en i onmen s and echnologies. Audiences also wo e a go-p o came a ha eco ded wha hey saw, and his oo age was used o p omp discussion du ing pos -show in e iews. I will discuss his wo k h ough an analysis o my own expe ience, and commen s made by audiences. Whe e quo ing di ec ly om in e iews, I ha e used my own ini ials (LT) o indica e mysel as in e iewe , and he le e s A-G o indica e di e en membe s o he audience. Two key aspec s o he pe o mance THE SCENOGRAPHIC SPACE(S) OF AUGMENTED REALITY 332 eme ged whe e he AR p oduced a scenog aphic spa iali y, and I will ocus on hese momen s in my discussion he e. The i s o hese was he expe ience o ‘ ollowing’ he ilmed pe o me h ough he space. Fo many audience membe s, his c ea ed he sensa ion ha he pe o me inhabi ed i ual and physical space simul aneously: A: I liked ha sensa ion o ollowing somebody. When I el com o able, when I knew whe e I was going. LT: Wha did you like abou i ? A: Tha i wasn’ eal, ha he wasn’ he e. Bu he kind o was. Tha was he…almos el mo e hea ical han he o he bi s and pieces. I el like he e was his eal pe son who wasn’ he e. LUCY THORNETT Figu e 1. E nes Remains augmen ed eali y ins alla ion a Banham Thea e, Uni e si y o Leeds (2019) by au ho . An audience membe iewing he ins alla ion h ough he sc een 333 The use o he wo d ‘ hea ical’ he e is in e es ing in ela ion o his slippage be ween i uali y and ac uali y. An onin A aud desc ibed hea e as a ‘ i ual eali y’, discussing he way ha i posi s an ‘illuso y wo ld’ (1958, p.49). A numbe o schola s ha e poin ed ou ha hea e is inhe en ly conce ned wi h i uali y (e.g. Popa , 2016; Thomas and Glowacki, 2018). Fo B ian Massumi, he i ual is no illuso y, bu on ologically eal. He de ines he i ual as he ‘“ eal bu abs ac ” inco po eali y o he body’ (2002, p.21). Fo Massumi, he e m i ual he e is no synonymous wi h digi al eali ies, howe e o he s including Ma k G imshaw ha e a gued ha digi al echnology migh p o ide us wi h new ways o accessing his i uali y (2014). Massumi’s wo k is use ul he e in unde s anding ha he i ual is no less eal han physical eali y. Ins ead i consis s o a di e en ma e iali y, one ha can s ill be el in he body. In his sense, we can ela e i ual eali y o hea ical eali y in ha hough i is specula i e, i is simul aneously on ologically eal, and pe cei ed as such h ough THE SCENOGRAPHIC SPACE(S) OF AUGMENTED REALITY Figu e 2. E nes Remains augmen ed eali y ins alla ion a Banham Thea e, Uni e si y o Leeds (2019) by au ho . Audience ollowing a ilmed pe o me h ough he space 334 he sensing body o he audience. I will u n now o ano he momen in he pe o mance o illus a e his u he . In he inal pa o he pe o mance, audiences we e guided o e u n o he space hey had s a ed in. While hey we e elsewhe e in he space, he cu ains had opened o e eal pos e s o ees on he audi o ium loo . A an had been u ned on, c ea ing a gen le b eeze ha mo ed he cu ains. When audiences di ec ed he able a he ees, li e-sized i ual ees appea ed ha emained in place i he sc een was mo ed up owa ds he lea es a he op. Fo some audiences, he ees egis e ed as a solid p esence in he space. Fo hese pa icipan s he e was an embodied ins inc o a oid walking h ough he ees, despi e knowing ha hey we en’ physically he e, as seen in he commen s om wo audience in e iews below: B: Yeah he ees we e, ha was a e y cool eeling, so o walking in and… knowing whe e I could mo e and whe e I could walk suddenly became a bi o like a ‘Oh, wai ’. I almos ques ioned whe he I could walk o e he ee… Yeah, i el de ini ely el like physical obs acles and I was so o almos … mo ing a ound hem a he han so o con iden ly mo ing h ough hem. LT: Oh, you mo ed you body o ge a ound hem? C: Yeah, I wen like ha o y and ge ound i . And I was like (imi a es mo emen ) ... ha ’s idiculous. LT: Tha ’s qui e in e es ing. So hen he e’s a kind o bodily eeling ha he ee is in he space? C: Yeah. E en hough I knew i wasn’ . The i ual ees he e a e isually seen on-sc een, bu el o be co-p esen in he same physical space as he audience – hey ha e a i ual ma e iali y. The ac ha his slippage o i ual in o ac ual was desc ibed by he pa icipan abo e as p oducing a ‘cool eeling’ poin s o he a ec i e na u e o his expe ience. Fo LUCY THORNETT 335 Massumi, a ec is empo al – a ‘s a e o suspense’ (1995, p.86) o p e-conscious in ensi y ha allows he i ual o en e , b ie ly, in o ou conscious expe ience. In ac , he a gues ha i can only be unde s ood empo ally and no spa ially (2011, p.16). Howe e , an a mosphe ic and scenog aphic pe spec i e migh e eal his a ec i e expe ience o he i ual o be spa ial as well as empo al. The e is a g owing body o schola ship on he spa iali y o hese a ec i e p ocesses. De ek P. McCo mack uses he e m ‘a ec i e space imes’ o a icula e he insepa abili y o spa iali y and empo ali y in his p ocess: Space, in o he wo ds, is ne e a backd op o some hing mo e dynamic. No indeed, can i o should i be jux aposed o p ocess o empo ali y in a way ha p i ileges he la e . Ins ead, i is always mo e accu a e o speak o space, o spaces, as mul iple: spaces p oduced ia a ange o echnologies and expe ienced h ough di e en senso y egis e s; spaces wi h a iable eaches and in ensi ies; and spaces ha can o en only be app ehended in and h ough he assemblages o mo emen and s illness o which hey a e composed. (2014, p.2) This discussion o spaces as mul iple, as p oduced by bodies in mo emen and h ough echnologies unde sco es he way ha he AR echnology in E nes Remains enac ed specula i e scenog aphic space(s). Ka hleen S ewa simila ly speaks o a mosphe ic ‘a unemen s’ – ‘an a en ion o he… complex eme gen wo lds…’, which, ollowing Heidegge , she calls ‘wo lding’ (2011, p.445). Like McCo mack, she discusses he embodied p ac ices h ough which such wo ldings eme ge (p.446). This helps o hink h ough he way in E nes Remains, he mo ing body in conce wi h handheld sc een, digi al con en and headphones – p oduced a pa icula space ime o a mosphe e. The e a e a numbe o ways ha his spa iali y was seen o mani es in he pe o mance. One o hese was a pa icula di ec ionali y necessi a ed by he THE SCENOGRAPHIC SPACE(S) OF AUGMENTED REALITY