Examining User Performance in the presence of Latency and Jitter in Distributed Interactive Applications

ISSC 2004, Bel as , June 30 - July 2 Examining Use Pe o mance in he p esence o La ency and Ji e in Dis ibu ed In e ac i e Applica ions Declan Delaney φ , Paul Meeneghan φ , Tomás Wa d* and Séamus Mc Loone* φ Depa men o Compu e Science, NUI Maynoo h Co. Kilda e, I eland E-mail: φ [email p o ec ed]ay.ie * Depa men o Elec onic Enginee ing, NUI Maynoo h Co. Kilda e, I eland E-mail: * oma[email p o ec ed] __________________________________________________________________________________________ Abs ac – La ency and ji e inhe en ly limi he main enance o consis ency in Dis ibu ed In e ac i e Applica ions such as compu e games, dis ibu ed whi eboa ds and eal- ime, collabo a i e en i onmen s. Al hough he e has been much esea ch in o me hods o main aining consis ency, he e is a dis inc lack o esea ch explo ing he connec ion be ween la ency, ji e and he end use expe ience in Dis ibu ed In e ac i e Applica ions. We ha e de eloped an applica ion ha allows us o conduc ials unde con olled la ency and ji e condi ions. This p o ides da a, which can be analysed o cha ac e ise how people adap o a ious deg ees o la ency and ji e . We p esen esul s ha highligh how an inc ease in la ency and ji e a ec he end- use expe ience, hus con i ming he need o echniques o comba la ency and ji e in Dis ibu ed In e ac i e Applica ions. We alsoe no e ha he e ec s o ji e a e signi ican ly g ea e han hose o la ency. Keywo ds – Dis ibu ed In e ac i e Applica ions, Ji e , La ency. __________________________________________________________________________________________ I INTRODUCTION Dis ibu ed In e ac i e Applica ions (DIAs) a e a class o compu e p og ams in ol ing eal- ime use s ac ing in a i ual en i onmen o execu e some collabo a i e, coope a i e o in e ac i e ask. The dis ibu ed na u e o such applica ions means ha a consis en wo ld iew ac oss all pa icipan s is impossible o achie e. This esul s in each pa icipan seeing sligh ly di e en e en s a di e en imes. In such a case he end-use expe ience can be des oyed. Al hough he e a e a numbe o issues ha a ec he consis ency o a dis ibu ed applica ion, he ul ima e limi a ion is imposed by la ency e ec s. La ency has been de ined in a ious ways [1-3]. In his pape we de ine i as he ime aken om s a o ansmission o he i s bi o a da a packe a he applica ion laye o one pa icipa ing node o he eceip o he las bi o he same packe a he applica ion laye o a second pa icipa ing node. A numbe o delays combine o cause la ency. The e a e queuing and p ocessing delays a ou e s, b idges, ga eways and a packe sou ce and des ina ion; delays in anscoding he da a (enc yp ion / dec yp ion and comp ession / decomp ession); p opaga ion and ansmission delays due o he speed o ligh and he speed o he communica ions link. The e has been a lo o esea ch ca ied ou in o echniques and mechanisms o comba he e ec s o la ency. Mos o hese echniques seek o educe he numbe o packe s ha need o be ansmi ed be ween pa icipan s. By so doing, he la ency associa ed wi h queuing, p ocessing and anscoding is educed. Some echniques include he ollowing: • Clien -side p edic ion con ac mechanisms: These include echniques such as dead eckoning [4, 5] and s a egy-based modelling [6, 7]. • Rele ance Fil e ing echniques: da a is il e ed based on c i e ia such as geog aphical p oximi y o a e o change [8]. Ano he o m o ele ance il e ing is isibili y culling [9]. • Ne wo k ansmission p o ocol: Mul icas ing o g oups o pa icipan s wi h he same da a equi emen s. The In e ne o e s e y limi ed suppo o mul icas ing p o ocols [10]. • Packe bundling: Inc ease he packe size by sending mul iple packe s unde he one heade . This may also in oduce la ency i no used wi h ca e [8]. • Da a Comp ession: Reduce he size o he in o ma ion packe being ansmi ed, using o example, del a comp ession [8]. • Time Managemen : Synch onize e en s o wall clocks among pa icipan s. E en s can be p e-emp ed o delayed o main ain consis ency [11, 12]. • P io i y Scheduling: Da a essen ial o main aining consis ency is sen wi h he highes p io i y [13]. • Quali y o Se ice (QoS) Techniques: ne wo k pa ame e s such as bandwid h and la ency a e gua an eed by he se ice p o ide [14, 15]. Despi e he esea ch ca ied ou on sol ing he la ency p oblem, e y li le wo k has been ca ied ou on he e ec s o la ency a ia ion o ji e on use s o dis ibu ed applica ions. An in e es ing sui e o expe imen s was pe o med by Vaghi e al. in which hey examined he e ec s o g adually inc easing la ency on he in e ac ion o wo pa icipan s in a i ual ball game [16]. Thei esul s indica ed ha he use s modi y hei playing s a egies as hey a emp o cope wi h la ency and hey iden i ied se e al ypes o adap a ion s a egies. The ocus o his pape is on examining he accu acy o use pe o mance in he p esence o la ency. Fu he mo e, we ex end ou wo k o also include he e ec s o ji e . We in end o show ha hese e ec s can’ be simply igno ed. In he ollowing sec ion we de ail he de elopmen o he applica ion ha will be used o conduc expe imen s unde con olled la ency and ji e condi ions. The expe imen s pe o med a e ou lined in sec ion 3, while he esul s a e p esen ed and analysed in sec ion 4. The inal sec ion indica es ou conclusions and he u u e di ec ion o he esea ch. II TEST PLATFORM The es pla o m was designed o measu e he abili y o use s o pe o m a single ask in he p esence o a ying deg ees o la ency and ji e and a di e en speeds o ope a ion. The applica ion was de eloped in Ja a e sion 1.4.2 (ja a.sun.com), employing a Ja a class lib a y o gene a ing cha s called JF eeCha e sion 0.9.16 (j ee.o g/j eecha /). The Eclipse In eg a ed De elopmen En i onmen (IDE) e sion 2.1 was used o de elop he applica ion. The choice o ja a was made so ha he applica ion would be independen o he unde lying ope a ing sys em, hus allowing he applica ion o be made a ailable o mul iple pla o ms o e he In e ne . The g aphical use in e ace (GUI) was designed Figu e 1: G aphical Use In e ace ha is in ui i e and allows he expe imen al pa ame e s o be se . Use sees his en i y Loca ion o en i y ha he use does no see. Di e ence in loca ion is due o la ency and ji e . Mo emen T ajec o y wi h wo objec i es in mind: (1) o allow he expe imen e se he expe imen al pa ame e s easily and hide hem om he use and (2) o p o ide an easily na igable layou and game-like en i onmen wi h bo h audio and isual eedback. The GUI is shown in Figu e 1. The main ea u es o he so wa e a e: • T ajec o y, la ency and ji e alues can be se using slide ba s; • Feedback o he use can be u ned on/o ; • All expe imen al da a is sa ed o a ime- s amped di ec o y in ex o ma o acili a e expo o o he applica ions; • A se ies o 15 plo s a e au oma ically gene a ed when he use exi s he applica ion. A sample o one o hese plo s is shown in Figu e 2. The expe imen al pla o m simula es he local ende ing o a emo e en i y ( ep esen ed as a disk), which is mo ing along one o se e al possible ajec o ies. The use can only see he posi ion o he disk compu ed locally, al hough he ue posi ion o he emo e en i y can also be shown i desi ed. The ask pe o med by he pa icipan s in each expe imen is o a emp o p edic he ac ual loca ion o he emo e en i y’s posi ion unde a ying deg ees o la ency and ji e . All mouse clicks a e eco ded and when a co ec ‘hi ’ is made he applica ion p o ides audio and isual eedback, as well as inc emen ing a hi coun e . Al hough he e is no limi o he numbe o mouse clicks ha can be made, a hi can only be sco ed once o each simula ion ime uni . The da a p oduced by each expe imen is eco ded in ASCII o ma in a unique ime s amped olde . In all 15 di e en a iables a e eco ded such as ime sample, hi coun , ji e alue, la ency alue, cumula i e e o and o al mouse clicks. The es pla o m was used o eco d da a om a numbe o use s. De ails o hese expe imen s a e gi en in he ollowing sec ion. III EXPERIMENTS The es pla o m was employed o pe o m an expe imen in ol ing a o al o eigh olun ee s. Each olun ee pe o med a single expe imen consis ing o a se ies o i e a ions du ing which he la ency and ji e alues we e a ied au oma ically by he applica ion acco ding o he pa ame e s de ined. The ec angula ajec o y was employed. Use s began a he op le -hand co ne o he ec angle and mo ed in a clockwise di ec ion. The la ency and ji e alues emained in a ian o each i e a ion o he ec angle. A o al o 12 indi idual i e a ions o he ec angle we e pe o med wi h di e en se ing o la ency and ji e – see he lis o i e a ions o he expe imen in Table 1. Bo h audio and isual eedback we e u ned on bu use s we e unawa e o he la ency and ji e pa ame e s. The objec i es o he i e a ions a e desc ibed in he ollowing pa ag aphs. I e a ions La ency Ji e 1 0 0 2 1 0 3 2 0 4 3 0 5 4 0 6 5 (max) 0 7 3 0 8 3 10 9 3 15 10 3 20 11 3 25 12 3 30 (max) Table 1: A se ies o 12 i e a ions we e pe o med. Ji e and la ency we e a ied each ime he use passed h ough he op le -hand co ne o he ec angle. I e a ion 1: Du ing his i e a ion we measu e he pa icipan ’s pe o mance when he e is no ji e o la ency. Bo h he local and emo e posi ions a e iden ical and he use only has o mo e he mouse cu so o e he mo ing a ge and click. Each mouse click should esul in a sco e wi h any e o s being a esul o use inaccu acy and no a esul o la ency o ji e . An e o alue o 11 o unde esul s in a ‘hi ’ o sco e. This alue co esponds o he size o he objec on he sc een. This p o ides a benchma k o baseline measu emen , agains which o he measu emen s can be accu a ely compa ed. I e a ions 2 o 6: He e ji e was kep a a cons an alue o ze o and la ency was inc eased in s eps om Figu e 2: A se ies o g aphs a e au oma ically gene a ed when he use exi s he p og am – a sample g aph is shown he e. ze o o i s maximum alue o he gi en applica ion on e e y i e a ion o he ec angle. This p o ides a measu e o he e ec s o la ency in he absence o ne wo k ji e . I e a ion 7: This i e a ion is pe o med wi h a medium alue o la ency and ze o ji e . This la ency alue will be main ained in he subsequen i e a ions. This i e a ion is iden ical o i e a ion 4 and ma ks he ansi ion om la ency- a ying o ji e - a ying i e a ions. I e a ions 8 o 12: In hese i e a ions, la ency is main ained a a medium alue o 3 and use pe o mance unde p og essi ely inc easing ji e is measu ed. This p o ides da a ela ing o he e ec s o ji e unde condi ions o cons an la ency. An analysis o he esul s a e p esen ed in he ollowing sec ion. IV ANALYSIS OF RESULTS The p esen ed esul s a e p elimina y and o m he basis o unde s anding he beha iou o use s unde con olled condi ions o la ency and ji e . The esul s a e shown o a ypical use ha is ep esen a i e o all pa icipan s in he i e a ion. Figu es 3(a)-(d) illus a e a ep esen a i e pe o mance o one o he pa icipan s in e ms o hi coun sco e as la ency and ji e inc ease om 0 o hei maximum alues. Figu es 3(b) and 3(d) a e he no malized e sion o Figu es 3(a) and 3(c) espec i ely. The no maliza ion is pe o med wi h espec o he o al numbe o mouse clicks made by he use . As la ency inc eases, Figu es 3(a) and 3(b) show a all o in he sco e he use achie es, educing om an absolu e alue o abou 30 o unde 15 when la ency is a i s maximum alue. Wi h la ency main ained a a cons an alue, ji e ends o esul in a poo e o e all use pe o mance wi h all sco es lying be ween 10 and 20. In addi ion Figu e 3(c) shows ha he a ia ion in sco es is mo e andom, compa ed o he g adual all o in pe o mance wi h inc easing la ency. I ’s o in e es o no e ha in Figu e 3(a) when la ency is 3 and ji e is ze o, he hi coun is 24. In Figu e 3(c) when la ency is educed o 3 and ji e is s ill ze o, he hi coun is only 12. This was o be expec ed because he use would be expec ing he end o inc easing la ency o con inue and he adjus men in la ency was g ea e han be ween any o he p e ious i e a ions. The ins an aneous e o e e s o he sum o he absolu e dis ance om he posi ion he mouse was clicked a o he ac ual posi ion o he emo e en i y – see Figu e 4. The cumula i e e o e e s o he sum o he ins an aneous e o wi h ime. A ypical plo o cumula i e e o agains ime is shown in Figu e 5 o h ee scena ios: (1) Ji e and la ency bo h se o ze o; (2) la ency se o i s maximum alue and ji e se o ze o and (3) la ency a a medium alue and ji e se o maximum. In scena io (1) he cu e may be app oxima ed as a s aigh line, indica ing ha he accumula ion in e o is a linea p ocess. The o he scena ios illus a e he a e o accumula ion o e o in he p esence o high la ency is mo e han doubled Figu e 4: The ins an aneous e o is compu ed be ween he mouse click posi ion and he ac ual posi ion o he emo e en i y. Figu e 5: As ime p og esses he cumula i e absolu e e o inc eases bu does so mo e apidly in he p esence o ji e han ha o la ency. Figu e 3(a)-(d): As la ency is inc eased, he pa icipan ’s success a sco ing a hi educes. In oducing ji e also esul s in a signi ican educ ion. (x 1 ,y 1 ) (x 3 ,y 3 )(x 2 ,y 2 ) mouse click posi ion Local ende ing o emo e posi ion Ac ual emo e posi ion E o , ∆E 2 = (x 2 -x 1 ) 2 +(y 2 -y 1 ) 2 (x 1 ,y 1 ) (x 3 ,y 3 )(x 2 ,y 2 ) mouse click posi ion Local ende ing o emo e posi ion Ac ual emo e posi ion E o , ∆E 2 = (x 2 -x 1 ) 2 +(y 2 -y 1 ) 2 Ze o la ency, ze o ji e Max la ency, ze o ji e La ency = 3, Max ji e Hi Coun Time Ze o la ency, ze o ji e Max la ency, ze o ji e La ency = 3, Max ji e Hi Coun Time La ency Value Hi coun sco e La ency Value Ji e Value (a) (b) (c) (d) Hi coun sco e Hi coun sco e Hi coun sco e Ji e Value La ency Value Hi coun sco e La ency Value Ji e Value (a) (b) (c) (d) Hi coun sco e Hi coun sco e Hi coun sco e Ji e Value compa ed o he case o ze o la ency and ze o ji e . Finally we also no e ha he e was a u he inc ease in cumula i e e o due o he addi ion o ji e . Typical plo s o he ins an aneous e o s as a unc ion o ime a e shown in Figu es 6(a)-(d). Figu e 6(a) shows ha wi h no la ency o ji e he ins an aneous e o is low, usually below he expe imen al e o le el o 11. As la ency and ji e a e in oduced he ampli ude o e o a ia ion inc eases, wi h he wo se case scena io being he case o medium la ency and high ji e in Figu e 6(d). An examina ion o he dis ibu ion o ins an aneous e o s clea ly demons a es he e ec s o ji e and la ency on use pe o mance - see Figu e 7. The dis ibu ions become wide as la ency and ji e a e in oduced. The dis ibu ion o ze o la ency and ze o ji e has a maximum sp ead o abou 20. In he case o medium la ency and maximum ji e his inc eases o a sp ead o almos 100. The a e age ins an aneous e o o each i e a ion was calcula ed oge he wi h he a iance. These alues we e plo ed and a e shown in Figu e 8. The a e age e o inc eases s eadily and becomes e a ic when ji e is in oduced. The a iance is also much less p edic able when ji e is in oduced. I is wo h obse ing ha use s ind i easie o adap o cons an ly inc easing la ency bu s uggled o cope wi h he e ec s o ji e as indica ed in Figu e 8. I is also in e es ing o no e he impo ance o eedback o he use du ing he expe imen . A low alues o la ency and ji e , use s adap ed o hei p esence by elying on he audio eedback. Howe e , a high alues o ji e hey elied mo e on he isual eedback mechanism illus a ed in Figu e 9 because hey could hea ha hey we en’ making any hi s and hey needed some addi ional in o ma ion o help hem compensa e. Visual eedback is p o ided in a small indica o sc een showing he posi ion o he use ’s ac ual mouse click and a line om he e o he co ec sco ing posi ion. I is ob ious ha eedback becomes mo e impo an a high la ency and ji e alues, bu his is o be expec ed as he use ’s ask becomes mo e di icul as la ency and ji e a e in oduced. These indings con i m ou in ui ion and a e no al oge he unexpec ed. V CONCLUSIONS and FUTURE WORK A es pla o m and an expe imen designed o allow us de e mine he e ec s o la ency and ji e on use pe o mance ha e been desc ibed. While he esul s p esen ed in his pape a e o one use only, i was ound ha all eigh use s showed simila beha iou . They illus a e ha use pe o mance, e en in a ela i ely simple applica ion, de e io a es apidly in Figu e 8: The a ia ion in a e age e o alue and a iance o each expe imen Figu e 9: The Visual Feedback Mechanism. Figu e 7: The dis ibu ion o e o s inc eases wi h inc easing la ency and ji e . Figu e 6(a)-(d): Plo s o ins an aneous e o o di e en la ency/ji e combina ions Va iance Value A e age Value Expe imen no. Expe imen no. (a) A e age o e o s (b) Va iance in e o s Va iance Value A e age Value Expe imen no. Expe imen no. (a) A e age o e o s (b) Va iance in e o s local posi ion mouse click posi ion line connec ing o co ec posi ion local posi ion mouse click posi ion line connec ing o co ec posi ion (a) No la ency o ji e (b) Maximum la ency, No ji e (c) Medium la ency, No ji e (d) Medium la ency, maximum ji e Coun E o Value E o Value E o Value E o Value Coun Coun Coun (a) No la ency o ji e (b) Maximum la ency, No ji e (c) Medium la ency, No ji e (d) Medium la ency, maximum ji e Coun E o Value E o Value E o Value E o Value Coun Coun Coun Coun E o Value E o Value E o Value E o Value Coun Coun Coun (a) No la ency o ji e (b) Maximum la ency, No ji e (c) Medium la ency, No ji e (d) Medium la ency, maximum ji e Time Time Time Time E o E o E o E o (a) No la ency o ji e (b) Maximum la ency, No ji e (c) Medium la ency, No ji e (d) Medium la ency, maximum ji e Time Time Time Time E o E o E o E o he p esence o la ency and ji e and hus hese ac o s canno be simply igno ed. The inc ease in esea ch o la ency masking echniques he e o e needs o be ma ched by esea ch in o comba ing he e ec s o ji e . The expe imen s conduc ed o da e indica e a numbe o new a enues o explo e. Se e al ea u es will be added o he es pla o m such as eco ding o use mouse click coo dina es, use o a sc ip ing language o se up he expe imen s and inc eased size o he isual eedback sc een. Ou objec i e is o de e mine accep able le els o ji e and la ency alues so ha he use ’s psycho-pe cep ual expe ience emains accep able. R EFERENCES [1] J. C. de Oli ei a, S. Shi mohammadi, and N. D. Geo ganas, "Collabo a i e Vi ual En i onmen S anda ds: A pe o mance E alua ion," p esen ed a P oceedings o he 3 d IEEE In e na ional Wo kshop on Dis ibu ed Simula ion and RealTime Applica ions, Oc obe 23-24, College Pa k, Ma yland, 1999. [2] S. K. Singhal and M. Zyda, Ne wo ked Vi ual En i onmen s. New Yo k: ACM P ess, 1999. [3] J. Smed, T. Kauko an a, and H. Hakonen, "Aspec s o Ne wo king in Mul iplaye Compu e Games," p esen ed a P oceedings o In e na ional con e ence on applica ion and de elopmen o compu e games in he 21s cen u y, China, 2001. [4] IEEE, "IEEE S anda d o Dis ibu ed In e ac i e Simula ion - Applica ion P o ocols," IEEE 1995. [5] C. Du bach and J. M. Fou neau, "Pe o mance E alua ion o dead eckoning mechanism," p esen ed a P oceedings o he 2nd IEEE In e na ional Wo kshop on Dis ibu ed In e ac i e Simula ion and RealTime Applica ions, May 1-4, A lan a, Geo gia, 1999. [6] D. Delaney, T. Wa d, and S. Mc Loone, "On Reducing En i y S a e Upda e Packe s in Dis ibu ed In e ac i e Simula ions using a Hyb id Model," p esen ed a P oceeding o he 21s IASTED In e na ional Mul i- con e ence on Applied In o ma ics, Feb ua y 10-13, Innsb uck, Aus ia, 2003. [7] D. Delaney, T. Wa d, and S. Mc Loone, "Reducing Upda e Packe s in Dis ibu ed In e ac i e Applica ions using a Hyb id Model," p esen ed a 16 h In e na ional Con e ence on Pa allel and Dis ibu ed Compu ing Sys ems, Augus 13-15, Reno, USA, 2003. [8] M. A. Bassiouni, M. H. Chiu, M. Lope , M. Ga nsey, and J. Williams, "Pe o mance and eliabili y analysis o ele ance il e ing o scalable dis ibu ed in e ac i e simula ion," ACM T ansac ions on modeling and compu e simula ion, ol. 7, pp. 293-331, 1997. [9] C. Faiss naue , D. Schmals ieg, and W. Pu ga ho e , "P io i y Round-Robin Scheduling o e y la ge Vi ual En i onmen s," p esen ed a P oceedings o he IEEE Vi ual Reali y 2000 Con e ence, Ma ch 18-22, New B unswick, New Je sey, 2000. [10] J. M. Pullen, "Reliable Mul icas ne wo k T anspo o Dis ibu ed Vi ual Simula ion," p esen ed a P oceedings o he 3 d IEEE In e na ional Wo kshop on Dis ibu ed Simula ion and RealTime Applica ions, Oc obe 23-24, College Pa k, Ma yland, 1999. [11] B. G. Wo hing on and D. J. Robe s, "Encapsula ing Ne wo k La ency Compensa o s in VRML," p esen ed a P oceedings o VWSIM (Vi ual Wo lds and Simula ion Con e ence), San Diego, USA, 2000. [12] D. J. Robe s and P. M. Sha key, "Maximising Concu ency and Scalabili y in a Consis en , Causal, Dis ibu ed Vi ual Reali y Sys em, whils minimising he e ec o Ne wo k Delays," p esen ed a P oceedings o he 6 h Wo kshop on Enabling Technologies In as uc u e o Collabo a i e En e p ises, Camb idge, MA, 1997. [13] C. Faiss naue , D. Schmals ieg, and W. Pu ga ho e , "Scheduling o e y la ge Vi ual En i onmen s and Ne wo ked Games using Visibili y and p io i ies," p esen ed a P oceedings o he 4 h IEEE In e na ional Wo kshop on Dis ibu ed Simula ion and RealTime Applica ions, Augus 24-26, San F ancisco, Cali o nia, 2000. [14] A. S. Tanenbaum, Compu e Ne wo ks, 3 ed: P en ice Hall, 1996. [15] C. Chasso , A. Lozes, F. Ga cia, M. Diaz, L. Dai aine, and L. Rojas, "QoS equi ed by a DIS applica ion in a new gene a ion In e ne ," p esen ed a IEEE Sp ing Simula ion In e ope abili y Wo kshop, Ma ch 14-19, O lando, Flo ida, 1999. [16] I. Vaghi, C. G eenhalgh, and S. Ben o d, "Coping wi h inconsis ency due o ne wo k delays in collabo a i e i ual en i onmen s," p esen ed a P oceedings o he ACM symposium on Vi ual eali y so wa e and echnology, London, England, 1999.