Network traffic characterisation, analysis, modelling and simulation for networked virtual environments

Network traffic characterisation, analysis, modelling and simulation for networked virtual environments Juan Luis Font Calvo July 2019 Departmento de Arquitectura y Tecnología de Computadores Escuela Técnica Superior de Ingeniería Informática Doctoral dissertation Network traffic characterisation, analysis, modelling and simulation for networked virtual environments Juan Luis Font Calvo Supervisor: José Luis Sevillano Ramos July 2019 Juan Luis Font Calvo Network traffic characterisation, analysis, modelling and simulation for networked virtual environments Doctoral dissertation, July 2019 Supervisor: José Luis Sevillano Ramos Universidad de Sevilla Escuela Técnica Superior de Ingeniería Informática Departmento de Arquitectura y Tecnología de Computadores Avd. Reina Mercedes S/N 41700 Sevilla Abstract Networked virtual environment (NVE) refers to a distributed software system where a simulation, also known as virtual world, is shared over a data network between several users that can interact with each other and the simulation in real-time. NVE systems are omnipresent in the present globally interconnected world, from entertainment industry, where they are one of the foundations for many video games, to pervasive games that focus on e-learning, e-training or social studies. From this relevance derives the interest in better understanding the nature and internal dynamics of the network traffic that vertebrates these systems, useful in fields such as network infrastructure optimisation or the study of Quality of Service and Quality of Experience related to NVE-based services. The goal of the present work is to deepen into this understanding of NVE network traffic by helping to build network traffic models that accurately describe it and can be used as foundations for tools to assist in some of the research fields enumerated before. First contribution of the present work is a formal characterisation for NVE systems, which provides a tool to determine which systems can be considered as NVE. Based on this characterisation it has been possible to identify numerous systems, such as several video games, that qualify as NVE and have an important associated literature focused on network traffic analysis. The next contribution has been the study of this existing literature from a NVE perspective and the proposal of an analysis pipeline, a structured collection of processes and techniques to define microscale network models for NVE traffic. This analysis pipeline has been tested and validated against a study case focused on Open Wonderland (OWL), a framework to build NVE systems of different purpose. The analysis pipeline helped to defined network models from experimental OWL traffic and assessed on their accuracy from a statistical perspective. The last contribution has been the design and implementation of simulation tools based on the above OWL models and the network simulation framework ns-3. The purpose of these simulations was to confirm the validity of the OWL models and the analysis pipeline, as well as providing potential tools to support studies related to NVE network traffic. As a result of this final contribution, it has been proposed to exploit the parallelisation potential of these simulations through High Throughput Computing techniques and tools, aimed to coordinate massively parallel computing workloads over distributed resources. v Resumen Un entorno virtual en red (networked virtual environment, NVE) hace referencia a un tipo de software distribuido donde una simulación, también denominada mundo virtual, es compartida a través de una red de datos entre varios usuarios que pueden interactuar entre ellos y la propia simulación en tiempo real. Los sistemas NVE son omnipresentes en el mundo actual, global e interconectado, desde la industria del entretenimiento, donde son uno de los fundamentos sobre los que se asientan muchos video juegos, a los juegos ubicuos centrados en e-learning, e-training o estudios sociales. Debido a esta relevancia se deriva el interés en comprender mejor la naturaleza y dinámicas internas del tráfico de red que vertebra estos sistemas, de utilidad en campos como la optimización de infraestructuras de red o el estudio de Quality of Service yQuality of Experience asociados a servicios basados en sistemas NVE. El presente trabajo pretende profundizar en esta comprensión del trafico de red generado por sistemas NVE mediante la definición de modelos de tráfico de red que lo describan de forma precisa y que puedan ser usados como base para herramientas útiles para los campos de estudio mencionados anteriormente. La primera contribución del presente trabajo es una caracterización formal para sistemas NVE, la cual proporciona una herramienta para determinar que sistemas pueden considerarse NVE. En base a esta caracterización ha sido posible identificar numerosos sistemas, como por ejemplo varios videojuegos, que cualifican como NVE y tienen una importante literatura asociada centrada en el análisis de tráfico de red. La siguiente contribución ha sido el estudio de esta literatura desde la perspectiva de los sistemas NVE y la propuesta de un proceso de análisis(analysis pipeline), un conjunto estructurado de procesos y técnicas para definir modelos a nivel de micro-escala para el tráfico de red de sistemas NVE. Este proceso de análisis ha sido probado y validado contra un caso de estudio centrado en Open Wonderland (OWL), una herramienta (framework) para construir sistemas NVE de diverso propósito. El proceso de análisis ha ayudado a definir modelos de red partiendo de tráfico experimental de OWL y a evaluar su precisión desde el punto de vista estadístico. La última de las contribuciones ha sido el diseño e implementación de herramientas de simulación basadas en los anteriores modelos para OWL y la herramienta de simulación de redes ns-3. El objetivo de estas simulaciones ha sido confirmar la validez de los modelos de OWL y el proceso de análisis, a la vez que proporcionar vii herramientas potencialmente útiles en las áreas de estudio relacionadas con el tráfico de sistemas NVE. Como resultado de esta última contribución se ha propuesto explotar el potencial de paralelización de estas simulaciones a través de técnicas y herramientas de computación de alto rendimiento, (High-Throughput computing), cuyo objetivo es coordinar tareas de cómputo masivamente paralelas sobre recursos distribuidos. viii Agradecimientos La realización de esta tesis doctoral ha sido un largo viaje que ha abarcado años, todo tipo de eventos vitales y giros de guión. Me alegra poder agradecer por fin a todos aquellos que son casi tan responsables como yo de que haya sido posible. En primer lugar agradecer a mi director de tesis José Luis Sevillano por todos sus esfuerzos, ayuda y confianza en que esta tesis podía materializarse, a Christian Callegari del Dipartimento di Ingegneria dell’Informazione de la Università di Pisa por hospitalidad y ayuda durante mi estancia en Italia, y a mis antiguos compañeros del Departamento de Arquitectura y Tecnología de Computadores Daniel Cascado por sus esfuerzos y tutela, y a Rosa Yañez por la ayuda, cordura y amistad aportadas a la causa. En segundo lugar, agradecer a mis padres Concepción y Juan Luis, por apoyarme incondicionalmente, no sólo durante este proceso, sino durante toda mi vida. Mamá, Papá, os quiero. En tercer lugar, agradecer a todos esos amigos incondicionales repartidos por el camino entre Los Palacios (y Villafranca) y Ámsterdam con escala en Pisa y otros lugares igualmente interesantes. Sois geniales. ix in different contexts, ranging from military to academic and scientific purposes, as well as been an important part of a multibillion global business as it is that of video games. The following sections will provide a historical overview focused on these different fields, referencing some of the most significant milestones and their significance from the perspective of the NVEs. 1.1.1 Military use NVE systems were initially sought as cost effective alternative training with real combat vehicles and equipment while avoiding all the implicit risks associated to such an activity. Thus, military organisation have sponsored and hosted research projects to produce training technologies based on NVE systems [Rhe91], [Arm]. SIMNET “was the first successful implementation of large-scale, real-time, man-in- the-loop simulator networking for team training and mission rehearsal in military operations” [MT95]. The SIMNET research project was developed for the DARPA between 1983 and 1990, it provided a real-time distributed simulator for combat simulation. SIMNET had the features that characterize a NVE: it was designed to support hundreds of concurrent users using network technology initially based on 56 Kbit/s dial-up lines and moving to faster T-1 links in later stages of the project. Users were accommodated in the simulation stations, which provided an on-screen graphical representation of the virtual world as well as the required peripherals to interact with it. The simulation station was represented by a vehicle within the virtual world capable of hosting several human crew members. Audio generated by crew members was also transmitted within the virtual world. The Distributed Interactive Simulation (DIS) is an IEEE standard [Dis12] for conducting real-time platform-level wargaming across multiple host computers. DIS started as a follow-up project of SIMNET aimed to document its communication protocol, describing a fully distributed architecture for heterogeneous clients scalable up to 500 users. Another descendant of the SIMNET project was the Mounted Warfare TestBed (MWTB), located at Fort Knox, Kentucky, started in 1986 as the initial site for the SIMNET-D [Gar89] program, a spin-off of the original SIMNET project. The MWTB was the premier site for distributed simulation experiments in the US Army for over 20 years. It used simulation systems to perform experiments that examined current and future weapon systems, concepts, and tactics. Some of the technologies developed within the MWTB include the ACRT virtual simulator [CFM94] and the Virtusphere, a device that allows the user to walk around the simulated battlefield. 2Chapter 1 Introduction SIMNET also founded the basis of several companies and professional careers that transfer part of the knowledge and technology developed during the project to other non-military fields. Examples of this process are MetaVR Inc. (W. Garth Smith) [Met] and MaK Technologies [MAK] (Warren Katz and John Morrison), companies that still provide simulation and training software and resources for civil and engineering usage; Zipper Interactive founded by Brian Soderberg [Int12], a defunct video game company part of Sony Computer Entertainment, responsible of the development of the SOCOM (U.S. Special Operations Command) peer-to-peer (P2P) game series; or Wiz!Bang, a video game development company founded by Drew Johnston [Mob], who has developed a lengthy career in the video game industry working for the Microsoft Windows Gaming Platform team, among others. In recent years, this technology transfer has worked in the opposite direction, where well established technologies from the gaming consumer computer industry have found their way into military training programs and serious games, games not focused on entertainment purposes, but training and education. One example is the DARWARS [Vot04] research program started in 2003, intended to develop and deploy military training systems based on low-cost, mobile, web-centric, simulationbased, lightweight systems taking advantage of the ubiquitous presence of the PC, multi-player games, virtual worlds, off-the-shelf PC simulations, intelligent agents, and online communities. Another good example of technology transfer are the Virtual Battlefield Systems 1 (VBS1) released in 2002, and its successors VBS2 released in 2007 and VBS3 released 2014 [Int], all of them military simulators that heavily rely on modern gaming technology and generally considered as serious games. VBS1 is based on the first-person shooter (FPS) video game Operation Flashpoint developed by Bohemia Interactive Australia. The system allows training in military tactics in an interactive multiplayer 3D world. This combination of military simulator functionality and modern gaming technology proved to be a success and resulted in a broad military customer base. Besides training, NVE technologies and system have been also used as strategic communication tools. The first usage of gaming technologies in support of U.S. Army recruiting and strategic communication has been America’s Army [Arm], a platform to develop first-person shooter games released in 2002 by the U.S. Army, based in the Unreal game Engine and designed to allow young Americans to virtually explore the Army and helping them to determine if soldering matches their expectations. An evolution of America’s Army is the Virtual Army Experience (VAE) [Rel07], a mobile U.S. Army simulator created in 2007 by the Army development team which brought the army’s computer game, America’s Army: Special Forces (Overmatch) to a life-size, networked environment to provide visitors with an experience of soldiering. 1.1 Historical overview 3 1.1.2 Entertainment industry and video games A video game is defined as an electronic game that involves human interaction with an user interface to generate visual feedback on a video device (TV, computer monitor, etc.) [BRH72]. The main goal of video games is providing a rewarding experience to the player. This reward does not only lie in the final goals of the game but also the process itself to achieve them [MSB13]. Video games have evolved from a very focused market in mid 1970s to important industry (sometimes referred as interactive entertainment industry) billing more than $80 billions [BV14] worldwide in 2014 and jumping up to $138 billions in 2018 [BV18]. This growth has sustained the evolution and proliferation of numerous genres [Wol02], game mechanics and topics that currently conform a rich gaming ecosystem. In a classification attending technical features, video games relying on networking capabilities define the online games category, those played over a data network [Ada10]. These games have become ubiquitous thanks to the universalization of the Internet. Early NVE-compliant video games As discussed in section 1.1.1, NVE systems appeared in the context of the military context as a simulation and instruction tool. However, there are many video game genres and game dynamics based on the NVE concept (see Section 2.1 for a formal characterisation of NVE). Thus, there is a significant overlapping between the NVE paradigm and the characteristics of online video games: both consist in electronic, specifically digital , systems where several users engage in a virtual world simulation shared over a data network and can interact between them in real-time. Thus, online games which dynamics involve the recreation of a virtual world shared over a data network where users can interact and undertake challenges, are de-facto NVE systems designed to provide an entertaining experience. Some of the milestones in the convergence of video games into true NVE systems are linked to the technical advances making possible the incorporation of the defining characteristics of the NVEs into the consumer video game world. In the canonical accounts of the history of video games, Maze War (1974) is regarded as one of the earliest examples of FPS, as well as inventing or disseminating several important concepts that are used by many of the video games to follow [RP09] [Ars09]. Gameplay consisted in players roaming around a maze, other players are represented by eyeballs, when the player encounters another players in the maze, the player 4Chapter 1 Introduction can gain or lose points by shoot or been shot during the match. There are several pioneering features in Maze Wars that are closely linked to the NVE concept: • Initially developed to run on the Imlac PDS-1, a popular graphical display system which provided input and output peripherals. • First-person 3D perspective, players are provided with a visual representation of the maze (their shared virtual world) from their point of view. • Avatars, players are represented by eyeballs. Previous games usually represented players as spaceships and other kind of vehicles. This is one of the earliest examples where players are represented by organic beings. • Network play, an early example of P2P computers, unlike previous games played within the same minicomputer or mainframe using different terminals. • Client-server networked play, this feature achieved in later improved versions allowed to play the game over ARPANET in 1977, making it one of the earliest examples of Internet gaming. Moria [s15], [Add] (1975), an early role-playing game (RPG) game, borrowed concepts from Maze War. It was developed to run on the Programmed Logic for Automatic Teaching Operations (PLATO) system [SE79], the first generalized computer assisted instruction system. Moria took advantage of the many multi-user and networking features of PLATO to define a gameplay where parties of up to ten players to travel as a group, while exploring dungeons and featuring a wireframe first-person perspective display. An feature that makes Moria to further converge towards the NVE definition is the ability of players to communicate among them using text messages during the gameplay, adding a social factor. MUD1 (1978), initiator of the so-called Multi-User Dungeon (MUD) genre, was initially developed on a DEC PDP-10 at Essex University. MUD1 a multiplayer game where users interact in real-time in a Dungeon & Dragons inspired virtual world. MUD1 is text based, so the user interacts with the system typing combination of commands and keywords and receives a description of the virtual world and its events in text format. MUD1 can be considered a milestone in the convergence of video games into NVEs because it became the first online Internet role-playing game once the Essex University connected its internal network to ARPANet [MP03]. Moreover, it is considered one of the virtual world game still in existence, being playable via telnet protocol [Tot]. 1.1 Historical overview 5 Evolution and popularisation After early games defined and most of the key elements found in NVE systems, later titles pushed forward these features, improving the graphic representation of the virtual world, pushing the boundaries of connectivity to the range of local area networks (LANs) or popularizing among large audiences genres build around the NVE concept. Dogfight [SGI93] (1985) was developed by Silicon Graphics, Inc. (SGI) as a demo program for their workstations. The game is a multiplayer flight simulator where the player has a first person view of a cockpit, other users in the game are represented by planes which play the role of avatars. The game is considered a technical landmark because of its 3D render capabilities and especially networking features, being the first game to use the TCP/IP protocol. Games pioneering in technical features that would define the NVE experience (graphic representation of the virtual world, avatar, networking) gave way during mid and late 80s to games that contributed to the popularisation and expansion of different NVE-related genres, especially the FPS and the massively multiplayer online role-playing game (MMORPG) ones. Both genres will prove increasingly popular during the whole decade of 1990s and booming in 2000s with the ubiquity of Internet and improvements in network technologies. Habitat (1985) developed by Lucasfilm Games Group, is another milestone in the video game convergence towards the NVE paradigm. The game has been defined in [MF91] as a ““multi-player online virtual environment”, a cyberspace. Each player uses his or her home computer as a front end, communicating over a commercial packet-switching data network to a centralized back end system. The front end provides the user interface, generating a real-time animated display of what is going on and translating input from the player into requests to the back end. The back end maintains the world model, enforcing the rules and keeping each player’s front end informed about the constantly changing state of the universe”. The back end enables the players to interact not only with the world but with each other. The game was available for the Commodore 64 platform, one the most widespread 8-bit home computer of the 1980s, relying on the online service Quantum Link, the corporate progenitor of America Online (AOL). These factors made Habitat the first attempt at a large-scale commercial virtual community that was graphically based [Rob94]. Attending to the Habitat’s description provided in [MF08] and [MF91], it can be determined that includes most of the defining NVE characteristics. 6Chapter 1 Introduction Neverwinter Nights (1991) was the first MMORPG to display graphics [Bai04]. The gameplay is divided between a screen providing information about the game party members, current status and geographical location within the virtual world and the screen for combat mode where each character is represented by an avatar and battles take place. During its lifespan running on AOL network from 1991 to 1997, the capacity of the Neverwinter Nights servers grew from 50 to up to 500 concurrent users per server. Near the end of its run in 1997, the game had 115 000 subscribed players and an average of 2000 concurrent players during peak hours. Much of the game’s popularity was based on the presence of active and creative player guilds, who staged many special gaming events online for their members. Doom [Sof93] (1993), a science fiction FPS video game by id Software, is considered one of the most significant and influential titles in the video game industry, responsible of ushering the mainstream popularity of the FPS genre. While Maze War was innovative by defining and shaping many video game concepts, Doom helped to bring into wide audiences many video game concepts and features that are current staples in the genre. Among others, it provided immersive 3D graphics, support for customized additions and content via data files known as WADs (ludicrous acronym for Where’s All the Data?) and networking capabilities allowing different online multiplayer modalities on both LAN and the Internet [Kus04]. Internet and the global scale of NVE-based games The Internet explosion during the later half of the 1990s greatly expanded the target audience for all kind of online games during that period and the next decade of the 2000s. The shared virtual worlds expanded from LANs to the Internet, breaking the barrier of locality and allowing interactions over a global and much more heterogeneous data network. Some of the genres that gained great popularity was MMORPG, a combination of classic RPG and multiplayer online games where players interact between them in a virtual world. Titles such as Ultima Online or EverQuest were among the first to popularize the genre, while the massive commercial success arrived in 2004 with World of Warcraft (WoW) by Blizzard Entertainment, having 11 million subscribers at its peak in 2010 [Ent10]. MMORPGs games in general and WoW in particular not only become a powerhouse in the video game industry, but also sparked academic interest. With an audience of millions of players all around the world a revenue in the range of billions of dollars, studying the relationship between network quality of service (QoS) and quality of experience (QoE) inspired studies such as [FRS05], [Che+06b] and [RSR08]. The technical challenges posed by such a number of concurrent players all around the world was inspired studies such as [PKK05], [Kim+05], [Che+05], [Che+06a] and [SKR07], also addressing network 1.1 Historical overview 7 traffic optimisation in [Sal+12]. Another interesting aspect of the relationship of MMORPG games and academia has been their usefulness as tools for other disciplines such as epidemiology, as shown in [Bal07] and [GG15]. The FPS genre based on online multiplayer mechanics also flourished during this period, with sagas such as Counter-Strike (2000-present,Valve Corporation), Halo (2001-present, published by Xbox Game Studios), Quake series (1996-present, id Software), Call of Duty (2003-present, distributed by Activision), PlayerUnknown’s Battlegrounds (2017, PUBG Corporation) or 2016, Overwatch (Blizzard Entertainment) becoming extremely popular and contribution to the raise in popularity of the Esports, a form of competition using video games [HS17]. But while MMORPGs involved challenges about scaling up to millions of concurrent users and synchronisation of the virtual worlds across many servers, online FPS games posed technical challenges related to tight synchronisation constraints. Modelling their network traffic was addressed in studies such as [Ami+13], [CB07] focused on the network traffic modelling, or [Fen+02], which used network modelling to help with the provisioning of network infrastructures to successfully accommodate large number of concurrent and distributed users. QoE-related studies such as [Wat+06], dealt with player’s perceived quality, others such as [ZA04] focused on the impact of network QoS on the user’s QoE. The study performed in [LC17] goes further in problematics tightly associated to the strict synchronisation requirements of FPS games, such as the so-called “shot around the corner”, where synchronisation inconsistencies can lead to the kill of a player even if it is properly covered. During the decade of 2010s, the predominance of online multiplayer titles even pushed popular open world game series, focused on offline single-player dynamics where the player can freely explore a virtual world, to expand their features or release titles supporting online multiplayer mode. Some of these examples are Fallout 76 (2018, Bethesda Game Studios), Grand Theft Auto Online (2013, Rockstar Games) or Red Dead series (2010 and 2018, Rockstar Games). This trend reflects the popularity of the networked virtual world paradigm in the game industry and the economical weight associated to it. Due to the involved game dynamics, the above titles combine the technical challenges of huge open world scenarios and multiple synchronised server of the MMORPG games with the tight timing constrains and real-time synchronisation of FPS titles, posing more questions for the fields of study around NVE network traffic and providing new tools for others ([MH12], [FLK17]). 8Chapter 1 Introduction 1.1.3 Academia and research NVEs have been associated to research from their early inception, sometimes as a canvas where to build and perform experiments, sometimes as subject of study themselves. One of the earliest academic NVE was NPSNET [SZ99], developed in 1986 by the Department of Defense (DoD) of the U.S. as an evolution of the previously mentioned SIMNET, focused on implementing large-scale virtual environments (LSVEs) at LAN network level. The decade of 1990s witnessed the emergence of several more research NVE. Distributed Interactive Virtual Environment (DIVE), released in 1991 by the Swedish Institute of Computer Science, was an Internet-based multi-user virtual reality (VR) system where participants navigate in 3D space where they can interact with other users and applications [Hag96]. Another example of academic NVE was MASSIVE-1 [GB95], developed by the Computer Research Group at the University of Nottingham. It was a distributed multi-user VR system running on high end Sun Microsystem and SGI platforms. It had textual, graphical and audio client programs. Later MASSIVE-2 and MASSIVE-3 improved scalability and moved to a TCP-based client-server model. The initial trend in the development of NVE platforms was to design and build specific-purpose systems with specific research questions in minds. For example in [YK13] the authors provide a comprehensive list of NVEs systems which use different P2P techniques for virtual world synchronisation, many of them just build as a proof of concept or to study a very specific feature instead of trying to provide a complete virtual world experience to end users. The increase in complexity of the NVE systems together with the advances in computing power, 3D graphics and networking experimented during the decade of 1990s, propitiated the emergence of new generic NVE platforms. The openness of these systems, in both license and technical terms, made them attractive frameworks for scientific and researchers that started to use them as a canvas. Specialised NVE frameworks The initial trend in the development of NVE platforms was to design and build specific-purpose systems with specific research questions in minds. For example in [YK13] the authors provide a comprehensive list of NVEs systems which use different P2P techniques for virtual world synchronisation, many of them just build as a proof of concept or to study a very specific feature instead of trying to provide a complete virtual world experience to end users. The increase in complexity of the NVE systems together with the advances in computing power, 3D graphics and networking experimented during the decade of 1990s, propitiated the emergence 1.1 Historical overview 9 of new generic NVE platforms. The openness of these systems, in both license and technical terms, made them attractive frameworks for scientific and researchers that started to use them as a canvas. Open Cobalt is a free and open-source (MIT license) multi-platform software for constructing, accessing, and sharing virtual worlds (virtual workspaces in the terminology of the project) both on local area networks or across the Internet, with no need for centralized servers. Developed at Duke University and initially released in 2007, it is based on Squeak, a dialect of Smalltak, including features such as integrated web-browsing, voice and text chat, and other embedded applications accessible in-game. Open Cobalt aims to create different virtual worlds hyperlinked between them thus constituting a metaverse. Most of the studies performed using Open Cobalt as experimental platform are focused on e-learning and e-training ([FB14], [TL14]). As of 2019, the Open Cobalt project is currently dormant. The OpenSimulator project [Fis09], sometimes referred as just OpenSim, was founded in January 2007 by Darren Guard with the goal of providing an open-source 3D NVE server that could become a standard framework to build NVEs. Initially based on the open-source Second Life client and messaging protocols, it currently provides a multi-platform, multi-user 3D application server. OpenSimulator allows a high degree of flexibility to virtual world developers to customize their worlds thanks to its extensibility. Its core is written in C# under a BSD License, a commercially friendly license to embed OpenSimulator in products. Academic studies based on or using OpenSimulator range from e-learning topics ([Vil+10]), to archaeology ([SM13]), robotics ([JEP10]) or network scalability of NVE systems ([LH13]). Another noteworthy mention of general purpose NVE with strong rooting in academia is Open Wonderland (OWL), a Java open-source toolkit for creating collaborative 3D virtual worlds. OWL will be subject of study in the present work as a study case of NVE system. Further details about OWL can be found in Section 2.2. Commercial video games as academic NVE frameworks The second half of the 2000s saw a new trend where popular video games that qualified as NVE started to be extended and used for academic and research purposes. Using commercial video games as platform to build research NVE systems is not a new trend. Projects such as Colyseus ([BPS06] and [BAS13]), a NVE platform based on P2P synchronisation techniques based on Quake II to study network scalability and load balancing, or the use of BrowserQuest as platform for health-related projects such as [GG15] are noteworthy examples. 10 Chapter 1 Introduction Second Life is an online virtual world, developed and owned by the San Franciscobased firm Linden Lab and launched on June 23, 2003. Reporting up to 1 million of regular users in 2013 and around 800 000 active user accounts in 2017 [Lab13], Second Life enjoyed great popularity by the end of the decade of 2000s and beginning of the 2010s. While sharing many characteristics with the MMORPG genre, their creators have always emphasised that Second Life is not a game, not having a set objective. This virtual world comprised the technology to articulate it, such as client, messaging protocols and so, as well as the server infrastructure. Thus, Second Life only exists as long as their creators keep the service alive. Despite being a service provided by a third-party entity, the social nature and in-game degree of freedom offered by Second Life, together with its popularity among wider audiences, made it a popular research tool in several fields. Second Life proved to be a interesting study case for technical NVE-related topics such as avatar traffic analysis and mobility ([KC08], [Lia+09]), dynamic balancing of virtual worlds ([LB10]) or traffic modelling ([Fer+07]). Health-related applications were also noteworthy as shown by studies on health-related activities ([Bea+09]) or the potential of virtual worlds in medical education ([BHW07]). But where Second Life has been especially popular, like happened with other NVE systems, is in the context of e-learning, with multiple studies dealing with its potential applications in the field, such as [Hay06], [GHT08], [WI09] or [Get+10]. It is noteworthy though the popularisation in recent years, both among the gaming and research communities of the sandbox games. A sandbox game is a “style of game in which minimal character limitations are placed on the gamer, allowing the gamer to roam and change a virtual world at will. In contrast to a progression-style game, a sandbox game emphasizes roaming and allows a gamer to select tasks. Instead of featuring segmented areas or numbered levels, a sandbox game usually occurs in a “world” to which the gamer has full access from start to finish” [Tec]. While the terms sandbox and open world are sometimes used interchangeably, they refer to different concepts and they are not synonymous. It can be argued that the first example of sandbox-like game was the text adventure Colossal Cave Adventure [RB06] (1976) which allowed free-roaming exploration. In the 2000s several titles such as Dwarf Fortress [Rab15] (2006, by Tarn Adams) or Infiniminer [NSR16] (2009, by Zachtronics LLC) further popularized game dynamics built around the sandbox concept and also influencing Minecraft (2009, by Mojang) a landmark in the massive popularization of the genre. Minecraft does not only offer a sandbox experience, but incorporates multiplayer features which fully qualify it as a potential NVE system. Minecraft provided freedom to explore and the ability for the player to modify at will vast procedurally generated 1.1 Historical overview 11 as performance evaluation, network infrastructure sizing or quality of experience (QoE) derived from network quality of service (QoS). 2.1 Networked virtual environment As stated in Chapter 1, a NVE can be broadly defined as a software system providing a simulation shared over a network where users located in different geographical places can interact among them in real-time. The existing literature dealing with NVEs provides a sparse set of characteristics that are common to this kind of systems [Tre03]. These characteristics can be broken down into three main categories. First, the NVE software, which implements the logic of the NVE system, it defines, hosts, updates and synchronises the simulation shared among the users. Second, the NVE hardware, which constitutes the physical layer, runs the NVE software and provides the input and output devices that allow users to interact in real-time with the shared simulation and also the network hardware required to access the communication channels. And third, the data network, which provides a communication channel that interconnects the hardware devices where the NVE software is run. These categories are described in further detail below. 2.1.1 NVE software The NVE software orchestrates the whole simulation share among the NVE users. It keeps a representation of this simulation, its entities, properties and states. It also updates the virtual world with the input provided by users while trying to keep a consistent and synchronised view of the virtual world for all the users participating in it. Most modern NVE software are applications that run on top of a complete software stack comprising an operating system, hardware drivers and network protocol stacks. The software architecture for NVE software can greatly vary, using different techniques to host the virtual world, provide persistence and synchronize and propagate state changes of its entities. Regarding the software structure, most of the software for NVEs can be classified as client-server [SZ99] or peer-to-peer [BAS13], [YK13]. The first rely on a central machine running the server software which hosts the virtual world, gather the updates and synchronises the views of all the NVE clients. The peer-to-peer ones host the shared simulation in a collective and non-centralised way, every peer acting as client and also server and using different synchronisation approaches to guarantee coherence. 18 Chapter 2 Networked virtual environments: characterisation and case study Below is an enumeration of the defining elements and characteristics that can be found in NVE software. Virtual World: is a computer simulation shared by users over a data network. This simulation is the representation of a reality following a set of rules that define the possible interactions of the entities contained within the simulation and how their states can evolve regarding these interactions. The NVE software implements the logic that defines the virtual world, from basic physics rules that govern the virtual world to the mechanics that shape the interaction, goals and rewards that users can experience within the virtual world. Avatar: is a graphical representation used by a person to identify himself or herself in the context of a communication network or virtual community, providing the sense of presence [Çap99]. In the context of the NVEs, the avatar represents the user inside the virtual world. This entity will be controlled by the user and it will be its proxy between him/her and the rest of the entities within the virtual world. In the avatar model, the user interacts with the game through a single representative character and the player actions are defined in terms of commanding the character. The avatar exists at a particular location in the virtual world and can influence only the immediate locality. Software built around this avatar interaction model typically have either a first person perspective, where the player looks through the eyes of the avatar, or a third person perspective where the player follows an avatar in the virtual world. FPS games, RPG, action games, sports games and racing games are all examples of game genres that have an Avatar interaction model. Real-time interaction between the users: but also between users and virtual world entities. An action on the user side involves an immediate interaction and result within the simulation that can be perceived by the user him/herself and the rest of participants. We must differentiate the concept of "real-time" depending on the context. From the user perspective, real-time refers to the reaction to events at the same time as they unfold, thus a change in the virtual world as consequence of the user input . From a computing perspective, real-time refers to performing actions within the system matching certain operational deadlines. Events, i.e. user input, have to be processed and the virtual world updated within defined time boundaries by the NVE system to minimise latency between user actions and system reaction, providing final users with the sensation of instant responsiveness [RG14], [Fin13], [Bru+13]. Communication channels: the simulation is shared among users over a data network where users not only can see each other’s representation (or avatar) within the game, but they can also communicate beyond visual ways. The most common com- 2.1 Networked virtual environment 19 munication channels are written text in the form of chat rooms or instant messaging, and audio in the form of public conversations or private voice calls. 2.1.2 NVE hardware The NVE user will require a computing system to access the virtual world, a device with a certain degree of computing capabilities to be able to run the required software to join the NVE virtual word, interact with it and receive a representation of it. This device acts as client of the NVE system and provides the link between the human user and the rest of the NVE. The most common option is the usage of microprocessorbased architectures equipped with RAM memory and storage memory for persistence. Moreover, due to graphical requirements it may be necessary to include dedicated hardware such as graphics processing units (GPUs). Personal computers have been the traditional client devices for NVE systems. Both desktop workstation and laptop actor provide all the hardware requirements to run the NVE client software, being usually equipped with input devices such as keyboard and mouse, as well as connectivity to different network technologies. Later years have witnessed the emergence of handheld devices such as tablets and smartphones that can be also suitable NVE clients due to their computing capabilities or as input accessories, bringing new input methods into scene such as touch screens, a range of built-in sensors such as accelerometers and gyroscopes [Geb+14]. Below is an enumeration of the defining elements that can be found in Control and input devices: the computing system has to count with input devices to allow the user to interact with the virtual world and control his/her avatar within it. These devices will translate the user actions into virtual world events. The most common user interactions involve avatar control and communication between NVE users using text or voice. Avatar control can be performed using keyboards, gamepads or pointing devices such as mouses or trackballs. Keyboards also allow to perform text input while microphones together with audio hardware allow audio input. Touchscreens are the norm in modern tablets and smartphones. Dedicated keyboards and pointing devices are usually replaced in these devices with controls directly displayed and operated from the tactile screen. Moreover, the new batch of handheld devices open new input possibilities thanks to the incorporation of built-in sensors. Moreover, later years have witnessed an irruption of the Internet of things (IoT) [WCJ14] or virtual reality (VR) technology [Gun+17], [He+17] to broader audiences, the later aimed to take immersive virtual world experiences to a new level. 20 Chapter 2 Networked virtual environments: characterisation and case study Below is an enumeration of the defining elements and characteristics that can be found in the hardware supporting a NVE system. Output devices: every user has a representation within the virtual world hosted by the NVE system. This virtual world stores all the entities within the simulation and all the details about location, properties and state [XS16], [Dai+09]. However, users are usually provided with just a partial representation of the virtual one, capturing the simulation area that can be encompassed from his/her location within the virtual world and following a set of physical rules of the simulation, like the field of vision. The most common trend is providing the user with a 3D render image of the virtual world from his/her perspective, this may require the presence of dedicated hardware such as GPUs. Other possible virtual world representations such as 2D or even ASCII based can be found on niche NVE, responding a technical limitations (underpowered graphic hardware by the time of their implementation) or merely aesthetic considerations (like retro game revival trends). Audio is provided as an extra to enrich the immersive experience within the virtual world, a computing device will require specialized audio hardware to decode and play sounds part of the virtual world environment, as well as peripherals such as speakers or headphones. Data network: interconnects the hardware components of the NVE system, allowing the NVE software to share the virtual world among users. There are numerous physical channels that can be used to implement modern data networks, from wired technology such as Ethernet, to wireless networks based on WiMAX, Wi-Fi or G3/G4/G5 technology. The geographical extension of the network can also range from Local Area Networks confined to a room to world-wide ones, being the Internet the maximum exponent. On top of these technologies there is a network protocol stack where TCP/IP is the current de-facto standard. The network communications of a NVE system can be structured following a classic client-server architecture or peer-to-peer (P2P) approach [HL04] [SJT06] [BAS13]. 2.2 Open Wonderland Open Wonderland is a Java open-source toolkit for creating collaborative 3D virtual worlds, originally conceived as a tool for collaborative working by Sun employees [Yan+04]. It has been chosen in the context of the present work as a representative example of NVE and it will be used in later chapters as a case study to test the proposed analysis and modelling procedures defined for NVE network traffic. According to the characterisation of NVE proposed in Section 2.1, OWL meets all the requirements to be considered as a representative of the NVE systems: it is based on 2.2 Open Wonderland 21 software components running on computer hardware and provides a virtual world shared over a data network where users are represented by avatars that can engage in real-time interaction between them and also with other elements of the virtual world. In addition of being an example of NVE software, OWL is also an interesting case study due to its expandability and suitability as a tool in research fields related to the motivations of the present work, enumerated in Section 1.2. For example, OWL has been used as foundation to build persuasive systems, focused on motivating healthy lifestyle habits [Fog02], such as Virtual Valley [Rom+10], [Cas+10]. OWL has been also used for e-learning studies such as [Gar+11]. Other studies such as [FSC12], [Fon+12a] or [FCS13] have used OWL as a case study for performance evaluation of NVEs, focusing on network and server resources respectively. Thus, choosing OWL as case study for the present work is twofold: on the one hand due to its NVE condition it will help to prove the validity of the analysis and modelling process that will be proposed in the present work. On the other hand, if meaningful network traffic models are generated as result of this analysis process, they can prove useful in any of the fields that motivate the study and where OWL is a relevant tool, further confirming the interest in the study of NVE network traffic. OWL has several other technical characteristics that make it especially suitable for its study and the design of experiments based on it. These characteristics mostly derive from its open source nature and design choices, making it an open platform for new developments [KY11], allowing the publication and distribution of derived works and giving organisations and individuals the possibility to deploy the system on their own infrastructures without having to rely on third-party services, which is especially crucial in applications that may involve highly confidential data, such as medical records [Gar+09]. 2.2.1 Architecture and internal components The OWL version used in the present work is Project Wonderland 0.5, its architecture is shown in Figure 2.1 [Proc]. OWL is subdivided in several components and thirdparty projects that work together to deliver a NVE experience. These elements are enumerated and detailed below. Wonderland: comprises both the core of OWL client and server as well as a set of modules that provide key functionalities such as security, shared applications, avatars and so on. It also contains the web administration server. Specifically, the shared application feature allows sharing applications among different users. Some of these 22 Chapter 2 Networked virtual environments: characterisation and case study Audio Client Obj.Sync. Client Audio Client Obj.Sync. Client Audio Client Obj.Sync. Client IPv4 Stack Audio [OWLAudioClient] Obj.Sync. [OWLMovClient] Node Socket CSMA Channel Node Socket IPv4 Stack CSMA Device Audio [OWLAudioServer] Obj.Sync. [OWLMovServer] CSMA Device Applications [ns3::application] Clients [ns3::node] Server [ns3::node] Socket Socket Fig. 2.1.: Open Wonderland client-server architecture applications are already integrated in Wonderland, like the multi-user PDF viewer and the SVG White board. Users can also share additional external applications installed in the server (like Firefox or OpenOffice) using the Shared Applications Server. Wonderland Modules: a repository for OWL extension modules which expand its functionalities. Some of these modules are shipped by default within the OWL binary releases. There are also experimental modules provides by both project developers and community members. MTGame Graphics Engine: a high-performance graphics engine that extends jMonkeyEngine. MTGame adds multi-threading capabilities for improved graphics performance [Foud]. jVoiceBridge: a pure-java audio mixing platform providing real-time immersive audio (via VoIP) with time distance attenuation and a selectable range of qualities as well as a companion software phone, called softphone’ that allows phone calls between users within the virtual world. It supports mixing high-fidelity, stereo audio at up to CD quality [Fouc]. Moreover, OWL includes several open source projects as internal components, such as graphic or audio engines, applications servers and so on. Some of these third-party components are listed below. Darkstar: a Java platform created by Sun Microsystems for scalable communications and persistence in games. OWL includes a Darkstar service that manages all 2.2 Open Wonderland 23 client and world state. Nowadays the project development is officially halted but a community fork has been created, called RedDwark Server [Proe]. Glassfish: is highly scalable, open source pure-Java application server that provides several functionalities for the Java EE platform such as RMI, XML and web server. OWL is based on an embedded instance of the Glassfish server. Wonderland web applications include web-based management of the server and worlds, a content repository for hosting all world data, and an integrated single-sign on system used to maintain identity across Wonderland services [Cor]. jMonkeyEngine: a 3D game engine written entirely in Java. It provides core graphics APIs, including graphics primitive and shader support. The OWL graphics system is based on these core APIs, with some extensions from MTGame to support multithreading [Teab]. 2.2.2 In-game elements and interactions Within the OWL virtual world, the user is graphically represented by a 3D object known as avatar, the virtual projection of the user perceived by other players within the same gaming session. Its movements and behaviour are determined by the user behind it, using his/her keyboard, mouse or any other computer input peripherals. The virtual world can contain all kind of objects, which can be 3D objects like pieces of furniture, buildings, etc. or 2D objects like screens with embedded applications (web browsers, word processors, and so on). The usual way to model the spatial relationships between objects is using a scene graph. Each object is a node (or cell in OWL terminology) in this graph. The cells, representing any volume of space of the virtual world, are organized in a graph with a tree hierarchy [Prod]. Figure 2.2 shows an example of the cell tree. The in-game interactions within OWL can be divided into three categories: avatar movement, audio communication and interaction with embedded apps. Avatar movement and object interaction: these interactions are related to the location of the avatar within the game scenario and the other avatars and objects in it. The result of these interactions are the update and synchronization of the shared simulation between all the game users and their graphic representation. 24 Chapter 2 Networked virtual environments: characterisation and case study Cell 1 House 1 Road House 2 Room 1 Lamp Room 1 Sofa Car 1 Car 2 Room 2 Chair Table Fig. 2.2.: Structure of a Open Wonderland cell Audio communication: involves the transmission of audio between users in the form of private conversations or audio being broadcast to all the users within the listening range. Embedded applications: these are applications within the game, such as browsers, digital boards or PDF readers. These kind of interactions are very heterogeneous since they are strongly linked to the nature of the application. Their study and modelling are out of the scope of the present work. Each one of the above type of interactions generate a characteristic and distinctive type of network traffic. Each type of interaction has optimized network flows designed to better suit their requirements: reliability, data integrity, real-time and so on. These network flows are further discussed in section 2.2.3. 2.2.3 Networking OWL is based on a client-server architecture [Fouf]. The OWL server must have an IP address known and reachable by the clients. Initially, a client is disconnected from the server until it calls a WonderlandSession.login(). If this succeeds, the session goes into the CONNECTED state. A client may connect multiple sessions to the same server. Once a session is connected to a server, connections may be added to it. Each connection in Wonderland has a unique type for sending different types of data. For example, a client may use one connection for sending cell data, and another one for sending voice communications data. Clients may use as many connections as they need for their interaction with the server. The only limitation is that a client may 2.2 Open Wonderland 25 only have a single connection of a given type connected to a given session. Clients may also use multiple sessions to get multiple copies of a single connection type. Once in the CONNECTED state, messages can be passed from a client to the server, from the server to one client (not necessarily in response to a client message or request) and also the same message can be sent by the server to a number of clients. From version 0.5 on, there is no more client-client communication, that is, all messages from clients go directly to the server. The traffic between clients and server/s can be broadly divided into object synchronisation,audio, and application data [She+03]. The object synchronisation traffic uses TCP protocol to ensure the virtual world updates are received by all the clients engaged in the same session, this type of traffic will be subject of further analysis in Chapter 4. The audio traffic uses UDP for the propagation of voice and audio content. While UDP does not guarantee the delivery of the packets, it is acceptable a certain degree of packet loss without compromising and spoiling the user experience. The studio of the audio traffic has been left out of the scope of the present work. Finally, the application data traffic is highly dependant of the nature of the embedded applications that generate it and it has also been left out of the scope of the present work. 2.3 Experimental testbeds based on Open Wonderland Using OWL as case study for NVE network traffic modelling requires to have available experimental network traces generated and captured under controlled circumstances. To this end, two testbeds based on OWL have been defined, the first one, labeled as Open Wonderland TestBed 1 (OWTB1), follows one of the classic approaches on literature where human players volunteer to participate in gaming sessions which network traffic is captured an later analysed.The second testbed, labeled as Open Wonderland TestBed 2 (OWTB2), is an attempt to be more scalable by using basic bots to control the avatar, replacing human operators and the constrains associated to the availability of volunteers. Nonetheless, both testbeds are client-server installations where clients connect to a standalone OWL server. Both testbeds are described in further detail below in the next sections. 2.3.1 Open Wonderland Testbed 1 Open Wonderland Testbed 1 (hereafter OWTB1) was the first testbed to be deployed in the laboratory of the Department of Computer Technology and Architecture of the University of Seville. This testbed was composed by heterogeneous hardware and 26 Chapter 2 Networked virtual environments: characterisation and case study operated by a group of volunteers. This testbed was aimed to capture initial OWL network traffic to help in the first evaluations and the initial definition of the analysis and modelling processes. Therefore, this is a heterogeneous platform in both hard and soft terms. The structure and topology of OWTB1 is shown in Figure 2.3. Open Wonderland Testbed 1 router client client server client client client User User User User User Fig. 2.3.: Structure of Open Wonderlad Testbed 1 Hardware: OWTB1 was implemented using a set of PC machines from different generations and specifications. The first testing session followed the bring your own computer (BYOC) principle [RHS10], where all the volunteers brought their own machines and connected to the Ethernet infrastructure and OWL server provided for the experiment. This situation does not differ greatly from any other regular OWL deployments observed in the wild which deal with different hardware settings. Despite the differences in computing and graphic resources, all the machines met the minimum requirements to run the OWL client software [FSC12]. As for the server hardware, it had enough resources to not constraint the performance of the system under the planned workloads. Table 2.1 shows a summary of the hardware settings of the OWTB1 computers. Network: the OWTB1 nodes were interconnected using Ethernet technology. The network configuration consisted in a single Ethernet 10/100Mbps switch, provided by a Linksys WRT54G v7.0 with stock firmware. The switch interconnected the server and clients in a single collision domain. The usage of wired technology and ISO Layer 2 device aimed to provide an isolated network scenario that minimised 2.3 Experimental testbeds based on Open Wonderland 27 world environment to each client, not being significant for the stationary scenario of the traffic. Due to the simplicity of the in-game interactions it was not required a significant warm-up period to reach stationary state. A 10 min trace capture is representative enough of the network traffic dynamics to be studied [Mac87]. 2.4 Summary The present chapter provides a characterisation of NVE systems, a tool to identify software systems that can be categorised as NVE. The immediate application of this characterisation is to identify NVE systems in areas such as video games, and therefore being able to reuse and reinterpret the existing literature focused on them from the perspective of NVE. These allows to identify a lot of material to synthesize the analysis and modelling processes for NVE network traffic proposed in the present work. The characterisation of NVE systems describe them as software providing a simulation, or virtual world, shared over a data network that is powered by computing devices and peripherals. The representation of the users engaged in the virtual world experience is done by the usage of an avatar. These avatars interact with each other and with other elements within the virtual world in real-time basis. OWL complies with the NVE characterisation proposed in this chapter. This, together with its use as research platform in fields related to the motivation of the present work, make OWL a good candidate to test and validate the analysis and modelling procedures for NVE platforms later developed in this work. The open source nature of OWL and its architecture and technical characteristics make it also a good alternative to design experiments and generate network traffic to be used for modelling purposes. Having chosen OWL as a case study for NVE network traffic analysis and modelling, the chapter also describes the testbeds built around it, OWTB1 and OWTB2. While they have some differences, such as hardware setup or the way avatars are controlled in each one, they still share the same client-server topology over an Ethernet network, network traffic capture procedure and the same guidelines to conduct experiments, where certain types of user interactions are especially favoured due to its interest from the modelling point of view. 34 Chapter 2 Networked virtual environments: characterisation and case study 3 Analysis and modelling pipeline for NVE network traffic But magicians have calculated that million-to-one chances crop up nine times out of ten Terry Pratchett, Mort This chapter provides an overview on previous work analysing and modelling network traffic generated by different types of video games, using them to extrapolate and propose a statistical analysis pipeline specifically devised to analyse and model network traffic generated by networked virtual environment (NVE)s. The reason to rely on video game-related literature is its abundance and the fact that many video games overlap with the NVE definition provided in Section 2.1. This chapter is an effort to identify and filter the statistical devices that can help to shape the statistical analysis of the network traffic generated by the NVE systems defined the present work. For convenience reasons, when the term video game is referred it is implicit that refers to those which share characteristics with the NVEs. An analysis pipeline consists in a number of interconnected processing steps that receive a certain data input and perform an analysis to generate some sort of output. The pipeline proposed in this chapter is a collection of steps and statistical tools that can be automated partially automated using a programming language (i.e. R [Fouh] or Python [Foug]), and workflow technologies [Sha+14]. The modelling strategy implemented through this pipeline is the so called microscale modelling [GS10], which models the network traffic based on two random variables, inter-arrival time (IAT) and packet size (PS). The aim of the proposed statistical analysis pipeline is to describe the steps researchers can follow to determine a model for each one of the random variables (IAT and PS) that describe the traffic at microscale level. Some of the steps will require the researcher to make decisions, such as the most promising theoretical distribution to fit into the experimental network traffic. The pipeline also include steps to determine how optimal those decisions have been, i.e. evaluation of Goodness of Fit, allowing the researcher to reconsider and iterate again over the process. 35 3.1 Background Network traffic models have been traditionally developed under the assumption that packets or flows arrive according to a memoryless Poisson process [Kle75]. This implies that observed network traffic exhibits only short-term autocorrelations where the autocorrelation function of the observations becomes negligible after a certain finite lag. Later, self-similar models of network traffic became widespread in literature [Lel+93] [FM94], being successfully applied to different real-world traffic traces, such as variable bitrate (VBR) video streams [Ber+95] and World- Wide-Web traffic [CB97]. These models could capture the long-term autocorrelations measured in local area network (LAN) and backbone traffic, however there is no concise relation that describes how self-similar traffic affects queueing at switches or end-user performance.[GB96]. There have been other approaches for the video game network modelling, such as the study in the frequency domain [Fen+05], or using time series such as Markov chains or autoregressive-moving-average (ARMA) models [CB07] [CBA07]. The present work follows the microscale approach for network traffic modelling, consisting in examining the inter-arrival dynamics between individual packets withing a gaming session. Microscale models have been used previously to model the packet inter-arrival periods of telnet [Pax94] [PF95] and FTP [Dan+01] traffic. Some of the earliest applications of the microscale modelling to the video game network traffic can be found in [BDA99] and [BD00], but it is the paper from Michael S. Borella, “Source Models of Network Game Traffic” [Bor00], the work that defined the basic guidelines for the microscale network modelling as well as the measurement of discrepancy of these models against the empirical network traffic. This work laid the groundwork for many later papers on this subject, are numerous the literature applying these guidelines to video games from first-person shooter (FPS) genre, such as Half-Life [HB01] [Lan+03], Quake3 [LHR02] [LBA04], Counter-Strike [Fär02] [Fär04], Halo2 [ZA05], Unreal Tournment 2004 [Hüb08]. Network traffic generated by massively multiplayer online role-playing game (MMORPG)s has been also covered in works such as ShenZhou Online [Che+06a]. The real-time strategy (RTS) genre has been also subject of study from the microscale network modelling with representatives such as StarCraft [DPV05]. 3.2 Analysis pipeline for microscale modelling The microscale modelling for video game network traffic popularised in [Bor00] considers traffic as a flow of packets of different sizes, each packet separated from the previous and next by certain amount of time. Thus, the magnitudes that focus 36 Chapter 3 Analysis and modelling pipeline for NVE network traffic attention for the microscale modelling are IAT /inter-departure time (IDT), period of time separating consecutive incoming / outgoing network packets and PS, the number of bits composing the network packet. These magnitudes can be considered as random variables which behaviour can be described using stochastic models. The basic steps for the analysis pipeline for microscale modelling of the random variables IAT and PS, are enumerated below. Determining the model for each variable requires to execute the pipeline for data input specifically filtered and adequated to model such variable. Each one of these steps are described in further detail in the following sections. Figure 3.1 describes a flow chart for the pipeline. 1. Data filtering: network packets useful to study IAT or PS will be identified and isolated during these step. 2. Data preview: initial graphical representations and analysis of the raw data will help to determine how later steps will be performed. Statistical elements such as the plot of the Empirical Cumulative Distribution can help with this step. 3. Autocorrelation: determining the presence of self-similarity (or lack of) allows to determine the right modelling approach for the raw data. Proving the randomness of the studied parameter allows to go on with the microscale analysis pipeline, otherwise, other approaches such as time-series may be more appropriated for traffic of this nature. 4. Data Modelling: one or more statistical models will be evaluated to determine the most promising to describe the studied network parameter. This step will be conditioned by the discrete or continuous nature of the values observed during the preview step. 5. Goodness of Fit: the fitness of the models proposed in the previous step will be determined against several goodness of fit (GoF) techniques such as Q-Q plots or numeric discrepancy metrics. 3.3 Data Filtering The Data Filtering step aims to extract the relevant network traffic from the captured traces for later statistical modelling. In the case of IAT traffic, the filtering will select the arrival times of the packet traces in the capture which will allow to calculate the time between consecutive arrivals of similar packets. The filtering for the study of PS only require to focus on the packet size of the payload exchanged through the NVE network traffic. 3.3 Data Filtering 37 Data filtering outliers distribution tail Data preview ECDF plots Autocorrelation ACF random? time-series modelling Data modelling (distribution fitting) MLE binomial exponential normal Weibull ... distribution type? discrete splitcontinuous Proposed model(s) Statistical discrepancy Q-Q λ² Pearson's r ... Network traffic Fig. 3.1.: Workflow for the NVE analysis and modelling pipeline 38 Chapter 3 Analysis and modelling pipeline for NVE network traffic There are both technical and statistical considerations to take into account during this step. Technical considerations are related to the nature of the traffic itself, allowing to determine which data flows are of interest for the later stages of the statistical analysis. The following list of considerations have been used to determine how to filter experimental in the context of the present world: • network traffic source: client-generated traffic Vs server-generated one • type of transport layer protocol: TCP vs UDP • nature of the packet payload regarding the game stage: scenario load, user login, regular gameplay, etc. • nature of the packet payload regarding the event it codifies: avatar sync, audio, app-related traffic, etc. • evaluation of complete packet size Vs trace payload • details related to the traffic captured tools used, i.e. lack of frame check sequence (FCS) field in traces captured by traffic analyser Wireshark. The statistical considerations are mostly related to the statistical concept of outlier, that can be defined as an observation point that is distant from other observations. Outliers may be due to variability in the measurement or they may indicate experimental error. In case of the latter, the outlier can be removed from the data set [DGK75]. Moreover, some of the statistical tests may be overly sensitive to outliers (i.e. λ2 [Bor00]). Another statistical consideration is related to the distribution tail and its impact on later steps of the analysis process [BD00] [Fär04] [BA06]. Later statistical analysis of experimental network traffic will include the criteria followed regarding outliers and their handling. By the end of this step the raw experimental data will have been transformed in a data set containing the values that will allow to model the desired micromodelling parameters (IAT or PS) in the following steps of the pipeline. 3.4 Data preview and preliminary analysis The Data preview and preliminary analysis step aims to provide a initial graphical preview of the experimental data used to model the random variable. Other statistical tools, such as applying basic distribution fitting can also be performed during this step to better identify potential models. This preview will help researchers to decide how to conduct later steps of the analysis pipeline. The cumulative distribution function (CDF) [SW09] describes the probability that a random variable X with a given probability distribution will be found at a value 3.4 Data preview and preliminary analysis 39 less than or equal to x . For continuous distributions, it provides the area under the probability density function from −∞ to x. The empirical cumulative distribution function (ECDF) [Vaa00] is the cumulative distribution function associated with the empirical measure of the sample. This CDF is a step function that jumps up by 1/n at each of the n data points. The empirical distribution function estimates the true underlying CDF of the points in the sample and converges with probability 1 according to the Glivenko–Cantelli theorem [Tuc59]. While it is possible to base the measurement of discrepancy on the ECDF [Ste74], in this step it will be only used to provide hints whether a given data set can be described by a theoretical distribution based on the comparison between the data ECDF and the distribution CDF. While simple, this approach may prove to not suffice in situations where it is necessary to determine which distribution better describes the given data set. By the end of this step, a graphical preview of the data set will be available, acting as input for later steps in the pipeline, specifically the Selection of a probability distribution. 3.5 Correlation and Autocorrelation The Correlation and Autocorrelation step aims to evaluate the randomness of the data set thus determine if it is suitable for later microscale modelling based on probability distributions. The term correlation is used in statistics and probability theory to describe the dependence between the values of two random variables [Haz95]. It is used to include a standardising factor ranging between −1 and 1 . The formula for correlation is defined in Equation (3.1). cor(X, Y ) = cov(X, Y ) σXσY (3.1) cov(X, Y ) = N X i=1 (xi−¯x)(yi−¯y) N(3.2) 40 Chapter 3 Analysis and modelling pipeline for NVE network traffic The autocorrelation concept is present both in the fields of signal analysis and statistics. For the fist one, it is the cross-correlation of a signal with itself, describing the similarity between observations separated between them by a certain amount of time. In statistics, the autocorrelation of a random process describes the correlation between values of the process at different times, as a function of the two times or of the time difference. Let X be a repeatable process and i some point in time after the start of that process. i may be an integer if X is a discrete-time process or a real number for a continuous-time process. Xi is the value for a given run of the process at time i . It is assumed that the process X has defined values for mean µi and variance σ2 i for all the times i . For this scenario, the autocorrelation between times s and t is defined in Equation (3.3), where Eis the expected value operator. R(s, t) = E[(Xt−µt)(Xs−µs)] σtσs (3.3) If the function R is well-defined (there are no variance values equal to zero nor infinite), the autocorrelation value is in the range [−1,1] . A value of 1 indicates a perfect correlation, the series matches perfectly with itself after being shifted a determinate amount of time. On the other hand, an autocorrelation value of -1 means perfect anti-correlation or inverse correlation. If Xt is a wide-sense stationary process then the mean µ and the variance σ2 are time-independent, and further the autocovariance function depends only on the lag between t1t2 : the autocovariance depends only on the time-distance between the pair of values but not on their position in time. This further implies that the autocovariance and auto-correlation can be expressed as a function of the time-lag, and that this would be an even function of the lag τ=t2−t1 . This gives the more familiar forms for the autocorrelation function (ACF) shown in Equation 3.4. RXX(τ) = E[XtXt+τ](3.4) The presence of autocorrelation in the studied network traffic can help to direct the modelling approach and the type of model to be used. While models based on time-series [CBA07] [CB07] rely on the presence of autocorrelation to determine if a Markov chain or ARMA model is suitable to describe the nature of the network traffic, works following a microscale approach such as [Bra06] or [DPV05] start measuring autocorrelation in the studied network parameters to determine the 3.5 Correlation and Autocorrelation 41 random nature of the traffic and the suitability of a statistical model derived from maximum likelihood estimation (MLE). According to [Bra06], “an indication of whether or not a traffic exhibits Markov characteristics is the rate at which correlation between two samples decreases as the distance between the samples increases [Pax93]. For a Markov process, the correlation between two samples decreases exponentially as the distance (or lag) time between the samples increases. For a Process showing longer range dependence the correlation decreases more slowly.” Thus, if the autocorrelation function decays more quickly than the exponential function then that is some evidence of randomness. If the autocorrelation function is approximately exponential, then that is evidence that the autocorrelation behaviour of the studied parameter can be captured with a Markov model. If the autocorrelation function decays much more slowly than the exponential function then that is evidence of a more complex, long rang dependence. A valuable tool in the study of autocorrelation is the autocorrelation plot, also known as correlogram, a graphical representation of statistical correlation. In time series analysis, this plot shows the sample correlation rh versus the time lag h . If crosscorrelation is used, the result is called a cross-correlogram. The correlogram is a commonly used tool for checking randomness in a data set. This randomness is ascertained by computing autocorrelations for data values at varying time lags. If random, such autocorrelations should be near zero for any and all time-lag separations. If non-random, then one or more of the autocorrelations will be significantly non-zero. In the context of the present work, autocorrelations should be near-zero for randomness; if the analyst does not check for randomness, then the validity of many of the statistical conclusions becomes suspect. The correlogram is an excellent way of checking for such randomness. The example in Figure 3.2 shows the autocorrelogram for a series resulting from adding a periodic series with an uniform random sequence. Must be noted that the first vertical line in each autocorrelation plot correspond to lag = 0 , in other words, the autocorrelation of the time-series with itself, which is always 1. In the case of a data set that shows strong signs of autocorrelation, it would be advisable to explore a different modelling approach other than the microscale one. Some examples of such analysis based on time-series can be found in [CB07] and [CBA07]. Time-series modelling is out of the scope of the present work and it is not covered by the proposed analysis pipeline. On the other hand, if there is enough evidence about the randomness of the data set it will be possible to move to the next step of the pipeline. 42 Chapter 3 Analysis and modelling pipeline for NVE network traffic ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0 20 40 60 80 −1.0 −0.5 0.0 0.5 1.0 Index x ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0 20 40 60 80 −1.0 −0.5 0.0 0.5 1.0 Index noise ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0 20 40 60 80 −2 −1 0 1 2 Index data 0 5 10 15 −0.5 0.0 0.5 1.0 Lag ACF Series data Fig. 3.2.: ACF for uniform random sequence plus periodic series 3.5 Correlation and Autocorrelation 43 ● ● ● ● ● ● ● ● ● −1.0 −0.5 0.0 0.5 1.0 −1.5 −1.0 −0.5 0.0 0.5 1.0 Q−Q plot Theoretical quantiles Empirical quantiles Fig. 3.3.: Q-Q plot, normal sample vs. normal quantiles observations in the i−th bin and Ti the number of observations expected for that bin according to the given theoretical distribution. χ2= N X i (Ei−Ti)2 Ti (3.12) The original formula for λ2 proposed by Pederson and Johnson in [PJ90] is shown in (3.13) , where n is the number of observations in the data set and df is the number of degrees of freedom of the test. The formula for parameter K is shown in Equation (3.14) , where N is the number of bins, Ei is the number of observations in the i−th bin and Tithe expected number of observations for that same bin. λ2=χ2−K−df n−1(3.13) K= N X i Ei−Ti Ti (3.14) 50 Chapter 3 Analysis and modelling pipeline for NVE network traffic Previous works addressing λ2 state that the measurement is not reliable when the number of bins is too big. Trying to fit very heavy-tailed distributions generate very large λ2 values. In this case there are two possible approaches: [Pax93] proposes to log-transform the data to reduce the tail and number of bins. On the other hand, [Bor00] opts for truncating the upper tail when it prevents a good fit. Negative or close to 0 λ2 values indicate a high degree of fitting between the data sample and the fitting distribution. On the other hand, λ2 values of several orders of magnitude are a sign of lack of fitting. Even so, λ2 cannot be considered an absolute fitting metric, but a tool to compare the fitting of samples from a common population. Thus, a λ2 value in the order of thousands indicates a worse fitting that a value close to 0, but there is not necessarily a direct relationship between the absolute value of λ2and the goodness of the fit. The λ2 calculation process may encounter divide-by-zero situations when the expected number of observations Ti is 0 for the given theoretical distribution. To avoid such a situation, Borella [Bor00] proposed an alternative definition, ˆ λ2 which inherits the advantages of λ2 while avoiding divide-by-zero situations. To solve the opposite situation where the empirical observations Ei are zero for a given bin, this bin is just ignored and no computed. The formula for ˆ λ2 metric in Equation (3.15) relies on ˆχ2 (Equation (3.16) ) and ˆ K (Equation (3.17)), reformulations of χ2and Krespectively. ˆ λ2=ˆχ2−ˆ K−df n−1(3.15) ˆχ2= N X i=1 (Ti−Ei)2 Ei (3.16) ˆ K= N X i=1 Ti−Ei Ei (3.17) Both χ2 and λ2 as well as their variants ˆχ2 and ˆ λ2 require a binning method for continuous distributions. [PJ90] includes a discussion about some of the available methods. This work uses the binning technique introduced in [Sco79] and also employed [Pax93] and [Bor00], based on choosing an optimal number of fixed-size 3.8 Statistical Discrepancy 51 bins of width w given by Equation (3.18) where σ is the empirical standard deviation and nis the number of observations in the data set. w= 3.49σn−1/3(3.18) Fixed-width bins have been chosen due to the latter case, a long-tailed distribution will often have its entire tail contained into a single bin, which would provide a too optimistic estimation about the GoF between the two compared distributions. It is important to understand the impact of bin size on the GoF discrepancy metrics. χ2 is very sensitive to bin choice while λ2 is relatively insensitive, only fluctuation in a factor of 2 or 3 if the bin size is close to the optimal value w. ˆ λ2 has been chosen as one of the quantitative GoF metrics in the present study regarding its capacity to quantitatively compare the GoF of two different models for a given data sample or population. Appendix Section C includes the implementation for λ2 and ˆ λ2 in R language used in the data analysis performed for the present study. 3.9 Summary The present chapter includes an overview of statistical methods used in the literature for network traffic modelling of video games, choosing those compatibles with the study of NVEs. The chapter deals with: • Overview of the literature focused on modelling of network traffic for video games that share characteristics with the definition of NVE, those sharing a simulation over a network environment such as FPS, MMORPG or certain RTS titles. The approach chosen for the modelling of NVE network traffic has been the microscale one, popular in the literature dealing with the previous genres, which is based on modelling the IAT and PS of the traffic generated by the video game / NVE. • Definition of the basic workflow for an analysis pipeline for statistical analysis following the microscale modelling approach. Such a pipeline can be automate up to certain degree, but some of the steps require the evaluation of the researcher performing it, i.e. choosing the best candidate models to proceed with the model fitting process. • Guidelines for the initial data filtering plus considerations related to the management of outliers and distribution tail. The filtering will be performed depending of the type of traffic to be studied. 52 Chapter 3 Analysis and modelling pipeline for NVE network traffic • Data preview to provide information to the researcher applying the pipeline to make technical decisions such as the nature and type of models to evaluate to generate the final one. Data preview heavily relies on the graphical representation of the ECDF for the input data. • Study of autocorrelation as a necessary test to determine the randomness of the data input and the suitability of using probabilistic models to describe it. A strongly autocorrelated data traffic may require using different models such as time-series or Markov ones, out of the scope of the proposed pipeline. • Selection of the probabilistic models that will be shaped and optimised into the final ones. The data previews generated during early steps will help to determine the nature of the data model (discrete vs. continuous) or the necessity of using a split model. The candidate models will get values for their parameters using the MLE method. In both the literature and the present study the exponential and Weibull model has been widely applied, thus including an overview of their main characteristics. • Study of statistical discrepancy to determine the degree of fitness or GoF of the generated models. The two chosen methods are Q-Q plots, to provide a visual and intuitive measurement of the GoF and the family of λ2 , a refinement of χ2 with desirables properties for data collections with similar probabilistic nature than NVE network traffic and similar video games. 3.9 Summary 53 4 Analysis and modelling of NVE network traffic: Open Wonderland In other words, the given time of arrival is the one moment of time at which it is impossible that any member of the party will arrive. Douglas Adams, Life, the Universe and Everything This chapter aims to apply the pipeline described in Chapter 3 to the network traffic generated by Open Wonderland (OWL), the networked virtual environment (NVE) system chosen as case study as a representative example of NVE. This pipeline has been designed based on previous works on video game network traffic modelling, and taking into consideration the special characteristics of NVEs to determine the most appropriate approach to the NVE network traffic modelling. The goal of applying the analysis pipeline to the OWL network traffic is to prove its suitability to generate accurate models that can be used in later activities such as simulation, network traffic extrapolation or sizing network infrastructures for NVE systems. This chapter relies on the analysis pipeline described in chapter 3 to analyse the empirical network traffic generated by the NVE subject of study in the present work, OWL. This traffic has been generated in the testbeds Open Wonderland TestBed 1 (OWTB1) and 2 Open Wonderland TestBed 2 (OWTB2), both described in chapter 2. The guidelines used to perform the OWL testing gaming sessions and capture the empirical network traffic data have been described in 2.3.3. Each section details the results obtained for OWTB1 and OWTB2 described in 2.3.1 and 2.3.2 respectively. The chapter narrows the scope to traffic associated to object synchronisation within OWL sessions. Audio traffic has been analysed and modelled in previous papers derived from the present work, showing a highly periodic packet rate and almost constant bandwidth. Due this simplicity and the space constraints, it has been left out of the scope of the present document. Details about the audio traffic study can be found in the article [Fon+12b]. 55 The chapter also provides a description of the variables that define it at microscale model level, associated to inter-arrival time (IAT) and packet size (PS) respectively. The modelling of each variable is performed by applying the analysis pipeline previously defined, and the details about the modelling process are included. After applying the pipeline for each of the microscale variables, the resulting statistical models are presented, including a range of valid values of their parameters. 4.1 Object synchronisation in Open Wonderland Objects within the OWL virtual world have a set of attributes that can have different values: for instance, coordinates in the virtual world, velocity (for moving objects), etc. The state of each object (or cell) is defined as the values of these attributes at a given time. In Wonderland, the server keeps a copy of all the world data, with data about the cells stored in XML files. When a client connects to Wonderland, it obtains from the server the information about the visible objects (cells). These objects can be classified as static, with fixed attributes that do not vary in time, as it is the case of a mountain or a building; or dynamic, with attributes that can vary in time, like an avatar moving from one region to another [Lui01]. Within the shared simulation provided by OWL, every client should have a consistent view of the virtual world, and therefore a mechanism must exist to ensure synchronisation between clients. When an object moves in this virtual world all clients that want to view the dynamic object must provide the same sequence of state changes. When something changes within OWL, such as the position of an object, its associated data are updated on the server and then forwarded to all the clients, which then update their own local views of the world. For instance, if a client moves their avatar, the client notifies the server and sends the new state of the cell to the server. Then, the server sends the new state of this object to all other clients, which then update their copies of the cell [Fouf]. The object synchronisation within OWL is performed over TCP protocol. While other game genres, such as first-person shooter (FPS), sacrifice the reliability of TCP over the speed of UDP, OWL has chosen to use TCP, which ensures the object updates are properly propagated and confirmed over the networked simulation. This TCP traffic is exchanged by default through port 1139 [Com]. The inspection of OWL network traffic at packet level using Wireshark [Tea19b] reveals that the typical OWL network packet exchange associated to Object Synchronisation happens as follows: 56 Chapter 4 Analysis and modelling of NVE network traffic: Open Wonderland 1. The OWL client sends a MOVE_REQ message to the server when the position of an object changes. 2. The server propagates the update above by sending a MOVED message to every other client, one per client. 3. Every client responds to the update with an ACK message. The server does not send any other MOVED message to a client until this responds with an ACK to the previous update. Although these MOVE_REQ messages are sent every time the object moves, in principle they are limited by the MoveableComponent. According to the Open Wonderland Project Forum [Foue], this limit is 5 updates per second, although this statement could not be confirmed in our experiments as we will discuss later on. 4.2 Microscale modelling and analysis pipeline The analysis pipeline for NVE network traffic modelling defined in Chapter 3 follows a microscale approach that aims to fully characterise the intra-session dynamics of the NVE network traffic by means of two variables, the IAT (or inter-departure time (IDT), depending on the observation point of view) and the PS. The empirical network traffic for modelling has been captured on the server side, as described in section 2.3.4. Moreover, this study focuses on the outgoing client traffic /outgoing server traffic. Thus, the text addresses the time between consecutive incoming packets that arrive to the OWL server from the different clients as Inter-Arrival Time (IAT), in the same way, Packet Size (PS) refers to the packet size of the incoming network traffic relative to the OWL server. 4.3 Data filtering criteria The object synchronisation traffic generated by OWL is composed of TCP packets that can be identified by certain tokens contained in their payload. Specifically, the update request contains the plain text MOVE_REQUEST token, while their confirmation packets contain the text token MOVED. Network traffic captured during the first and last minute of each session for both testbeds was discarded to avoid patterns generated by the login and logout process. All the traffic was captured using Wireshark[Tea19b], an open-source packet analyzer, as well as tshark[Tea19a], its CLI counterpart and better suited for scripting and automation of the packet capture. Further details about the packet capture process and parametrization can be found in Appendix B, Section B.1. 4.2 Microscale modelling and analysis pipeline 57 Once the OWL object synchronisation packets have been isolated from the raw network capture, the data set is ready for calculating the IAT for consecutive packets of the same type. This data set will be used by the analysis pipeline to provide a model that describe its behaviour. 4.4 Object synchronisation inter-arrival time The term object synchronisation inter-arrival time (OSIAT) defined in the present work refers to the time between consecutive object synchronization packets arriving to a given host, in this case, the server. The OSIAT is one of the two variables used in the microscale modelling approach for modelling NVE network traffic. The following sections describe how the OSIAT has been measured and analysed to produce statistical models. 4.4.1 Data preview The preliminary study of OSIAT values has consisted in a preview of the empirical cumulative distribution function (ECDF) for the object synchronization traffic received from each client within the session. This preview shows all the incoming client traffic OSIAT follows a similar pattern. Figure 4.1 shows the ECDF plots belonging to Client 2from the 2-client session and Client 3 from the 5-client session (see Table 2.1 and Section 2.3.1 for further details on the client’s hardware and software). Further ECDF plots for OSIAT traffic can be found in Appendix D, specifically the Figures D.1, D.2 and D.3 which show the ECDF plots for OSIAT values for clients engaged in the 2, 3 and 5-player sessions respectively. ECDF plots for OWTB1 clients, as displayed in Figure 4.1, show a continuous nature in the values for IAT, as well as two differentiated sections divided at 0.5 s . Values below 0.5 s seem to follow an exponential or Weibull distribution. This exponential nature is plausible if we assume the fact that outgoing client traffic can be considered as a Poisson Process where the waiting time between consecutive packets follows an exponential probability distribution [Ros95]. OSIAT values greater than 0.5 s represent a small fraction of the total and varies between clients and sessions. Although the ECDF curves have very similar shapes, they are not completely the same. Further tests within the testbeds showed that IAT distribution depends on session parameters such as user activity, number of players or client resources. Section 4.4.3 discusses the nature of these values and their relationship with user activity during the gaming session. 58 Chapter 4 Analysis and modelling of NVE network traffic: Open Wonderland Fitting for Client 2 from 2−client session Packet IAT(s) F(x) 0.0 0.2 0.4 0.6 0.8 1.0 0 0.2 0.4 0.6 0.8 1 0.5s Empirical Exponential (rate= 4.52 packet/s) Fitting for Client 3 from 5−client session Packet IAT(s) F(x) 0.0 0.2 0.4 0.6 0.8 1.0 0 0.2 0.4 0.6 0.8 1 0.5s Empirical Exponential (rate= 2.68 packet/s) Fig. 4.1.: ECDF for OSIAT and CDF for exponential distribution, OWTB1 There is a probability saturation around 0.5 s with an important percentage of OSIAT values concentrated around a narrow range centered on this value. This percentage varies between clients; in Figure D.2 the ECDF for OSIAT values generated by Client 1has a lower step around 0.5 s than that of Client 3. In both cases, values greater than 0.5 s are relatively unlike and their frequency also varies between clients. According to the information published in the OWL documentation [Proc], clients limit the synchronisation traffic to avoid rates greater than 5 packets per second. This would imply a minimum OSIAT value of 0.2 s . However, it was found a big incidence of values around 0.5 s . A possible explanation for this concentration of OSIAT values around 0.5 s is object synchronisation packets follow an exponential-like distribution in optimal conditions, but when there is a saturation of synchronisation packets, the OSIAT values fall into the worst-case scenario for the system, this is 0.5 s. Each ECDF curve in Figure 4.1 is represented by a solid line, accompanied by a dashed one corresponding to the cumulative distribution function (CDF) F(x) for an exponential distribution calculated by maximum likelihood estimation (MLE). The visual inspection of the fitting suggests an exponential nature for the OSIAT values below 0.5 s . The rate parameter λ determines the exponential behaviour and can be interpreted in the IAT context as the number of packets per second generated by an OWL client. Although this exponential functions are a rudimentary approximation, they support the hypothesis of an exponential distribution of part of the OSIAT values. The OSIAT values for OWTB2 clients display similar ECDF plots to those from OWTB1. Figures D.4, D.5, D.6 and D.7 show the ECDF plots for OSIAT from 2, 3, 5 4.4 Object synchronisation inter-arrival time 59 been chosen as representative statistical parameter of the interval due to its lower sensitivity to extreme values. Taking into account that the OSIAT values from OWTB1 and OWTB2 are just a representation of the all possible values that can be observed in the OWL traffic, together with the errors inherent to the traffic capture and measurement process, tthres = 0.5s will be used in the present work to determine the boundary between the two parts the OSIAT distribution is divided. Using further decimals could just fall into model overfitting for the given experimental data while not providing any tangible benefit. 0.490 0.495 0.500 0.505 0.510 0 50 100 150 200 250 300 350 PDF Object Sync. IAT values OWTB1 t(s) Density median = 0.5011s Fig. 4.5.: PDF for the aggregation of OSIAT values from OWTB1 Activity Correction Parameter The Activity Correction Parameter, Cacp , aims to weight the exponentially distributed OSIAT values regarding the overall OSIAT distribution. Section 4.4.3 discussed the relationship between the rate of user activity within the gaming session (movement and audio transmission) and the percentage of OSIAT values equal or greater than 0.5 s . The ECDF plots for the user activity experiment showed that high inactivity periods imply heavy tailed OSIAT distributions. Thus, depending on the activity rate displayed during the gaming session and the associated distribution tail, the fitting curve obtained using MLE to model values in interval [0, tthres] will be required to be adjusted. 66 Chapter 4 Analysis and modelling of NVE network traffic: Open Wonderland 0.490 0.495 0.500 0.505 0.510 0 1000 2000 3000 4000 PDF Object Sync. IAT values OWTB2 t(s) Density median = 0.5010s Fig. 4.6.: PDF for the aggregation of OSIAT values from OWTB2 The client 1 from the 5-client session performed on OWTB1 is used as reference to define the formula for Cacp . Figure 4.7 shows the ECDF for the OSIAT values for this client (solid black line). The red dashed line corresponds to the exponential distribution calculated using MLE. It can be appreciated that probability saturation around 0.5 s diverts the fitting curve from the empirical results. The goal of the Cacp is to correct this deviation by escalating the fitting curve. The scaling factor Cacp is calculated by measuring the difference between the ECDF and the fitting curve at PU= 0.49s . The choice of this value correspond to the initial of the interval where the vast majority of OSIAT values around 0.5 s are concentrated (Section 4.4.4), being a convenient point of union between the two parts of the OSIAT model Figure 4.8 zooms over the area where the studied ECDF and fitting curve meet, highlighting the curve points used to calculate the value of δ . The definition of Cacp implies that its value for a given model depends on the fitting curve for interval [0, tthres], at point PU= 0.49s. The difference between these curves at PU= 0.49 is expressed in Equation (4.1) by δ as the difference between the ECDF value for 0.49 s and the value PFIT (PU,{Prate_mle}) , the cumulative probability of the fitting curve PFIT with parameters {Prate_mle} at point PU= 0.49s . Thus, the Cacp is a scale factor described in Equation (4.2) . The Figure 4.8 highlights the point pexp(0.49s, rate = 3.74) , the value given by the fitting exponential CDF with rate parameter λ= 3.74 (the PFIT (X, ratemle)function 4.4 Object synchronisation inter-arrival time 67 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 ECDF Object Sync. IAT Client 01, 5−client session, OWTB1 t(s) ECDF(X) Fig. 4.7.: ECDF and fitting CDF, Client 1,5-client session, OWTB1 0.40 0.45 0.50 0.55 0.60 0.70 0.75 0.80 0.85 0.90 0.95 1.00 ECDF Object Sync. IAT (Detail) Client 01, 5−client session, OWTB1 t(s) ECDF(X) P_u=0.49s ●ECDF(0.49s) ● pexp(0.49, rate=3.74) δ Fig. 4.8.: Detail of ECDF and fitting CDF, Client 1,5-client session, OWTB1 68 Chapter 4 Analysis and modelling of NVE network traffic: Open Wonderland represented by the dashed red curve) for the point 0.49 s , and point ECDF(0.49s) , part of the ECDF curve obtained for the sample values. δ=PFIT (PU,{Prate_mle})−ECDF(PU)(4.1) acf = 1−δ PFIT (PU,{Prate_mle})!(4.2) The Cacp values calculated for the OSIAT models based on exponential and Weibull distributions for each client and session from OWTB1 and OWTB2 are shown in Tables D.2 and D.3 respectively, included in the Appendix D. According to these values, Cacp = 0.8 is a reasonable approximation that works well with most of the observations from the testbeds. Fitting curve for first interval of the split distribution The data preview from Section 4.4.1 showed that ECDF plots for OSIAT values from the activity evaluation sessions (Figure 4.4), OWTB1 multiplayer sessions (Figures D.1, D.2 and D.3), and OWTB2 multiplayer sessions (Figures D.4, D.5, D.6 and D.7), suggest that the OSIAT values follow an exponential-like distribution for values between 0 and ≈0.5 s before transitioning into a tail which weight decreases with the increase in player activity rate during the game session (Section 4.4.3). The most promising continuous probability distributions to model these values in the interval [0,0.5] are the Exponential and Weibull distributions. The present section will propose models based on each distribution. Determining the analytical expression for the exponential section of OSIAT presents the following problem: if all the empirical data is used to calculate the exponential (or Weibull) parameters using MLE, the OSIAT values around 0.5 s may lead to a poorly fitting distribution that tries to both accommodate exponentially distributed values with others that do not follow this trend. Figure 4.4 shows a preliminary fitting attempt for the IAT values from the active/inactive sessions from OWTB1, using a pure exponential distribution as fitting distribution. In this plot each ECDF is accompanied by two CDF plots for exponential distributions. The ones labeled as Raw Fitting (dashed lines) result from applying MLE to the whole set of OSIAT values from each one of the active/inactive sessions. The Filtered Fitting CDF plots (dotted lines) result from applying MLE to the set of 4.4 Object synchronisation inter-arrival time 69 OSIAT values after removing the tail values, this is those values greater than 0.51 s , as suggested in Section 4.4.3. At these point it is useful to briefly introduce a quantitative way to determine the goodness of fit (GoF) of a given model for an empirical data set, which will numerically measure how well the proposed model describes the empirical data. The evaluation of the GoF for the proposed models is further developed and discussed in more detail in Section 4.4.5, this section will use λ2 to determine if tail values would be included or not when using MLE to calculate the fitting curve for interval [0,0.5]. The exponential models for Raw and Filtered OSIAT values from the inactive client perform differently in terms of GoF, λ2 RAW = 0.776 versus λ2 FILT. = 4.66 . On the other hand, the exponential models proposed for the values from the active client session show very similar values, λ2 RAW = 4.12 and λ2 FILT. = 4.08 respectively, which is consistent with the smaller percentage of outliers and lighter OSIAT distribution tail associated to high activity rates (Section 4.4.3). It is noteworthy that while λ2 provides a quantitative measure for GoF, its value does not provide an absolute measurement of how good or bad the fitness is, it just allows to compare and sort how different theoretical models approximate a given data set. While λ2 helps to determine which model describes better than others the empirical data set, it does not provide information about how much better it describes it, so very different λ2 do not necessarily imply quantitative measurement of the GoF. In the case of λ2 RAW = 0.776 versus λ2 FILT. = 4.66 for the proposed CDFs for the inactive client session, it can be concluded that filtering tail and outlier OSIAT values worsens for the modelling purposes the fit of the distribution calculated using MLE. Removing the distribution tail values for the different testing sessions (values greater than 0.5 s ) helps to get better fitting distributions. Based on the distribution of values around 0.5 s shown in Figures 4.5 and 4.6, there is a low density of values greater than 0.51 s , this can be used to delimit the distribution tail. Thus, the analytical expressions for the range [0, tthres] for the split distribution for OSIAT values will be obtained by applying MLE to those values from the testing sessions lower or equal than 0.51 s. In conclusion, the analysis suggests that OSIAT values in the interval between 0 s and 0.5 s can be potentially modelled using an exponential or a Weibull distribution. It will be required to resort to the evaluation of the statistical discrepancy between these candidate models and the experimental data set to determine which is the most suitable to describe the OSIAT for OWL. 70 Chapter 4 Analysis and modelling of NVE network traffic: Open Wonderland Proposed models for Object Synchronisation Inter-Arrival Time The result of putting together the model elements defined throughout Section 4.4.4 are Equation (4.3) and Equation (4.4) , the two candidate models to describe the OSIAT values where t is the threshold for the split expression (established in t= 0.5s using the median for OSIAT values concentrated around 0.5 s , as described in Section 4.4.3), Cacp is the activity correction parameter which value is linked to the activity rate associated to the player within the gaming session, formulated in Section 4.4.4. λ parameter in Equation (4.3) represents the rate for the Exponential distribution, while λ and κ parameters in Equation (4.4) represent the scale and shape parameters respectively for Weibull distribution. F(x;λ, Cacp, tthres) =        1,x > tthres Cacp ·1−e−λx,tthres ≥x≥0 0,x < 0 (4.3) F(x;λ, κ, Cacp, tthres) =        1,x > tthres Cacp ·1−e−(x/λ)κ,tthres ≥x≥0 0,x < 0 (4.4) The parameters for the exponential distribution in Equation (4.3) calculated for each client and session from OWTB1 and OWTB2 are shown in Tables D.4 and D.6 in Appendix D. Values from Table D.6 have been used to generate the Figure 4.9, a surface plot where axis X (labeled as session) represents the experimental sessions from OWTB2, axis Y (labeled as client) represents the nth client from a given session, and axis Z (labeled as Exp. lambda) is the parameter value obtained by MLE. Thus, a point (x, y, z) represents the λ exponential parameter of value z calculated applying MLE to the OSIAT values from the yth client in the xth experimental session performed in OWTB2. The surface shows the MLE values are around 0.47. The parameters for the Weibull distribution in Equation (4.4) calculated using for each client and session from OWTB1 and OWTB2 are shown in Tables D.5 and D.7 in Appendix D. Values from this last table have been used to generate the surface plots in Figure 4.10. As in the case of Figure 4.9, each point (x, y, z) represents the parameter calculated using MLE with value z for the yth client in the xth session performed in OWTB2. These surfaces show that the shape and scale parameters for Weibull distribution take values around 0.51 and 0.14 respectively. 4.4 Object synchronisation inter-arrival time 71 session 246810 client 2 4 6 8 10 Exp. lambda 3.0 3.5 4.0 4.5 5.0 Fig. 4.9.: Exponential λrate values for OWTB2 clients and sessions session 246810 client 2 4 6 8 10 Weibull shape 0.0 0.2 0.4 0.6 0.8 1.0 session 246810 client 2 4 6 8 10 Weibull scale 0.00 0.05 0.10 0.15 0.20 Fig. 4.10.: Weibull λscale and κshape values for OWTB2 clients and sessions 72 Chapter 4 Analysis and modelling of NVE network traffic: Open Wonderland Table 4.3 contains a summary of the values from tables D.4, D.6, D.5 and D.5. Tab. 4.3.: Summary of MLE parameters for OWTB1 and OWTB2 Distribution Parameters OWTB1 OWTB2 µ σ2µ σ2 Exponential λrate 3.63 0.74 4.76 0.10 Weibull λscale 0.26 0.05 0.14 0.01 κshape 0.87 0.11 0.51 0.03 The λ parameter values obtained by MLE for the exponential-based model are quite similar between the OSIAT data flows generated by each one of the clients engaged in the same gaming session. These same λ values seem to be correlated to the number of players within a given session. The same trends are observed for parameters λ and κ for Weibull-based model. This suggests that the number of simultaneous players engaged in the same OWL gaming session has an impact on the ratio of object synchronisation packets that are received from each client on the OWL server side. 4.4.5 Statistical discrepancy The statistical discrepancy or GoF for the OSIAT is evaluated using the following methods suggested by the analysis pipeline (must be noted that Pearson correlation coefficient (PCC) has been also included): • Q-Q plots to provide an intuitive and qualitative preview of GoF. • Pearson Correlation Coefficient to measure the similarity between values generated using the ECDF and the proposed OSIAT models. •ˆ λ2 to quantitatively compare the GoF provided by the Exponential and Weibullbased OSIAT models formulated in Section 4.4.4. Q-Q plots Q-Q plots (Section 3.8.1) make possible to qualitatively compare quantiles from two distributions, in the current case, the quantiles derived from the ECDF for the OSIAT versus the quantiles for the models Equation (4.3) and Equation (4.4) proposed in Section 4.4.4. These models only describe the exponential-like interval of OSIAT values in interval [0, tthres] and the probability saturation point at tthres , leaving out the tail of OSIAT values. Thus, it is expected that the Q-Q plots will show a poor correspondence between empirical and model quantiles (points largely 4.4 Object synchronisation inter-arrival time 73 deviating from the plot diagonal). In the present study Q-Q plots are used to provide a qualitative measure of GoF for the OSIAT quantiles in the interval [0, tthres]. Due to space limitations, the present section only includes the Q-Q plots for the 3-client sessions from OWTB1 and OWTB2. A greater selection of OSIAT can be found in Appendix D, Section D.6. Figure 4.11 suggests a good fit between the empirical quantiles and those calculated for Equations (4.3) and (4.4) for OWTB1. The plot includes the values used for the model parameters. 0.0 0.1 0.2 0.3 0.4 0.0 0.1 0.2 0.3 0.4 Q−Q plot for Object Sync. IAT, Client 01 Empirical quantiles Model quantiles ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ●● ●● ● ● ● ● ● ●● ●●● ● ● ●● ●●● ● ● ● ●● ● ●● ● ● ●● ● ●● ● ●●● ●●● ●●●●● ●●●● ●●●●●●●●● ● ● Model Exp. rate=4.57, acp=0.9 Model Wei. k=1.05, scale=0.22, acp=0.9 0.0 0.1 0.2 0.3 0.4 0.0 0.1 0.2 0.3 0.4 Q−Q plot for Object Sync. IAT, Client 02 Empirical quantiles Model quantiles ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ●●● ●●● ● ● ●●●● ● ● ●●●●● ● ●●●●● ● ●● ●● ●●●●●●●● ●●● ● ● ● Model Exp. rate=4.76, acp=1.0 Model Wei. k=0.98, scale=0.21, acp=1.0 0.0 0.1 0.2 0.3 0.4 0.0 0.1 0.2 0.3 0.4 Q−Q plot for Object Sync. IAT, Client 03 Empirical quantiles Model quantiles ● ● ● ● ●●● ● ●● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ●● ● ● ● ●●● ● ● ● ● ● ● ●●● ● ●●●●● ●●●●● ●●●●●●●●● ●●●● ●● ● ● ● ● ● ● ● Model Exp. rate=3.77, acp=0.8 Model Wei. k=1.19, scale=0.28, acp=0.8 Fig. 4.11.: Q-Q for OSIAT models, t3-player session, OWTB1 Figure 4.11 shows a slightly worse fit for the OWTB2 sessions. It can be observed that both the Weibull and Exponential-based models make the points in the Q-Q plot fall above the diagonal, which means they overestimate the weight of the interval [0, tthres] in the overall model. This deviation from the diagonal stays constant for many of the quantiles, which suggests the possibility to improve the fitting by correcting such an offset in the proposed OSIAT models. On the other hand, Figure 4.12 shows the Q-Q plots for the 3-client session from OWTB2. These plots show points run parallel to the bisector of the first quadrant. The offset making the model quantiles (axis Y) being above the bisector suggests the 74 Chapter 4 Analysis and modelling of NVE network traffic: Open Wonderland model slightly overestimates the values for the cumulative probability for empirical results from OWTB2. 0.0 0.1 0.2 0.3 0.4 0.0 0.1 0.2 0.3 0.4 Q−Q plot for Object Sync. IAT, Client 01 Empirical quantiles Model quantiles ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ●●●● ●●●● ●●● ●●●●●●● ●●●●● ● ● ● ● ● ● Model Exp. rate=4.58, acp=0.8 Model Wei. k=0.55, scale=0.16, acp=0.8 0.0 0.1 0.2 0.3 0.4 0.0 0.1 0.2 0.3 0.4 Q−Q plot for Object Sync. IAT, Client 02 Empirical quantiles Model quantiles ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ● ● ●●●● ● ●●●● ●●●● ●●●● ●●● ●●● ● ● ● ● ● ● Model Exp. rate=4.64, acp=0.8 Model Wei. k=0.54, scale=0.15, acp=0.8 0.0 0.1 0.2 0.3 0.4 0.0 0.1 0.2 0.3 0.4 Q−Q plot for Object Sync. IAT, Client 03 Empirical quantiles Model quantiles ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ● ● ● ●●●● ●●●●● ●●● ●●●●● ●●●● ●●●● ● ● ● ● ● ● ● ● ● ● ● ● ● Model Exp. rate=4.50, acp=0.8 Model Wei. k=0.57, scale=0.17, acp=0.8 Fig. 4.12.: Q-Q for OSIAT models, 3-player session, OWTB2 While providing an intuitive estimation of the GoF, the Q-Q plots do not allow to easily determine which of the proposed OSIAT models (exponential vs. Weibull) provides a better fit for the empirical data, being advisable to resort to quantitative methods to be able to compare the level of GoF provide by each candidate model. Pearson Correlation Coefficient The PCC or Pearson’s r measures the linear correlation between two variables x and y , where r measures the trend relative magnitude of the fit [Fis15], [DGK75]. A r value equal to 1 indicates that the equation describes the relationship between x and y perfectly, while a r equal to 0 implies the lack of correlation between empirical data and the model. The Pearson’s r is accompanied by r2 , which indicates the proportion of the total variance that is explained by the prediction. While PCC is not a robust statistic due to its sensibility to outliers, it is used in the present study to provide a quick and simple estimation of GoF by measuring the correlation between empirical values and those derived from the proposed theoretical distributions. 4.4 Object synchronisation inter-arrival time 75 Tab. 4.7.: Relative frequency for OSPS, OWTB2 Session N. Packets 293 B 304 B 01-client 2809 0.41 0.59 02-client 6358 0.42 0.58 03-client 9074 0.44 0.56 04-client 12087 0.43 0.57 05-client 14524 0.43 0.57 06-client 17776 0.43 0.57 07-client 19977 0.43 0.57 08-client 22636 0.43 0.57 09-client 26101 0.42 0.58 10-client 27736 0.43 0.57 Total 159078 0.43 0.57 P(x) = (0.43 ,x= 293 bytes 0.57 ,x= 304 bytes (4.7) Choosing one model over the other will depend on the specific setup that is going to be modelled. OWTB1 is a more heterogeneous testbed, where avatars where operated by human players, while OWTB2 is an homogeneous testbed where avatars performed a high rate of activity and interaction within the sessions due to the automatic control performed with scripts, providing a relatively synthetic setup which on the other hand allows to evaluate the features and performance of the system. Due to the simplicity of the OSPS models in Equations (4.6) and (4.7) , the study of GoF has not been included. 4.6 Summary The present chapter details the analysis details and results from applying the pipeline proposed in Chapter 3 to the experimental network data generated for the NVE software OWL in the testbeds from Chapter 2 (OWTB1 and OWTB2). The goal of this exercise is to determine the suitability of the pipeline to model the network traffic generated by a system, OWL, fitting into the definition of NVE provided in Section 2.1. Thus, the analysis pipeline has been used to model the random variables OSIAT and OSPS which describe the OWL network traffic at microscale level. 82 Chapter 4 Analysis and modelling of NVE network traffic: Open Wonderland This chapter provided a technical description of the object synchronisation traffic generated by OWL, where the packets belonging to these data flows will be filtered and object of study using the analysis pipeline. It was also provided a naming convention and definition for the random variables associated to the object synchronisation traffic, OSIAT and OSPS, which describe the behaviour of the network traffic at microscale level. It is also included the detailed study and modelling of the OSIAT variable for the OWL object synchronisation traffic from the testing testbeds, performed using the analysis pipeline proposed in the previous chapter. This study included: • The data preview of the OSIAT revealed an exponential-like distribution of values and a probability saturation around value 0.5 s. • Autocorrelation study showed OSIAT values are not autocorrelated and match the requirements to be modelled as a random variable. • The study of the user activity within the gaming sessions revealed that there is a correlation between the activity and the size of the tail of the distribution of OSIAT, the higher the activity, the smaller the distribution tail. • The data modelling step proposed a split distribution to model the OSIAT, defining all the parameters that determine the behaviour of the model and providing its analytical expression. Because of the exponential-like look of the data preview, two variations of the OSIAT model were proposed: one based on the exponential distribution for values between 0 and 0.5 s (Equation4.3) and another based on the Weibull distribution for the values in this same range (Equation 4.4). The models were accompanied by the parameters for the exponential and Weibull components, calculated by applying the MLE method to the experimental data obtained from the testbeds (Table 4.3). • The measurement of the statistical discrepancy was performed using several methods: Q-Q plot to have an intuitive estimation of the GoF, PCC as a simple numeric metric for fitness, and λ2 / ˆ λ2 a more robust metric for statistical discrepancy which allows comparing the degree of fitting between the experimental data and the proposed models. The Q-Q plots did not show a clear advantage to any of the two proposed models. According to the ˆ λ2 results, the GoF for the Weibull-based OSIAT was better than that for the exponential-based model (Tables 4.4 and 4.5). The study and modelling of the microscale variable OSPS, performed using the analysis pipeline included the following elements: • The data preview showed that OSPS values are a small set of discrete values, which made evident the convenience of modelling the variable with a discrete probability model. 4.6 Summary 83 • The autocorrelation study showed little evidence of self-similarity, so OSPS can be considered as a random variable and modelled as such. It was also observed that the number of number of discrete values for OSPS varied from one testbed to another, a side effect of the different rates of user activity and network traffic present in each one. • The data modelling step provided a simplified discrete model for OSPS described in Equation 4.7. 84 Chapter 4 Analysis and modelling of NVE network traffic: Open Wonderland 5 Simulation based on NVE models Maybe the only significant difference between a really smart simulation and a human being was the noise they made when you punched them. Terry Pratchett, The Long Earth This chapter describes the design, implementation and execution of simulations based on the object synchronisation inter-arrival time (OSIAT) models proposed in Chapter 4, Section 4.4.4 for Open Wonderland (OWL) network traffic. These models were generated using the analysis pipeline for networked virtual environment (NVE) network traffic defined in Chapter 3. Results from these simulations will help to validate the models implemented by the simulation and also the pipeline that generated those models. Moreover, these validated models will be ready for further use in research of performance evaluation of NVE systems in the lane of OWL. Simulation is a key tool in networking research due to its inherent advantages over testing with real hardware [RPW09]. Matters such as budget and cost of time make simulation attractive for research, while flexibility, scalability and virtually no cost expansion are also valuable advantages over other testing methods [OB10]. Network simulators allow us to implement and study different network entities in a simulated environment, providing a high degree of flexibility to test new protocols, technologies, conceptual models and topologies. 5.1 Simulation framework: ns-3 The implementation of the models for OSIAT and object synchronisation packet size (OSPS) proposed in Chapter 4 (Sections 4.4.4 and 4.5.3 respectively) is based on the simulation tool and framework ns-3 [Pro19]. The version used in the present work is ns-3.29, last stable release at the time of writing. The choice of ns-3 is derived from the author’s previous work in [Fon+10] and [Fon+11a]. These works approach the comparison of ns-2 and ns-3, two of the most popular network simulation frameworks 85 at the time of writing them, from the perspective of software engineering by using complexity and code quality metrics. The chosen network simulation framework, ns-3, provides a modular and well tested simulation framework, it is an open source community-driven project and provides a well defined building process, comprehensive documentation and the possibility of coding in C++ or Python [Fon+10], [Fon+11a]. Ns-3 is devised to be executed in GNU/Linux and Unix-like environments, being also possible to build and run it in Windows systems using Cygwin. Ns-3 was conceived as ns-2’s [Proa] [LA03] successor: its developers have tried to solve or mitigate many of ns-2’s well-known drawbacks as well as apply new concepts, such as validation and software engineering techniques, to produce a more reliable simulation tool to support academic and industrial research [Hen14]. Ns-3 is conceived to be distributed in source code form though some GNU/Linux distributions such as Debian or Arch provide pre-compiled packages. Due to users have access to source code, they can modify and extend their features and optimize their binaries as needed. Source code can be fetched from the Git [Prob] repository maintained by the ns-3 project. This repository contains all the official releases, and publicly available development branches. The programming language used for the ns-3 core and model components is C++, while simulations that put to work these components can be written in C++ or Python, an interpreted language which presents some advantages about development speed and code readability. Regarding the architecture of the framework, ns-3 emphasizes source code hierarchical structuring by defining several basic network entities present in every single simulation. Specific models are a refinement of these generic entities [Pro19]. • Node, represents the basic computing device. It acts as container of the elements of the network stack and interacts with other nodes by the communication channel. Nodes require net devices in order to be able to use the physical channel. • Application, sets up on top of the network stack within the node, playing the role of packet consumer or packet generator. • Channel, provides the media to interconnect nodes and to allow data traffic exchange. • Net Device, abstracts the network hardware that makes communication possible through a channel. 86 Chapter 5 Simulation based on NVE models • Topology Helper, auxiliary class that makes the generation of complex topologies easier by automating the creation of network elements, their configuration and interconnection. • simulation, a program that defines, configures and launches the simulation experiment based on the models and helpers. Ns-3 gathers all source code files of both simulator core and models in the ns-3/src folder, separating them from the rest of auxiliary tools, building scripts, documentation and examples. The ns-3/src folder is also subdivided, each subfolder containing the files that implement the different models shipped with ns-3 by default, plus the simulator core components. On the other hand, simulation are placed in the ns-3/scratch folder. 5.2 Simulation implementation Ns-3 code is divided into models, helpers and simulations. The models are classes that contain the functionality of the modelled system, such as network nodes, data packets, protocol stacks or network devices. On the other hand helpers are auxiliary classes that allow to create and configure several objects of the same class in an easy and concise way, such as defining several interconnected nodes, initialising a large amounts of network devices with consecutive IP addresses, and so on. Finally, a simulation is a piece of code that defines, configures and launches the simulation experiment based on the models and helpers provided by ns-3. While ns-3 simulations can be written in C++ or Python, the former is the language for model and core components. The ns-3 models, helpers and simulation coded and used in the context of the present work have been written in C++. Many of the classes mentioned below contain the prefix ns3::, which is the C++ namespace for all ns-3 classes. 5.2.1 Implementation and code structure Figure 5.1 contains a simplified diagram of the different entities that are part of the OSIAT simulation. The simulation rely on OWL client and server, ns-3 nodes each one sporting a different collection of models to simulate the object synchronisation exchange of traffic. These nodes relies on the ns-3 TCP/IP stack and carrier sense multiple access (CSMA) models to exchange information between them. The dashed rectangles within the figure represent the helpers that assist in the creation and configuration of the different kind of nodes in the simulation. 5.2 Simulation implementation 87 configure OWLServer Helper OWLServer Helper OWLClientNode OWLClientApp OSIATmodel OSPSmodel IPv4stack CSMAchannel OWLServerNode OWLServerApp IPv4stack configure Fig. 5.1.: Structure of the ns-3 simulation for OSIAT Below there is an enumeration of the main object oriented (OO) classes that constitute the building blocks for the ns-3 simulation based on the OSIAT and OSPS models from Chapter 4. Each class includes a brief description and relationships with other classes. ns3::Node acts as a container entity that represents a network node. It contains objects that simulate network devices, protocol stacks or applications that generate or consume network packets. Auxiliary classes such as ns3::NodeContainer help to manage a large number of ns3::Node objects. ns3::Application represents elements that generate and consume network traffic following the logic coded by the programmer, e.g. a Telnet server. WonderMovClient extends ns3::Application class, it uses the models for OSIAT and OSPS proposed in Sections 4.4.4 and 4.5.3 to generate traffic the way OWL clients do. It relies on ns-3 models for the TCP/IP stack, such as ns3::Socket and Ipv4Address. This application requires to be connected to an object of class WonderMovServer which plays the role of OWL server. Figure 5.2 shows the relationship of this class with the rest of classes in the simulation. The operation of the application is shown in the sequence diagram in Figure 5.3. Once the client is associated to the server object, it starts a loop that determines the size of the packet to be sent using the OSPS model, then proceeds to send the packet. The server receives this packet, processes it and proceeds to send an ACK message to the client. The client uses a OSIAT model to determine when the next packet will be sent to the server, starting a new iteration of the loop. 88 Chapter 5 Simulation based on NVE models WonderMovClient uint32_t*m_data uint32_tm_dataSize uint32_tm_size boolm_connected uint32_tm_readpackets uint32_tm_sentpackets Ptr<Socket>m_socket Ipv4Addressm_peerAddress uint16_tm_peerPort Ptr<Socket>m_socket_tcp Ipv4Addressm_ipaddr Ptr<OsPs>ops_model Ptr<OsIat>osiat_model +OWonderMovclient()<<constructor>> +~OWonderMovclient()<<destructor>> StartApplication() StopApplication() Send() HandleRead() +GetTypeId():Typeid +GetInstanceTypeId():TypeId +SetRemote(Ipv4Address,uint16_t) +SetDataSize(uint32_t) +GetDataSize():uint32_t +GetNReadPackets():uint32_t +GetNSentPackets():uint32_t +SetConnected() +SetNode(Ptr<Node>) Ipv4Address Socket Application Extends OsIat OsPs Use Use Use Use WonderMovServer uint32_tm_readpackets uint32_tm_sentpackets Ptr<Socket>m_socket Ipv4Addressm_ipaddr uint16_tm_port std::map<uint32_t,Address>m_clientlist std::list<Ptr<SocketList>>m_socketlist +WonderMovServer()<<constructor>> +~WonderMovServer()<<destructor>> StartApplication() StopApplication() HandleRead(Ptr<Socket>) HandleAccept(Ptr<Socket>),Address) +GetTypeId():Typeid +GetInstanceTypeId():TypeId +GetNReadPackets():uint32_t +GetNSentPackets():uint32_t Extends Use Use Fig. 5.2.: Class diagram for WonderMovClient and WonderMovServer :WonderMovServer loop :WonderMovClient objectsyncupdate ACK Fig. 5.3.: Sequence diagram for OWL traffic models 5.2 Simulation implementation 89 WonderMovServer extends ns3::Application class. Objects of this class are associated to WonderMovClient objects which send them object synchronisation packets. The class models the very basic functionality of the OWL server, only reading packets and proceeding to acknowledge them. The class is also capable of propagating the object synchronization updates to the rest of the client objects associated to the server. Figures 5.2 and 5.3 show its class and sequence diagram respectively. It must be noted that the object synchronisation traffic sent by the OWL server is mostly reactive, without signs of periodicity nor bursty behaviour. This detail has been taking into account when implementing the WonderMovServer class. WonderMovClientHelper and WonderMovServerHelper helper classes that provide auxiliary methods to help with the configuration of several nodes running the WonderMovClient and WonderMovServer applications described above. The helpers take care of installing the corresponding application in each one of the provided nodes. Figure 5.4 shows the class relationships and a summary of the methods and attributes for these classes. WonderMovServerHelper ObjectFactorym_factory() +WonderMovServerHelper(uint16_t)<<constructor>> +~WonderMovServerHelper()<<destructor>> +SetAttribute(std::string,AttributeValuevalue) +Install(Ptr<Node>):ApplicationContainer +Install(std::string):ApplicationContainer +Install(NodeContainer):ApplicationContainer InstallPriv(Ptr<Node>):Ptr<Application> ObjectFactory Use WonderMovClientHelper ObjectFactorym_factory() +WonderMovClientHelper(Ipv4Address,uint16_t)<<constructor>> +~WonderMovClientHelper()<<destructor>> +SetAttribute(std::string,AttributeValuevalue) +Install(Ptr<Node>):ApplicationContainer +Install(std::string):ApplicationContainer +Install(NodeContainer):ApplicationContainer InstallPriv(Ptr<Node>):Ptr<Application> Use Fig. 5.4.: Class diagram for WonderMovClientHelper and WonderMovServerHelper OsIat, OsIatExp and OsIatWei OsIat is an generic class to describe the common functions to all the OSIAT models. Specific ones such as OsIatExp (based on the Exponential distribution) or OsIatWei (based on the Weibull distribution) extend this class. The WonderMovClient application is implemented to work with any class that implements the methods defined in this class, making it easy to incorporate further OSIAT models to the simulation. OsIatExp and OsIatWei classes rely on several ns3::RandomVariableStream derived classes such as ns3::ExponentialVariable, ns3::UniformVariable or ns3::WeibullVariable to implement the models described in Section 4.4.4. Methods, attributes and relationships between classes are shown in the class diagrams in Figures 5.5 and 5.6. 90 Chapter 5 Simulation based on NVE models OsIatExp lambda:double rexp:Ptr<ExponentialRandomVariable> runi:Ptr<UniformRandomVariable> +OsIatExp()<<constructor>> +~OsIatExp()<<destructor>> +SetLambda(double) +GetLambda():double OsIat c_acp:double t_thres:double +OsIat()<<constructor>> +~OsIat()<<destructor>> +GetTypeId():Typeid +GetInstanceTypeId():TypeId +GetValue():double +SetAcp(double) +GetAcp():double RandomVariableStream UniformRandomVariable ExponentialRandomVariable Extends Extends Extends Extends Use Use Fig. 5.5.: Class diagram for exponential-based OSIAT model OsIatWei lambda:double kappa:double rwei:Ptr<ExponentialRandomVariable> runi:Ptr<UniformRandomVariable> +OsIatWei()<<constructor>> +~OsIatWei()<<destructor>> +SetLambda(double) +GetLambda():double +SetKappa(double) +GetLambda():double WeibullRandomVariable Extends Use Use RandomVariableStream UniformRandomVariable OsIat c_acp:double t_thres:double +OsIat()<<constructor>> +~OsIat()<<destructor>> +GetTypeId():Typeid +GetInstanceTypeId():TypeId +GetValue():double +SetAcp(double) +GetAcp():double Extends Extends Extends Fig. 5.6.: Class diagram for Weibull-based OSIAT model 5.2 Simulation implementation 91 6 Conclusions and future work I may not have gone where I intended to go, but I think I have ended up where I needed to be. Douglas Adams, The Long Dark Tea-Time of the Soul The present chapter contain the conclusions derived from the study of NVE network traffic as well as the lines of work that can be addresses as future work. 6.1 Conclusions The main conclusions derived from the work on NVE network traffic performed in the present work are the following. • NVE systems have a relevant role in many fields in a global and highly interconnected world. The historical overview of NVE systems in Section 1.1 provided a sense of the importance the paradigm in fields such as entertainment, training or e-learning. Moreover, it has been observed that current trends are shifting from NVE frameworks such as OpenSimulator or OWL towards adapting mainstream video games such as Minecraft as canvas to develop NVE systems for research and scientific purposes. • The NVE characterisation defined in Section 2.1 captures all the elements and characteristics that define a NVE and has proven to be key for the literature review allowing to define the analysis pipeline proposed in Chapter 3. • OWL matched the NVE characterisation and proved to be technically viable study case on which to build experimental testbeds Open Wonderland TestBed 1 (OWTB1) and OWTB2 from Section 2.3 to get the necessary network traffic captures to validate the analysis pipeline. 99 • The study of the literature on network traffic of video games that qualify as NVE helped to define the analysis pipeline from Chapter 3, which follows a microscale modelling approach focused on IAT and PS to describe the network traffic, also providing methods metrics to assess statistical discrepancy, such as Q-Q plots or ˆ λ2discrepancy metric. • The analysis pipeline defined in Chapter 3 applied to the data from testbeds described in Section 2.3 generated the results discussed in Chapter 4. The pipeline helped to define two candidate models for OSIAT described in Section 4.4.4, both split distributions based respectively on the exponential and Weibull distributions. The OSPS values followed a simple discrete distribution described in Section 4.5.3. • The models proposed in Chapter 4 for OSIAT and OSPS were implemented in simulations based on ns-3, an open-source discrete-event network simulation framework. Simulation results confirmed the similarity between experimental network traffic from the testbeds and the traffic generated by he simulators, in both terms of bandwidth and packet rate, being the Weibull-based OSIAT model slightly closer to the original OWL network traffic. 6.2 Future work There are several lines of work that can be further developed as continuation of the present work. On the one hand, it is possible to further extend the analysis pipeline for modelling NVE network traffic proposed in Chapter 3. This pipeline only addresses network traffic which packets present randomly distributed IAT and PS values, leaving cases with strong autocorrelation out of the scope. Exploring the feasibility of applying time-series models to describe NVE traffic can be an valuable addition to the pipeline, and may help to identify traffic dynamics that are not fully explored in the current literature about NVE or video games network traffic. A second potential line of work is further work in metrics for statistical discrepancy. The present work partly relies on qualitative methods to assess this discrepancy, such as Q-Q plots, techniques that require the human expertise to be of any use. The metric ˆ λ2 has proven to be a promising quantitative metric but with some limitations about the information that can reveal and some weakness regarding its performance on situations like heavy tailed distributions. Thus, refining the measurement of statistical discrepancy of microscale network models can yield useful results for the modelling of network traffic. 100 Chapter 6 Conclusions and future work Finally, due to the nature of the ns-3 simulations proposed in Chapter 5 and some of their technical features, like the control of the pseudo-random seeds that control the execution of the simulation, it is worth to explore the possibility of adapting these processes to run on high-throughput computing (HTC) environments to exploit their potential parallelization. These concepts are further discussed in Appendix A. 6.2 Future work 101 A NVE simulations and HTC environments The present appendix deals with the particularities of the ns-3 simulations for networked virtual environment (NVE) network traffic presented in Chapter 5 and specifically the problems associated to software distribution and provisioning time and how they can be minimising by running these simulations in a high-throughput computing (HTC) environment assisted by modern software distribution services. The content in this appendix is based on the author’s experience as system administrator in the Distributed Data Processing Unit at SURFsara, the Dutch national high-performance computing and e-Science support center in Amsterdam, Netherlands. A.1 Running NVE simulations in HTC environments The NVE traffic simulations implemented and executed in Chapter 5 using the ns-3 framework were mono-core processes, meaning that the simulation was running in a single core, the default behaviour for ns-3 simulations unless developers decide to include parallelisation techniques. Moreover, the inclusion of arguments that allow to specify pseudo-random seeds to control the execution of the simulation (Section 5.2.2) allows to exactly reproduce runs of the simulation by providing the same seed or, more interesting from the point of view of distributing and parallelising the simulation runs, by executing multiple times the simulation making sure each run is different from the rest by providing a unique seed to each one. Single-core processes and the possibility to differentiate each run of the simulation make these simulations based on ns-3 a suitable candidate to be executed in HTC environments [LH13], computing environments were available resources work together to achieve the execution of long-running tasks taking advantage of parallelisation to achieve high degree of throughput in the results generated. It must be noted that while loosely related fields, HTC and high-performance computing (HPC) are different in many ways, while HPC tasks are characterized as needing large amounts 103 of computing power for short periods of time, the HTC is more interested in how many jobs can be completed over a period of time than in how fast those jobs are individually. Depending on the scale of available computing resources, the HTC environment can be a traditional cluster where all the machines are in the same geographical location, orchestrated by a job scheduler such as HTCondor [Faj+15] or SLURM ([YJG03]). In case where the amount of resources is far the reach of individual institutions, it is possible to scale out HTC infrastructure by using grid computing, which can be viewed as a federation of individual HTC cluster distributed around remote geographical locations, sharing resources to achieve common goals. This federation is accomplished by using a grid middleware software that orchestrates and distributes the workload all over the grid infrastructure. An example of grid middleware is gLite, used by the CERN LHC experiments and other scientific domains. It was implemented by collaborative efforts of more than 12 different academic and industrial research centers in Europe and it has been deployed in more than 250 institutions [And+08]. A.2 Software distribution in HTC environments One of the technical challenges to solve when computing in distributed environments is the distribution of data and the software to process them. A classical approach has been provisioning the worker nodes (nodes that perform the actual computation in the HTC infrastructure) with local copies of the software required to process the data. While straightforward, this approach has the disadvantage of increasing the complexity of maintaining the worker nodes. More software means more maintenance work for the operators of the infrastructure, who have to be up to date with security alerts, apply security patches and bug fixes, make sure updates do not break the system or cause regressions. Moreover, this means each worker node has to be prepared beforehand to run jobs that require some specific software, reducing the flexibility of the infrastructure to host new workloads. In the specific case of simulations based on ns-3, system administrations have two options when provisioning worker nodes, on the one hand they can install the software from source code, making sure the binaries they generate are compatible with the target system and all the dynamic libraries required by these binaries are present on each node. On the other hand, some GNU/Linux distributions (the most extended operating system in the HPC/HPC environments) provided packages for ns-3 in their official repositories. 104 Chapter A NVE simulations and HTC environments A more flexible approach for software distribution is making the job locally install the required software, using scripts that automate the download, configuration and compilation (if necessary) of the software. In the case of ns-3, in addition to the transfer time required to fetch the source code from the repositories of the ns-3 project or a local proxy (being the second option highly advised to avoid generating an accidental distributed denial-of-service (DDoS) attack by having hundreds of machines with high speed connections requesting copies of the repository), each job will have to spend a considerable amount of time compiling its instance of ns-3. The experimental measurements performed in this study show that compiling ns-3.29 in an machine equipped with an Intel i5-4690 CPU at 3.5 GHz takes up to 20 min and up to 1 GB RAM when using a single dedicated core at 100 % load. The reason to limit the compilation tests to one single core is because ns-3 simulations are single-core processes by default and they will likely be encapsulated into single-core HTC jobs to ensure an adequate resource assignment. In this approach, the ns-3 binaries generated cannot be directly reused between jobs, even if they run on the same worker node. A previous approach can be improved by distributing pre-compiled ns-3 binaries compatible with the software base of the worker nodes. Thus, jobs can fetch these binaries from a local repository within the cluster, or they can be shared through a network file system. Traditional network file systems such as NFS are an alternative to implement this approach, but there are other solutions specifically designed and optimised for the distribution of software in HPC and HTC environments, such as CernVM File System (CVMFS), a read-only POSIX network file system based on the HTTP protocol that can benefit from using HTTP caching proxies such as Squid [Prof] for better performance and provides authentication and authorisation based on X.509 standard for public key certificates [Agu+08] [Wei+17]. Distributing pre-compiled software from a repository or a shared network file system still has to deal with the problem of library dependencies in the target worker node where the job is going to run. Different versions of the same software may depend in different versions of the same dynamic library, which may not be trivial in many Linux/Unix systems. One traditional solution to this problem has been using tools that manage the shell environment to make this possible, such as Environment Modules [FO96]. While a widespread solution, it is not specially robust and require a good understanding of the library dependencies of the software and the shell environment where software will run. Recent years have witnessed the popularisation of container technologies as an efficient and platform-agnostic way to distribute and deploy software on heterogeneous systems. Containers encapsulate and isolate one or various processes from the rest of the host where they are running. This level of isolation is not as strong as in virtual A.2 Software distribution in HTC environments 105 machines, but enough to make the software within the container totally independent from the user space software of the host while being much more lightweight in resource terms than virtual machines. Thus, containers provide a true homogeneous runtime environment isolated from the particularities of the host, such as package manager, hardware devices or library versions, and a convenient way to distribute software in the form of images than can run on any GNU/Linux host with the required container runtime. While Docker is becoming a de-facto standard for containers [Ber14], it does not fully match the security requirements to run on grid computing environments. This has motivated the creation of Singularity, a container technology developed with federated, research and grid infrastructures in mind [KSB17] [ADS17]. Some of the features that make Singularity ideal for HTC and grid computing are running as nonprivileged user to minimise security risks, capability of directly using Docker images or converting them to Singularity Image Format (SIF), support for image registries and the possibility to distribute images as single files, making straightforward their execution on hosts supporting Singularity containers. A.3 Summary This chapter discussed the suitability of running the NVE simulations implemented in Chapter 5 on distributed HTC environments. These simulations can be encapsulated in HTC jobs and distributed across this kind of infrastructures. It is also discussed the technical challenge posed by software distribution in distributed environments. Making the software available on the worker nodes is key to make possible running the simulations anywhere on a distributed infrastructure. In the case of ns-3, the compilation time is specially time consuming, so different alternatives to achieve the goal “compile once, run many times” are described. The chapter also illustrates how the distribution of binaries using optimized network file systems such as CVMFS together with tools such as Environment Modules are giving way to more modern distribution channels like the container technologies. In this context, Singularity is becoming a standard in the academic and research context. 106 Chapter A NVE simulations and HTC environments B Testbed Appendix This appendix details the software tools and configuration used in Open Wonderland TestBed 1 (OWTB1) and Open Wonderland TestBed 2 (OWTB2), the experimental testbeds defined in Section 2.3. B.1 Traffic capture with Wireshark and tshark The packet capture, filtering and export to comma-separated value (CSV) has been performed using Wireshark [Tea19b], an open-source packet analyzer, and its command-line interface (CLI) counterpart tshark [Tea19a], more suitable for scripting and automation. Wireshark has been used to capture all the incoming and outgoing traffic. These raw packets has been stored in format pcap and later filtered and processed with both Wireshark and tshark. The object synchronisation inter-arrival time (OSIAT) traffic coming from a specific IP address can be filtered using Wireshark with a filter expression as follows: tcp contains "MOVE" && ip . src ==192.168.1.2 These kind of filters can be also used with tshark: tshark −r raw_traces . pcap \ −R " tcp contains "MOVE" && \ ip . src ==192.168.1.2 && \ frame . time_relative >= 20 && \ frame . time_relative <=220" \ −w f il te r ed _t r ac es . pcap The above tshark command filters all the OSIAT packets from the given .pcap file that have been originated in the client with IP 192.168.1.2, selecting only those captured after 20 s and no later than 220 s. The conversion from the pcap format, which contains most of the raw packet information, to plan-text CSV can be performed using Wireshark, by opening the pcap file, filtering the desired packets and proceeding to save the results with the menu option Export packet dissections, selecting CSV as output format. 107 C.2 Exponential-based OSIAT model for OWL Rimplementation of the exponential-based OSIAT model for OWL defined in Section 4.4.4, Equation (4.3). # ’ Cumulative P r o b a b i l i t y f un ct i on f o r Truncated E xpo ne nti al # ’ # ’ C a l c u l a t e s p r o b a b i l i t y f o r giv en v e c t o r x us in g th e # ’ tru nc ate d e xp one nci al d i s t r i b u t i o n adjusted f or # ’ OSIAT modelling . # ’ @param x v e c to r of va lu es which p r o b a b i l i t y w i l l be c a l cula t e d # ’ @param r at e parameter f or the e xp on en ti al s e c t i o n # ’ @param thr es ho ld d e lim i ti n g the truncate d d i s t r i b u t i o n s e c t i o n s # ’ @param acp , c o r r e c t s a c i t i v i t y −induced d e v i a t i o n s # ’ @return ve c t o r with the p r o b a b i l i t y va lu es #’ @export pexp . trun <−function(x , rate , thres =0.5, acp=1){ r<−sapply(x , pexp . trun . aux , rate=rate , thres=thres , acp=acp) return( r ) } # ’ A u x i l i a r y f u n c t i o n f o r pexp . tru n # ’ # ’ Th is f u nc t i o n implements the pexp . tru n l o g i c , a ll ow in g to # ’ use sap ply f or b e t t e r performance . pexp . trun . aux <−function(x , rate , thres =0.5, acp=1){ i f (x < thres ){ r<−pexp(x , rate ) *acp }else { r<−1 } return( r ) } # ’ Quantile f un ct io n fo r Truncated E xpon entia l d i s t r i b u t i o n # ’ # ’ @param q v e c t o r o f q u a n t i l e s to be c a l c u l a t e d # ’ @param e xp on en ti al ra te parameter f or the e xp on en ti al s e c t i o n # ’ @param thr es ho ld d e lim i ti n g the truncate d d i s t r i b u t i o n s e c t i o n s # ’ @param acp , c o r r e c t s a c i t i v i t y −induced d e v i a t i o n s # ’ @return v e c t o r with q u a nt i l e v a lu e s #’ @export qexp . trun <−function(q, rate , thres , acp ){ r<−sapply(q,qexp . trun . aux , rate=rate , thres=thres , acp=acp) return( r ) } # ’ A u x i l i a r y f u n c t i o n f o r qexp . trun # ’ # ’ Th is f u nc t i o n implements the qexp . trun l o g i c , a ll ow in g to # ’ use sap ply f or b e t t e r performance . qexp . trun . aux <−function(q, rate , thres , acp ){ 114 Chapter C R implementation of OWL network traffic models and discrepancy metrics i f (q< thres ){ y<−qexp(q/acp , rate ) }else { y<−thres } return(y) } # ’ Random number generato r f or Truncated E xpon entia l d i s t r i b u t i o n # ’ # ’ F unction u se s i n v e r s e t ra ns fo rm in g sampling f o r th e g en er at io n # ’ of random numbers using the Truncated Exp onen tial d i s t r i b u t i o n # ’ adapted f o r OSIAT . # ’ @param n of random numbers to ge generated # ’ @param e xp on en ti al ra te parameter f or the e xp on en ti al s e c t i o n # ’ @param thr es ho ld d e lim i ti n g the truncate d d i s t r i b u t i o n s e c t i o n s # ’ @param acp , c o r r e c t s a c i t i v i t y −induced d e v i a t i o n s # ’ @return v e c t o r with random v al ue s #’ @export rexp . trun <−function(x , rate , thres , acp ){ p<−runif(x) r<−qexp . trun (p , rate , thres , acp) return( r ) } C.3 Weibull-based OSIAT model for OWL Rimplementation of the Weibull-based OSIAT model for OWL defined in Section 4.4.4, Equation (4.4). # ’ Cumulative P r o b a b i l i t y Function f o r Truncated Weibull # ’ # ’ C a l c u l a t e s p r o b a b i l i t y f o r giv en v e c t o r x us in g th e # ’ tru nc ate d we ib ul l d i s t r i b u t i o n ad ju sted f or # ’ OSIAT modelling . # ’ @param x v e c to r of va lu es which p r o b a b i l i t y w i l l be c a l cula t e d # ’ @param shape parameter for Weibull d i s t r i b u t i o n s e c t io n # ’ @param s c a l e parameter f or Weibull d i s t r i b u t i o n s e c t io n # ’ @param thr es ho ld d e lim i ti n g the truncate d d i s t r i b u t i o n s e c t i o n s # ’ @param acp , c o r r e c t s a c i t i v i t y −induced d e v i a t i o n s # ’ @return ve c t o r with the p r o b a b i l i t y va lu es #’ @export pweibull . trun <−function(x , shape , scale , thres =0.5, acp=1){ r<−sapply(x , pweibull . trun . aux , shape=shape , scale=scale , thres=thres , acp=acp ) return( r ) } C.3 Weibull-based OSIAT model for OWL 115 # ’ A u x i l i a r y f u n c t i o n f o r p w ei bu l l . t ru n # ’ # ’ Th is f u nc t i o n implements the p we i bu l l . tr un l o g i c , al lo wi ng to # ’ use sap ply f or b e t t e r performance . pweibull . trun . aux <−function(x , shape , scale , thres =0.5, acp=1){ i f (x<thres ){ r<−pweibull (x , shape , scale )*acp }else { r<−1 } return( r ) } # ’ Quantile f un ct io n fo r Truncated Weibull d i s t r i b u t i o n # ’ # ’ @param q v e c t o r o f q u a n t i l e s to be c a l c u l a t e d # ’ @param shape Weibull shape parameter f or the Weibull s e c t i o n # ’ @param s c a l e Weibull s c a l e parameter f or the Weibull s e c t i o n # ’ @param thr es ho ld d e lim i ti n g the truncate d d i s t r i b u t i o n s e c t i o n s # ’ @param acp , c o r r e c t s a c i t i v i t y −induced d e v i a t i o n s # ’ @return v e c t o r with q u a nt i l e v a lu e s #’ @export qweibull . trun <−function(q, shape , scale , thres , acp ){ r<−sapply(q, qweibull . trun . aux , shape=shape , scale=scale , thres=thres , acp=acp ) return( r ) } # ’ A u x i l i a r y f u n c t i o n f o r q w ei b ul l . trun # ’ # ’ Th is f u nc t i o n implements the q we i bu ll . t ru n l o g i c , a ll ow in g to # ’ use sap ply f or b e t t e r performance . qweibull . trun . aux <−function(q, shape , scale , thres , acp){ i f (q< thres ){ y<−qweibull(q/acp , shape , scale ) }else { y<−thres } return(y) } # ’ Random number generato r f or Truncated Weibull d i s t r i b u t i o n # ’ # ’ F unction u se s i n v e r s e t ra ns fo rm in g sampling f o r th e g en er at io n # ’ of random numbers using the Truncated Weibull d i s t r i b u t i o n # ’ adapted f o r OSIAT . # ’ @param n of random numbers to ge generated # ’ @param shape Weibull shape parameter f or the Weibull s e c t i o n # ’ @param s c a l e Weibull s c a l e parameter f or the Weibull s e c t i o n # ’ @param thr es ho ld d e lim i ti n g the truncate d d i s t r i b u t i o n s e c t i o n s 116 Chapter C R implementation of OWL network traffic models and discrepancy metrics # ’ @param acp , c o r r e c t s a c i t i v i t y −induced d e v i a t i o n s # ’ @return v e c t o r with random v al ue s #’ @export rweibull . trun <−function(x , shape , scale , thres , acp ){ p<−runif(x) r<−qweibull . trun (p , shape , scale , thres , acp ) return( r ) } C.4 Statistical discrepancy metrics Rimplementation of the statistical discrepancy metrics λ2 and ˆ λ2 described in Section 3.8.2. This implementation is distribution agnostic, receiving as parameter a couple of data sets, functions representing the cumulative distribution function (CDF) of a function or any combination of the former two. The functions will proceed to calculate the statistical discrepancy between the data sets or distributions. # ’ C a l c u l a t e s [ hat ] lambda square GoF m et ri c f o r a g ive n e m pi r i ca l # ’ data s e t and Cumulative D i s t r i b u t i o n Function (CDF ) . # ’ # ’ The fu nc tio n measures the Goodness of F i t between the e mp iri ca l # ’ data and the proposed CDF . The parameters determine how the # ’ lambda c a l c u l a t i o n i s performed , as w el l as i nc l ud i ng the # ’ parameters th at determine the behaviour o f the g iven CDF . # ’ @param emp . data v e c t o r with e m pi r i ca l measurements # ’ @param t he . d i s t R f u n c t i o n mode ll in g th e CDF f o r th e # ’ t h e o r e t i c a l d i s t r i b u t i o n which GoF f or the emp . data # ’ w i l l be evalu at ed # ’ @param breaks d etermines the number of " breaks " or bin s to be # ’ used within the hat lambda square algorithm . −1 # ’ i n d i c a t e s the fu nc tio n w i l l c a l c u l a t e the optimal # ’ number of b i ns f o r t he g iv en p aramet ers # ’ @param mode sw itc hs between standard lambda square and hat # ’ lambda square m etr ic s # ’ @param . . . a r b i t r a r y l i s t o f para meters r e qu i re d by th e # ’ the . d i s t fun ction , i . e . exponen ti al rate , Weibull ’ s # ’ shape and shape # ’ @return t u p l e o f v a lu e s ( [ hat ] lambda square , var ) #’ @examples # ’ > x <−rexp (1000 , ra te =5) # ’ > lambda . square (x , pexp , −1, " hat " , r at e=5) # ’ [1] 0.9155332 3.5994222 #’ @export lambda . square <−function(emp. data , the . dist , breaks , mode, . . . ) { # c a l c u l a t e bin width [Pax94 ] min . aux <−min(emp. data ) max. aux <−max(emp. data ) i f (breaks == −1){ # optimal width acc or di ng to [ Bor00 ] and [ Pax94 ] w<−3.49 *sd (emp. data )*(length (emp. data))^(−1/3) C.4 Statistical discrepancy metrics 117 # number of b ins n . bins <−ceiling((max. aux −min . aux) /w) # f i x the bin width w.new <−(max. aux −min . aux) /n . bins bins <−seq (min . aux , max. aux , by=w.new) }else { w<−(max. aux −min . aux)/breaks bins <−seq (min . aux , max. aux , by=w) } # c a l c u l a t e s t a r t and end f o r bin bin . a <−bins[−length ( bins )] bin . b <−bins[−1] # c a l c u l a t e d ex p ec te d ( t h e o r e t i c a l ) v al ue s z<−the . d i s t ( bin . b , . . . ) −the . d ist ( bin . a , . . . ) # observed/e mpi ric al v al ue s y<−(hi st (emp. data , breaks=bins , plot=FALSE))$counts; i f (mode == " default " ) { i f (min(z) == 0) { y<−y[z !=0] z<−z[z !=0] } }else { i f (min(y) == 0){ y<−y[y!=0] z<−z[y !=0] } } # number o f o b s e r v a t i o n s and b in s n . obs <−sum(y) n . bins <−length (z) # number o f o b s e r v a t i o n s by t h e o r e t i c a l f r e q u e n c i e s # ex pe ct ed va lu es accor ding t h e o r e t i c a l d i s t r i b u t i o n npi <−n . obs *z ; i f (mode == " default " ) { x2 <−sum(( y −npi)^2 /npi ) K<−sum(( y −npi ) /npi ) D <−y−npi E<−npi }else { # t h i s i s hat lambda square x2 <−sum(( npi −y)^2 /y) K<−sum(( npi −y) /y) D <−npi −y E<−y } n. param <−nargs() −4 df <−n . bins −1−n.param lambda2 <−(x2 −K−df )/(n . obs−1) # lambda−square v aria nce T<−sum( ( D^3 −2*D*E + (5/2)*(D^2) + (3/2)*y) /(E^2) ) aux <−(2*df + 4*n . obs*lambda2 + 4*n . obs*lambda2^2 + 4*T) var . lambda2 <−aux /n . obs return(c( lambda2 , var . lambda2 )) } 118 Chapter C R implementation of OWL network traffic models and discrepancy metrics D Derivatives from the analysis of Open Wonderland network traffic D.1 ECDF and fitting CDFs for OSIAT The present section contains the empirical cumulative distribution function (ECDF) plots for the object synchronisation inter-arrival time (OSIAT) values measured in the testing sessions from testbeds Open Wonderland TestBed 1 (OWTB1) and Open Wonderland TestBed 2 (OWTB2). Each ECDF plot is accompanied by the cumulative distribution function (CDF) curves for the proposed exponential and Weibull-based OSIAT models (Equations (4.3) and (4.4) respectively). 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 ECDF for Object Sync. IAT, Client 01 Object Sync. IAT(s) F(x) 0.5002s ECDF Exp.T. rate=4.81 Wei.T. k=1.05 scale=0.21 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 ECDF for Object Sync. IAT, Client 02 Object Sync. IAT(s) F(x) 0.5021s ECDF Exp.T. rate=4.86 Wei.T. k=0.97 scale=0.20 Fig. D.1.: ECDFs OSIAT values, 2-player session, OWTB1 119 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 ECDF for Object Sync. IAT, Client 01 Object Sync. IAT(s) F(x) 0.5018s ECDF Exp.T. rate=4.57 Wei.T. k=1.05 scale=0.22 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 ECDF for Object Sync. IAT, Client 02 Object Sync. IAT(s) F(x) 0.5026s ECDF Exp.T. rate=4.76 Wei.T. k=0.98 scale=0.21 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 ECDF for Object Sync. IAT, Client 03 Object Sync. IAT(s) F(x) 0.5011s ECDF Exp.T. rate=3.77 Wei.T. k=1.19 scale=0.28 Fig. D.2.: ECDFs OSIAT values, 3-player session, OWTB1 120 Chapter D Derivatives from the analysis of Open Wonderland network traffic 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 ECDF for Object Sync. IAT, Client 01 Object Sync. IAT(s) F(x) 0.5013s ECDF Exp.T. rate=4.34 Wei.T. k=1.14 scale=0.24 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 ECDF for Object Sync. IAT, Client 02 Object Sync. IAT(s) F(x) 0.5020s ECDF Exp.T. rate=4.35 Wei.T. k=0.97 scale=0.23 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 ECDF for Object Sync. IAT, Client 03 Object Sync. IAT(s) F(x) 0.5013s ECDF Exp.T. rate=3.71 Wei.T. k=1.32 scale=0.29 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 ECDF for Object Sync. IAT, Client 04 Object Sync. IAT(s) F(x) 0.5015s ECDF Exp.T. rate=4.40 Wei.T. k=0.62 scale=0.19 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 ECDF for Object Sync. IAT, Client 05 Object Sync. IAT(s) F(x) 0.5010s ECDF Exp.T. rate=3.35 Wei.T. k=1.23 scale=0.31 Fig. D.3.: ECDFs OSIAT values, 5-player session, OWTB1 D.1 ECDF and fitting CDFs for OSIAT 121 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 ECDF for Object Sync. IAT, Client 01 Object Sync. IAT(s) F(x) 0.5010s ECDF Exp.T. rate=4.80 Wei.T. k=0.55 scale=0.15 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 ECDF for Object Sync. IAT, Client 02 Object Sync. IAT(s) F(x) 0.5010s ECDF Exp.T. rate=4.59 Wei.T. k=0.57 scale=0.16 Fig. D.4.: ECDF plots for OSIAT, 2-player session, OWTB2 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 ECDF for Object Sync. IAT, Client 01 Object Sync. IAT(s) F(x) 0.5010s ECDF Exp.T. rate=4.58 Wei.T. k=0.55 scale=0.16 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 ECDF for Object Sync. IAT, Client 02 Object Sync. IAT(s) F(x) 0.5010s ECDF Exp.T. rate=4.64 Wei.T. k=0.54 scale=0.15 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 ECDF for Object Sync. IAT, Client 03 Object Sync. IAT(s) F(x) 0.5010s ECDF Exp.T. rate=4.50 Wei.T. k=0.57 scale=0.17 Fig. D.5.: ECDF plots for OSIAT, 3-player session, OWTB2 122 Chapter D Derivatives from the analysis of Open Wonderland network traffic 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 ECDF for Object Sync. IAT, Client 01 Object Sync. IAT(s) F(x) 0.5010s ECDF Exp.T. rate=4.84 Wei.T. k=0.51 scale=0.14 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 ECDF for Object Sync. IAT, Client 02 Object Sync. IAT(s) F(x) 0.5010s ECDF Exp.T. rate=4.63 Wei.T. k=0.53 scale=0.15 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 ECDF for Object Sync. IAT, Client 03 Object Sync. IAT(s) F(x) 0.5010s ECDF Exp.T. rate=4.65 Wei.T. k=0.52 scale=0.15 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 ECDF for Object Sync. IAT, Client 04 Object Sync. IAT(s) F(x) 0.5010s ECDF Exp.T. rate=4.78 Wei.T. k=0.55 scale=0.15 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 ECDF for Object Sync. IAT, Client 05 Object Sync. IAT(s) F(x) 0.5010s ECDF Exp.T. rate=4.72 Wei.T. k=0.54 scale=0.15 Fig. D.6.: ECDF plots for OSIAT, 5-player session, OWTB2 D.1 ECDF and fitting CDFs for OSIAT 123 0 5 10 15 20 25 30 0.0 0.2 0.4 0.6 0.8 1.0 Lag ACF ACF for Object Sync. IAT, Client 01 0 5 10 15 20 25 30 35 −0.2 0.0 0.2 0.4 0.6 0.8 1.0 Lag ACF ACF for Object Sync. IAT, Client 02 0 5 10 15 20 25 30 35 −0.2 0.0 0.2 0.4 0.6 0.8 1.0 Lag ACF ACF for Object Sync. IAT, Client 03 0 5 10 15 20 25 30 35 −0.2 0.0 0.2 0.4 0.6 0.8 1.0 Lag ACF ACF for Object Sync. IAT, Client 04 0 5 10 15 20 25 30 35 −0.2 0.0 0.2 0.4 0.6 0.8 1.0 Lag ACF ACF for Object Sync. IAT, Client 05 0 5 10 15 20 25 30 35 −0.2 0.0 0.2 0.4 0.6 0.8 1.0 Lag ACF ACF for Object Sync. IAT, Client 06 0 5 10 15 20 25 30 35 −0.2 0.0 0.2 0.4 0.6 0.8 1.0 Lag ACF ACF for Object Sync. IAT, Client 07 0 5 10 15 20 25 30 35 −0.2 0.0 0.2 0.4 0.6 0.8 1.0 Lag ACF ACF for Object Sync. IAT, Client 08 Fig. D.15.: ACF for OSIAT, 8-player session, OWTB2 130 Chapter D Derivatives from the analysis of Open Wonderland network traffic 0 5 10 15 20 25 30 35 −0.2 0.0 0.2 0.4 0.6 0.8 1.0 Lag ACF ACF for Object Sync. IAT, Client 01 0 5 10 15 20 25 30 35 −0.2 0.0 0.2 0.4 0.6 0.8 1.0 Lag ACF ACF for Object Sync. IAT, Client 02 0 5 10 15 20 25 30 35 −0.2 0.0 0.2 0.4 0.6 0.8 1.0 Lag ACF ACF for Object Sync. IAT, Client 03 0 5 10 15 20 25 30 35 0.0 0.2 0.4 0.6 0.8 1.0 Lag ACF ACF for Object Sync. IAT, Client 04 0 5 10 15 20 25 30 35 −0.2 0.0 0.2 0.4 0.6 0.8 1.0 Lag ACF ACF for Object Sync. IAT, Client 05 0 5 10 15 20 25 30 35 −0.2 0.0 0.2 0.4 0.6 0.8 1.0 Lag ACF ACF for Object Sync. IAT, Client 06 0 5 10 15 20 25 30 35 −0.2 0.0 0.2 0.4 0.6 0.8 1.0 Lag ACF ACF for Object Sync. IAT, Client 07 0 5 10 15 20 25 30 35 −0.2 0.0 0.2 0.4 0.6 0.8 1.0 Lag ACF ACF for Object Sync. IAT, Client 08 0 5 10 15 20 25 30 35 −0.2 0.0 0.2 0.4 0.6 0.8 1.0 Lag ACF ACF for Object Sync. IAT, Client 09 0 5 10 15 20 25 30 35 −0.2 0.0 0.2 0.4 0.6 0.8 1.0 Lag ACF ACF for Object Sync. IAT, Client 10 Fig. D.16.: ACF for OSIAT, 10-player session, OWTB2 D.2 Autocorrelation functions for OSIAT 131 D.3 Tail quantiles for OSIAT The present section contains the Table D.1 showing the percentiles from Q95 to Q99 for the OSIAT values from OWTB2. Tab. D.1.: Quantiles for OSIAT, OWTB2 sessions Session Client Q0.95 Q0.96 Q0.97 Q0.98 Q0.99 1-client 1 0.5011 0.5011 0.5011 0.5011 0.5012 2-client 1 0.5011 0.5011 0.5011 0.5014 0.8761 2 0.5011 0.5011 0.5012 0.5015 0.8874 3-client 1 0.5011 0.5012 0.5014 0.8467 1.0020 2 0.5011 0.5012 0.5021 0.8570 1.0021 3 0.5012 0.5012 0.5039 0.7525 1.0020 4-client 1 0.5012 0.5012 0.6374 0.9036 1.0021 2 0.5012 0.5014 0.7634 1.0012 1.0021 3 0.5012 0.5012 0.6161 0.9030 1.0020 4 0.5012 0.5016 0.7210 1.0015 1.0021 5-client 1 0.5016 0.6682 0.8812 1.0020 1.0022 2 0.5012 0.6077 0.8323 1.0019 1.0021 3 0.5013 0.6620 0.8820 1.0020 1.0021 4 0.5013 0.6602 0.9151 1.0020 1.0021 5 0.5013 0.7945 0.9563 1.0020 1.0026 6-client 1 0.5012 0.5914 0.8489 1.0020 1.0021 2 0.5012 0.5828 0.8298 1.0018 1.0021 3 0.5012 0.6014 0.9441 1.0020 1.0022 4 0.5012 0.7122 0.9197 1.0019 1.0021 5 0.5012 0.5991 0.9102 1.0019 1.0022 6 0.5012 0.5044 0.8209 1.0015 1.0021 7-client 1 0.5013 0.6584 1.0014 1.0021 1.1838 2 0.5013 0.7243 0.9919 1.0021 1.2692 3 0.5029 0.7989 1.0020 1.0022 1.5029 4 0.5013 0.7990 1.0020 1.0022 1.5030 5 0.5013 0.6951 1.0019 1.0021 1.3579 6 0.5012 0.6285 0.9846 1.0020 1.3066 7 0.5017 0.7928 1.0010 1.0021 1.3921 132 Chapter D Derivatives from the analysis of Open Wonderland network traffic Tab. D.1.: Quantiles for OSIAT, OWTB2 sessions Session Client Q0.95 Q0.96 Q0.97 Q0.98 Q0.99 8-client 1 0.5013 0.6898 0.9661 1.0021 1.4502 2 0.5013 0.5842 0.9394 1.0020 1.2642 3 0.5016 0.7414 1.0017 1.0021 1.2715 4 0.5012 0.6075 0.9892 1.0020 1.2969 5 0.5013 0.6614 1.0018 1.0021 1.4325 6 0.5013 0.6814 0.9305 1.0020 1.4570 7 0.5014 0.8151 1.0017 1.0022 1.4040 8 0.5014 0.8001 1.0020 1.0022 1.3933 9-client 1 0.5013 0.7713 1.0009 1.0021 1.3308 2 0.5055 0.8814 1.0020 1.0085 1.4663 3 0.5013 0.7494 1.0020 1.0021 1.3609 4 0.5013 0.7735 1.0019 1.0023 1.3463 5 0.5074 0.8006 1.0019 1.0157 1.4069 6 0.5013 0.6734 1.0007 1.0021 1.3046 7 0.5013 0.6439 1.0019 1.0022 1.2888 8 0.5088 0.7630 1.0011 1.0020 1.1466 9 0.5013 0.6391 0.9940 1.0022 1.2887 10-client 1 0.7465 0.9848 1.0021 1.1544 1.5031 2 0.6953 1.0018 1.0021 1.2069 1.5029 3 0.5665 0.9171 1.0020 1.0625 1.5030 4 0.6937 0.9626 1.0021 1.0417 1.4494 5 0.5458 0.9747 1.0020 1.2116 1.5031 6 0.5480 0.9319 1.0020 1.0022 1.5030 7 0.6248 0.9014 1.0020 1.0584 1.4882 8 0.6353 0.9501 1.0021 1.1182 1.5030 9 0.6784 1.0009 1.0021 1.1058 1.5028 10 0.6236 0.9826 1.0020 1.0817 1.5030 D.3 Tail quantiles for OSIAT 133 D.4 Activity Correction Parameter for OSIAT models The present section contains the Tables D.2 and D.3 of Activity Correction Factor values ( Cacp , Section 4.4.4) calculated for each one of the experimental sessions from OWTB1 and OWTB2 respectively. Tab. D.2.: Cacp values for exponential and Weibull-based models, OWTB1 Session Client acpexp acpwei 2-client 1 0.95 0.94 2 0.96 0.97 3-client 1 0.94 0.93 2 0.95 0.95 3 0.83 0.81 5-client 1 0.91 0.90 2 0.92 0.92 3 0.84 0.81 4 0.87 0.91 5 0.79 0.78 Tab. D.3.: Cacp values for exponential and Weibull-base models, OWTB2 Session Client acpexp acpwei 1-client 1 0.82 0.87 2-client 1 0.82 0.87 2 0.80 0.85 3-client 1 0.79 0.84 2 0.79 0.84 3 0.78 0.83 4-client 1 0.79 0.84 2 0.79 0.84 3 0.79 0.84 4 0.79 0.84 5-client 1 0.79 0.84 2 0.77 0.82 3 0.78 0.83 4 0.79 0.84 5 0.79 0.83 134 Chapter D Derivatives from the analysis of Open Wonderland network traffic Tab. D.3.: Cacp values for exponential and Weibull-base models, OWTB2 Session Client acpexp acpwei 6-client 1 0.77 0.82 2 0.79 0.84 3 0.78 0.84 4 0.78 0.83 5 0.79 0.85 6 0.79 0.84 7-client 1 0.78 0.84 2 0.79 0.84 3 0.77 0.82 4 0.78 0.83 5 0.78 0.83 6 0.79 0.85 7 0.78 0.83 8-client 1 0.78 0.84 2 0.79 0.85 3 0.78 0.83 4 0.79 0.84 5 0.78 0.84 6 0.78 0.83 7 0.80 0.85 8 0.78 0.84 9-client 1 0.79 0.85 2 0.78 0.84 3 0.79 0.84 4 0.79 0.84 5 0.78 0.84 6 0.80 0.86 7 0.78 0.83 8 0.79 0.85 9 0.80 0.85 D.4 Activity Correction Parameter for OSIAT models 135 Tab. D.3.: Cacp values for exponential and Weibull-base models, OWTB2 Session Client acpexp acpwei 10-client 1 0.78 0.83 2 0.79 0.85 3 0.78 0.83 4 0.78 0.84 5 0.77 0.83 6 0.77 0.83 7 0.78 0.84 8 0.78 0.84 9 0.78 0.84 10 0.77 0.83 D.5 MLE parameters for OSIAT models The present section contains the Tables D.4 and D.6, which contain the values for λ rate for the exponential-based distributions calculated for OSIAT using maximum likelihood estimation (MLE) on the traffic captured in OWTB1 and OWTB2 respectively. Tables D.5 and D.7 contain the values for λ scale and κ shape for the Weibull-based distributions calculated using MLE on the traffic captured in OWTB1 and OWTB2 respectively. Tab. D.4.: MLE exponential, λrate for OWTB1 sessions Session Client λ σ loglik 2-client 1 4.36 0.09 1165.42 2 4.52 0.09 1336.87 3-client 1 3.83 0.08 881.08 2 4.50 0.08 1582.61 3 2.99 0.07 198.52 5-client 1 3.74 0.07 898.33 2 3.93 0.07 1092.78 3 2.68 0.06 -28.49 4 3.15 0.06 360.67 5 2.57 0.06 -111.05 136 Chapter D Derivatives from the analysis of Open Wonderland network traffic Tab. D.5.: MLE Weibull, κshape and λscale for OWTB1 sessions Session Client κ σκλ σλloglik 2-client 1 0.94 0.01 0.22 0.00 1174.80 2 0.89 0.01 0.21 0.00 1367.35 3-client 1 0.86 0.01 0.24 0.01 950.09 2 0.96 0.01 0.22 0.00 1587.37 3 0.89 0.01 0.31 0.01 236.05 5-client 1 0.95 0.01 0.26 0.01 907.20 2 0.88 0.01 0.24 0.01 1135.38 3 0.87 0.01 0.34 0.01 27.47 4 0.59 0.01 0.21 0.01 1194.90 5 0.85 0.01 0.35 0.01 -30.30 Tab. D.6.: MLE exponential λrate for OWTB2 sessions Session C λ σ loglik 1-client 1 3.90 0.07 1015.6 2-client 1 4.49 0.08 1624.5 2 4.33 0.08 1456.2 3-client 1 4.19 0.08 1305.9 2 4.26 0.08 1379.5 3 4.15 0.08 1265.6 4-client 1 4.20 0.08 1315.3 2 4.13 0.08 1240.8 3 4.26 0.08 1379.5 4 4.21 0.08 1324.7 5-client 1 4.06 0.08 1174.4 2 4.05 0.08 1164.2 3 4.04 0.07 1148.5 4 4.03 0.07 1145.0 5 4.01 0.07 1116.9 6-client 1 4.03 0.07 1140.3 2 4.18 0.08 1288.0 3 4.13 0.08 1245.4 4 4.06 0.08 1174.0 5 4.14 0.08 1252.1 6 4.17 0.08 1287.7 D.5 MLE parameters for OSIAT models 137 Tab. D.6.: MLE exponential λrate for OWTB2 sessions Session C λ σ loglik 7-client 1 4.05 0.08 1158.6 2 4.09 0.08 1196.5 3 3.83 0.07 947.4 4 3.84 0.07 949.3 5 3.91 0.07 1022.3 6 4.08 0.08 1186.4 7 3.99 0.07 1103.5 8-client 1 3.35 0.07 501.34 2 4.08 0.08 1190.84 3 3.99 0.07 1101.46 4 4.02 0.07 1131.93 5 4.02 0.07 1134.74 6 3.97 0.07 1084.86 7 4.10 0.08 1211.33 8 3.93 0.07 1038.27 9-client 1 4.09 0.08 1202.88 2 3.94 0.07 1053.50 3 4.04 0.07 1148.73 4 3.99 0.07 1097.91 5 3.95 0.07 1063.74 6 4.13 0.08 1246.69 7 3.96 0.07 1067.68 8 4.04 0.07 1153.53 9 4.12 0.08 1232.84 10-client 1 3.78 0.07 895.47 2 4.02 0.07 1134.87 3 3.85 0.07 959.81 4 3.73 0.07 852.60 5 3.76 0.07 876.66 6 3.86 0.07 977.42 7 3.92 0.07 1037.01 8 3.90 0.07 1016.04 9 3.87 0.07 983.40 10 3.83 0.07 940.43 138 Chapter D Derivatives from the analysis of Open Wonderland network traffic Tab. D.7.: MLE Weibull, κshape and λscale for OWTB2 sessions Session Client κ σκλ σλloglik 1-client 1 0.55 0.01 0.16 0.01 2230.3 2-client 1 0.54 0.01 0.16 0.01 2635.9 2 0.57 0.01 0.17 0.01 2282.2 3-client 1 0.55 0.01 0.17 0.01 2219.7 2 0.54 0.01 0.17 0.01 2358.7 3 0.57 0.01 0.18 0.01 2042.2 4-client 1 0.53 0.01 0.16 0.01 2377.6 2 0.55 0.01 0.17 0.01 2131.0 3 0.55 0.01 0.17 0.01 2308.2 4 0.54 0.01 0.17 0.01 2304.7 5-client 1 0.51 0.01 0.16 0.01 2330.0 2 0.53 0.01 0.17 0.01 2172.5 3 0.52 0.01 0.17 0.01 2198.2 4 0.54 0.01 0.17 0.01 2111.8 5 0.54 0.01 0.18 0.01 2046.2 6-client 1 0.53 0.01 0.17 0.01 2090.9 2 0.52 0.01 0.16 0.01 2398.9 3 0.50 0.01 0.16 0.01 2437.4 4 0.54 0.01 0.17 0.01 2149.0 5 0.51 0.01 0.16 0.01 2408.7 6 0.53 0.01 0.17 0.01 2293.5 7-client 1 0.50 0.01 0.16 0.01 2333.1 2 0.50 0.01 0.16 0.01 2434.3 3 0.50 0.01 0.17 0.01 2045.8 4 0.51 0.01 0.17 0.01 2064.8 5 0.50 0.01 0.17 0.01 2171.2 6 0.48 0.01 0.16 0.01 2543.8 7 0.50 0.01 0.16 0.01 2275.6 8-client 1 0.48 0.01 0.16 0.01 1953.6 2 0.49 0.01 0.16 0.01 2446.1 3 0.49 0.01 0.16 0.01 2362.2 4 0.52 0.01 0.17 0.01 2239.8 5 0.49 0.01 0.16 0.01 2393.7 6 0.49 0.01 0.16 0.01 2301.6 7 0.49 0.01 0.16 0.01 2535.7 8 0.51 0.01 0.17 0.01 2137.5 D.5 MLE parameters for OSIAT models 139