Study and development of navigation techniques for virtual reality

Bachelor thesis Degree in Informatics Engineering Specialization in Computing Study and development of navigation techniques for virtual reality Author: Sergi ´ Avila Sang¨ uesa Thesis supervisor Nuria Pelechano Gomez (Computer Science) Thesis co-supervisor Alejandro R ´ ıos Jerez (Computer Science) Defense date: 28/1/2020 Contents 1 Introduction 4 1.1 VirtualReality ............................ 4 1.2 Context and main goal . . . . . . . . . . . . . . . . . . . . . . . . 5 1.3 Projectstructure........................... 6 1.4 Software,hardware and environment . . . . . . . . . . . . . . . . 6 2 Background and related work 10 2.1 Virtual camera movement methods . . . . . . . . . . . . . . . . . 10 2.2 WiPtechniques............................ 11 3 Improving the original project 14 3.1 Understanding the base paper . . . . . . . . . . . . . . . . . . . . 14 3.2 Changesmade ............................ 14 4 Creating new WiP modes 19 4.1 Need for slow and fast WiP . . . . . . . . . . . . . . . . . . . . . 19 4.2 The state machine and new mapping function . . . . . . . . . . . 21 5 Designing a training stage 28 5.1 Project state so far . . . . . . . . . . . . . . . . . . . . . . . . . . 28 5.2 Training stage development . . . . . . . . . . . . . . . . . . . . . 28 5.3 Finaltrainingstage.......................... 33 6 A new WiP method 35 6.1 Step frequency and feet movement . . . . . . . . . . . . . . . . . 35 6.2 Collecting people feet data . . . . . . . . . . . . . . . . . . . . . . 36 6.2.1 Experiment method . . . . . . . . . . . . . . . . . . . . . 36 6.2.2 Initial result discussion . . . . . . . . . . . . . . . . . . . . 40 7 Project management 45 7.1 Methodology ............................. 45 7.2 Initial Planning and budget . . . . . . . . . . . . . . . . . . . . . 45 7.2.1 Task definition . . . . . . . . . . . . . . . . . . . . . . . . 45 7.2.2 Tasksummary ........................ 48 1 7.2.3 Budget Table and explanation . . . . . . . . . . . . . . . 49 7.2.4 Budget control . . . . . . . . . . . . . . . . . . . . . . . . 50 7.3 Changes in planning . . . . . . . . . . . . . . . . . . . . . . . . . 51 8 Sustainability report 52 8.1 Survey auto evaluation . . . . . . . . . . . . . . . . . . . . . . . . 52 8.2 EconomicDimension......................... 52 8.2.1 Economic cost of the project . . . . . . . . . . . . . . . . 52 8.3 Comparison to other solutions . . . . . . . . . . . . . . . . . . . . 52 8.4 Environmental Dimension . . . . . . . . . . . . . . . . . . . . . . 53 8.4.1 Environmental cost of the project . . . . . . . . . . . . . . 53 8.4.2 Comparison to other solutions . . . . . . . . . . . . . . . 53 8.5 SocialDimension........................... 54 8.5.1 Personal growth . . . . . . . . . . . . . . . . . . . . . . . 54 8.5.2 Impact of our project . . . . . . . . . . . . . . . . . . . . 54 9 Conclusions 56 10 Future work 56 2 Resum La navegaci´o en un entorn virtual ´es actualment una de les tasques m´es complicades. Real walk (caminar natural) ´es la millor interf´ıcie, per`o nom´es ´es possible si l’entorn virtual i l’espai f´ısic son de mides semblants. La majoria de sistemes amb navegaci´o per grans escenaris provoquen marejos o desorientaci´o. Walk in Place (caminar al lloc) ´es una t`ecnica en qu`e l’usuari es mou de forma semblant al caminar natural per`o sense avan¸car f´ısicament, de forma que es redueix el mareig. Quan el moviment generat pel sistema no ´es mateix que l’usuari espera, l’usuari es pot frustrar o marejar. Aquest document comen¸ca mostrant millores sobre un sistema WiP ja existent. Tot seguit mostra una expansi´o als sistemes actual permetent un moviment r`apid, anomenat Run in Place (c´orrer al lloc) i un lent, de forma que es permet interactuar amb l’entorn de noves formes. Despr´es es descriu una fase d’entrenament per tal d’obtenir informaci´o especifica de cada usuari que serveix al sistema per adaptar-se als moviments de l’usuari i poder generar una velocitat de sortida millor ajustada a cada persona. Finalment, s’explica un experiment fet amb diversos usuaris per obtenir informaci´o del moviment dels peus durant WiP amb diferents velocitats de c`amera, creant les bases per desenvolupar un nou m`etode de WiP. Abstract In Virtual Reality navigating through an environment is currently one of the most challenging problems. Real Walking is the preferred interface, but it is only feasible when the virtual environment is similar to the physical space available. Most systems that allow movement in large scenes cause either motion sickness or disorientation. Walk in Place (WiP) is a technique that reduces motion sickness in a virtual environment by introducing user gestures similar to real walk but that does not result in a physical displacement. When the system generated forward movement does not match the user’s expected output, the user can become frustrated or feel dizziness. This document first presents improvements over an existing WiP system. Then presents an expansion to current WiP system by creating a system that allows a fast movement called Running in Place and a slow WiP movement, allowing for different ways to interact with the environment. After that the document explains the design of a training stage that uses user-specific data to prepare the system for the user’s inputs in order to better match the expected virtual speeds. It finally lays the groundwork to developing a new WiP method by showing a experiment with different users to collect feet data from performing WiP at different speeds Paraules clau realitat virtual, caminar, c´orrer, navegaci´o, entorn virtual Keywords VR, virtual reality, WiP, walk in place, run in place, walk, run, navigation, virtual environment 3 1 Introduction 1.1 Virtual Reality Virtual reality, or VR for short, allows the user to interact with a 3D computer generated environment. VR offers an immersive or semi-immersive experience as opposed to simply having a screen monitor and interacting with keyboard, mouse or joystick. VR usually attempts to simulate as many senses as possible with the intention to make the user feel as if he/she were in a real environment. In contrast to Augmented Reality, which overlaps information or graphics with the real world, in VR everything the user experiences and sees is computer generated. Currently, the most popular equipment for VR setups is a Head Mounted Display (HMD). This device is worn on the head, similar to a helmet, and has a screen located right in front of the user’s eyes. The screen is separated in two halves, one for each eye. Each one showing the virtual environment from a slightly different viewpoint, creating the illusion of 3D. Other common devices used alongside the HMD include headphones, that may be integrated in the HMD itself, and controllers, usually one for each hand so that the user can interact with the surroundings using each hand independently in a natural manner. Figure 1: HTC vive HMD and trackers. Images from the VIVE product webpage [HTC] VR can be applied to a large range of fields. For most people, its main use is entertainment, as it can be used for video games. In medicine and others fields of research it can be used for visualising things that are hard to imagine as a 3D object unless you actually see them, such as the organs of the human body. When you need to build something, for example a car or a building, VR can also help to see how the final product will look like and allows for changes to be made without spending resources on actually building it. Navigating in VR is probably the most challenging problem when it comes to natural interaction in VR. This is mostly because many navigation methods cause motion sickness. For example, techniques that use some kind of joystick or controller to move the camera, while the user’s body remains static, can cause motion sickness due to the inconsistencies between what the user eyes are seeing, the optical flow, and the user proprioceptive system which is aware of the lack 4 of movement. This problem can be lessen by having the user physically moving, ideally with normal walk, but this is not always possible because the virtual world can be much larger than the available physical space. In Walk in Place techniques, WiP, the user moves the legs up and down in order to simulate walking while not actually changing position. WiP techniques manage to provide the proprioceptive awareness of movement while not needing a large physical space. However if the mapping between leg movement and camera movement is not well adjusted, then some users will still experience this undesirable effect. Figure 2: Applications of VR in the medical field and in the assembly process of a car. Images from the ViRVIG webpage [ViR] 1.2 Context and main goal This project has been developed in the ViRVIG Research Center for Visualization, Virtual Reality and Graphics Interaction, a research center hosted at UPC that works very closely with the FIB, teaching most of the graphics courses in both the bachelor and the masters. The project started as an extension of the work shown in the paper Smooth transitioning between two walking metaphors for Virtual Reality applications written by professors Alejandro R´ıos and Nuria Pelechano and Isadora Salvetti, a master student [SRP19].1The paper describes a technique to switch between walking normally around a room with the HMD on, and Walking in Place in larger environments. This project focused on improving the WiP part. Walking in place is very different to performing a normal walk, since the feet move up and down instead of up-forward-down. Therefore, finding a good mapping between the up/down movement of the feet and the movement of the virtual camera is not trivial. Any inconsistencies between what the user expects to observe and how the camera moves, can cause the user to suffer from motion sickness. Thus the main goal of the project is to research methods to adjust the camera movement to the natural movement of each user in order to improve the immersion and to reduce motion sickness. In order to adjust the virtual camera movement to each person we decided to 1Since the paper will be referenced multiple times throughout this document and also for better readability, it will be referred to as ”the base paper” from this point on. 5 study the biomechanics of our gait cycle (e.g. the speed and frequency of our feet when walking). With this new method, we want to allow anyone who wants to develop VR application involving large scenes, to be able to have a navigation method that feels more natural and provides correct proprioceptive cues. 1.3 Project structure The work done in this project can be divided in 4 separate stages. All of them added new research or functionalities to the previous stages, but they are different enough to deserve a different section in this document. In the first stage Improving the original project the goal was to make small improvements to the original Unity project so that it better reflected the method shown in the base paper. Some internal usability tests with other ViRVIG members were also performed in order to decide what parts of the project were more interesting to research next. The second stage Creating new WiP modes consisted on allowing to Walk in Place at different paces with a virtual flow that matched the user’s expectations. Little research has been conducted on slow and fast WiP, which we will call Run in Place or RiP 2, and the base project did not perform well when trying to move at different paces. This stage required to visualize and analyze the data of our gait cycle in order to create a state machine that detected how the user was walking and to create a new function to map the user movement when Running in Place. While gait data was used to develop slow WiP and RiP during this stage, by the end of it the computer was still not able to adapt the movement to each user’s specific natural movement. In the third stage Designing a training stage we designed a training stage that adjusts the parameters that decide if the user is performing WiP or RiP. The stage required several tests to decide what parameters to adjust, what biomechanical information was required and how to extract it. In the final project the user first undergoes the training stage and then starts navigating the virtual environment with the adjusted virtual camera movements. Finally we conducted an experiment to extract feet data from users Walking in Place with different optical flows. The experiments will serve as the groundwork to develop a new WiP method. 1.4 Software,hardware and environment To develop the project, the normal version of the HTC VIVE was used, alongside the bases and the original 2017 version of the trackers. The pc where all the project was developed had and Intel Core i7-4970k CPU, NVIDIA GeForce GTX 970 gpu and a 16GB RAM. PC specifications are important for this project since VR devices have high PC requirements. 2During the rest of the document, we will use the terms ”WiP” and ”RiP” when talking about the technique itself, and ”Walk in Place” and ”Run in Place” when referring to the movement performed by the user in the techniques. 6 Software-wise, the main program we used is Unity game engine [Unia]. Current game engines offer a set of tools designed to allow users to easily develop any graphical application, including games and VR environment. The tools allow the users to abstract themselves from the code required to do a lot of the most common tasks when working with graphics by providing a simple graphical interface. Some of the tasks that are easy to include in a project are: defining the geometry in a scene, materials, movement, position and properties of lights, and defining interactions between objects. Thanks to all this tools that Unity provides, the programmer can focus exclusively in developing new algorithms and implementing other aspects of the project that are not yet included in the game engine. Unity, as all other engines, provides a development environment in which users can control all aspects of the project from a graphical interface. It also provides a scripting API that allows programmers to write scripts to better define behaviours and interaction between objects. Unity requires the user to program in C#, an object-oriented programming language strongly influenced by Java and C++ and developed by Microsoft in the year 2000. Finally we also needed to use Steam and Steam VR[Val]. Steam is a digital video game distribution storefront, created by Valve, developers of the HTC Vive. Steam VR is the program needed to use the Vive’s HMD and other devices. It controls their state, behaviour and provides programmers with an API to interact with the devices Figure 3: Unity environment with the project opened The feet movement was captured by the bases using Vive’s trackers attached to the feet. For most of the project, the space available to perform the tests was relatively small, with a distance between the 2 bases of around 5 meters, and a usable area space of 3.5m x 1.7m, limited by walls and tables, figure 4. The space allowed us to record our movements when walking normally, but it was very small in terms of gathering usable data for analysis, because we needed 7 to discard the acceleration and deceleration phase of a normal walk. Therefore, this limitation left us with only a handful of steps. Those steps, while they were good enough for most internal testing, were too small to perform the final experiment. Therefore in order to carry out the experiment and also gather important recordings from the training stage, we moved the capture area to a larger space. In this second setup of figure 5 the space between the bases is 7 meters and the usable area 6m x 2m. This larger space allowed us to capture better and more accurate recordings. The only problem during the development of this project, is that this larger room was not always available, since it is shared with other student projects and it is were the ViRVIG shows most of their projects to visitors. 8 that by the time we had reached a robust software solution, a new hardware version without said problems would probably already be on sale. A small but important change introduced during this stage was multiplying the final camera speed. In [NSN14] it is argued that virtual camera speed in a treadmill or WiP system need to be multiplied by a gain factor because we percieve virtual speeds as slower than they really are. Using an implementation of the WiP from [WWB10] results showed that a WiP system requires a gain factor of around 1.8 for virtual speeds to feel natural. After some tests we determined that in our system a gain factor of 1.5 was enough. In all the charts of this document that show virtual camera speeds, the value shown is already multiplied by the gain factor. In order to further smooth the user experience, we then made some changes to make sure the system achieved similar goals as LLCM-WIP [FWW08]. The goals that our base system did not satisfy when this TFG started were: •1-Low latency (when stopping) Shown in figure 9 •4Incorporation of real-world turning and short-distance maneuvering into virtual locomotion. Shown in figure 10 15 Figure 9: Example of the high latency when stopping. After 3 steps the user stops completely but the virtual camera keeps moving for 1 second (waiting for a possible next step). The camera movement while the user has stopped is very likely to cause motion sickness Figure 10: Example of not allowing small movements. Even very small feet movement cause a small camera movement. While the camera movement seems insignificant, the user feels that there is a movement even when the motions performed were not meant to generate any. The high stopping latency was fixed by implementing a linear equation to slow down when no movement has been detected for a certain period of time instead of keeping a constant movement, while maintaining the same total time before virtual camera speed reaches 0 (figure 11). The short distance maneuvering was allowed by introducing a height threshold of 2cm, value extracted from 16 [WWB10] (figure 12). If the feet do not move above the threshold, the camera does not move, and thus some maneuvering and small movements are allowed without affecting the visual flow. Another improvement that we noticed was needed, was to make the transition from WiP to normal walk more responsive. The base method would switch between WiP and normal walk, based exclusively on the feet movement. However, in reality, for this particular movement the head is translated because the feet have moved along the horizontal plane (one feet is separated from the other). Therefore, we can extend the previous method and include also feet translation. The new equation to determine the walking state is shown in equation 1. In this new equation we have added an additional condition to change the state based on the feet distance over the horizontal plane: S=NW if pH t−pH t−1>(δ+)or kpL t−pR tk> dmax WIP if pH t−pH t−1<(δ−)(3) Where pL tand pR tare the Left and Right foot position projection on the horizontal plane, kpL t−pR tkis the distance between the feet, and dmax is a distance threshold that we set to 0.4m (we empirically found that when users perform WiP, this distance never goes beyond that value). The slow down stage of WiP is an important and troublesome part of mapping feet movement to camera speeds. In normal walk, we do not halt instantly, we have to first slow down. In WiP however, there is no slow down, we finish a step and simply do not move again. Therefore we have to decide how much time to wait when both feet are on the floor before starting to halt, since maybe the user has not stopped, and is going to move the other foot next. If we wait too little the movement does not feel smooth since the camera speed changes too much (e.g. it could drop velocity drastically between consecutive steps), if we wait too long, the system feels unresponsive (e.g.the camera keeps moving forward when the user wants to stop, making the user lean forward unconsciously). Both cases are potential sources of motion sickness. Finding a good trade-off between latency and responsiveness is a problem that will persist in the next stages of the project. 17 Figure 11: Fixed stopping latency. After 0.3 seconds of no feet movement the camera reduces the speed and starts slowing down. The time it takes to completely halt remains 1 second, but the slow down makes it feel more natural. Figure 12: After adding a height threshold, small movements do not move the virtual camera. 18 4 Creating new WiP modes 4.1 Need for slow and fast WiP While researching for the creation of the base paper, it was found that current WiP systems may not be sufficient for all situations that require movement. A clear example is putting the user in an emergency situation like a fire in a building and telling him/her to escape using a WiP interface. In real life, a person would run or walk quickly in a significantly different way compared to how he/she was moving before, and so in a WiP system the user should also be able to Run in Place. If the user keeps just walking in place, the sense of urgency is likely to be lost, thus reducing immersion and presence. Slow WiP may also be needed for some situations, for example walking in a crowd or in an environment with a lot of elements that the user may want to look at or do some maneuvering to avoid. In that situation the user does not want to completely stop, since it would delay completing main goal, but it cannot move too fast either since it could lead to collisions. In the Future Work section of [WWB10] it was stated that: “GUD WIP – as with most WIP systems – only accepts Walking-In-Place inputs. If a user’s gestures indicate running, our GUD WIP implementation prints an error to the screen. In our experience achieving consistent in-place running is considerably more difficult than consistent in-place walking – more so than the apparent differences between Real Running and Walking would suggest. However, VE users almost certainly wish to run through virtual environments. ‘Running-In-Place’ systems are very researchworthy.” Their statement further reinforces the idea that investigating different WiP modes is interesting and can allow for more WiP applications to be developed. First of all, we tried to Walk in Place slowly and fast in our system to see how well it was able to handle an input that it was not designed for. The results in figures 13 and 14 show that indeed, our base system also did not respond well to attempting to run or walk slowly in place, and thus studying the differences between walking in place at different paces is a good research topic. 19 Figure 13: Attempt to Run in Place in the original project. While the optical flow is faster than when walking in place, it is still much slower than the output we expect when running in place. The result is a confusing experience where we see movement, but not the movement we expect. In the recording of this data sample, the virtual camera speed caused irregular feet speed and frequency due to the user trying to keep running in place. The virtual camera did not match the user’s expectations. Figure 14: Attempt to walk in place slowly in the original project. The virtual camera slows down significantly during feet change due to the user staying with both feet on the ground for a longer period of time. The change from moving one feet up to moving it down also takes more time than the system expects so it also reduces the speed. The user experiences very sharp changes in the camera speed, which tend to produce motion sickness. 20 Figure 15: Normal WiP feet and virtual camera speed. 4.2 The state machine and new mapping function After seeing that the WiP system could not handle fast and slow WiP there were two ways to continue: either to change the mapping function from the feet movement to the virtual camera speed in order to better adapt to the user’s variety of movements or to create a state machine with 3 states: slow, normal and fast each with a different mapping function. The original mapping function used feet speed as the only biomechanical information to determine the speed magnitude. This way of calculating the virtual camera speed had two problems when handling Running in Place: 1. Feet speed when performing a Run in Place is not significantly higher than when doing a Walk in Place 2. The combination of the room setup, the Vive’s bases precision and tracker’s wobbling caused a significant number of incorrect readings during Run in Place as shown in figure 16 The reason behind all the noise that appears in the tracker, is that the VIVE trackers combine two technologies to determine the position of trackers. Firstly, the bases (light houses) send a moving infrared beam which is detected by each tracker and used to calculate its position in global coordinates. Secondly the trackers use inertia sensors to update their position at a higher rate that infrared updates. Inertia sensors are very sensitive to fast movements and to the strong contact forces that appear after stomping the feet on the floor, which is what causes the errors in the Y values. While feet speed alone could not be used in RiP, we noticed that feet frequency did change a lot compared to WiP . If we compare the Run in Place and the 21 Figure 16: Feet position when running in place and trackers position when feet are on the floor. If all readings were correct, the minimum height should only be slightly below resting position due to wobbling, however values go far beyond that point and sometimes even show negative values despite the floor being calibrated as height 0. Walk in Place of figure 17 we can see that normal Walking in Place shows some resting time between steps, however fast Running in Place shows little to no resting. After some further testing we concluded that we could reliably measure step frequency by adding a filter to discard false steps produced by noise and if we only used step frequency in RiP, feet position readings below the resting height would not affect the speed. Thus this is the state machine that we developed based on frequency: WiP start RiP flast > fmax flast < fmin Figure 18 where flast is the frequency calculated from the last step, flast is the inverse of time since the last step happened (1/tlast) and fmax and fmin are thresholds to change the state. Therefore the transition from WiP to RiP can take place every time the user completes a step, but the transition from RiP to WiP will happen mid-step to avoid the need to wait for a whole slow step. The state machine is simple and its performance relies on how accurately the thresholds are adjusted to each user, since it was already designed with a future training stage in mind. The goal of the training is to adjust the values of the thresholds to each user style of walk and run in place. 22 Figure 17: Comparison between WiP and RiP feet movements To the best of our knowledge, a RiP system has not been developed yet. Thus we have to create a new feet movement to camera speed function for RiP, and use only step frequency since, as previously argued, it is the most reliable biomechanical parameter we are able to use. Since our ultimate goal is to mimic real world speeds as closely as possible, we will first define what ”real world running speeds” are. We want to allow the creation of applications where the user can run for a long period of time and not just a few seconds. Therefore we will first determine the speed range of an adult running at an aerobic intensity, which can be performed for an extended period of time. An adult person running with aerobic intensity in a flat terrain will run at a speed of around 2.2m/s (8km/h) to 5.6m/s (20km/h) with the first being a very light pace, and the second being a very intense pace that only professional athletes can keep for extended periods of time. We don’t expect a user to Run in Place at the equivalent pace of running at 5.6m/s, but the value will serve as an upper bound. Thus our RiP system will be designed to output speeds of 2.2m/s when stepping at relatively low frequencies and output 5.6m/s when stepping at very intense frequencies. After some preliminary tests we determined that a good match to speeds of 2.2m/s is a step frequency of 1.5s−1and an intense and fast stepping was of around 6s−1, which will be the match for 5.6m/s. Finally, we have to decide how to match the values in the middle. Literature suggests that the speed of real walk is a square root function of the energy spent on the motion [Bob60]. In the article, for some people step frequency was also measured, which showed a logarithmic relation with speed. During RiP, the way to increase speed is to increase step frequency, and thus we believe that using a logarithmic function will be a better match to real world movement. The final function used in Run in Place is: vt= 3.156 ln (1.338 flast) (4) 3.156 and 1.338 were obtained by calculating a logarithmic function with speed = 2.2 when frequency = 1.5 and speed = 5.6 when frequency = 6 23 During RiP, the camera speed only updates after a full step has been completed. In WiP mid-step adjustments were necessary. However since RiP mode only activates when step frequency is already high, the latency is already low and there is no need for further adjustments during the steep Figure 19: Run in Place Function. 24 Figure 26: Different users syncing their feet to the virtual training camera in different ways, the first user syncs the step end to the camera’s lowest speed while the second user syncs to the camera’s highest speed. The vertical lines indicate the synchronization points 31 Figure 27: Running takes some time to reach a constant speed. The chart only shows an acceleration and deceleration and thus we can not extract useful information from it regarding constant run speed. When replicating the natural movement of a participant (e.g. without such noticeable speed changes) as the input camera speed, no user synced their steps to the steps of the movement. We believe this is due to the fact that even if we can see patterns in the movement, natural walk speeds are very close to being constant. Figures 24 and 25 show this: users could feel a change from 1.2m/s to 0.4m/s but not from 1.2m/s to 1.1m/s. Therefore, the movement felt like it had a constant speed and we decided to drop the idea of replicating the exact recorded speed. The process to save a movement to replicate required to record the movement and then manually cut it and edit it slightly so that it would feel natural when it looped, making the process automatic would have required too much effort. Now that this was not needed we could simply record the user’s average speed and then replicate a constant movement that would still be based on the way we move. The most important part of the training stage was deciding what biomechanical parameters we needed to extract and what we would use them for. We wanted to at least adjust the WiP and RiP transition thresholds and regulate the speed in a way that would depend on the frequency and the feet height during each step. Since the state transition thresholds seemed to be easier to adjust we first designed a complete training stage with only those parameters in mind and then expand upon it to also adjust the speed. In order to extract the information to train the RiP mode we would have ideally recorded the user running, however due to space limitations, even in the room shown in figure 5 the data we tried to record was not accurate, as in figure 27. Thus we had to record a fast walk and apply a bigger gain factor to the speed shown in order to match the speed that the user would have when running. 32 5.3 Final training stage The final training stage has two parts, first the user performs normal walk in 3 speeds, normal, fast and slow through the room and then Walks in Place and Runs in Place using the speeds obtained from the previous parts. To get more natural recordings in the normal walk parts, the user has the HMD on top of the head, so that he/she is not watching the screen. This is better to take realistic measurements because the user’s movement are affected by seeing the virtual environment instead of the real world, and there is plenty of evidence in the literature that humans tend to walk slower in virtual reality even when the virtual world is an exact match of the real one [NSN14] [Jan+19] . The following diagram shows the full training stage: NW-normal start NW-fast NW-slow WiP-normal WiP-fast Figure 28 •NW-normal: Average head speed is recorded to be used in WIPnormal. •NW-fast: Average head speed is recorded to be used in WIP-fast •NW-slow: Average head speed is set as the minimum camera speed in the WiP system. •WIP-normal, WIP-fast: feet frequency is used to calculate WiP and RiP thresholds using equation 5 During the WiP parts, the virtual camera automatically moves at a constant speed and the user is asked to perform WiP motions until it feels that the movements match the virtual camera, it is at that moment that the recording starts. We found that a recording duration of 15 seconds is enough to obtain the information needed, but short enough so that the user is not tired after the training. To calculate the thresholds between WiP and RiP using the data from the training stage we use the middle point between the frequency of walking in place and running in place. To further adjust to each person and avoid flickering we introduce a window. The window size is calculated from the standard deviation of the step frequency data samples obtained after each step is completed. This way 33 the window size increases if the user’s step frequency can change significantly . The final equation is thus: fmid =avg(fnorm) + (avg(ffast)−avg(fnorm)) /2 fmax =min(fmid +stdev(fnorm), avg(ffast) ) fmin =max(fmid −stdev(ffast), avg(fnorm) ) (5) Where fnorm and ffast are the recordings of the frequencies resulting from the user’s steps during the WiP and RiP training respectively. The average of the frequencies also acts as upper and lower bounds in the case that the standard deviations are too big. 34 6 A new WiP method The system described so far is the final completed WiP and RiP interface developed in this project. The system allows to Walk in Place, Walk in Place slowly and Run in Place, and contains a training stage that adjusts the state transition thresholds to each user. The goal of the training stage was also to change the virtual camera speed for each user, and initially we were going to try to adjust some parameters in order to achieve this goal. But as the design of the training stage advanced and more tests were performed, we started to consider the possibility to design a completely new WiP method instead of just doing some adjustments to the current one. Developing a completely new method takes much longer and exceeds the scope and time frame of this project and therefore, at the time of writing this document, it is still a work in progress. However, the development of this project allowed us to run several user studies to investigate in depth the different biomechanical aspects that should be taken into account for the development of a new model. In the final stage of the project, instead of performing a usability study of the current system, we have conducted an experiment with users from both inside and outside the ViRVIG group in order to collect WiP data that will be later used to create a new mapping function for WiP. The following subsections argue the need for a new WiP function and explain in detail how the experiments were performed. 6.1 Step frequency and feet movement Throughout all this project, our goal has been to simulate our natural movement during normal walk to navigate in a virtual environment but without the need to physically move. In normal walk the movement speed (|v|) is produced by the step frequency (f) and the step length (l) [WWB10]: |v|=f×l(6) Of course, there is no direct equivalent of this equation for WiP, since there is no physical displacement over the horizontal plane. WiP that are based on walking biomechanics aim to find a similar function so that camera movement feels natural, but each method tries to do so in a different way. In [FWW08] and the base paper [SRP19] the main parameter used was feet speed. In [WWB10] however, it is argued that what we perceive to be more important in movement speed is feet frequency. In the final version of this project, feet speed is used in WiP, but feet frequency was found to be more useful in RiP, so both parameters have been useful. Maximum step height is another parameter that is mostly overlooked in WiP methods and could also be important, for example to replace the concept of step length in the normal walk equation. During the design of the training stage, we argued and theorised about how users would adapt their WiP to a camera movement. Even within the small amount of data we obtained, we started to think that maybe neither of the two paths was completely correct. As camera speed changed users adapted their 35 movement but they didn’t just vary the frequency or feet movement, but a combination of frequency, feet speed and maximum step height. We started to theorize that if we obtained a bigger sample, and it included people with little to no experience with VR we would see different ways to adapt to the change in camera speed. We believed that most people would only increase step rate, but some would keep the frequency and instead increase step height and speed. With the purpose of understanding how Walking in Place changes as camera speed is increased, we designed and conducted an experiment. 6.2 Collecting people feet data The goal of the experiment performed is to collect data of how each user Walks in Place when exposed to different camera speeds in order to understand how feet movement is adapted to virtual camera speed changes, and whether different users react in different ways or not. Analyzing the collected data will help us understand how the output virtual camera speed should be influenced by different user motion. As opossed to previous work, where biomechanical data is gathered from normal walk, the novelty of our study is that we gather biomechanical data directly from a walk in place movement. To be able to do so, we provide a fixed camera velocity to the Head Mounted Display and ask the user to perform a WiP that feels comfortable with the observed camera speed. 6.2.1 Experiment method Design: After equipping the subject with an HMD and trackers on the feet as shown in figure 29, the virtual environment from [RP18] was shown to the user (see Figure 32) with the following virtual camera speeds: 0.5, 0.8, 1.1, 1.4, 1.7, 2.0 and 2.3 in ascending order. The user was told to perform a WiP motion until he felt that his motion matched the optical flow. The feet data was then recorded for 15 seconds for each speed. Subjects: 13 physically unimpaired subjects performed the user study. Ages 20 to 48 (average = 26.6, median = 23). 9 male, 4 female. The user’s heights ranged from 158cm to 183cm ( average = median = 171cm). All participants had at least some experience with videogames and computers. 4 participants had extensive experience with VR, 1 had some experience, and for the other 8 this was their first contact with VR. No user had any experience with WiP systems. All subjects volunteered and were not paid to perform the experiment. Procedure: The procedure is illustrated in figure 31. Before starting , participants received information about the purpose of the experiment and were explained the procedure. Subjects were told about space constraints in VEs, and introduced to the idea of Walking in Place interfaces to remove space constraints. Users were told that they would see different camera speeds, but not that the speeds would go from slowest to fastest. Participants who had never used any VR device before were left interacting with the Steam VR home simulation (Figure 30) for a few minutes until they felt comfortable performing normal walk on a virtual environment. A very general description of what the WiP motion is (moving one foot up and down, then the other foot) was given 36 Figure 29: User with the Vive HMD and trackers to the users. The experimenter did at no point perform a WiP motion nor were the users shown any video or image depicting the movement. Some videos of the user’s feet were recorded while they had the HMD on. Participants were informed that a video showing only their feet and legs may be recorded during the experiment, but were not told when that would happen. In the experiment participants were located in the middle of a virtual train station. On the floor there was a yellow line showing the path of the virtual camera (screenshot in figure 32). Participants were not required to be on the line and its purpose was just to show the movement direction. After the experiment, users filled one questionnaire to collect demographic data and another about simulation sickness. 37 Figure 30: HTC Vive’s home environment. The user moves through normal walk+teleport, can pick and throw some objects and can paint in 3D and use some other tools Figure 31: Experiments procedure flowchart 38 Figure 32: Experiment environment. The screenshot shows the yellow line that indicated the movement direction 39 6.2.2 Initial result discussion While the data collected from the experiment still requires more study, our initial analysis of the data is already helping us to improve the system and to start to develop a new mapping function between feet movement and virtual camera speed. Fixing the system sampling rate: The recording of the WiP movement such as in figure 15, hinted that there was something that was causing changes in the speed to be sharper than they should be if we use the equations described in the base paper. Since the change was not too significant and it was hard to notice without looking at the chart from the recording, we put the issue aside and continued with the project. After applying the method on the data from the recording, we found that there was indeed something different in the code implementation of the equation. The issue was found to be a sampling rate problem in the original project. In the code that was already written at the start of this thesis, the virtual camera speed was updated once every 5 frames, probably because doing so every frame resulted in some problems (please not that this is visible on the velocity graphs, but not perceptually noticeable for the user during the VR experience). Despite the recording system being changed to save values every frame at some point during this project, the 5 frame delay was kept. Changing the code to make the speed update every single frame resulted in a more smooth movement. A recording of normal WiP with the change is shown in figure 33. The recording is still not completely smooth, as it shows some small dents in the curves, but they are small enough to be near to impossible to notice. Comparing the results to literature: In the motivation to create a new WiP system, we discussed our belief that WiP literature was missing something to truly match normal walk to WiP. We have already started to compare the obtained results with equations from WiP literature. In [WWB10] it was stated that biomechanics studies support the following equation: |v|=f 0.157 ×h 1.722 (7) Where |v|is walking speed, fis step frequency and his subject height. The equation serves as the basis for them to only use step frequency to determine camera speed. We used the data from 3 participants of our experiments with the same height (183cm) to eliminate the variable, and plotted the step frequency and feet speed at different camera speeds to see if only step frequency would change or if feet speed would also increase. The charts are shown in figures 34, 35 and 36. While indeed, there seems to be a relation between movement speed and step frequency, the feet speed (and thus step height) also seem to increase in a similar fashion. Another important result shown in the charts is that for each person, the importance given to step frequency and feet speed when adapting to the camera speed is different. To compare it we will use the coefficients from the trend lines, since the coefficients indicate the steepness of the slope. For user 10, 40 Deep learning method MST: Study of unity’s learning methods Dependencies: BST Estimated duration: 20 hours Description: Studying the options that Unity offers regarding deep learning and reinforcement learning and understanding them in dephth to determin how they could be applied to our problem. MI: Implementation of the machine learning method Dependencies: DST Estimated duration: 20 hours Description: Programming the code that extracts the parameters chosen from the trackers to train unity. Running the code to extract enough data from a person walking normally. Testing Tasks required to test the project’s usability with other people, both in short cycles to test the new functionalities and in a final experiment to extract the results of the project PiE: Pilot experiments performed with ViRVIG members Dependencies: None Estimated duration: 10 hours Description: Usability tests performed regularly to test the changes. The test subjects will be the thesis supervisors and other ViRVIG members. There will not be a statistical study of the results, but the feedback will be taken into account to modify the code. EP: Planning the final experiments to test the developed methods Dependencies: None Estimated duration: 15 hours Description: Planning the experiments that will be performed. Including: the procedure, virtual environment that will be used,picking participants and creating the surveys that participants will have to fill. Defining what formulas and statistic methods will be used to extract the conclusions from the results of the tests. EXP: Performing the experiments Dependencies: IT, EP, DSI, BSI Estimated duration: 10 hours Description: Performing the experiments defined in EP and saving the results. The experiments will require to meet with different persons, introducing them to VR if necessary, performing the tests, and handing them surveys to know how they felt throughout the experiment. Project management tasks DEF: Defining the project and its exact goals,scope,planning,etc. Dependencies: None Estimated duration: 27 hours Description: Initial documentation of the project. Defining the key aspects of the project such as the scope, goals, methodology, planning and budget. 47 Required to start any project. FREP: Writing the thesis report Dependencies: None Estimated duration: 60 hours Description: Writing the thesis report of the whole project. MEET: Meetings Dependencies: None Estimated duration: 14 to 20h Description: Meetings with the thesis supervisors every two weeks (Scrum methodology). Each meeting takes about 2h. This task has a variable length since more meetings may be required depending on the progress and problems of the project. 7.2.2 Task summary Task Description Estimated duration(h) Dependencies DEF Defining the project and its exact 27 - goals,scope,planning,etc. MEET Meetings 20 - FREP Writing the thesis report 60 - PiE Pilot experiments performed with ViRVIG 10 - members STP Reading the base paper 10 - STU Learning to use the Vive HMD 5 - and trackers, Unity and SteamVR STC Learning C# and unity scripting 3 - PRB Understanding the code of the project 5 STP STU STC and fixing bugs BST Study of the biomechanics literature 40 STP,STU,STC BI Implementation of the biomechanics 30 BST method MST Study of unity’s learning methods 20 BST MI Implementation of the machine learning 20 DST method EP Planning the final experiments that 15 - test the developed methods EXP Performing the experiments 10 PiE EP MI BI 48 7.2.3 Budget Table and explanation Activity CosteNotes Defining the project and its exact 661.76 goals,scope,planning,etc. Meetings 2007.47 Writing the thesis report 1470.59 Pilot experiments performed with ViRVIG 245.10 members Reading the base paper 73.53 Learning to use the Vive HMD 122.55 and trackers, Unity and SteamVR Learning C# and unity scripting 122.55 Understanding the code of the project 490.20 and fixing bugs Study of the biomechanics literature 980.39 Implementation of the biomechanics 735.29 method Study of unity’s learning methods 490.20 Implementation of the machine learning 490.20 method Planning the final experiments 367.65 Performing the experiments 245.10 TOTAL CPA 8502.57 HTC vive HMD 50 2 Vive trackers 20.95 PC 61.90 Transport 125.4 Electricity 40 Space 40 TOTAL GC 338.26 TOTAL GC + CPA 8840.82 CONTINGENCY 1326.12 15% of Total CPA + Total GC Mappings for slow and fast WiP required 367.65 50% risk Extra time biomechanics method 49.02 20% risk Extra time machine learning method 49.02 20% risk Extra WiP method developed 61.27 5% risk Redirect method developed 220.59 30% chances TOTAL INCIDENTIALS 747.55 TOTAL BUDGET 10166.95 Staffing costs Because I am doing research, we will assume that I am working under ”Personal d’ajut a la recerca” contract. The contract cost 42.000e annually for a workload of 37,5 weekly hours. The price includes paying taxes and social security. As for the time my supervisors will spend in meetings with me, we will use the amount it costs the UPC to have a teacher, around 65.000e paying taxes and security. Material and transport The cost of buying a new HTC vive is 600e[HTC] 49 The estimated useful life of the headset is 4 years, keeping in mind that its life can end either for becoming outdated of due to a malfunction. The trackers cost 120e. They are more resistant to breaking and less likely to become outdated. Therefore, the estimated useful life is 7 years. The PC needs to be of at least a modern middle to high end in order to play virtual reality applications with the necessary resolution and screen refresh rate. We estimate the total cost of the PC to be around 1300etaking into consideration the rest of peripherals, namely the screen, keyboard and mouse. We also assume it was bought as a whole and not in separate components. The estimated useful life of high end PC is around 7 years. Regarding the transport, the price shown is the cost of me going to ViRVIG and Fib for around 4 months. 7.2.4 Budget control Risks The most important risk found in the project was the need to develop a new mapping function for walking in place slow and fast. Creating the functions would require 30h. This task is the core element of our work, so it cannot be removed in case of failure. We must find a solution even if the improvement compared to the current method are not very significant. We would like to innovate with learning techniques to find the best mapping function, and this is the step where we may not succeed and need to go back to a manual mapping. So in the unlikely case that we require new functions for walking in place slow and fast, and 20h are not enough to create the new function with reinforcement learning techniques, will only implement a mapping that can adjust the value of the parameters to the biomechanical information gathered. in order to not further increase the budget. The risks of the project would only have an effect on working hours. There would not be a need for more equipment or other extra costs. Thus, our main method to control the budget is tightly related to controlling the schedule of the project. Through the project, we will track the extra hours dedicated to each task, and the extra tasks needed. If in the first half of the project we have spent more than 30 extra hours, we will avoid doing any extra task even if they were planned. Instead, we will explain the need to reduce the scope in the final report. This way we assure that we do not increase the budget too much. Extra costs In the case that the project advances smoothly and we finish our objectives before the scheduled finish time, we considered the option to increase the scope. The secondary objective would be to develop a method to redirect the user when it gets close to the edges of the physical room. The estimated total duration was 30h. While developing the method would increase the budget, it would not be by an excessive amount, 187.5e. If we are able to develop this extra, it would mean that we had little to no extra costs in the rest of the project. Thus, we would have money to spend on the extra feature, and it would be worth the price. 50 7.3 Changes in planning During this project we followed the scrum framework described in the planning. There were regular meetings with my tutors, and internal tests were performed every time a new feature was added. Thanks to choosing an agile methodology we were able to add and remove goals, and invest more or less time in adding a feature or investigating as we saw necessary. However we never lost our original main goal, all our efforts were directed towards developing a biomechanics based WiP system. But still, looking back at the original tasks planned can show us whether the project advanced and concluded smoothly or had a lot of changes and problems. Indeed, the biomechanics based WiP system was developed, however at the start we underestimated the scope and time needed to develop it. RiP and slow WiP at first were thought to be small tasks that would only require to adjust the original WiP system, however as we investigated more we found them to be interesting due to how our system handled the movements and how little research there was about RiP. In the end developing the new modes required a lot of experiments, programming and adjustments, but we believe that the time was well invested, and that the project would have been worse with RiP and slow WiP only half developed and left as future work. As for the Deep learning method, as we investigated and started to see how we would develop it, we noticed how time consuming it would be. As time went on, and the WiP system became more complex, we realized that if we had tried to develop a similar system but using machine learning and we wanted get results the project would not finish in time. Thus, since we were obtaining results from the biomechanical method, we decided to use the time left to improve it even more instead of trying to build a new system. Even if it not in the original planning, the design of the training stage also took more time than expected. As we started performing experiments and recording data we saw that there was a lot of potential and kept changing the parts of the training stage and the way the camera moved almost each week. The time frame that we initially wanted to use on the user study was in the end spent on performing an experiment and collect feet data. While the results are not used to develop a new system in this project, will be much more useful in the future. The project still obtained results and finished on time. The time spent on the project remained about the same as originally planned and the budget didn’t increase, we just redistributed the time available to what we thought was best at any moment, which is expected when working under an agile methodology. 51 8 Sustainability report 8.1 Survey auto evaluation The survey ( goo.gl/kWLMLE ) asks about my knowledge regarding sustainability and my ability to develop a project that takes into account the different dimensions of sustainability. I think the results of the survey were generally positive. There were a lot of questions about my knowledge regarding sustainability, economy and project management. I positively answered most of those questions, since during the bachelor I have been taught about those subjects in some courses. The survey had a few questions about my ability to analyze different aspects of the project, such as the impact in society, economy, ethics and environment. I believe I can think critically when I’m in a project or I’m trying to analyze it. I can identify the different aspects of the project that affect sustainability and evaluate the impact on the environment and society. However, I am a bit lacking when I have to apply the principles myself. I am about to end a bachelor and I have little experience in the working world. While I know how to evaluate a project, once I’m actually inside one I don’t think that at this moment I would be able to make sure it’s sustainable. I lack the experience to know what methodologies and what techniques we would have to use to develop a sustainable project. I could a bit help giving some ideas when talking about the sustainability aspects of the project, but I would not have the initiative to talk about these topics. If I had to start a real project on my own, I believe I would need a little help to make it sustainable. 8.2 Economic Dimension 8.2.1 Economic cost of the project The cost of developing the project is shown in the budget table. While tasks performed in the project changed, the total development time remains aproximately the same, and therefore the budget of the project remains the same. This project does not aim to generate any kind of revenue, instead its goal is to help other people that want to develop a WiP system. As a result, for this project to be economically profitable, the only condition is that the research performed and the results obtained are used on the developement of some aplication or in another person’s research. 8.3 Comparison to other solutions In our solution, we aim to find a middle ground between cost and performance. The setup is not cheap, but its neither so expensive that only a few people and research groups could use it. We believe the results it can produce are worth the economic cost. There has been research on both ends, using cheaper and more expensive tech52 nology. On the side of cheaper technology, there are methods for navigating using a HMD consisting of a phone inside a headset or HMD that do not need a pc or any other component to work. Most common locomotion techniques use a hand held controller to move the virtual camera. Methods for WiP use the head bobbing to detect steps. However they are limited, because only the head orientation can be used to determine walking direction, you can not walk and look at different directions. Also, the lack of positional trackers makes it impossible to use for normal walk. On the other side, some techniques use more complex setups. Some solutions use even more trackers to more accurately detect movement, usually positioned in the knees,legs, or chest. There are solutions that use a small room with projectors, such as a CAVE. And others that use a whole large room to create a bigger virtual room. Our project aims to achieve the same level of immersion and precision as those bigger, more expensive setups. 8.4 Environmental Dimension 8.4.1 Environmental cost of the project In order to estimate the cost of developing the project we will use the following values: Activity Energy usage PC 0,2 kWh Screen 0.015 kWh Vive setup 0.016 kWh [Red] The whole project will make use of the PC and screen. Some tasks use the Vive for about 50% of the time, And the economic cost of the development of the project will be of about 62,59kWh. The finished project itself will not have a direct environmental impact. The results wil allow other people to develop other projects or applications. Nevertheless, we hope that since the results will be used in other WiP systems, not needing to create a completely new RiP system or training stage will reduce the economical impact of other projects. 8.4.2 Comparison to other solutions In the environmental dimension we find a similar case as in the economic one. Our solution does not attempt to improve other systems in this dimension, but to find a balance between the cost and the usability. Using a phone or a simple HMD with low energy usage will have a lower environmental impact. However using more devices or other, bigger setups will have a greater energy usage. 53 8.5 Social Dimension 8.5.1 Personal growth Initial thoughts In the first phase of the project, while investigating, I realized the scope of the project and the investigation regarding VR and specifically WiP. Looking at different papers with various navigation techniques, I saw that there is still a lot of investigation needed in this field. For modern videogames and virtual environments that do not use VR there are some well established rules regarding movement. The joystick is always the best method to move the camera, and in some applications, there is automatic camera movement. In VR, full room setup seems to be the optimal way to move in a small area. However when we need to move in a large environment, there is no consensus regarding what technique to use. The added challenge of reducing motion sickness due to the use of a HMD makes the task much more difficult than when interacting with just a flat screen. When this project concludes, I feel like I will have a greater understanding of the challenges of navigating a virtual environment and I hope will be able to continue investigating this field or that I will be a useful member of a team of developers of Virtual Reality Applications. While we’ve had modern HMD for a while, they are still at an early stage of development, and when in the future they become more widely used I hope that I will be ready to keep innovating and creating new games and applications. Final thoughts After finishing the project I think that I do have a much better understanding of navigation in virtual reality and the many problems it still has. After seeing many people react to different camera movements I was able to see how different they react, for some motion sickness is not a problem at all, while for others even small camera movements that they don’t expect, or in our case that doesn’t match the feedback they expect from their movements, cause motion sickness. In a time where videogames are starting to commit to being more accessible , VR is still quite behind and still doesn’t even have a universally accepted good navigation method that doesn’t cause motion sickness nor disorientation. I think that now I’m able to keep investigating to help improve it. 8.5.2 Impact of our project Motion sickness is a problem of VR and a barrier for some people to start using it. With the techniques developed in this project we hope to reduce the likeliness of suffering from motion sickness compared to other solutions, thus improving the well-being of VR users. The techniques developed will be available for anyone to use. Developers will be able to use our research into their VR applications, and thus it will have an impact on the quality of the final product. The project could also be used by other people to create their own methods of movement, allowing for further research to be done. Because there is still not a standard way to move in large virtual environments, more research and techniques are required to some day find one method that can be 54 universally used. After finishing the project, we believe we have helped reduce the motion sickness problems of WiP and laid the groundwork for future investigation. The problem is still not solved and there are probably still a lot of improvements that can be made, but our results are step in the right direction to improving VR systems and allowing anyone to use them without feeling sick. Since VR has a lot of uses, our efforts can be used by a wide range of people, including fields outside of entertainment, where people who use VR are likely not familiar with graphical applications and are more likely to suffer from motion sickness. 55 9 Conclusions During the development of this project we have improved current WiP systems by adding functionalities to allow users to interact with the environment in different ways while also reducing motion sickness. In the first stage we successfully improved the original project by adding small modifications that made the system more responsive but also smoother. We created a RiP and slow WiP system, and got results that showed that Running in Place is significantly different to Walking in Place and require a different movement to virtual camera mapping function. The combined system increased immersion by allowing to move large distances with in-place motions and also allowed navigation in crowds or other environments where slow movements are required. We found a relation between slowing down during normal walk and during WiP or RiP that can help reduce motion sickness. During the creation of a training stage we discovered a way to manipulate the user’s WiP frequency. Some functions were created in order to adjust certain WiP and RiP parameters to each user’s natural movement, thus improving their experience when using the system. Finally we succesfully conducted an experiment to collect feet data that will be used to develop a new WiP method. 10 Future work While a lot of experiments were performed, and a lot of aspects of WiP were studied, some work was left for a future projects due to time constraints. Developing a WiP and RiP system but using some form of machine learning was the biggest goal of the project that we were not able to reach. The scope of developing the system was deemed to be much bigger than expected. We believe that using machine learning to let the computer either decide if we are performing WiP or RiP or even decide the output virtual camera speed is worth investigating, and has enough depth to require an entire thesis. It would require to find a way to generate training data for the machine, either by recording or by generating artificial data, creating a correct output, and making sure the computer is trained properly. During the development of this project, unity’s ML-Agents [Unib] were looked into, and some tutorials and tests were performed. We saw how powerful the tool can be, but using it correctly requires an in depth understating of how the plugin and algorithms it uses work. The main problem we found was the amount of data it required for even the most simple artificial intelligence, which would be very hard to manually generate in our case. Thus it would be required to either extract large amounts of data from real people or to find a way to generate walking data that is very similar to real walking, which is probably a huge challenge by itself. A side optional objective of this project was to find a natural way to redirect the user’s physical displacement during WiP to avoid that they go out of the physical play area and are and hit some person or piece of furniture. After 56