Exploring how charts are perceived in 3D

LG   (;3/25,1*+2:&+$576$5(3(5&(,9(',1 ' +(/(1$)(55$13$'8$ 7KHVLVVXSHUYLVRU3(5(3$89È=48(=$/&2&(5'HSDUWPHQWRI&RPSXWHU6FLHQFH 'HJUHH%DFKHORUV'HJUHHLQ'DWD6FLHQFHDQG(QJLQHHULQJ 7KHVLVUHSRUW )DFXOWDWGLQIRUPjWLFDGH%DUFHORQD),% (VFROD7qFQLFD6XSHULRUG(QJLQ\HULDGH7HOHFRPXQLFDFLyGH%DUFHORQD(76(7% )DFXOWDWGH0DWHPjWLTXHVL(VWDGtVWLFD)0( 8QLYHUVLWDW3ROLWqFQLFDGH&DWDOXQ\D83&%DUFHORQD7HFK Date: 26/6/2023 1 Exploring how charts are perceived in 3D Abstract The increasing popularity of Virtual Reality (VR) has opened up new possibilities for data visualization, allowing the incorporation of an additional dimension for feature encoding. However, the understanding of how data is perceived in these environments remains unclear. This thesis aims to evaluate the e↵ectiveness of bar chart perceptions in VR by conducting a comparative analysis of various basic configurations, which are chosen along with the experiment conditions. Through a user study employing a VR headset, estimations of the heights are assessed to evaluate the accuracy of the perceptions. Additionally, eye tracking data is collected to identify the elements that most contribute to the performance and are more relevant to the user. The findings reveal that, under the particular condition set, grid lines play a crucial role in enabling accurate estimation of values within the immersive environment. 2 Exploring how charts are perceived in 3D Resum El recent increment de la popularitat de la Realitat Virtual (VR) ha obert noves possibilitats en quant a la visualitzaci´o de dades. Aix`o ´es degut a la incorporaci´o d’una tercera dimensi´o que ens permet codificar caracter´ıstiques addicionals. No obstant aix`o, encara no s’ent´en completament com es perceben les dades en aquests entorns. Aquest projecte t´e com a objectiu avaluar l’efectivitat de la percepci´o de gr`afics de barres en VR mitjan¸cant una an`alisi comparativa de diverses configuracions b`asiques. Aquestes s´on seleccionades meticulosament juntament amb les condicions de l’experiment. A trav´es d’un estudi d’usuari realitzat mitjan¸cant l’´us d’un casc de VR, s’avaluen les estimacions de les al¸cades per determinar la precisi´o de les percepcions. A m´es a m´es, es recopilen dades de seguiment ocular per identificar els elements que milloren el rendiment i s´on m´es rellevants per a l’usuari. Els resultats revelen que, sota les condicions espec´ıfiques establertes, les l´ınies de quadr´ıcula juguen un paper crucial a l’hora de realitzar estimacions precises dels valors dins de l’entorn immersiu. 3 Exploring how charts are perceived in 3D Resumen El reciente incremento de la popularidad de la Realidad Virtual (VR) ha abierto nuevas posibilidades en cuanto a la visualizaci´on de datos debido a la incorporaci´on de una tercera dimensi´on apta para codificar caracter´ısticas adicionales. Sin embargo, a´un no se comprende completamente c´omo se perciben los datos en estos entornos. Este proyecto tiene como objetivo evaluar la efectividad de la percepci´on de gr´aficos de barras en VR mediante un an´alisis comparativo de varias configuraciones b´asicas. Estas son seleccionadas meticulosamente junto con las condiciones del experimento. A trav´es de un estudio de usuario realizado mediante el uso de un casco de VR, se eval´uan las estimaciones de las alturas para determinar la precisi´on de las percepciones. Adem´as, se recopilan datos de seguimiento ocular para identificar los elementos que mejoran el rendimiento y son m´as relevantes para el usuario. Los hallazgos revelan que, bajo las condiciones espec´ıficas establecidas, las l´ıneas de cuadr´ıcula desempe˜nan un papel crucial a la hora de realizar estimaciones precisas de los valores dentro del entorno inmersivo. 4 Exploring how charts are perceived in 3D Acknowledgements First and foremost, to express my appreciation to my research tutor, Pere-Pau V´azquez Alcocer, who has made this project possible and has guided me through the di↵erent steps. I am grateful for the knowledge I have been able to acquire thanks to him. I would also like to show gratitude to the people in the CRV department for their welcome and kindness. Their willingness to assist whenever I needed their help has been appreciated. Finally, I would like to acknowledge my friends and family for their support and encouragement, but mainly for taking their time to come to the department and perform the experiment, considering the e↵ort it may have entailed for some of them. I am particularly grateful for those who not only did the experiment themselves, but also helped by bringing their friends or family. 5 Exploring how charts are perceived in 3D Contents 1 Introduction 7 1.1 Structure of the Bachelors’ Thesis ........................... 8 2 Goals of the project 9 2.1 Tasks and milestones .................................. 9 2.2 Previous work ..................................... 10 3 Basic concepts 11 3.1 Virtual Reality ..................................... 11 3.2 Data visualization ................................... 11 4 Project Development and Results 13 4.1 Bar charts setup .................................... 13 4.1.1 Exploration ................................... 13 4.1.2 Unity ...................................... 13 4.1.3 Configurations ................................. 20 4.2 Virtual Rality ...................................... 23 4.2.1 OpenXR .................................... 23 4.2.2 Eye tracking .................................. 23 4.3 Data Generation .................................... 24 4.4 Data collection ..................................... 25 4.4.1 Form ...................................... 26 4.4.2 Bar height ................................... 26 4.4.3 Time of response ................................ 26 4.4.4 Focus ...................................... 26 4.5 Procedure ....................................... 28 4.5.1 Pilot study ................................... 28 4.5.2 Users ...................................... 29 4.6 Analysis ........................................ 30 4.6.1 Time ...................................... 30 4.6.2 Height ..................................... 31 4.6.3 Eye tracking .................................. 34 5 Conclusions and Further Research 39 5.1 Conclusions ....................................... 39 6 Exploring how charts are perceived in 3D 5.2 Future Work ...................................... 39 7 Exploring how charts are perceived in 3D 1. Introduction In recent years, virtual reality technologies have gained popularity and have transformed various fields regarding multiple industries. One particular field that has emerged is immersive analytics, which combines immersive environments with data analysis. Together, they aim to use the potential of virtual reality to enhance data visualization experiences and provide new insights into complex datasets. In immersive analytics, the visualization of data using 3D bar charts in virtual reality environments is a captivating area of exploration. The usage of the third dimension to encode additional data increases the range of possibilities and changes the perspective used in traditional 2D visualizations. However, there is still much to uncover regarding the e↵ectiveness and perception of 3D bar charts in the immersive context of virtual reality. This project seeks to explore this area and identify the key elements that contribute to an enhanced visualization experience in the context of 3D bar charts. By understanding these insights, valuable guidance for designing and optimizing the presentation of bar charts within virtual reality can be provided, which leads to an improvement on the comprehension and interpretation of data for users. There is multiple existing research on human perception in 2D visualization systems, which consider amodelofperceptualprocessing[13]oraccuracyintheperceptionsaccordingtosomegraphical and spatial elements[9]. On the other hand, the e↵ectiveness and perception of 3D bar charts in virtual reality environments remain relatively unexplored. Therefore, the project aims to address the open questions surrounding the optimal design and presentation of 3D bar charts in virtual reality, as well as the potential benefits and challenges they bring. To achieve these objectives, an experiment is conducted to explore various configurations and features of 3D bar charts. By analyzing their impact on data perception, accuracy, and user experience, we can gain comprehensive insights into the strengths and weaknesses of di↵erent visualization approaches. However, relying solely on estimations may provide limited information about perceptions and only allow conclusions about accuracy rather than actual user behavior. Then, to also approach the behaviour and gain further insights, eye tracking technology is employed. It enables to investigate features that attract users’ focus when estimating data representations. By understanding which elements capture the attention, new possibilities for immersive data exploration can be unlocked, enhancing the relationship between data visualization, virtual reality, and human perception, and bring these disciplines closer together. Through this approach, we can obtain new possibilities for immersive data exploration and enhance our understanding of how individuals perceive and interact with visualized information in virtual reality environments. 14 Exploring how charts are perceived in 3D functionality to easily create their projects. However, to implement more complex components and interactions, scripting using the C# programming language is required. As an engine widely employed in video game development, one of its advantages is the availability of assets and prefabs. These are user-created templates that can be integrated into a scene, eliminating the need to start from scratch and accelerating the game development process. While assets for characters, objects, and natural elements are easily found due to their common usage in games, getting a suitable prefab to design data visualizations is a greater challenge. Eventually, an existing bar chart asset was discovered and initially used. However, customizing it and adding the desired features proved to be a complex task. Given the significance of carefully addressing every detail of the chart to assess its impact on the final perception, and considering the need to still create the additional features from scratch, the decision was made to construct the chart from the beginning. This approach involved using cubes and planes instead of loading a pre-existing asset. While it may be more time-consuming, it o↵ers the advantage of providing complete control and customization over the chart. On the other hand, Unity also has an intuitive and user-friendly interface, which allows fast creations of elements using basic geometric figures. However, structures created through the API are difficult to manipulate, since a small modification to one of the objects often requires readjusting the whole chart manually. This is not a reasonable approach when several changes need to be added for the di↵erent configurations. The need to automatically resolve all objects when features are added or modified, requires creating the chart through a script. This not only demands C knowledge, but also to be thoughtful and precise to correctly specify the positions and scales of every object so that they all adjust to the chart. The script contains the following functionalities to create the base chart, which includes only the bars without any additional features (Figure 1): •Read: Unity is thought for games, which do not usually use external files. Then, the task of reading the values was not trivial and also required its function. •Scale bars: When a cube is created without any further specifications, its default size is set to one unit per edge, which is already relatively large. Consequently, if the values read have a significantly di↵erent magnitude, they may not fit the scene properly. To address this, it is desirable to scale the values before plotting the data. The scaling process takes into consideration a maximum value, which corresponds to the maximum height of all bars, if it is not specifically provided. In this project, a limit of four units was set for all the charts, which appears to be an appropriate measure. •Plot bars: This function plots as many vertical bars as values included in the data file. The width and depth of the bars are parameters of the function, as well as the spacing between them. This allows for easy comparison of di↵erent dimensions of the bar, so that the most suitable option is chosen. 15 Exploring how charts are perceived in 3D Figure 1: Base Chart with its Collider •Add Collider: AcolliderisacomponentcommonlyusedinUnityGameObjects. Itsmain purpose is to represent the object’s boundaries. In this case, it is useful to group all bars together as a single object and include a box-shaped collider that includes all of them. A single bounding box implies that there is only one position and scale of reference, which simplifies the adjustment of additional features. This collider is removed once the graphic is finished for eye tracking purposes (see Section 4.2.2). In addition to the bars, the aim is to include references that ease the task of the user by making a relative judgement. Then, the script also has the following functions that create the extra desired features: •Boxes: Aboxisusedtocompletetheremainingpartofthebarneededtoreachthemaximum value (Figure 2). The box is constructed out of four planes. These are appropriate for this element, since they are only visible from one side. This allows only the interior of the box to be seen. For instance, when viewing the chart from the left, the left plane will be invisible and will not obstruct the visualization of the bar from that perspective. The choice of the material for these boxes is left to the user but, for this experiment, the opacity of the selected color and set the rendering mode (available option when creating a material) to transparent. This enhances the understanding of the chart by making it easier to distinguish the part that actually corresponds to the actual bar. •Top line: In this function, a horizontal line is added at the top, which will provide clearer reference for the maximum value. The distance between the line and the leftmost bar, as well as the length of the top line, can be adjusted as desired. Since the bars have three dimensions, there are several options for these lines (Figure 3). The first one is for them to be displayed as a plane to include the depth. Alternatively, they can be shown as a line with customizable thickness. In this case, there is also the possibility of 16 Exploring how charts are perceived in 3D Figure 2: Boxed Bar Chart Figure 3: Three alternatives for topline: front, back and plane displaying them in front or at the back of the bars. Both alternatives might be perceived as significantly di↵erent, especially if the bar depth is high. •Base: Abaseforthebarsisincludedasasupporttohelpwithperceptions(Figure4). Placing the bars isolated with irregular backgrounds might make the estimation task more challenging. Therefore, it is possible to add a base in any of the dimensions. This means having a plane at the bottom of the chart to visualize the origin, one at the back to assist with depth perception, and even one at the side. These three bases can be combined, and their lengths can be modified. •Grid: This feature includes a grid behind the bars. It consists of adding several horizontal lines that can be unified with a vertical line at the left. The length of the lines beyond the bars can also. Furthermore, it is possible to exclude the bottom line, which is recommended when combined with a base. The standard way of adding a grid involves getting the maximum value (either specified by the user or obtained from the heights) and determining the most suitable lines according to this 17 Exploring how charts are perceived in 3D Figure 4: Bar Chart with bottom and back bases in an irregular background Figure 5: Grid Lines adjusted to a maximum value of 100 value. For this purpose, there is an additional function that complements this one by computing the distance between lines, considering that the desired number of lines is approximately five. This value might increase depending on the maximum, since the distance between ticks must be an easily perceived number. We are used to recognizing values that follow simple and familiar patterns (such as 1, 2 or 5 and its respective magnitudes 10, 200, 5000...), so these values should be the ones represented by the lines. Alternatively, a second method to compute the grid requires the user to specify the maximum value and the number of lines as desired. Then, it distributes them evenly across the available space. •Tufte: This function generates a bar chart inspired by the Tufte style, introduced by Edward Tufte [12]. This approach is similar to grid lines, but aims to maximize the data-ink ratio. This is, no extra ink is used, and the whole line is not shown, indeed, only the portion of the line that intersects with the bar is shown. In this style, the lines are positioned in front of the bars to ensure they are not obscured (Figure 6). This type of chart can also use the same methods as the grid to determine the spacing between horizontal lines. These functions enable to customize and create the desired chart (Figure 7). Thus, the script 18 Exploring how charts are perceived in 3D Figure 6: Bar Chart in Tufte style functions as a tool to display several types of bar charts in Unity. To use it, the script just has to be added as a component to one of the existing elements in the scene. Then, the chart will automatically be created as its object when entering play mode (which would be similar to executing the program). All features are adjusted to ensure that each one is slightly positioned in front of the others, preventing collision and unwanted e↵ects. To maintain the organization, the created objects for each of the features are automatically set as children of a parent object that encompasses them, which is usually an empty object. This allows easy management of each part and of the final graphic, which works as a single element (Figure 7). To customize the bar chart, it is necessary to access the parameters of them from the script itself. Materials are the only variables that are declared as public so that users can modify them and change colors or textures through the Unity interface. Interaction To facilitate the comparison of di↵erent configurations, it is desired that they are displayed immediately one after the other, without the need to start and stop the program each time. To achieve this, it is important to check the organization of a Unity script. It usually contains two main functions: Start,whichisexecutedonceatthebeginningoftheexecution,andUpdate,which is called every frame. Then, the latter one will allow to constantly check if the spacing bar from the keyboard is released. If so, the next chart is displayed as seen in Algorithm 1 It is also important to perform the action when the button is released and not pressed, since the first option can only happen once, while pressing the key for a bit too long could perform the action twice and lead to undesired behavior. Since Unity is a game engine, it was also quick to create a player character that could be controlled 19 Exploring how charts are perceived in 3D Figure 7: Chart including all features. Object organization is seen at the left Algorithm 1 Pseudo-Code for Changing Charts A [a1,a2, ..., aN] B [b1,b2, ..., bN].Values for the parameters to be modified i 0 function Update if key.spacebar = release and i<Nthen Destroy(previous chart) create chart(A[i], B[i], ...) i i+1 end if end function 20 Exploring how charts are perceived in 3D using the keyboard and mouse. The camera moved along with this character, which enabled the possibility of exploring the scene and seeing the chart from di↵erent perspectives before setting up the virtual reality environment. 4.1.3 Configurations Once many configurations have been seen through Unity, a final set of them is selected for the experiment. Bars It is not only interesting to study the features that might a↵ect perception, but also to observe how do neighbor bars impact the estimation. This is, to understand if a value is perceived di↵erently when its adjacent bars fall within di↵erent value ranges. It is important to consider that adding an extra bar increases a lot the amount of possible combinations of values. Therefore, the most simple case that allows to study the e↵ect of neighborhood is using two bars. Then, 6 di↵erent charts are considered, which will be referred to as di↵erent height arrangements. Further details about these arrangements can be found in Section 4.3. In terms of bar dimensions, it was a matter of interest to analyze if bar size a↵ects perceptions. In particular, since depth it is a feature that does not exist in 2D, and it would be interesting to evaluate its behavior. Nevertheless, when sequentially displaying charts with di↵erent depth values, it was observed that changes were not very noticeable. Variations are only e↵ortlessly perceived when charts are being looked at from a lateral or when extreme depth measures are used. As it is not sure whether a population of about fifteen users would be enough to perceive the di↵erences in perception through the depth configurations, it was decided to opt for options with more pronounced distinctions. Hence, the parameter to estimate will be the bar height and its width and depth will be fixed through all configurations. In particular, these values were chosen to be visually pleasing, as recommended in [6]. However, for a chart of only two bars, the height-width ratio suggested is 10 : 1, which seemed excessive, so 5 : 1 is used instead. This is, since the maximum height of a bar in Unity units is set to 4, its width should be 0.8. As for the spacing between bars, the proposed range of 50% to 75% of the width is used and the final value chosen is 0.5 In terms of depth, a perfectly shaped prism appeared to have an unfavorable data-ink ratio, as too much ink is used without purpose. On the other hand, small values were not visually pleasing, as the chart is perceived almost like a 2D object, which does not align to the intended environment. Consequently, depth is finally set to the middle value, which is half the width, equivalent to 0.4. Environment 21 Exploring how charts are perceived in 3D It is desired that the conditions of the environment have a minimal impact on the perceptions, so certain adjustments are made. Firstly, the default scene environment displayed by Unity is not uniform. This might a↵ect the final results and estimations, so an empty room with a plain background is set up. With this addition, all configurations that include bases become less relevant and not as beneficial as other alternatives; therefore, they are finally discarded. Secondly, a Unity game often aims to be a representation of the reality, which means that there are shadows and di↵erent illuminations depending on where the focuses are placed. One more time, it is wanted that as less external matters as possible a↵ect the charts, so the options of casting and receiving shadows are disabled for all objects. Moreover, two additional focuses are placed in the room to ensure uniform illumination across the entire graphic. Final features To get the final set of configurations, values are attributed to all the modifiable parameters mentioned in the di↵erent functions. The di↵erent results are compared between them and combined ones with others. At first, all configurations that required a maximum value were discarded because they are not suitable for representing data sets without a defined maximum value. However, to make the estimations, a reference value is needed. Then, an appropriate simplification of the reality is to consider the maximum value as reference. This ensures that all values across di↵erent configurations are compared to the same reference point, and do not variate depending on the height of the bars. For this same reason, grid and Tufte use their second method of line generation (4.1.2). This consists of fixing a maximum value, 100, and a specific number of lines, 4. The standard method is actually easier to use due to not having to set any parameters. However, this alternative leads to variations depending on the bar heights and is not appropriate for the experiment. All things considered, five final configurations are chosen, one for each of the functions presented in section 4.1.2 except from bases. These are: the baseline, boxed bars, top line, Tufte and grid (Figure 8). The baseline does not include anything but the two bars and a vertical line, which is also accompanied by a label. The inclusion of text required the additional installation of the package TextMeshPro. The vertical line showing the maximum value is considered to be the minimum reference to estimate the value. It is maintained through all the remaining configurations except for the grid, which already includes a vertical line and labels itself. In terms of style, the modifiable parameters for these configurations need to be defined. In par- 22 Exploring how charts are perceived in 3D Figure 8: Final configurations used in the experiment In order: baseline, boxed, top line, Tufte, grid 23 Exploring how charts are perceived in 3D ticular, the positioning and thickness of the lines are considered. About positions, the vertical line is placed half the size of the chart (which includes both of the bars together as a single object) to the left. The top line has the center aligned with the chart center, and its size is double of the size of the chart. Line thickness is set to 0.05 for vertical and top lines. On the other hand, Tufte and grid lines were initially set to small values to be more precise, nevertheless, they had to be increased up to 0.04, since thin lines had issues with pixelation clarity within the virtual environment. Lastly, no colors are added to avoid a↵ecting perceptions. The bars and vertical lines are set to white, while the grid and Tufte lines have a grayish color. 4.2 Virtual Rality Once the configurations are determined and implemented, it is time to see them in a virtual reality environment. To accomplish this, the HTC VIVE Pro Eye headset is used. All the setup had already been done, so connecting the headset to the computer only required plugging it in and inserting the cables to the corresponding ports. To detect it, two bases are installed and configured, but these were also ready to use, and it was only necessary to plug them in too. 4.2.1 OpenXR Although the headset is connected to the computer, it is not as trivial to connect it to the Unity project. The XR Plug-in Management installation is required, where OpenXR is selected. This can be done through the edition of the project settings. Then, the scene seen in Unity is also shown in the virtual reality environment, but there is still one step left. The camera is not moving along with the headset unless it is specified to do so, then, although the user might be moving its head, the same scene is always displayed as a static image. To solve this issue, the XR available option in the camera object is changed to specify that the main camera is wanted to be converted to XR Rig. Once this is set, he user can already see the bar charts through the headset and is ready to interact with the virtual environment. 4.2.2 Eye tracking Adding the eye tracking was, by far, the most challenging part of the project. It requires the installation and load of the package VIVE SRanipal SDK and also the Import the Tobii XR SDK as stated in their development guide [1]. However, this did not manage to fulfill the task. It turns out that there are version incompatibilities [2]whenusingthelatestversionofall 30 Exploring how charts are perceived in 3D perform the experiment with them on. In the analysis of the results, the fact of wearing glasses or contact lenses is not taken into account. If they affirmed to see the chart correctly and the highlighting confirmed that the eye tracking was working as expected, there is no reason to think this might be an issue. Positioning Participants were not specifically instructed neither to keep still nor to adopt a particular position. In fact, they were informed that they could move around the room as much as desired. However, all of them decided to do the whole experiment while seated. What is more, the majority of of them maintained a stationary head position throughout the experiment, and only a couple of participants occasionally adjusted their posture to view the graph from a slightly di↵erent angle. This helped them to accurately perceive depth, as they stated later on. 4.6 Analysis The three types of data gathered in the experiment (excluding the form) are individually analyzed. 4.6.1 Time In this section, the time of response for each of the configurations is analyzed. On average, it took users 10 seconds to estimate both bars of the chart. However, the range of values observed is large, since the fastest response took 3 seconds, whereas the slowest answer was not given until 54 seconds had passed. While no specific hypotheses were made regarding this data, it is still interesting to observe if the range of response times is related to the di↵erent configurations. The results are shown in Figure 10. It is seen that the baseline charts were, on average, the fastest to be answered. This configuration, along with the top line one, had the most constant behavior among users. On the other hand, the grid and Tufte features resulted in higher time response and were the ones with more variation among users. The reason behind this behavior might be related to the information presented in the configuration. Since Tufte and the grid contain more elements to analyze, they also require more time to make the pertinent estimations. It is possible that when there are fewer reference points, users tend to guess rather than estimate, being the second one clearly more time-consuming than the former one. Nevertheless, this is just an educated speculation. In addition to this, it is worth noting that it took a long time for a particular user to estimate the bars in the grid configuration. This is an outlier, which has a lot of weight due to the small sample size. As a result, the mean is skewed upwards and the median might be a more representative measure. 31 Exploring how charts are perceived in 3D Figure 10: Response time for every configuration Therefore, the time response for the grid configuration may actually be faster than the Tufte one, but still slower than the other configurations. 4.6.2 Height About the estimations, there are several hypotheses: H1: Top line and Boxed bar charts improve baseline estimations. H2: Top line and Boxed bar charts have similar behavior. H3: Tufte and Grid charts improve top line and Boxed bar charts estimations. H4: References being upwards or downwards influence on values being over or underestimated. The first three hypotheses are tested using an average absolute error plot (Figure 11). It is strongly believed that the addition of extra features improves the accuracy of estimations by serving as references. Then, boxed bar charts and top lines are expected to obtain better results than the baseline, since they both provide additional information. At the same time, the extra information provided is the same for both configurations, since they both aim to indicate the location of the maximum value. Therefore, they are expected to get similar performances. On the other hand, boxed bar charts and top lines only provide the top references, which leads to the thought that Tufte and grid lines should work better than them for giving more specific information. When looking at the results, it is confirmed that adding extra features helps with the estimations, since all configurations managed to improve the results obtained by the baseline. Nevertheless, top-line charts did not have the expected behavior, which led to the rejections of H2andH3. 32 Exploring how charts are perceived in 3D Figure 11: Absolute error for every configuration The display of the maximum value was thought to enhance the results slightly, as the feature does not provide much information. However, top line managed significantly to improve the estimations, not only regarding the baseline but also in comparison to the boxed chart, which provides the same information. Despite its results in accuracy, its distribution does not give as successful results, since it is the configuration that shows higher variance among users. On the other hand, Tufte and grid configurations also encode similar information, which narrows the range of possible values for the estimations. As a result, it was anticipated that both of them would enhance the performance of the 3 prior configurations. However, due to their setting di↵erences, it was their setting is slightly di↵erent, so it is not surprising that they yield di↵erent outcomes. What was unexpected is that only the grid managed to achieve better results than the top line. In fact, the configuration that clearly outperforms is the grid. Not only it has the lowest mean time of response, but it also exhibits the least variance. In other words, all users were able to perform well and provide highly accurate estimations of the heights in this configuration. To support this results, a Repeated Measures ANOVA analysis is performed. This test allows to compare data that is not independent, which is the experiments’ case as the same individuals are being used through all charts. The test computes if the sample di↵erence between the averages of some configurations is big enough to be statistically significant. Since the p-value is 3.873e8, which is much smaller than the significance level, the null hypothesis (H0: Averages for all groups are considered to be equal) is rejected. Similarly, the test statistic Fequals 11.8221, which is not in the 95% region of acceptance: [1,2.51]. Bonferroni’s test shows that grid’s estimations have a significantly di↵erent average than every other configuration, whereas these are considered equal to each other. Therefore, grid is, in fact, the only feature that managed to improve the baseline results. 33 Exploring how charts are perceived in 3D Taking part of the process has allowed me to observe the behavior of the users while performing the experiment. In this section, I wanted to make an additional comment of something I noticed during the data collection. Regarding the Tufte configuration, it was difficult for users to compute the values. Most of the participants were thinking out loud while performing the task, and the sentence ”This line is 75 and the bar exceeds it by a few units, so let’s say 72” was surprisingly heard more than once. Another example is the observation with more error, that also corresponded to one of the Tufte charts. The value is underestimated by exactly 25 units, but what actually happened is that the user mixed the values of each line. Although these might seem like punctual lapses, it actually happened to more users than one could imagine. Most of them were able to correct themselves on time, others did not manage to find their mistake at all or realized it many charts later. It is important to remark that this did not happen even once to any of the participants when estimating values of the grid charts. This does not mean that values should be modified. The estimations are incorrect although the user did not actually mean to say that value but a more accurate one. However, this might reveal that the potential of Tufte was not exploited to its fullest and that grid charts might have obtained better results due to the specification of the value of each line. This being said, we might proceed with the examination of the last hypothesis (H4). Previously, our focus was on the extent of deviation between estimations and the real values. Now, we will also take into consideration whether the configurations lead to consistent underestimations or overestimations (Figure 12). Specifically, baseline and grid are anticipated to be the most balanced configurations, since they contain the same amount of information upwards and downwards. On the contrary, top line and boxed charts are expected to lean towards one direction. For instance, they might be overestimated due to a tendency to approach the boundary as much as possible when it is present. The opposed behavior is expected for Tufte, which only has the lower bound. It turns out that this hypothesis is also refuted. Once again, the top line outperformed our expectations, since its errors are perfectly balanced. As stated, boxed charts exhibit a tendency to be overestimated, and Tufte follows the opposite behavior. Baseline is the configuration with more variety, which is also not centered. In fact, its values are commonly underestimated, and its behavior does not resemble the grid one at all. Time VS Error H5: Larger time of response results in better estimations An additional hypothesis that combines the two types of data is included. In particular, it is thought that the longer the time spent estimating a value, the better the outcome. However, the correlation index between both variables is calculated to be 0.07, which indicates a weak or negligible relationship. Therefore, the hypothesis is rejected, and no further study is considered with the gathered data. 34 Exploring how charts are perceived in 3D Figure 12: Error for every configuration 4.6.3 Eye tracking Chart1 :High-High Looking at Figure 13 it can already be seen that the most important element when estimating the height of a bar is the position of its top. However, we can recognise di↵erent behaviours. For example, users mainly focused on the left bar when the baseline was presented. The second bar received as much focus as the 100 label. Although we had to reject H2 in the previous section, for this particular chart it seems to be accurate, since there are not relevant di↵erences between Topline and Boxed behaviours. The reference for these two configurations was to indicate the maximum. It seems that it might have been slightly helpful for the second bar, which is also higher, but in general the feature is unnoticed. It is relevant to notice that Tufte is the only configuration on which the first bar does not get more attention than the second one. Moreover, the top of the bar on the right loses a lot of importance and users mainly focus on the Tufte line. Despite having di↵erent height and their top being at di↵erent cells, the user looked at the same element for both of them. In addition to this, it seems that it is the configuration that leads users to look to further values from the top, but this fact is not determinant unless it is repeated in other charts. Finally, the grid configuration has its focus more uniformly distributed among cells. In particular, the labels on the left gain a lot of attention despite not providing much relevant information. Therefore, the users did not only check the grid line next to the bar, but frequently went to check for its corresponding value. Additionally, grid lines also seem to be more relevant to the user than the top of the bar, but the results are not as evident as Tufte’s. 35 Exploring how charts are perceived in 3D Figure 13: Heatmap of the seconds spent looking at each cell for chart1 Chart2 :Low-High In Figure 14 it can be noticed a constant behaviour for the small bar that is consistent through all configurations. Participants do not only focus on the cell including the top of the bar, but also on the one above it. However, the cell at the bottom is ignored. For this chart, Top line and Boxed also resembled each other, mainly in the fact that both give considerable attention to the cell above the highest bar. This behaviour is not present in baseline, so might be a consequence of the maximum value reference. Nevertheless, Boxed also has some similarities with the baseline that are not present in Top line. For instance, the Boxed heatmap shows that the focus is more distributed through central cells that do not contain any elements, whereas Top line mostly ignores them. What is curious for Tufte is that, although the shorter bar does not contain any reference lines, the adjacent bar does. Therefore, users seem to use that one to estimate the value. For the rightest, the main focus is retained in the cell that contains both the top of the bar and the Tufte line. However, it is unclear why participants also gave significant importance to the cells below. When looking at the grid configuration, the focus is distributed among a few cells only. For the left bar, users put their attention on the cells corresponding to its top and to the reference line above it. For the right one, the line and the top of the bar fall into the same cell, so it keeps all the focus. As it happened in the previous chart, users seem to repeatedly look at the label of the line. 36 Exploring how charts are perceived in 3D Figure 14: Heatmap of the seconds spent looking at each cell for chart2 Chart3 :Low-Low This chart is shown in Figure 15.Themaincharacteristicisthatlowvaluesseemtobehavealmost identically for all configurations except the grid. It also might be seen that the bar on the left lead users to look to the cell below and ignored the one above. Taking the conclusions of Chart2, we might think that this fact is only related with proximity. This is, if the value is placed at the top of the cell, users also look to the cell above. Similarly, if it is placed at the bottom, they will consider the cell below. In particular, the fact that none of the values reached the first Tufte line made this configuration and the baseline look exactly the same. Consequently, they are estimated similarly. Both of them seem to give significantly more importance to the bar on the left than other configurations, but, excluding the grid, the results are practically indistinguishable. About the grid configuration, the behaviour remains the same as in previous arrangements. However, the grid line above the bar on the left seems to not get much attention and the origin line appears to be more relevant. 37 Exploring how charts are perceived in 3D Figure 15: Heatmap of the seconds spent looking at each cell for chart3 Chart4 :High-Medium In Figure 16 we can see that Boxed and Top line have similar behaviours, which was actually stated in H2. However, from what can be observed, top references are ignored, as these configurations also resemble Baseline a lot. This fact is not much compatible with H1 (these hypotheses are presented in Section 4.4.2). Tufte has a clear behaviour in this particular chart, undoubtedly users mainly focus on the tufte line rather than on the top of the bar. For the grid, it is evident that lines are useful. Moreover, the leftest bar shows that the line below the top of the bar is as relevant as the one above. Figure 16: Heatmap of the seconds spent looking at each cell for chart4 Chart5 :Medium-Medium 38 Exploring how charts are perceived in 3D Figure 17 shows consistent conclusions to the ones stated in previous charts and does not provide any additional information. Figure 17: Heatmap of the seconds spent looking at each cell for chart5 Chart6 :High-Medium Finally, in Figure 18 it might be seen that, for the left bar, the top reference of Boxed and Topline is considered. On the other hand, labels did not get much importance in the grid configuration. Figure 18: Heatmap of the seconds spent looking at each cell for chart6 39 Exploring how charts are perceived in 3D 5. Conclusions and Further Research 5.1 Conclusions In this project, an experiment was performed in order to see how charts are perceived in 3D. To accomplish this, 5 di↵erent configurations have been compared. The analysis show that the addition of a grid considerably improves the estimations. Even though Tufte is a similar feature, it did not accomplish as satisfactory results. The references provided received highly focus, but this was not reflected on the estimations. When looking at the eye tracking data, it is seemed that Grid usually takes advantage of the labels, which might have influenced the di↵erence between the two configurations. Furthermore, adding a horizontal line at the top encodes similar information to using boxed bars. Although the former accomplished better estimations with a lower time response, none of them accomplishes significantly di↵erent results that the ones obtained without any additional features. In fact, the top reference only manages to get focus when the top of the bar is close to it. Overall, despite the fact that grid lines are placed at the back, which makes them be perceived di↵erently depending on the view point, for the particular depth value chosen in the experiment they are still the best option among the studied ones. 5.2 Future Work There are two types of future work that might be interesting to consider. On the one hand, the collected data has not been exploited to its fullest. For instance, to explore the features that are first looked at, or more detailed analysis of the right bar a↵ecting the left one, which is possible due to the leftest being asked to be estimated first. Time and eye tracking data might be merged and consider which elements were looked at first. On the other hand, these conclusions might lead to further experiments. For example, comparing in more profundity Tufte and grid lines to consider if the absence of labels a↵ected the results. In addition to this, if the eye tracking data is left a part, it would be interesting to perform the experiment using di↵erent values and consider if short-term memory actually a↵ected the results. This study aimed to get an overall idea to the most relevant elements for users when perceiving values on 3D, however, the use of narrower cells for focus and the gathering of more users could allow getting further conclusions about what elements have focus when values are better estimated. Other studies about the depth can also be performed.