Can Virtual Reality be used as a significant stressor for studies using ECG?

ScienceDirect Available online at www.sciencedirect.com Procedia Computer Science 176 (2020) 3255–3262 1877-0509 © 2020 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0) Peer-review under responsibility of the scientific committee of the KES International. 10.1016/j.procs.2020.09.123 10.1016/j.procs.2020.09.123 1877-0509 © 2020 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0) Peer-review under responsibility of the scientific committee of the KES International. Available online at www.sciencedirect.com ScienceDirect Procedia Computer Science 00 (2020) 000–000 www.elsevier.com/locate/procedia 1877-0509 © 2019 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of KES International. 24th International Conference on Knowledge-Based and Intelligent Information & Engineering Systems Can Virtual Reality be used as a significant stressor for studies using ECG? Wilhelm Daniel Scherza,*, Victor Corcoba Magañab, Ralf Seepolda,c, Natividad Martínez Madridc,d Juan Antonio Ortegae aUbiquitous Computing Lab HTWG Konstanz, Alfred-Wachtel-Str. 8, 78462 Konstanz, Germany bUniversity of Oviedo, Oviedo 33003, Spain cI.M. Sechenov First Moscow State Medical University, 2-4, Bolshaya Pirogovskaya st., 119435 Moscow, Russian Federation dReutlingen University, Alteburgstr. 150, 72762 Reultingen, Germany eUniversity of Seville, Avda. Reina Mercedes s/n, Seville, Spain Abstract In previous studies, we used a method for detecting stress that was based exclusively on heart rate and ECG for differentiation between such situations as mental stress, physical activity, relaxation, and rest. As a response of the heart to these situations, we observed different behavior in the Root Mean Square of the Successive differences heartbeats (RMSSD). This study aims to analyze Virtual Reality via avirtual reality headset as an effective stressor for future works. The value of the Root Mean Square of the Successive Differences is an important marker for the parasympathetic effector on the heart and can provide information about stress. For these measurements, the RR interval was collected using a breast belt. In these studies, we can observe the Root Mean Square of the successive differences heartbeats. Additional sensors for the analysis were not used. We conducted experiments with ten subjects that had to drive a simulator for 25 minutes using monitors and 25 minutes using virtual reality headset.Before starting and after finishing each simulation,the subjects had to complete a survey in which they had to describe their mental state. The experiment results show that driving using virtual reality headset has some influence on the heart rate and RMSSD, but it does not significantly increase the stress of driving. © 2019 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of KES International. * Corresponding author. Tel.: +49-7531-206-698; fax: +49-7531-206-8698. E-mail address: [email protected] Available online at www.sciencedirect.com ScienceDirect Procedia Computer Science 00 (2020) 000–000 www.elsevier.com/locate/procedia 1877-0509 © 2019 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of KES International. 24th International Conference on Knowledge-Based and Intelligent Information & Engineering Systems Can Virtual Reality be used as a significant stressor for studies using ECG? Wilhelm Daniel Scherza,*, Victor Corcoba Magañab, Ralf Seepolda,c, Natividad Martínez Madridc,d Juan Antonio Ortegae aUbiquitous Computing Lab HTWG Konstanz, Alfred-Wachtel-Str. 8, 78462 Konstanz, Germany bUniversity of Oviedo, Oviedo 33003, Spain cI.M. Sechenov First Moscow State Medical University, 2-4, Bolshaya Pirogovskaya st., 119435 Moscow, Russian Federation dReutlingen University, Alteburgstr. 150, 72762 Reultingen, Germany eUniversity of Seville, Avda. Reina Mercedes s/n, Seville, Spain Abstract In previous studies, we used a method for detecting stress that was based exclusively on heart rate and ECG for differentiation between such situations as mental stress, physical activity, relaxation, and rest. As a response of the heart to these situations, we observed different behavior in the Root Mean Square of the Successive differences heartbeats (RMSSD). This study aims to analyze Virtual Reality via avirtual reality headset as an effective stressor for future works. The value of the Root Mean Square of the Successive Differences is an important marker for the parasympathetic effector on the heart and can provide information about stress. For these measurements, the RR interval was collected using a breast belt. In these studies, we can observe the Root Mean Square of the successive differences heartbeats. Additional sensors for the analysis were not used. We conducted experiments with ten subjects that had to drive a simulator for 25 minutes using monitors and 25 minutes using virtual reality headset.Before starting and after finishing each simulation,the subjects had to complete a survey in which they had to describe their mental state. The experiment results show that driving using virtual reality headset has some influence on the heart rate and RMSSD, but it does not significantly increase the stress of driving. © 2019 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of KES International. * Corresponding author. Tel.: +49-7531-206-698; fax: +49-7531-206-8698. E-mail address: [email protected] 3256 Wilhelm Daniel Scherz et al. / Procedia Computer Science 176 (2020) 3255–3262 2 Wilhelm Daniel Scherz / Procedia Computer Science 00 (2020) 000–000 Keywords: Stress detection ; ECG, driving ; heart rate ; cardiovascular ; biovital signal 1. Introduction Awareness about health is of great importance in modern society. Collecting bio vital data became an essential part of it, and as a result, it changed the perception and understanding of stress [1]. Nowadays, there are several approximations for the estimation of stress. Currently, the most common approaches for detecting stress implement three different principles: The first one uses surveys or self-report questionnaires to evaluate the mental state; the second approach uses laboratory biomarkers like adrenaline and cortisol [2], behavior pattern analysis [3], and the latter approach uses physiological parameters like heart rate or skin conductivity [4] for stress detection. The approach that was used in this experiment uses a reduced data set that separates between more or less stress while using the simulation via monitors or virtual reality headset. The main challenges of detecting stress belong not only to its detection itself but also its separation from other situations like relaxation, physical activity. Currently, there is a big choice of consumer devices and portable EGC devices that measure heart rate and monitor activity, which could be used for stress detection. Nevertheless, the challenge of separation remains. It is well known that there are some long-term effects of stress on the cardiovascular system [5, 6, 7], stress affects the immune response of the body [8], it can trigger panic attacks [9], and it lowers the life quality [10, 11]. Additionally, stress makes it difficult to cope with physical and mental demands [12, 13, 14], and it has a harmful influence on the development of brain structures [10]. Simplifying, stress could be considered to be a negative influence on health and life quality, so it is important to preserve and quantify it and reduce the amount of stress that we experience. As an additional complication, we can name the misdetection of physical activity as stress. By contrast with stress, physical activity has positive long-term effects such as change of the lipase composition of the body [15] and improvements in the cardiovascular system. Like stress, physical activity has short-term effects. As we could observe in previous works, many additional stressors or influencers change the heart's behavior. As an example, relaxation changes the heart rate and systolic blood pressure [16, 17]. This makes it essential to find stressors that we can use to collect different datasets and improve stress detection. For this work, it is essential to understand and to define stress. In this case, we define stress not as the peak of a dramatic event but as a reaction to the situations that require a fast response or elevated activity or just in unusual situations. We can consider that stress is a part of our daily life; it is a mechanism that helps us to evaluate unknown and dangerous situations and give faster responses by increasing the efficiency of our reactions. As an example, in 'fight or flight' situations, stress reduces the time for reacting and decision-making, and as a consequence, in a dangerous situation, we choose between confrontation or escape [18]. Driving is a typical daily activity for many of us. A large amount of studies shows that driving efficiency and safety are strongly influenced by emotions and stress levels of the driver [19, 20, 21] . Several studies prove a strong relationship between traffic accidents and stress [22, 23]. That is why we have chosen this use case for the current study, as it is a common activity that is easy and safe to simulate. 2. Methods and materials In the following section, we describe the protocol followed during the experiment and the materials used. To record the heart rate, ECG, and stress, two devices were used: the chest band Polar H101 and the wristband Empatica E42. Additionally, to measure surrounding conditions such as temperature, humidity, and C02 levels during the experiment, we applied the Netatmo3 sensor. The environment conditions were not used in the measurement of stress because the conditions while the simulation where similar and did not change much. 1 https://www.polar.com/de/produkte/accessoires/herzfrequenz_sensor_h10. 2 https://www.empatica.com/research/e4/ 3 https://www.netatmo.com/en-us/weather/weatherstation/indoor-module Wilhelm Daniel Scherz et al. / Procedia Computer Science 176 (2020) 3255–3262 3257 2 Wilhelm Daniel Scherz / Procedia Computer Science 00 (2020) 000–000 Keywords: Stress detection ; ECG, driving ; heart rate ; cardiovascular ; biovital signal 1. Introduction Awareness about health is of great importance in modern society. Collecting bio vital data became an essential part of it, and as a result, it changed the perception and understanding of stress [1]. Nowadays, there are several approximations for the estimation of stress. Currently, the most common approaches for detecting stress implement three different principles: The first one uses surveys or self-report questionnaires to evaluate the mental state; the second approach uses laboratory biomarkers like adrenaline and cortisol [2], behavior pattern analysis [3], and the latter approach uses physiological parameters like heart rate or skin conductivity [4] for stress detection. The approach that was used in this experiment uses a reduced data set that separates between more or less stress while using the simulation via monitors or virtual reality headset. The main challenges of detecting stress belong not only to its detection itself but also its separation from other situations like relaxation, physical activity. Currently, there is a big choice of consumer devices and portable EGC devices that measure heart rate and monitor activity, which could be used for stress detection. Nevertheless, the challenge of separation remains. It is well known that there are some long-term effects of stress on the cardiovascular system [5, 6, 7], stress affects the immune response of the body [8], it can trigger panic attacks [9], and it lowers the life quality [10, 11]. Additionally, stress makes it difficult to cope with physical and mental demands [12, 13, 14], and it has a harmful influence on the development of brain structures [10]. Simplifying, stress could be considered to be a negative influence on health and life quality, so it is important to preserve and quantify it and reduce the amount of stress that we experience. As an additional complication, we can name the misdetection of physical activity as stress. By contrast with stress, physical activity has positive long-term effects such as change of the lipase composition of the body [15] and improvements in the cardiovascular system. Like stress, physical activity has short-term effects. As we could observe in previous works, many additional stressors or influencers change the heart's behavior. As an example, relaxation changes the heart rate and systolic blood pressure [16, 17]. This makes it essential to find stressors that we can use to collect different datasets and improve stress detection. For this work, it is essential to understand and to define stress. In this case, we define stress not as the peak of a dramatic event but as a reaction to the situations that require a fast response or elevated activity or just in unusual situations. We can consider that stress is a part of our daily life; it is a mechanism that helps us to evaluate unknown and dangerous situations and give faster responses by increasing the efficiency of our reactions. As an example, in 'fight or flight' situations, stress reduces the time for reacting and decision-making, and as a consequence, in a dangerous situation, we choose between confrontation or escape [18]. Driving is a typical daily activity for many of us. A large amount of studies shows that driving efficiency and safety are strongly influenced by emotions and stress levels of the driver [19, 20, 21] . Several studies prove a strong relationship between traffic accidents and stress [22, 23]. That is why we have chosen this use case for the current study, as it is a common activity that is easy and safe to simulate. 2. Methods and materials In the following section, we describe the protocol followed during the experiment and the materials used. To record the heart rate, ECG, and stress, two devices were used: the chest band Polar H101 and the wristband Empatica E42. Additionally, to measure surrounding conditions such as temperature, humidity, and C02 levels during the experiment, we applied the Netatmo3 sensor. The environment conditions were not used in the measurement of stress because the conditions while the simulation where similar and did not change much. 1 https://www.polar.com/de/produkte/accessoires/herzfrequenz_sensor_h10. 2 https://www.empatica.com/research/e4/ 3 https://www.netatmo.com/en-us/weather/weatherstation/indoor-module Wilhelm Daniel Scherz/ Procedia Computer Science 00 (2020) 000–000 3 2.1. Heart rate and ECG signal The heart signal's advantage is that it can be acquired in a non-invasive way, and the Heart Rate Variability (HRV) correlates with stress. In this study, two sensors were used to measure stress: the chest band Polar H10 and the wristband Empatica E4. There are two domains of how the heart rate variability can be analyzed: by time domain and by frequency domain. For the time domain analysis, quantifying the mean and the standard derivation of two consecutive heartbeats intervals has to be done. A heartbeat interval is defined as RR Interval, and an R peak represents the highest peak of a heartbeat on the QRS complex. An RR interval is the time between two heartbeats. The used measurements for the time domain are the following: average RR in milliseconds (ms), standard derivation of consecutive RR during the experiment (SDNN), average heart rate (HR) in ms, standard derivation heart rate (STD HR), the root mean square of successive differences (RMSSD) and the pNN50 (%) that indicates the proportion of consecutive RR intervals with the difference of more than 50ms. This value decreases when stress increases. For the frequency domain analysis, we have to calculate the power of the respiratory dependent higher frequency and the lower frequency components of the heart rate. For this, we use the following measurement: LF/HF indicates the ration of the low-frequency power (0.04-0.15 Hz) that is modulated by the sympatric and parasympathetic nervous system and the high-frequency power (0.14-0.4 Hz) related to parasympathetic activity. This value indicates a global sympatric vagal balance [24]. High results for LF/HF indicate that the sympathetic system is dominant. This happens when there is elevated stress. Nowadays, there is available a wide range of devices that enable the non-invasive recording of the heart. One of the most popular methods that are used in wearable devices is photo-plethysmography. This kind of device measures the blood volume pulse. The main drawback of this method is the sensitivity to movement and the changes in the device's contact pressure. [25]. An alternative that also allows the non-invasive measurement of the heart rate and, at the same time, provides higher accuracy are the chest bands with electrodes and ECG function. [26] . Polar H10 that was used for this study belongs to the second group of the devices and offers similar functionality as an ECG Holter [27]. 2.2. Driving simulator For this study, we used the driving simulator 'City Car Driving' [28]. The main reasons for choosing this simulator were the following: realistic driving experience based on advanced car physics, real traffic rules, adjustable traffic conditions, realistic pedestrian and car simulation, and high-quality graphics simulator. For this experiment, we have chosen the scene 'Old district' because it characterizes with narrow streets, simple crossing, clear traffic patterns. The software simulator was executed on a driving simulator with characteristics, as shown in Table 1. For driving without virtual Reality, three 27-inch screens were used. To operate simulation and to provide a realistic driving experience, we used the steering wheel Logitech G24 with gearbox and pedals. This steering wheel also offers force feedback. This allows us to achieve an immersive experience in the virtual environment. Table 1. Specifications of the PC in which the driving simulator is run Model Alienware Area-51 R4 Processor Intel Core i7-7800X Chipset Intel X299 PCH Memory 16 GB DDR4 2666 MHz GPU 2 X Geforce 1080 TI SLI Storage 128 GB SanDisk M.2 SSD The driving performance was recorded with a specially developed program that captures the rotation of the steering wheel, the pressure applied on the pedals, and the number of traffic infractions the test subjects realized. 3258 Wilhelm Daniel Scherz et al. / Procedia Computer Science 176 (2020) 3255–3262 4 Wilhelm Daniel Scherz / Procedia Computer Science 00 (2020) 000–000 2.3. Music tempo Music can influence human behavior. Several studies show that e.g., fast music reduces the amount of time that customers spend in supermarkets; in pubs, it makes people drink their beverages faster [29, 30] or while driving fast music can increase the readiness to drive faster [31]. For this study, the participants listened to the music using noise-canceling headsets, increasing the immersion into a simulation, and reducing external influences. Each participant could configure the volume individually according to their personal preferences. Two playlists were created for the participants. The first playlist consists of low tempo compositions (65-71 bpm), and the second playlist included songs with a faster tempo (155-188 bpm). The audio was reproduced with a quality of 320 bps. Aditional to the music, the driver hears the sounds of the virtual environment of the driving simulator. 2.4. Survey Before starting to drive, an initial survey had to be completed by the participants, as well as the second survey after driving. The first survey gives information about the emotional state, physical state, and the experiences of the participants. The final survey is aimed to collect the data about the emotional and physiological state of the participants after completion of the driving exercise and to observe if the driving task had any influence on the driver. Additional questions in the final survey refer to the realism of the simulation, own perception of the driving performance, and the environment perceptions such as temperature and noise. These additional questions allow us to compare their subjective perception of the situation with the measured data and to find out whether the simulator is realistic enough and usable as a tool for collecting data. 2.5. Procedure description The study is composed of 6 parts. First, the participant answers the survey and places the chest band and writs band. Then, the participant sits on a seat and places the headset and starts to listen to Mozart's Sonata for Two Pianos in D major. This first phase is used to relax the participant and stabilize the sensors and devices. After this, the participant starts driving for 25 minutes, and the heart signal, environment characteristics, and driving are recorded. As soon as the participant starts driving, the system proposes a route that he has to drive. The proposed route always has a length of 5 km. The amount of traffic is always the same and comparable. A counter starts at 20 points, for each infraction a point is taken, and if the counter reaches 0 points, the route has to be restarted. After 25 minutes, the participant can rest for 1-2 minutes. After the participant has set up comfortably and started the VR, the VR headset a new route is initialized with the segments of 5 km and 25 minutes. After this, the participant can rest 1-2 minutes and has to answer the final survey. 3. Results A total of 10 drivers with an average age of 25.4 (max: 33, min: 20; std. dev.: 3.47) and driving experience of 7 to 8 years (max 15, min: 2, std. dev: 3.62) participated in the experiment. First, the participant put on the sensors, completed the initial survey, and then listened to Mozart's Sonata for Two Pianos in D major using headphones. This track has been selected because it improves mental function [32]. The objective of this phase is to be relaxed before the driving test and to stabilize the sensors. Participants drove for 25 minutes. The drivers had to complete the routes proposed by the GPS of the driving simulator. Each route has a length of 5 km, and its level of difficulty is comparable because the concentration of vehicles and pedestrians is the same in all cases. The driving simulator assigns points to the participant at the beginning of the route. Each time an infraction is committed, points are deducted. When the score is zero, he or she must repeat the route. This allows the participant to focus on the driving task as if in a real environment [33]. The driving time is 25 minutes to ensure that there is enough time for the stress data to be valid [34]. Afterward, the participants relaxed for 5 minutes and started the same test again, but in this case, using virtual Reality. Wilhelm Daniel Scherz et al. / Procedia Computer Science 176 (2020) 3255–3262 3259 4 Wilhelm Daniel Scherz / Procedia Computer Science 00 (2020) 000–000 2.3. Music tempo Music can influence human behavior. Several studies show that e.g., fast music reduces the amount of time that customers spend in supermarkets; in pubs, it makes people drink their beverages faster [29, 30] or while driving fast music can increase the readiness to drive faster [31]. For this study, the participants listened to the music using noise-canceling headsets, increasing the immersion into a simulation, and reducing external influences. Each participant could configure the volume individually according to their personal preferences. Two playlists were created for the participants. The first playlist consists of low tempo compositions (65-71 bpm), and the second playlist included songs with a faster tempo (155-188 bpm). The audio was reproduced with a quality of 320 bps. Aditional to the music, the driver hears the sounds of the virtual environment of the driving simulator. 2.4. Survey Before starting to drive, an initial survey had to be completed by the participants, as well as the second survey after driving. The first survey gives information about the emotional state, physical state, and the experiences of the participants. The final survey is aimed to collect the data about the emotional and physiological state of the participants after completion of the driving exercise and to observe if the driving task had any influence on the driver. Additional questions in the final survey refer to the realism of the simulation, own perception of the driving performance, and the environment perceptions such as temperature and noise. These additional questions allow us to compare their subjective perception of the situation with the measured data and to find out whether the simulator is realistic enough and usable as a tool for collecting data. 2.5. Procedure description The study is composed of 6 parts. First, the participant answers the survey and places the chest band and writs band. Then, the participant sits on a seat and places the headset and starts to listen to Mozart's Sonata for Two Pianos in D major. This first phase is used to relax the participant and stabilize the sensors and devices. After this, the participant starts driving for 25 minutes, and the heart signal, environment characteristics, and driving are recorded. As soon as the participant starts driving, the system proposes a route that he has to drive. The proposed route always has a length of 5 km. The amount of traffic is always the same and comparable. A counter starts at 20 points, for each infraction a point is taken, and if the counter reaches 0 points, the route has to be restarted. After 25 minutes, the participant can rest for 1-2 minutes. After the participant has set up comfortably and started the VR, the VR headset a new route is initialized with the segments of 5 km and 25 minutes. After this, the participant can rest 1-2 minutes and has to answer the final survey. 3. Results A total of 10 drivers with an average age of 25.4 (max: 33, min: 20; std. dev.: 3.47) and driving experience of 7 to 8 years (max 15, min: 2, std. dev: 3.62) participated in the experiment. First, the participant put on the sensors, completed the initial survey, and then listened to Mozart's Sonata for Two Pianos in D major using headphones. This track has been selected because it improves mental function [32]. The objective of this phase is to be relaxed before the driving test and to stabilize the sensors. Participants drove for 25 minutes. The drivers had to complete the routes proposed by the GPS of the driving simulator. Each route has a length of 5 km, and its level of difficulty is comparable because the concentration of vehicles and pedestrians is the same in all cases. The driving simulator assigns points to the participant at the beginning of the route. Each time an infraction is committed, points are deducted. When the score is zero, he or she must repeat the route. This allows the participant to focus on the driving task as if in a real environment [33]. The driving time is 25 minutes to ensure that there is enough time for the stress data to be valid [34]. Afterward, the participants relaxed for 5 minutes and started the same test again, but in this case, using virtual Reality. Wilhelm Daniel Scherz/ Procedia Computer Science 00 (2020) 000–000 5 We have used the Sympathetic Tone Index (SNS) [35] to compare the stress suffered by the participant when they are driving using the driving simulator with and without virtual Reality. SNS index is calculated using Mean HR (bpm), Baevsky’s stress index, and LF power normal unit (n.u.). Figure 1 captures the values obtained by the participants without and with virtual Reality. The results show that there are no significant differences in the stress level when drivers do not use virtual reality (average SNS index = 0.898, SD = 1.23) with when they do use it (average SNS index = 0.784, SD = 1.16), t (9) = 1.210, p> 0.05. An SNS index value of zero means that the parameters reflecting sympathetic activity are, on average equal to the normal population average. Non-zero SNS index values describe how many SDs below (negative values) or above (positive values) the normal population average the parameter values are. Fig. 1. Comparison of Sympathetic Tone Index (SNS). Table 2 shows the results of the analysis of heart rate variability in time and frequency. We can observe that the participants who manifested a high level of stress during the driving test without virtual Reality also presented it when they used virtual Reality. This is the case of drivers 3, 5, 6, 7, and 10. Figure 2 captures the results of the survey completed by the participants before starting the driving test. Participants were asked about the level of stress they normally feel when driving in a real environment and the quality of sleep the last night. We have used a Likert type scale where ‘1’ has a very positive meaning (the driver says that he does not suffer stress when driving or he slept very well the night before) and ‘5’ is considered very bad (the driver suffers a lot of stress when driving in the real environment or did not sleep well the last night). The responses have been grouped according to the SNS index obtained when using the simulator. There are two groups, those that obtain an SNS index less than or equal to 0.76 (relaxed drivers) and those that obtain an SNS index higher than 0.76 (stressed drivers). In both cases, we are considering the values obtained with and without virtual Reality. The data indicates that there is a significant difference between both groups F (9) = 24.20, p <0.05 second the ANOVA test when we consider the stress question. The drivers who feel the most stress when driving in the real environment are the ones who show the most stress in the virtual environment, using virtual Reality and without it. In the case of the question about sleep quality, the drivers who presented the highest stress level during the driving test were the ones who slept the worst the night before. The difference between the two groups is significant. The ANOVA test result is F (9) = 12.25, p <0.05. -2,00 -1,00 0,00 1,00 2,00 3,00 4,00 12345678910 SNS Index Driver SNS Index (Without virtual reality) SNS Index (With virtual reality) 3260 Wilhelm Daniel Scherz et al. / Procedia Computer Science 176 (2020) 3255–3262 6 Wilhelm Daniel Scherz / Procedia Computer Science 00 (2020) 000–000 Table 2. Heart Rate Analysis Driver Virtual Reality Average RR (ms) SDNN (ms) Average HR (bpm) STD HR (bpm) RMSSD (ms) pNN50 LF/HF 1 No 813.11 23.90 73.79 2.21 24.72 4.10 3.36 Yes 743.71 42.41 80.67 4.72 31.80 10.56 2.48 2 No 860.17 32.67 69.75 2.66 24.38 3.98 3.96 Yes 862.86 30.54 69.53 2.47 22.49 2.57 4.76 3 No 674.51 51.95 88.95 6.85 32.15 8.72 6.29 Yes 686.20 61.92 87.48 7.86 40.04 13.53 5.04 4 No 813.95 43.33 73.71 4.02 35.54 13.96 3.35 Yes 884.46 58.33 67.83 4.57 46.81 22.78 2.78 5 No 625.47 29.776 95.92 4.57 19.29 1.88 2.93 Yes 650.04 30.624 92.30 4.31 20.15 1.53 2.38 6 No 693.55 28.961 86.51 3.63 18.07 1.01 7.62 Yes 697.13 37.552 86.06 4.58 20.12 2.65 8.78 7 No 700.52 55.332 85.65 6.80 47.19 16.23 1.78 Yes 740.45 40.64 81.03 4.36 33.94 13.20 2.05 8 No 1175.90 77.084 51.02 3.37 81.18 48.88 2.11 Yes 1133.00 82.77 52.91 3.92 83.59 50.36 2.17 9 No 804.11 44.07 74.61 4.11 32.63 9.91 5.93 Yes 769.06 68.41 78.01 6.87 48.95 19.22 5.27 10 No 729.06 28.20 82.29 3.19 19.32 1.09 3.56 Yes 743.37 28.34 80.71 3.09 20.15 2.20 3.36 Fig. 2. Results of the survey grouped by SNS index. 0 0,5 1 1,5 2 2,5 3 3,5 4 4,5 SNS index <= 0.76 SNS index > 0.75 Likert Scale (1: Very good, 5: Very bad) Sleep Quality before the driving experiment Stress in a real driving environment Wilhelm Daniel Scherz et al. / Procedia Computer Science 176 (2020) 3255–3262 3261 6 Wilhelm Daniel Scherz / Procedia Computer Science 00 (2020) 000–000 Table 2. Heart Rate Analysis Driver Virtual Reality Average RR (ms) SDNN (ms) Average HR (bpm) STD HR (bpm) RMSSD (ms) pNN50 LF/HF 1 No 813.11 23.90 73.79 2.21 24.72 4.10 3.36 Yes 743.71 42.41 80.67 4.72 31.80 10.56 2.48 2 No 860.17 32.67 69.75 2.66 24.38 3.98 3.96 Yes 862.86 30.54 69.53 2.47 22.49 2.57 4.76 3 No 674.51 51.95 88.95 6.85 32.15 8.72 6.29 Yes 686.20 61.92 87.48 7.86 40.04 13.53 5.04 4 No 813.95 43.33 73.71 4.02 35.54 13.96 3.35 Yes 884.46 58.33 67.83 4.57 46.81 22.78 2.78 5 No 625.47 29.776 95.92 4.57 19.29 1.88 2.93 Yes 650.04 30.624 92.30 4.31 20.15 1.53 2.38 6 No 693.55 28.961 86.51 3.63 18.07 1.01 7.62 Yes 697.13 37.552 86.06 4.58 20.12 2.65 8.78 7 No 700.52 55.332 85.65 6.80 47.19 16.23 1.78 Yes 740.45 40.64 81.03 4.36 33.94 13.20 2.05 8 No 1175.90 77.084 51.02 3.37 81.18 48.88 2.11 Yes 1133.00 82.77 52.91 3.92 83.59 50.36 2.17 9 No 804.11 44.07 74.61 4.11 32.63 9.91 5.93 Yes 769.06 68.41 78.01 6.87 48.95 19.22 5.27 10 No 729.06 28.20 82.29 3.19 19.32 1.09 3.56 Yes 743.37 28.34 80.71 3.09 20.15 2.20 3.36 Fig. 2. Results of the survey grouped by SNS index. 0 0,5 1 1,5 2 2,5 3 3,5 4 4,5 SNS index <= 0.76 SNS index > 0.75 Likert Scale (1: Very good, 5: Very bad) Sleep Quality before the driving experiment Stress in a real driving environment Wilhelm Daniel Scherz/ Procedia Computer Science 00 (2020) 000–000 7 In conclusion, virtual Reality did not cause additional stress when the participants drove in the driving simulator. The main advantage of this technology is that the simulation seems more realistic. However, its main drawback is that it causes dizziness. Participants filled out a survey where they evaluated using a Likert scale if they felt dizzy after completing the driving test. On this scale, '1' meant that they were not dizzy and '5' that they were very dizzy. The mean value was 3.1 ± 0.73 for drivers who used virtual Reality. In the case of 21 drivers who carried out the same test but without using virtual Reality, the mean value was 1.33 ± 0.96. Therefore, virtual Reality can be a headset learning tool for drivers to improve driving from an ecological and safety point of view. However, virtual reality devices must improve to avoid dizziness that discourages their use and reduces training time. 4. Conclusions and future work In previous studies, we collected data in four different situations, and we tried to describe stress using heart signals detection. In this study, we only observed the heart signals while driving a simulator with and without VR. The aim was to analyses the use of virtual reality headset as a possible stressor. For this, we compared the HRV of 10 participants while driving with and without virtual Reality. The obtained data showed that the participants had a slightly higher heart rate and other physiological parameters, but a proof that VR induces more stress could not be observed in such small group although we could observe some stress response and dizziness by some participants while using the VR. Future research should consider the potential effects of virtual Reality as a possible stressor more carefully, for example, by expanding the study to a bigger group of participants. If the results do replicate in a bigger study, different applications for the VR could be considered, such as teaching driving or other applications. Acknowledgments This research was partially funded by the EU Interreg V-Program "Alpenrhein-Bodensee-Hochrhein": Project "IBH Living Lab Active and Assisted Living", grants ABH040, ABH041, ABH066 and ABH068. References [1] L. Butler, K. Mercer, K. McClain-Meeder, D. Horne und M. Dudley, „Six domains of self-care: Attending to the whole person,“ Journal of Human Behavior in the Social Environment, Nr. 29, pp. 107-124, 2019. [2] H. Juliane und S. Melanie, „The physiological response to Trier Social Stress Test relates to subjective measures of stress during but not before or after the test,“ Psychoneuroendocrinology, Volume 37, Issue 1, 2012. [3] S. Chohen, T. Kamarck und R. Mermelstein, „A Global Mesure of Preceved Stress,“ Jurnal of Healt and Social Behavior , Bd. 24, Nr. 4, pp. 385-396, 1983. [4] M. Magno, L. Benini, C. Spagnol und E. Popovici , „Wearable Low Power Dry Surface Wireless Sensor Node for Healthcare Monitoring Application,“ nternational Conference on Wireless and Mobile Computing, Networking and Communications (WiMob), pp. 189-195, 2013. [5] Y. Mei, M. D. Thompson, R. A. Cohen und X. Tong, „Autophagy and oxidative stress in cardiovascular diseases,“ in Biochimica et Biophysica Acta (BBA) - Molecular Basis of Disease, 2015. [6] J. R. Zaffalon Junior, A. O. Viana, G. E. Lameira de Melo und K. De Angelis, „The impact of sedentarism on heart rate variability (HRV) atrest and in response to mental stress in young women,“ PHYSIOLOGICAL REPORTS, Bd. 6, Nr. 18, SEP 2018. [7] I. R. Barrows, A. Ramezani und D. S. Raj, „Inflammation, Immunity, and Oxidative Stress in Hypertension—Partners in Crime?,“ Advances in Chronic Kidney Disease, Bd. 26, Nr. 2, pp. 122-130, 2019. [8] E. S. Epel, E. H. Blackburn, J. Lin, F. S. Dhabhar, N. E. Adler, J. D. Morrow und R. M. Cawthon, „Accelerated telomere shortening in response to life stress,“ Proceedings of the National Academy of Sciences , Bd. 101, Nr. 49, pp. 17312-1731, 2004. [9] N. B. Schmidt, D. R. Lerew und R. J. Jackson, „The Role of Anxiety Sensitivity in the Pathogenesis of Panic: Prospective Evaluation of Spontaneous Panic Attacks During Acute Stress,“ Jurnal of Abnormal Psychology, Bd. Volumen 5, Nr. 2, pp. 355-364, 1997. [10] S. J. Lupien, B. S. McEwen, M. R. gunnar und C. Heim, „Effects of stress throughout the lifespan on the brain, behaviour and cognition,“ Nature Reviews Neuroscience 10, pp. 434-445, 06 2009. [11] T. Kidd, L. A. Carvalho und A. Steptoe, „The relationship between cortisol responses to laboratory stress andcortisol profiles in daily life,“ Biological Psychology, pp. 34-40, 25 02 2014. 3262 Wilhelm Daniel Scherz et al. / Procedia Computer Science 176 (2020) 3255–3262 8 Wilhelm Daniel Scherz / Procedia Computer Science 00 (2020) 000–000 [12] J. Martínez Fernández, J. C. Augusto, G. Trombino, R. Seepold und N. Martínez Madrid, „Self-Aware Trader: A New Approach to Safer Trading,“ in Journal of Universal Computer Science, 2013. [13] A. J. Tomiyama, „Stress and Obesity,“ Annual Review of Psychology, Bd. 70, pp. 703-718, 2019. [14] E. V. Goldfarb, T. Alexa, L. Davachi und E. A. Phelps, „Acute stress throughout the memory cycle: Diverging effects on associative and item memory,“ Journal of Experimental Psychology, Bd. 148, Nr. 1, p. 13–29, 2019. [15] B. Gutin, P. Barbeau, S. Owens, C. R. Lemmon, M. Bauman, J. Allison, H.-S. Kang und M. Litaker, „Effects of exercise intensity on cardiovascular fitness, total body composition, and visceral adiposity of obese adolescents,“ American Society for Clinical Nutrition, Bd. 75, Nr. 5, pp. 818-826, May 2002. [16] K. Wendy E. J. and R. Nikki S., “Relaxing Music Prevents Stress-Induced Increases in Subjective Anxiety, Systolic Blood Pressure, and Heart Rate in Healthy Males and Females,” Journal of Music Therapy, vol. 38, no. 4, pp. 254-272, 2001. [17] T.-J. Pia, T. Siiri, P. Tarja, L.-H. Minna-Johanna, L. Jari, H. Andreas, A.-T. Ansa, P. Satu, L. Marianne and H. Minna, "Effects of live music therapy on heart rate variability and self-reported stress and anxiety among hospitalized pregnant women: A randomized controlled trial," Nordic Journal of Music Therapy, no. 1, pp. 7-26, 2019. [18] A. S. P. Jansen, X. V. Nguyen, V. Karpitskiy, T. C. Mettenleiter und A. D. Loewy, „Central command neurons of the sympathetic nervous system: basis of the fight-or-flight response,“ Science , Bd. 270, Nr. 5236, pp. 644-646, 1995. [19] J. A. H. a. R. W. Picard, „Detecting stress during real-world driving tasks using physiological sensors,“ IEEE Transactions on Intelligent Transportation Systems, Bd. vol. 6, Nr. no. 2, p. 156–166, Jun 2005. [20] J. R. P. a. L. R. Bent, „A validation of techniques using surface EMG signals from dynamic contractions to quantify muscle fatigue during repetitive tasks,“ Journal of Electromyography and Kinesiology, Bd. 7, Nr. 2, p. 131–139, Jun 1997. [21] V. B. a. K. Adalarasu, „EMG-based analysis of change in muscle activity during simulated driving,“ Journal of Bodywork and Movement Therapies, p. 151–158, 2007. [22] J. Myounghoon , „Don’t Cry While You’re Driving: Sad Driving Is as Bad as Angry Driving,“ International Journal of Human–Computer Interaction, Bd. 32, Nr. 10, pp. 777-790, 2016. [23] T.-Y. Hu, X. Xie und J. Lib, „Negative or positive? The effect of emotion and mood on risky driving,“ Transportation Research Part F: Traffic Psychology and Behaviour, Bd. 16, pp. 29-40, Jan 2013. [24] J. Izquierdo-Reyes, R. A. Ramirez-Mendoza, M. R. Bustamante-Bello, J. L. Pons-Rovira und J. E. Gonzalez-Vargas , „Emotion recognition for semi-autonomous vehicles framework,“ International Journal on Interactive Design and Manufacturing (IJIDeM) , Bd. 12, Nr. 4, p. 1447–1454, November 2018. [25] K. Georgiou, A. V. Larentzakis , N. N. Khamis , G. I. Alsuhaibani , Y. A. Alaska und E. J. Giallafos, „Can Wearable Devices Accurately Measure Heart Rate Variability? A Systematic Review,“ Folia Med (Plovdiv), Bd. 60, Nr. 1, pp. 7-20, 2018. [26] D. Giles, N. Draper und W. Neil, „Validity of the Polar V800 heart rate monitor to measure RR intervals at rest,“ European journal of applied physiology, Bd. 116, Nr. 3, pp. 563-571, 2015. [27] R. Gilgen-Ammann, T. Schweizer und T. Wyss , „RR Interval Signal Quality of a Heart Rate Monitor and an ECG Holter at Rest and During Exercise,“ Eur J Appl Physiol, Bd. 7, p. 1525‐1532, 2019. [28] „City Car Driving - Car Driving Simulator, PC Game,“ 22 Juli 2019. [Online]. Available: https://citycardriving.com/. [29] . J. D. Herrington, „Effects of music in service environments: a field study,“ Journal of Services Marketing, Bd. 2, pp. 26-41, 1996. [30] A. C. North und D. J. Hargreaves, „Can Music Move People?: The Effects of Musical Complexity and Silence on Waiting Time,“ Environment and Behavior, Bd. 1, pp. 136-149, 1 January 1999. [31] W. Brodsky, „The effects of music tempo on simulated driving performance and vehicular control,“ Transportation Research Part F: Traffic Psychology and Behaviour, Bd. 4, Nr. 4, pp. 219-241, December 2001. [32] Y. Limyati, R. Wahyudianingsih, R. D. Maharani und M. T. Christabella, „Mozart’s Sonata for Two Pianos K448 in D-Major 2nd Movement Improves Short-Term Memory and Concentration,“ Journal of Medicine and Health, Bd. Vol. 2, Nr. No. 4, pp. 930-937, August 2019. [33] E. Rendon-Velez, P. van Leeuwenb, R. Happee, I. Horváth, W. van der Vegte und J. de Winter, „The effects of time pressure on driver performance and physiological activity: A driving simulator study,“ Transportation Research Part F: Traffic Psychology and Behaviour, Bd. 41, pp. 150-169, August 2016. [34] Task force of the European society of cardiology and the North American society of pacing and electrophysiology., „Heart rate variability – standards of measurement, physiological interpretation,and clinical use,“ European Heart Journal, Bd. 93, Nr. 5, pp. 354-381, 1996. [35] D. Nunan , G. R. H. Sandercock und D. A. Brodie, „A quantitative systematic review of normal values,“ Pacing Clin Electrophysiol. , Bd. 33, Nr. 11, p. 1407‐1417, November 2010.