Assessing ambient temperature measurements in road cycling races

Assessing ambient temperature measurements in road cycling races Aitor Pinedo-Jauregi 1,2,3 and Ander Romarate 4,5 Abstract The impact of heat stress on sports performance is a highly relevant topic. Environmental factors, such as temperature and humidity, play a significant role in determining athletic performance. Cycling computers are commonly used in road cycling to capture various data, including temperature. However, it is crucial to emphasize the importance of accurately measuring ambient temperature. This study aimed to assess the reliability and validity of the Garmin Edge 830 cycling computers in measuring ambient temperature during road cycling competitions. We analyzed temperature data collected by 12 different cycling computers. We focused on assessing these devices’reliability and validity compared to measurements obtained from an official meteorological station. The interclass correlation coefficient between the devices was 0.468 (95% CI: 0.254 to 0.626; p < 0.001). The temperature readings exhibited a significant variability across the different devices. Furthermore, when comparing the data with the readings from the meteorological station, there was no agreement, as indicated by a Lin’s Concordance Coefficient (LCC) of 0.398 and a Mean Absolute Percent Error (MAPE) of 8.87%. This research raises concerns regarding the reliability and validity of cycling computer devices for measuring ambient temperature. Keywords Heat, meteorological station, microclimate, technological device, weather Introduction The impact of heat stress on sports performance has received increasing attention. 1 Environmental factors such as temperature and humidity have a substantial impact on athletic performance, leading to a growing body of research in this field. 2–5 Among these factors, temperature is particularly important and can present significant challenges for athletes. 6 In endurance sports like cycling, temperature regulation is critical as it directly affects metabolic efficiency, cardiovascular strain, and the body’s ability to maintain optimal performance levels during prolonged efforts. 6–8 Currently, to measure the effects of temperature on performance, the wet bulb globe temperature (WBGT) is used to quantify thermal stress. 6,9 However, when WBGT measurements are unavailable, ambient temperature and humidity are used as alternative indicators. 9 In many cases, coaches only have temperature data and use this value as a reference of heat strain. For example, coaches can predict dehydration based on the ambient temperature. 6 In road cycling, cycling computers are expected to record data during races and training sessions to quantify internal and external load. These cycling computers are equipped with temperature sensors that allow the recording of temperature data throughout the entire route. Consequently, some studies have used these data to assess the impact on sports performance. 10 The authors suggested that the power values of the cyclists changed Reviewer: Andrew Grundstein (University of Georgia, USA) 1 Society, Sports, and Exercise Research Group (GIKAFIT), Department of Physical Education and Sport, Faculty of Education and Sport, University of the Basque Country UPV-EHU, Vitoria-Gasteiz, Spain 2 AKTIBOki: Research Group in Physical Activity, Physical Exercise and Sport, Department of Physical Education and Sport, Faculty of Education and Sport, University of the Basque Country (UPV/EHU), Vitoria-Gasteiz, Spain 3 Bioaraba, Physical Activity, Exercise and Health Group, Vitoria-Gasteiz, Spain 4 Department of Physical Education and Sport, Faculty of Education and Sport, University of the Basque Country UPV-EHU, Vitoria-Gasteiz, Spain 5 Emen4sport, Performance, Health, and Sport, Leioa, Spain Corresponding author: Aitor Pinedo-Jauregi, Department of Physical Education and Sport, Faculty of Education and Sport, University of the Basque Country (UPV/EHU), Portal de Lasarte, 71 - 01007 Vitoria-Gasteiz, Spain. Email: aitor[email protected] Original research International Journal of Sports Science & Coaching 2025, Vol. 20(2) 742–747 © The Author(s) 2024 Article reuse guidelines: sagepub.com/journals-permissions DOI: 10.1177/17479541241308504 journals.sagepub.com/home/spo depending on the ambient temperature and concluded that the optimal temperature range is between 10°C and 25°C. However, it must be taken into account that the cycle computers are placed on the handlebars of the bicycle. This placement is not the most suitable for measuring ambient temperature because the sensor is exposed to direct solar radiation, which may affect the measurement. In addition, for example, the technical specifications provided for Garmin Edge devices lack essential details such as accuracy data, sampling rate, and response time of temperature sensors. As a result, it is difficult to assess the reliability of the sensor right from the start. In previous studies of marathon running, researchers have discovered the presence of microclimates along the competition course. 11 While no specific research has been conducted on microclimates in cycling, it is reasonable to speculate that similar conditions could exist in cycling events due to their similarities with running. Accurately measuring the ambient temperature is essential to estimating the thermal stress experienced by cyclists. To ensure the reliability and validity of temperature measurements, it is important to consider both concepts. Reliability ensures consistent measurements, while validity confirms the accuracy of the device’s readings. Both factors are crucial when relying on device measurements, particularly in the case of ambient temperature. Moreover, it is important to consider that a standardized protocol must be followed when measuring ambient temperature. It is recommended to conduct temperature measurements in shaded areas, away from direct solar radiation, as it has been observed that sunlight can distort temperature readings. 12,13 Therefore, accurate temperature measurements are essential to obtaining unbiased and reliable data. The reliability and validity of measuring ambient temperature using cycling computers mounted on bike handlebars have yet to be established. Our main hypothesis is that cycling computers yield biased ambient temperature readings. Hence, the objective of this study was to analyze the reliability and validity of the Garmin EDGE 830 cycling computers in measuring ambient temperature during road cycling competitions. It was hypothesised that the Garmin EDGE 830 cycling computers are useful for measuring ambient temperature. Methods The data were obtained from 12 Garmin EDGE 830 (Garmin International Inc, Olathe, KS) devices used in the under-23 (U23) road cycling race in Amurrio (Araba, Basque Country, Spain). The road race took place on a circuit around the village. The researchers collected the “.fit”file, which was recorded using a cycling computer Garmin EDGE 830. All the devices were mounted in their standard location on the bike handlebar. All the data were recorded throughout the race. The temperature was recorded with an integrated sensor in a cycling computer. We analyzed the temperature data for the 2-h duration of the race. This research follows the ethical principles established by the Declaration of Helsinki and its subsequent revisions. As the University of the Basque Country ethics committee indicated, ethical approval is unnecessary for using ambient temperature data. The official meteorological station (Euskalmet, Basque Country) was used as a reference method. This station provided temperature data at 10-min intervals. The farthest point of the circuit and the meteorological station were approximately 5800 meters apart. The altitude of the meteorological station was 230 m, and the range altitude of the circuit was between 182 m and 352 m. On the race day, the sky was clear, and the temperature ranged between 21.6°C and 24.6°C. In order to synchronize both datasets, we extracted the temperature from the device every 10 min, resulting in a total of 12 time points. Statistical analysis Data analysis was performed using the R programming language. The normality of the data was assessed using the Shapiro-Wilks test. Means, standard deviations (SDs), and coefficients of variation (CVs) were calculated using standard statistical methods. Relative reliability was assessed using intraclass correlation coefficients (ICCs) with 95% CIs. ICC (two-way mixed model, single measures, absolute agreement) was selected based on guidelines published elsewhere. 14 One-way ANOVA was conducted, followed by a Bonferroni pos-test to examine pairwise differences. The effect size (ES) of one-way ANOVA effects was determined using partial eta squared (ηp2). Construct validity was assessed using independent Student’s T-test to compare mean differences between the two methods. The differences’magnitude was evaluated using a 95% Confidence Interval (CI) and Cohen’s effect size (ES). Agreement was assessed using Lin’s Concordance Coefficient (LCC), Mean Absolute Percent Error (MAPE) and Bland–Altman plots with limits of agreement (LoA). Homoscedasticity was assessed by visualizing Bland–Altman plots. Significance was set at p < 0.05. Results Reliability The cycling computer devices recorded a temperature range between 18°C and 33°C. Figure 1 shows the average of all devices at each time point. (Figure 1). ICC was 0.468 (95% CI: 0.254 to 0.626; p < 0.001). Significant differences were observed between devices (p Pinedo-Jauregi and Romarate 743 < 0.05; ηp2 =0.464). All pairwise combinations were significant except for two pairwise comparisons. The mean SD for all recorded time points was 2.14 °C, ranging from 1.66 °C to 3.08 °C. The mean CV was 9.09%, ranging from 6.80% to 11.60%. The mean range between the minimum and maximum recorded temperatures at each time point was 5.7 ±0.86°C (min =4°C; max =9° C). Table 1 presents the descriptive data of 12 time points recorded by the device. Validity We observed significant mean differences between the cycling computers and the meteorological station (p = 0.006, ES =0.255). The LCC was 0.398 (95% CI: 0.32 to 0.472), and MAPE was 8.87%. The agreement can be observed in Figure 2 and Figure 3. The mean bias was 0.55 ±2.36°C (Lower LoA =-4.07; Upper LoA =5.18). On Figure 3 Discussion Accurately measuring ambient temperature is essential for estimating the athlete’s performance in heat situations. 6,10 This study represents the first analysis of the reliability and validity of ambient temperaturerecordingsusing cycling computers. The findings revealed that the current recording method using the Garmin EDGE 830 is inaccurate. Figure 1. Temperature data from cycling computers during two-hour race time. Note: Mean: mean of all cycling computers per second. Max: maximum temperature registration from cycling computers. Min: minimum temperature registration from cycling computers. Range: Difference between maximum and minimum temperature registration. SD: standard deviation of all cycling computers. 744 International Journal of Sports Science & Coaching 20(2) Our findings revealed that these cycling computers recorded a wide range of temperatures at the same time point, indicating a low ICC and significant differences between variables. This lack of reliability presents a challenge when comparing data among cycle computers. Given that cycling performance is highly sensitive to environmental conditions, particularly temperature, effective thermoregulation strategies are essential for optimizing endurance and preventing heat-related impairments. 6 Studies have shown that even moderate increases in ambient temperature can exacerbate cardiovascular strain, reduce muscle blood flow, and accelerate the onset of fatigue. 7,15 Thus, with a valid device for measuring ambient temperature, coaches could make decisions based on ambient temperature. Despite participating in the same race, if we look at the temperature data, it may seem that each cyclist participated in a different race. For instance, in a study conducted by Table 1. Descriptive data of 12 time points from cycling computers. Mean (°C) SD (°C) CV (%) MIN (°C) MAX (°C) 1 22.08 2.11 9.55 19.00 25.00 2 21.00 2.22 10.55 18.00 24.00 3 21.08 1.98 9.37 19.00 24.00 4 22.42 2.07 9.21 20.00 25.00 5 22.25 2.34 10.52 19.00 26.00 6 23.25 2.09 9.01 21.00 26.00 7 24.92 2.19 8.80 22.00 28.00 8 23.50 2.15 9.16 21.00 26.00 9 25.42 1.93 7.59 23.00 29.00 10 25.17 2.04 8.10 22.00 28.00 11 25.58 2.11 8.24 23.00 28.00 12 28.08 2.11 7.51 25.00 31.00 Note: Mean: mean of all cycling computers at each point; Max: maximum temperature registration from cycling computers; Min: minimum temperature registration from cycling computers; SD: standard deviation; CV: coefficient of variation. Figure 2. Bland-Altman analysis to compare the agreement between cycling computers and the official meteorological station. Pinedo-Jauregi and Romarate 745 Valenzuela et al. 2022 10 cycling computers were used to examine the impact of ambient temperature on power output. However, based on our study, the results seem to indicate that these devices may not be valid. The results show lack of validity, as evidenced by low LCC and a high MAPE. Regarding agreement, although the mean bias is low, the LoAs are considerable. The temperature accuracy is set at 0.1°C below our mean bias. 12 Moreover, visual analysis revealed the presence of heteroscedasticity, as the mean bias increases with higher ambient temperatures. The discrepancy between the meteorological station and cycling computers may be attributed to external ambient variables that could influence the measurements. The lack of accuracy could be attributed to direct solar radiation, as observed in other studies. 12 A solution to the radiation issue could involve relocating the temperature sensor on the bike, for example, underneath the saddle. Furthermore, the discrepancy between the weather station and the EDGE devices could be caused by local microclimates. Similar to the marathon scenario, 11 there may be microclimates along the course that make it challenging to compare with the weather station. This lack of agreement presents challenges in understanding the influence of temperature on performance and hinders data comparison. Nonetheless, using cycling computers for temperature measurement remains prevalent among scientists and trainers. 10 However, these cycling computers seem to not be valid for quantifying WBGT during a race. It is crucial to consider that temperature accounts only for 10% of the WBGT equation. 13 As highlighted by Lemke and Kjellstrom 2012, 13 a one-degree change in air temperature roughly corresponds to a onedegree change in WBGT. Another current method for assessing heat strain involves considering temperature and humidity. 9 Nevertheless, these cycling computers are not suitable yet for accurately assessing heat strain. Our study found that temperature sensors on cycle computers, or their placement, may encounter issues when calculating ambient temperature. One proposed solution to this problem is for various companies to produce a small, portable weather station that can be placed in an optimal location on the bicycle. This idea has already been suggested for measuring thermal stress using external body temperature sensors. 16 The authors emphasize the importance of combining both measurements to predict an athlete’s thermal stress based on ambient temperature. The main limitation of this study is the “black box effect”, where there is a lack of knowledge regarding the specific mechanisms by which the Garmin EDGE 830 cycling computers record temperature. Understanding this process is crucial for comprehending the observed lack of accuracy. For instance, while we observed a 1°C accuracy in the device temperature output, we do not have information regarding the sampling frequency or the type of sensor used. Figure 3. The cycle computers and the meteorological station temperature recorded at 12 different time points. 746 International Journal of Sports Science & Coaching 20(2) Another limitation is the range of actual temperatures measured during the race. The presence of heteroscedasticity highlights the importance of analyzing accuracy under lower and higher temperature conditions. Additionally, since the official meteorological station reported data every 10 min, increasing the sampling frequency may lead to slightly different outcomes. Finally, the research was conducted only for one race and 12 devices. While this approach has facilitated the comparison with the same meteorological station, it is worth noting that the circuit’s characteristics could impact the measurements. To tackle this issue, considering the potential invalidity of the sensors, it is advisable for future research to adopt the methodology utilized by Kosaka et al. 2018 11 which compared the mobile weather station with the cyclo-switches. Moreover, it would be beneficial to include a wider array of devices from different manufacturers to enhance the sample size of device. Lastly, the sample size was limited, and despite efforts to reach out to more devices, it was not feasible. As a result, the anticipated statistical power was not attained. Nonetheless, the findings demonstrate the behaviour of these devices, which should be checked in future research. Conclusions In conclusion, this research raises concerns regarding the reliability and validity of the Garmin EDGE 830 cycling computer devices for measuring ambient temperature. Coaches and scientists should exercise caution when utilizing data from these devices, considering the observed discrepancies with the official meteorological station. Future research should clarify whether the results found in this study are generalizable to other models and brands of cycling computer devices. Also, future research should prioritize identifying optimal locations for bicycle temperature sensors to enhance data accuracy and validity by comparing with mobile weather stations. Acknowledgments The authors want to express their gratitude to the cycling teams for sharing the data and to Dr Txus for his valuable advice. Declaration of conflicting interests The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. Ethical approval As the University of the Basque Country ethics committee indicated, ethical approval is unnecessary. Funding The authors received no financial support for the research, authorship, and/or publication of this article. ORCID iD Aitor Pinedo-Jauregi https://orcid.org/0000-0002-7939-297X References 1. Kenney WL, Havenith G and Jay O. Thermal physiology, more relevant than ever before. J Appl Physiol (1985). 2022; 133: 676–678. 2. Charlot K, Faure C and Antoine-Jonville S. Influence of hot and cold environments on the regulation of energy balance following a single exercise session: A mini-review. Nutrients 2017; 9: 592. 3. Ruíz-Moreno C, Gutiérrez-Hellín J, González-García J, et al. Effect of ambient temperature on fat oxidation during an incremental cycling exercise test. Eur J Sport Sci 2021; 21: 1140–1147. 4. Gagnon DD, Perrier L, Dorman SC, et al. Ambient temperature influences metabolic substrate oxidation curves during running and cycling in healthy men. Eur J Sport Sci 2020; 20: 90–99. 5. Périard JD, Racinais S and Sawka MN. Adaptations and mechanisms of human heat acclimation: applications for competitive athletes and sports. Scand J Med Sci Sports 2015; 25: 20–38. 6. Périard JD, Eijsvogels TMH and Daanen HAM. Exercise under heat stress: thermoregulation, hydration, performance implications, and mitigation strategies. Physiol Rev 2021; 101: 1873–1979. 7. Nybo L, Rasmussen P and Sawka MN. Performance in the heat-physiological factors of importance for hyperthermiainduced fatigue. Compr Physiol 2014; 4: 657–689. 8. Cheung SS and Sleivert GG. Multiple triggers for hyperthermic fatigue and exhaustion. Exerc Sport Sci Rev 2004; 32: 100–106. 9. Armstrong LE, Casa DJ, Millard-Stafford M, et al. Exertional Heat Illness during Training and Competition. Med Sci Sports Exercise 2007; 39: 556–572. 10. Valenzuela PL, Mateo-March M, Zabala M, et al. Ambient temperature and field-based cycling performance: insights from male and female professional cyclists. Int J Sports Physiol Perform 2022; 17: 1025–1029. 11. Kosaka E, Iida A, Vanos J, et al. Microclimate variation and estimated heat stress of runners in the 2020 Tokyo Olympic marathon. Atmosphere (Basel) 2018; 9: 192. 12. Yang J, Liu Q, Chen G, et al. Optimized design and experimental validation of a temperature sensor for surface air temperature observation. Sens Actuators, A 2021; 323: 112646. 13. Lemke B and Kjellstrom T. Calculating workplace WBGT from meteorological data: a tool for climate change assessment. Ind Health 2012; 50: 267–278. 14. Koo TK and Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 2016; 15: 155–163. 15. Sawka MN, Leon LR, Montain SJ, et al. Integrated physiological mechanisms of exercise performance, adaptation, and maladaptation to heat stress. Compr Physiol 2011; 1: 1883–1928. 16. Verdel N, Podlogar T, Ciuha U, et al. Reliability and validity of the CORE sensor to assess core body temperature during cycling exercise. Sensors 2021; 21: 5932. Pinedo-Jauregi and Romarate 747