Mobile health requirements for the occupational health assessment of health care professionals: Delphi study

Original Paper Mobile Health Requirements for the Occupational Health Assessment of Health Care Professionals: Delphi Study Ana Belen Naranjo-Saucedo1, MSc; German Antonio Escobar-Rodriguez1, MSc; Carmen Tabernero2,3, PhD; Esther Cuadrado2,4, PhD; Carlos Luis Parra-Calderon1, MSc; Alicia Arenas2,5, PhD 1Institute of Biomedicine of Seville/Virgen del Rocío University Hospital, Consejo Superior de Investigaciones Científicas/University of Seville, Sevilla, Spain 2Maimonides Biomedical Research Institute of Córdoba, Córdoba, Spain 3Department of Social Psychology and Anthropology, Instituto de Neurociencias de Castilla y León, Campus of the University of Salamanca-Miguel de Unamuno, Salamanca, Spain 4Faculty of Educational Sciences and Psychology, University of Córdoba, Córdoba, Spain 5Department of Social Psychology, University of Seville, Seville, Spain Corresponding Author: German Antonio Escobar-Rodriguez, MSc Institute of Biomedicine of Seville/Virgen del Rocío University Hospital Consejo Superior de Investigaciones Científicas/University of Seville Hospital Universitario Virgen del Rocío - Servicio Andaluz de Salud Avda. Manuel Siurot S/N Sevilla, 41013 Spain Phone: 34 955013616 Email: [email protected] Abstract Background: In recent years, owing to the COVID-19 pandemic, awareness of the high level of stress among health care professionals has increased, and research in this area has intensified. Hospital staff members have historically been known to work in an environment involving high emotional demands, time pressure, and workload. Furthermore, the pandemic has increased the strain experienced by health care professionals owing to the high number of people they need to manage and, on many occasions, the limited available resources with which they must carry out their functions. These psychosocial risks are not always well dealt with by the organization or the professionals themselves. Therefore, it is necessary to have tools to assess these psychosocial risks and to optimize the management of this demand from health care professionals. Digital health, and more specifically, mobile health (mHealth), is presented as a health care modality that can contribute greatly to respond to these unmet needs. Objective: We aimed to analyze whether mHealth tools can provide value for the study and management of psychosocial risks in health care professionals, and assess the requirements of these tools. Methods: A Delphi study was carried out to determine the opinions of experts on the relevance of using mHealth tools to evaluate physiological indicators and psychosocial factors in order to assess occupational health, and specifically, stress and burnout, in health care professionals. The study included 58 experts with knowledge and experience in occupational risk prevention, psychosocial work, and health-related technology, as well as health professionals from private and public sectors. Results: Our data suggested that there is still controversy about the roles that organizations play in occupational risk prevention in general and psychosocial risks in particular. An adequate assessment of the stress levels and psychosocial factors can help improve employees’well-being. Moreover, making occupational health evaluations available to the team would positively affect employees by increasing their feelings of being taken into account by the organization. This assessment can be improved with mHealth tools that identify and quickly highlight the difficulties or problems that occur among staff and work teams. However, to achieve good adherence and participation in occupational health and safety evaluations, experts consider that it is essential to ensure the privacy of professionals and to develop feelings of being supported by their supervisors. Conclusions: For years, mHealth has been used mainly to propose intervention programs to improve occupational health. Our research highlights the usefulness of these tools for evaluating psychosocial risks in a preliminary and essential phase of approaches JMIR Form Res 2023 | vol. 7 | e40327 | p. 1https://formative.jmir.org/2023/1/e40327 (page number not for citation purposes) Naranjo-Saucedo et alJMIR FORMATIVE RESEARCH XSL • FO RenderX to improve the health and well-being of professionals in health care settings. The most urgent requirements these tools must meet are those aimed at protecting the confidentiality and privacy of measurements. (JMIR Form Res 2023;7:e40327) doi: 10.2196/40327 KEYWORDS occupational health assessment; psychosocial risks; health care professionals; mobile health; mHealth; Delphi method Introduction It is undisputed that health care professionals often experience high levels of work-related stress. They have a diversity of work schedules, high workloads, night shifts that lead to sleep deprivation, work-life imbalance, feelings of isolation, and low control over the content of tasks or decisions related to their work [1]. All of these require the ability to control and manage physiological and psychological stress, and they experience disturbances in their self-regulation capacities and sometimes fail to fulfill their duties and responsibilities [2]. Job stress causes the appearance of health problems in these professionals, leading to changes in their place of work and even creating the need for these professionals to leave their profession. This situation has worsened in recent years. The emotional well-being of health care professionals is critical. It is even more relevant today given the effects that the COVID-19 pandemic has had on health care management and patient care, especially in the work environment, due to both the psychological impact on professionals themselves and the possible consequences regarding job performance and patient care [3]. Studies have found that hospital personnel were significantly affected during the pandemic and reported high levels of concern given the fear of infection and the possible consequences of the disease on their physical and mental health [4,5]. These concerns included fear, insecurity, anxiety, and compassion fatigue, with the latter caused by the cost of worrying about others or their emotional pain associated with the aim of alleviating the suffering of patients [4]. On the other hand, workplace issues can play a crucial role in modifying or worsening the mental health of people who face this pandemic scenario. Among them, job insecurity, periods of isolation, and uncertainty are factors in the workplace that most affect health care workers [5]. To carry out appropriate interventions aimed at reducing work-related stress, the professional situation must first be adequately assessed and diagnosed. However, work overload is a barrier to performing work stress measurements, so it is essential to find a way to achieve such measurements with minimal harm to professionals and their work performance. The key may lie in technology. Currently, there are more research initiatives and initiatives involving the development of new technological tools in this specific population to assess the stress experienced by these professionals [6]. In this sense, we found some studies in which digital tools were used as applications to assess and improve the well-being of professionals [7,8]. These tools are often designed to offer training strategies to improve stress management at work. However, to our knowledge, tools focused on diagnosing the health of workers are scarce. In the scientific literature, there are studies focused on mobile health (mHealth) tools to assess the work environment and its psychosocial aspects, as well as how stress affects the health of professionals. The results of current research on the organizational health of hospital staff revealed high levels of stress [9] and burnout [10], as well as other psychological problems, such as anxiety and depression [11], leading to a high turnover of employees and a high number of sick leaves [12], and there have even been occasions where the treatment of patients or their family members has been considerably reduced [13]. Furthermore, the situation since the COVID-19 pandemic has further aggravated this reality [14]. Some tools offer individualized intervention programs for professionals [15,16]. There is limited literature on tools that perform occupational health measurements in professionals, and mHealth tools, such as mobile apps, often focus on the implementation of mindfulness intervention programs [17], breath control programs, or meditation intervention programs [18]. It should be noted that the use of wearable technology allows the recording of physiological variables in real time with great success [19]. These are small devices that capture biometric data from users over a prolonged period of time in natural environments and record variables [20], such as heart rate, respiratory rate, and hours of sleep, allowing the assessment of the stress levels and well-being of workers [21]. In contrast, mHealth tools focused on intervention and program implementation highlight approaches focused on mindfulness training [10], breathing control [22], and learning techniques to cope with stress and resilience [23] in difficult situations that arise on the job and in the lives of workers. Learning these techniques can help reduce stress levels among trainees and improve their well-being [24]. We were interested in exploring how mHealth tools should be focused on the identification and evaluation of psychosocial factors, such as stress and burnout, in an essential preliminary phase for interventions. We posed the following research question to experts: Do mHealth tools applied to occupational health allow you to understand and manage psychosocial risks among health care professionals in their organizations? Our study explored their opinions about the use of these tools in organizations to assess the occupational health of professionals and determined the characteristics that these tools must have to be accepted by workers. Methods Delphi Process The Delphi method was applied in a group setting to obtain consensus on the opinions of a group of experts through a series of intensive questionnaires with controlled opinion feedback [25]. The approach was applied to obtain expert opinion on the JMIR Form Res 2023 | vol. 7 | e40327 | p. 2https://formative.jmir.org/2023/1/e40327 (page number not for citation purposes) Naranjo-Saucedo et alJMIR FORMATIVE RESEARCH XSL • FO RenderX application of mHealth tools. It is commonly used in the fields of technological and social forecasting, social diagnosis, consensual interpretation of social or health realities, communication, and participation [26]. We believe that the Delphi methodology is suitable for answering our research question because of its ability to obtain expert consensus, its application in technological and social fields, and its ability to obtain accurate and reliable data. The response options in the Delphi method usually follow a 5-choice Likert scale [27]. Several steps were taken to carry out this study, as shown in Figure 1. Figure 1. Delphi process in this study. Design of the Expert Panel The first phase of the Delphi method consisted of identifying experts to create a coherent panel of experts according to the subject matter of the study. After a bibliographic review, a team of approximately 60 experts with knowledge and experience in occupational risk prevention, psychosocial work, and health-related technology, as well as health professionals was selected. For this purpose, professionals belonging to the academic fields of social psychology, computer engineering, and industrial engineering were identified. Health care and nonhealth care professionals were selected. Finally, experts from the private business sector were included. Different levels were considered, including company managers, middle managers, and operating core team members. In the case of experts from the labor or academic field, experience in the use of new technologies applied to health, the field of occupational risk prevention, or the psychosocial area was required. The characteristics of the experts who finally participated are detailed in Table 1. An average of 9 academic experts, 9 health experts, and 7.66 experts from the business sector participated in each round. In our study, 85% (23/27) of the respondents had more than 10 years of experience. In round 1, 26 experts participated (45% response rate), of whom 9 were from the academic field, 10 from the public health field, and 7 from the business field related to the health area, and 85% (22/26) had more than 10 years of experience. In round 2, 24 experts participated (41% response rate), with 8 from each area of participation (academic field, public health field, and business field), and 88% (21/24) had more than 10 years of experience. Finally, in round 3, 27 experts participated (46.5% response rate), of whom 10 were from the academic field, 9 from the health field, and 8 from the business field, and 85% (23/27) had more than 10 years of experience. These data are shown graphically in Figure 2. JMIR Form Res 2023 | vol. 7 | e40327 | p. 3https://formative.jmir.org/2023/1/e40327 (page number not for citation purposes) Naranjo-Saucedo et alJMIR FORMATIVE RESEARCH XSL • FO RenderX Table 1. Sociodemographic characteristics of the panel of experts. Respondents in the third round (n=27), n (%) Respondents in the second round (n=24), n (%) Respondents in the first round (n=26), n (%) Variable Sector 10 (37)8 (33)9 (35)Academic 3 (30)3 (38)3 (33)Psychology 2 (20)1 (13)1 (11)Industrial engineering 5 (50)3 (38)4 (44)Computer engineering 0 (0)1 (13)1 (11)Occupational risk prevention 9 (33)8 (33)10 (39)Health care 2 (22)6 (75)5 (50)Health care activities 7 (78)2 (25)5 (50)Nonhealth care activities 5 (56)4 (50)5 (50)Company managers 3 (33)3 (38)4 (40) Middle managers 1 (11)1 (13)1 (10) Operating core team members 8 (30)8 (33)7 (27) Business 4 (50)6 (75)3 (43)Company managers 4 (50)2 (25)4 (57)Middle managers 0 (0)0 (0)0 (0)Operating core team members Experience in the sector 2 (7)2 (8)1 (4)1-3 years 1 (4)0 (0)1 (4)3-5 years 1 (4)1 (4)2 (8)5-10 years 23 (85)21 (88)22 (85)>10 years Figure 2. Graphical representation of expert participation. JMIR Form Res 2023 | vol. 7 | e40327 | p. 4https://formative.jmir.org/2023/1/e40327 (page number not for citation purposes) Naranjo-Saucedo et alJMIR FORMATIVE RESEARCH XSL • FO RenderX Design of the Questionnaire A questionnaire was designed to include questions involving the most relevant categories or areas according to the findings obtained in our literature search. These categories were labeled as follows: “occupational health,” “procedure (applicability),” “security and privacy,” and “how should mHealth devices be.” To ensure that all relevant aspects had been included, the initial questionnaire included 4 open-ended questions to obtain the experts’opinions on general aspects. The objective was to check whether the information was included in the closed-ended question items or whether, in contrast, it would be convenient to add new categories or issues in the next round. After analyzing the open-ended questions, the category “adherence” was added. The items presented to the experts were designed according to an ordinal Likert scale with scores ranging from 1 to 5 (1=“strongly disagree” and 5=“strongly agree”). For facilitating the analysis of the results, the ratings were divided into the following 3 levels: (1) disagree (“strongly disagree” and “disagree” responses); (2) neither agree nor disagree (“neither agree nor disagree” responses); and (3) agree (“strongly agree” and “agree” responses) [28]. The questionnaires were developed using computer tools to facilitate the publication and collection of opinions. At the beginning of the surveys, consent was requested to participate. The instructions provided technical terms and their definitions to ensure that participants had the same conceptualizations (detailed in Multimedia Appendix 1). The items of each round were then included, and some questions about the sociodemographic profile of the experts (professional field, years of experience in the sector, and professional group) were provided. Description and Procedure of Delphi Rounds Once the opinions were collected, we analyzed the consensus for each item. Although there is no unanimous definition of consensus, one of the most commonly used is the percentage of agreement. Therefore, some authors, such as Humphrey-Murto et al [29], state that if 70% of experts agree on a response, the item has reasonably reached an agreement of experts to be eliminated for subsequent rounds. To achieve a more robust consensus, it was decided to consider that items with 75% agreement had reached consensus and therefore would not be presented in successive rounds. According to the literature consulted, it was decided to carry out a maximum of 3 iterative rounds, since this is the most frequent. It is estimated that if agreement has not been reached in these 3 rounds, it will not be reached later. Round 1 The objective, methodology, and theoretical background of the study were presented to the experts in the first contact. Similarly, the link to access the first round of the Delphi survey was included. The first round was launched on June 22, 2021. It consisted of 40 questions. These questions dealt with the following categories: “occupational health,” “procedure (applicability),” “security and privacy,” and “how should mHealth devices be.” Each category consisted of 10 items. Only in this initial round were 4 open-ended questions also included. They were taken into account and incorporated in the next round in case the aspects revealed by experts were not taken into account in the design of the original survey. These questions were as follows: (1) In what ways do you think mHealth systems can be used to assess occupational health? (2) What difficulties can be encountered when applying an occupational health program using mHealth in the health care setting? (3) How do you think the adherence of health care personnel to mHealth tools and interventions could be increased? and (4) What do you think these tools add to the conventionally used tools to assess occupational health in organizations? Round 2 Round 2 was launched on August 10, 2021. A report of the responses provided by the experts in the previous round was sent and included a link to the round 2 survey. It consisted of 15 items whose categories were the same as in the previous round. Based on the responses collected in the open-ended questions of round 1, some new items were added to existing categories in this round, and a new category called “adherence” was incorporated. Round 3 Round 3 was launched on October 5, 2021, and was accompanied by a detailed report of the responses from the previous round. The questionnaire consisted of 5 items belonging to the categories “occupational health” and “security and privacy,” which were the only categories yet to reach consensus. Ethical Considerations This study is part of the research projects “mPRL: mHealth tool for prevention in the area of psychosociology” (reference: PII2019SC0009) and “SALPRO, Analysis of physiological indicators and psychosocial factors in occupational health of health professionals.” Both projects have been approved by the Ethics Committee of the local government (Research Ethics Committee of Cádiz on March 28, 2019, and Research Ethics Committee of the Vírgen Macarena-Virgen del Rocío University Hospitals on December 3, 2019, respectively), and the study complies with the principles set out in the Declaration of Helsinki [30]. Participation was completely anonymous and confidential to guarantee the protection of the privacy of participants and was voluntary. Participants were informed of the aims of the research before they provided their consent to participate. No compensation was provided to study participants. Results Analysis of the Results of Each Round Round 1 In round 1, agreement was reached on 30 of the 40 items. By analyzing the different sections, the category “occupational health” reached consensus on 5 of the 10 items. Full consensus was reached for the category “procedure (applicability),” so it was not presented again in subsequent rounds. In the category JMIR Form Res 2023 | vol. 7 | e40327 | p. 5https://formative.jmir.org/2023/1/e40327 (page number not for citation purposes) Naranjo-Saucedo et alJMIR FORMATIVE RESEARCH XSL • FO RenderX “how should mHealth devices be,” consensus was reached on 11 of the 12 items. In the category “security and privacy,” 6 of the 10 items achieved consensus in this initial round. The analysis of open-ended questions was carried out by grouping the main themes of each response. Some responses covered aspects of the sections presented in the initial design, and corroborated that the initial design was adequate in those aspects. For example, some experts pointed out expressions, such as “They could be useful instruments for monitoring indicators related to health and stress in real time (P8), and encouraging or stimulating the adoption of healthy habits during working hours,” so we considered it in “occupational health” item OH4. Moreover, the response “one of the cross-cutting difficulties that can be encountered is the refusal by users to be continuously monitored” was found to be closely related to “security and privacy” items SP4 and SP6. On the other hand, some experts commented on aspects of the existing categories but pointed out new visions, so they were incorporated as new items in the following rounds. Statements, such as “The use of work-related mHealth tools should be integrated into workday tasks” and “To facilitate the use of mHealth tools during the workday, they should be brief in nature” (“how should mHealth devices be” items H11 and H12, respectively), are examples of this. Round 2 The second round consisted of 15 items, including 10 items not agreed upon in the previous round, 2 new items formulated from the answers to the previous round’s open-ended questions included in the section “how should mHealth devices be,” and 3 items included in the new “adherence” section. After the survey was presented to experts, the categories “how should mHealth devices be” and “adherence” achieved consensus. However, the categories “occupational health” and “security and privacy” did not achieve the minimum 75% consensus; thus, they were included in the third round. Round 3 The third round consisted of 5 items that did not reach a consensus in the previous round, including 3 items in the “occupational health” category and 2 items in the “security and privacy” category. Again, consensus was not achieved in this third round either. The results show disagreement given the wide dispersion of responses. Data Analysis In Tables 2-6, we present all the items in the 3 rounds (translated into English) and the results. The tables also show whether agreement was achieved. Moreover, the tables show which option achieved the most remarkable consensus and which of the options received the most votes. Furthermore, the arithmetic means that characterize the central tendency and the SDs that measure the dispersion are indicated. The original survey in Spanish is available in Multimedia Appendix 2. The results are discussed in greater depth in Multimedia Appendix 3. The statistical measures of mean, mode, maximum, and minimum for each item of all rounds are included in this appendix. Our data revealed that the categories “procedure (applicability)” and “how should mHealth devices be” most easily reached an agreement. The “procedure (applicability)” category reached absolute consensus in the first round, and only 1 item in the “how should mHealth devices be” category had to be resubmitted a second time before reaching the established consensus threshold. In the second round, 2 new items were added based on the responses obtained in the open-ended questions and were consensual in the second round. The “adherence” category, incorporated into the questionnaire in round 2, was created after analysis of the opinions contributed by the experts to the open-ended questions in round 1, and its items achieved consensus the first time they were presented. However, several items in the categories of “occupational health” and “safety and privacy” did not achieve consensus. In the “occupational health” category, 5 items reached consensus in round 1 and 2 more items reached consensus in round 2. After this second round, the experts continued to disagree on 3 items, and these items represented major points of disagreement. In the “security and privacy” category, 6 items reached consensus in round 1 and 2 more items reached consensus in round 2. After round 3, there were still strong disagreements for 2 items in the “security and privacy” category. Finally, after the 3 iterative rounds, the experts did not agree on 5 items. Analyses were carried out according to the field of knowledge to which the experts belonged, but there were no notable findings regarding these 5 items. For a better understanding, this complete process is graphically represented in Figure 3. Detailed results are provided for nonconsensus items in Figures 4-6. For item OH7 in the “occupational health” category, there seemed to be a trend toward dissensus. Thus, more iterations might result in dissensus, and “neither agree nor disagree” responses might grow with iterations. Therefore, it cannot be concluded that a consensus will be reached. For item OH9 in the “occupational health” category, there was no clear trend, but the majority agreed with the item. With more iterations, the 75% agreement required for consensus might not be reached. For item OH10 in the “occupational health” category, and items SP3 and SP4 in the “security and privacy” category, no clear trends were identified, mainly because a high percentage of respondents did not have a clear position between agreement and disagreement. Therefore, further iterations might not allow agreement to be reached. There is no apparent reason for the lack of consensus beyond the one identified above for item OH9 in the “occupational health” category and item SP4 in the “security and privacy” category. It may be necessary to conduct a further study to identify the reasons for not reaching a consensus on these items. JMIR Form Res 2023 | vol. 7 | e40327 | p. 6https://formative.jmir.org/2023/1/e40327 (page number not for citation purposes) Naranjo-Saucedo et alJMIR FORMATIVE RESEARCH XSL • FO RenderX Table 2. Items and results of the “occupational health” category (English version). ResultsOccupational health (OH) category items Round 3Round 2Round 1 Mean (SD) Consensus yes (≥75%) or no (A, NAND, or D %) Mean (SD) Consensus yes (≥75%) or no (A, NAND, or D %) Mean (SD) Consensus yes (≥75%) or no (Aa, NANDb, or Dc%) N/AN/AN/A N/Ad 4.65 (0.85) Yes (A=96.2%)OH1: It is crucial for employees to have occupational health information available to them. N/AN/AN/AN/A4.69 (0.84) Yes (A=96.2%)OH2: It is necessary to evaluate the occupational health of professionals. N/AN/AN/AN/A4.73 (0.84) Yes (A=96.2%)OH3: Professionals have the right to have their occupational health protected. N/AN/AN/AN/A4.65 (0.85) Yes (A=96.2%)OH4: Detecting stress levels can help improve the well-being of workers. N/AN/A3,88 (0.54) Yes (A=79.2%)3.73 (1.00) No (A=7.7%, NAND=42.3%, D=50.0%) OH5: Mobile health (mHealth) devices are more productive resources for conducting occupational risk prevention (ORP) assessments than conventional methods. N/AN/A3.79 (0.78) Yes (A=75.0%)3.62 (1.02) No (A=11.5%, NAND=30.8%, D=57.7%) OH6: mHealth devices have transformed the way professionals evaluate aspects of ORP. 3.30 (0.99) No (A=29.6%, NAND=18.5%, D=51.9%) 3.91 (1.12) No (A=41.7%, NAND=16.7%, D=41.7%) 3.00 (1.26) No (A=46.2%, NAND=15.4%, D=38.4%) OH7: Organizations are aware of the importance of taking care of the occupational health of employees. N/AN/AN/AN/A4.19 (0.94) Yes (A=84.7%)OH8: The fact that organizations evaluate the occupational health of their professionals gives them a sense of being taken into account. 2.56 (0.93) No (A=59.3%, NAND=18.5%, D=22.2%) 3.92 (0.93) No (A=66.7%, NAND=8.3%, D=25.0%) 2.81 (1.06) No (A=42.3%, NAND=34.6%, D=23.1%) OH9: The use of technology to evaluate occupational health and safety information is supported in organizations. 3.59 (0.80) No (A=55.6%, NAND=37.0%, D=7.4%) 3.95 (0.72) No (A=8.3%, NAND=45.8%, D=45.8%) 3.12 (0.86) No (A=26.9%, NAND=38.5%, D=34.6%) OH10: Currently, organizations take corrective actions when problems are detected in the occupational health of their workers. aA: agree. bNAND: neither agree nor disagree. cD: disagree. dN/A: not applicable. JMIR Form Res 2023 | vol. 7 | e40327 | p. 7https://formative.jmir.org/2023/1/e40327 (page number not for citation purposes) Naranjo-Saucedo et alJMIR FORMATIVE RESEARCH XSL • FO RenderX Table 3. Items and results of the “procedure (applicability)” category (English version). ResultsProcedure (applicability) (P) category items Round 3Round 2Round 1 Mean (SD) Consensus yes (≥75%) or no (A, NAND, or D %) Mean (SD) Consensus yes (≥75%) or no (A, NAND, or D %) Mean (SD) Consensus yes (≥75%) or no (Aa, NANDb, or Dc%) N/AN/AN/A N/Ad 4.69 (0.47) Yes (A=100%)P1: Technical assistance should be provided to professionals during the monitoring period if necessary. N/AN/AN/AN/A4.73 (0.45) Yes (A=100%)P2: Mobile health (mHealth) devices should be periodically checked to ensure proper functioning. N/AN/AN/AN/A4.35 (0.80) Yes (A=88.4%)P3: Professionals providing information about their occupational health should be included in the working day. N/AN/AN/AN/A4.77 (0.43) Yes (A=100%)P4: Professionals should receive an informative day where they are taught how to use mHealth devices for occupational health. N/AN/AN/AN/A4.77 (0.51) Yes (A=96.2%)P5: To encourage participation in the occupational health study, professionals should feel supported by their supervisors. N/AN/AN/AN/A4.73 (0.60) Yes (A=92.3%)P6: In the occupational health assessment, the adherence of workers can be increased by providing feedback about their participation. N/AN/AN/AN/A4.19 (1.06) Yes (A=77.0%)P7: The study of the occupational health of professionals should take into account the perspective of gender. N/AN/AN/AN/A4.62 (0.64) Yes (A=92.3%)P8: mHealth devices can be used to record variables that evaluate people’s occupational health. N/AN/AN/AN/A4.58 (0.50) Yes (A=100%)P9: Occupational stress can be detected by recording physiological variables and psychological questionnaires. N/AN/AN/AN/A4.42 (0.64) Yes (A=92.3%)P10: Thanks to mHealth devices, professionals will be able to visualize the results of their records and make changes in their work habits. aA: agree. bNAND: neither agree nor disagree. cD: disagree. dN/A: not applicable. JMIR Form Res 2023 | vol. 7 | e40327 | p. 8https://formative.jmir.org/2023/1/e40327 (page number not for citation purposes) Naranjo-Saucedo et alJMIR FORMATIVE RESEARCH XSL • FO RenderX Table 4. Items and results of the “security and privacy” category (English version). ResultsSecurity and privacy (SP) category items Round 3Round 2Round 1 Mean (SD) Consensus yes (≥75%) or no (A, NAND, or D %) Mean (SD) Consensus yes (≥75%) or no (A, NAND, or D %) Mean (SD) Consensus yes (≥75%) or no (Aa, NANDb, or Dc%) N/AN/AN/A N/Ad 4.85 (0.37) Yes (A=100%)SP1: The data collected must comply with current regulations on security and privacy (Constitutional Act 3/2018, of 5 December on Personal Data Protection and Guarantee of Digital Rights). N/AN/AN/AN/A4.92 (0.27) Yes (A=100%)SP2: Organizations must ensure the privacy of their workers in the recording of their physiological and psychological indicators. 3.52 (0.80) No (A=55.6%, NAND=33.3%, D=11.1%) 3.46 (0.88) No (A=8.3%, NAND=41.7%, D=50.0%) 3.62 (1.06) No (A=7.6%, NAND=46.2%, D=46.2%) SP3: Recording personal information through these mobile apps is secure. 2.63 (1.01) No (A=18.5%, NAND=44.4%, D=37.0%) 2.83 (0.56) No (A=25.0%, NAND=66.7%, D=8.3%) 3.15 (0.67) No (A=11.5%, NAND=65.4%, D=23.1%) SP4: Professionals agree to be monitored to assess their occupational health. N/AN/A3.88 (0.80) Yes (A=91.7%)3.85 (1.16) No (A=11.5%, NAND=15.4%, D=73.1%) SP5: Recording physiological information in an ongoing and outside workplace may be construed as an invasion of privacy. N/AN/A2.96 (0.46) Yes (NAND=72.9%)3.04 (0.72) No (A=15.4%, NAND=61.5%, D=23.1%) SP6: Practitioners agree to wear the wearable device throughout the day, if necessary, for the recording period. N/AN/AN/AN/A4.35 (0.98) Yes (A=80.8%)SP7: Data collected must be kept confidential and analyzed in aggregate to avoid the identification of participants. N/AN/AN/AN/A3.92 (0.74) Yes (A=77.0%)SP8: Employees may prefer to answer questions about their work reality via a mobile app rather than express it to a person, as might be the case with conventional methods. N/AN/AN/AN/A4.23 (0.51) Yes (A=96.2%)SP9: Workers may fear being identified when responding truthfully to questions about their work environment. N/AN/AN/AN/A4.46 (0.51) Yes (A=100%)SP10: Workers may be reluctant to share information about their psychological state. aA: agree. bNAND: neither agree nor disagree. cD: disagree. dN/A: not applicable. JMIR Form Res 2023 | vol. 7 | e40327 | p. 9https://formative.jmir.org/2023/1/e40327 (page number not for citation purposes) Naranjo-Saucedo et alJMIR FORMATIVE RESEARCH XSL • FO RenderX 11. Kaushik A, Ravikiran S, Suprasanna K, Nayak M, Baliga K, Acharya S. Depression, anxiety, stress and workplace stressors among nurses in tertiary health care settings. Indian J Occup Environ Med 2021;25(1):27. [doi: 10.4103/ijoem.ijoem_123_20] 12. Yang H, Lv J, Zhou X, Liu H, Mi B. Validation of work pressure and associated factors influencing hospital nurse turnover: a cross-sectional investigation in Shaanxi Province, China. BMC Health Serv Res 2017 Feb 03;17(1):112 [FREE Full text] [doi: 10.1186/s12913-017-2056-z] [Medline: 28158979] 13. Warlick CA, Van Gorp A, Farmer NM, Patterson T, Armstrong A. Comparing burnout between graduate-level and professional clinicians. Training and Education in Professional Psychology 2021 May;15(2):150-158. [doi: 10.1037/tep0000328] 14. Mira JJ, Vicente MA, Lopez-Pineda A, Carrillo I, Guilabert M, Fernández C, et al. Preventing and addressing the stress reactions of health care workers caring for patients with COVID-19: Development of a digital platform (Be + Against COVID). JMIR Mhealth Uhealth 2020 Oct 05;8(10):e21692 [FREE Full text] [doi: 10.2196/21692] [Medline: 32936769] 15. Serrano-Ripoll MJ, Ricci-Cabello I, Jiménez R, Zamanillo-Campos R, Yañez-Juan AM, Bennasar-Veny M, et al. Effect of a mobile-based intervention on mental health in frontline healthcare workers against COVID-19: Protocol for a randomized controlled trial. J Adv Nurs 2021 Jun;77(6):2898-2907 [FREE Full text] [doi: 10.1111/jan.14813] [Medline: 33675247] 16. Kloos N, Drossaert CH, Bohlmeijer ET, Westerhof GJ. Online positive psychology intervention for nursing home staff: A cluster-randomized controlled feasibility trial of effectiveness and acceptability. Int J Nurs Stud 2019 Oct;98:48-56. [doi: 10.1016/j.ijnurstu.2019.06.004] [Medline: 31295708] 17. Best NI, Durham CF, Woods-Giscombe C, Waldrop J. Combating compassion fatigue with mindfulness practice in military nurse practitioners. The Journal for Nurse Practitioners 2020 May;16(5):e57-e60. [doi: 10.1016/j.nurpra.2020.02.023] 18. Lehto R, Heeter C, Allbritton M, Wiseman M. Hospice and palliative care provider experiences with meditation using mobile applications. Oncol Nurs Forum 2018 May 01;45(3):380-388. [doi: 10.1188/18.ONF.380-388] [Medline: 29683121] 19. Khowaja SA, Prabono AG, Setiawan F, Yahya BN, Lee S. Contextual activity based Healthcare Internet of Things, Services, and People (HIoTSP): An architectural framework for healthcare monitoring using wearable sensors. Computer Networks 2018 Nov;145:190-206. [doi: 10.1016/j.comnet.2018.09.003] 20. Concheiro-Moscoso P, Groba B, Martínez-Martínez FJ, Miranda-Duro MDC, Nieto-Riveiro L, Pousada T, et al. Study for the design of a protocol to assess the impact of stress in the quality of life of workers. Int J Environ Res Public Health 2021 Feb 03;18(4):1413 [FREE Full text] [doi: 10.3390/ijerph18041413] [Medline: 33546392] 21. Peake JM, Kerr G, Sullivan JP. A critical review of consumer wearables, mobile applications, and equipment for providing biofeedback, monitoring stress, and sleep in physically active populations. Front Physiol 2018 Jun 28;9:743 [FREE Full text] [doi: 10.3389/fphys.2018.00743] [Medline: 30002629] 22. Hopkins L, Stacey B, Robinson DBT, James OP, Brown C, Egan RJ, et al. Consumer-grade biosensor validation for examining stress in healthcare professionals. Physiol Rep 2020 Jun 02;8(11):e14454 [FREE Full text] [doi: 10.14814/phy2.14454] [Medline: 32489016] 23. Jakel P, Kenney J, Ludan N, Miller P, McNair N, Matesic E. Effects of the use of the provider resilience mobile application in reducing compassion fatigue in oncology nursing. CJON 2016 Dec 1;20(6):611-616. [doi: 10.1188/16.cjon.611-616] 24. Roy A, Druker S, Hoge EA, Brewer JA. Physician anxiety and burnout: Symptom correlates and a prospective pilot study of app-delivered mindfulness training. JMIR Mhealth Uhealth 2020 Apr 01;8(4):e15608 [FREE Full text] [doi: 10.2196/15608] [Medline: 32234708] 25. Dalkey N, Helmer O. An experimental application of the DELPHI method to the use of experts. Management Science 1963 Apr;9(3):458-467. [doi: 10.1287/mnsc.9.3.458] 26. Landeta J. Current validity of the Delphi method in social sciences. Technological Forecasting and Social Change 2006 Jun;73(5):467-482. [doi: 10.1016/j.techfore.2005.09.002] 27. Ortega Mohedano F. El método Delphi, prospectiva en Ciencias Sociales a través del análisis de un caso práctico. Revista Escuela de Administración de negocios 2008 Aug 01(64):31-54. [doi: 10.21158/01208160.n64.2008.452] 28. Chegini Z, Shariful Islam SM. Expert perspectives on the active role of patients in their safety: Toward a framework using Delphi methodology. Nurs Forum 2021 Jul;56(3):490-499. [doi: 10.1111/nuf.12567] [Medline: 33665821] 29. Humphrey-Murto S, Varpio L, Gonsalves C, Wood TJ. Using consensus group methods such as Delphi and Nominal Group in medical education research. Medical Teacher 2016 Nov 12;39(1):14-19. [doi: 10.1080/0142159x.2017.1245856] 30. World Medical Association. World Medical Association Declaration of Helsinki. Ethical principles for medical research involving human subjects. Bull World Health Organ 2001;79(4):373-374. [Medline: 11357217] 31. Yang J, Tang S, Zhou W. Effect of mindfulness-based stress reduction therapy on work stress and mental health of psychiatric nurses. Psychiatr Danub 2018 Jun 19;30(2):189-196 [FREE Full text] [doi: 10.24869/psyd.2018.189] [Medline: 29930229] 32. Aguilar del Castillo MDC. El uso de la inteligencia artificial en la prevención de riesgos laborales [The use of artificial intelligence in the prevention of work risks]. Revista Internacional y Comparada de Relaciones Laborales y Derecho del Empleo 2020;8(1):262-293 [FREE Full text] 33. Chin WSY, Kurowski A, Gore R, Chen G, Punnett L, SHIFT Research Team. Use of a mobile app for the process evaluation of an intervention in health care: Development and usability study. JMIR Form Res 2021 Oct 28;5(10):e20739 [FREE Full text] [doi: 10.2196/20739] [Medline: 34709186] JMIR Form Res 2023 | vol. 7 | e40327 | p. 16https://formative.jmir.org/2023/1/e40327 (page number not for citation purposes) Naranjo-Saucedo et alJMIR FORMATIVE RESEARCH XSL • FO RenderX 34. Jimenez P, Bregenzer A. Integration of eHealth tools in the process of workplace health promotion: Proposal for design and implementation. J Med Internet Res 2018 Feb 23;20(2):e65 [FREE Full text] [doi: 10.2196/jmir.8769] [Medline: 29475828] 35. Khanade K, Sasangohar F, Sadeghi M, Sutherland S, Alexander K. Deriving information requirements for a smart nursing system for intensive care units. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 2017 Sep 28;61(1):653-654. [doi: 10.1177/1541931213601649] 36. Junker M, Dünnebeil S, Böhm M, Krcmar H. Usage of a workplace health promotion app: an evaluation of app usage data and medical check-up results. Health Informatics J 2023 Jan 27;29(1):14604582221148058 [FREE Full text] [doi: 10.1177/14604582221148058] [Medline: 36705467] 37. Mayer MA, Rodríguez Blanco O, Torrejon A. Use of health apps by nurses for professional purposes: Web-based survey study. JMIR Mhealth Uhealth 2019 Nov 01;7(11):e15195 [FREE Full text] [doi: 10.2196/15195] [Medline: 31682587] 38. Guo C, Tian P, Choo KR. Enabling privacy-assured fog-based data aggregation in e-healthcare systems. IEEE Trans. Ind. Inf 2021 Mar;17(3):1948-1957. [doi: 10.1109/TII.2020.2995228] 39. Ambientes de trabajo saludables: un modelo para la accio󸪢n: para empleadores, trabajadores, autoridades normativas y profesionales. World Health Organization. URL: https://apps.who.int/iris/handle/10665/44317 [accessed 2023-05-12] 40. Altindag O. Relationship between stress management and job performance in organizations. IJRBS 2020 Mar 14;9(2):43-49. [doi: 10.20525/ijrbs.v9i2.636] 41. Naranjo-Saucedo AB, Escobar-Rodriguez GA, Tabernero C, Cuadrado E, Parra-Calderon CL, Arenas A. Mobile health requirements for occupational health assessment of health care professionals: Delphi study. Zenodo. URL: https://zenodo. org/record/7936964 [accessed 2023-05-23] Abbreviations mHealth: mobile health Edited by A Mavragani; submitted 15.06.22; peer-reviewed by A Rosman, K Drude; comments to author 31.01.23; revised version received 21.02.23; accepted 22.02.23; published 31.05.23 Please cite as: Naranjo-Saucedo AB, Escobar-Rodriguez GA, Tabernero C, Cuadrado E, Parra-Calderon CL, Arenas A Mobile Health Requirements for the Occupational Health Assessment of Health Care Professionals: Delphi Study JMIR Form Res 2023;7:e40327 URL: https://formative.jmir.org/2023/1/e40327 doi: 10.2196/40327 PMID: ©Ana Belen Naranjo-Saucedo, German Antonio Escobar-Rodriguez, Carmen Tabernero, Esther Cuadrado, Carlos Luis Parra-Calderon, Alicia Arenas. Originally published in JMIR Formative Research (https://formative.jmir.org), 31.05.2023. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included. JMIR Form Res 2023 | vol. 7 | e40327 | p. 17https://formative.jmir.org/2023/1/e40327 (page number not for citation purposes) Naranjo-Saucedo et alJMIR FORMATIVE RESEARCH XSL • FO RenderX