Towards a data collection methodology for Responsible Artificial Intelligence in health: A prospective and qualitative study in pregnancy

Information Fusion 83–84 (2022) 53–78 Available online 25 March 2022 1566-2535/© 2022 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/bync-nd/4.0/). Towards a data collection methodology for Responsible Artificial Intelligence in health: A prospective and qualitative study in pregnancy A.M. Oprescu a , G. Mir´ o-Amarante a , L. García-Díaz b , c , V.E. Rey d , A. Chimenea-Toscano c , R. Martínez-Martínez e , M.C. Romero-Ternero a , * a Departamento de Tecnología Electr´ onica, Universidad de Sevilla, Avda. Reina Mercedes s/n, Seville 41012, Spain b Departamento de Cirugía, Universidad de Sevilla, Avda. S´ anchez Pizju´ an, s/n, Sevilla 41009, Spain c Hospital Universitario Virgen del Rocío, Av. Manuel Siurot, S/n, Sevilla 41013, Spain d Clínica Victoria Rey, Calle Luis Montoto, 81, Seville 41018, Spain e Departamento de Derecho Constitucional, Ciencia Política y de la Administraci´ on de la Universitat de Valencia, Avinguda dels Tarongers, s/n, Val` encia 46022, Spain ARTICLE INFO Keywords: Responsible Artificial Intelligence (RAI) Explainable Artificial Intelligence (XAI) Emotional computing Affective computing User-centered design Human-Centered Design Privacy Security Pregnancy ABSTRACT A medical field that is increasingly benefiting from Artificial Intelligence applications is Gynecology and Obstetrics. In previous work, we exposed that Artificial Intelligence (AI) technology and obstetric control by physicians can enhance pregnancy health, leading to better pregnancy outcomes and overall better experience, also reducing any possible long-term effects that can be produced by complications. This work presents a data collection methodology for responsible AI in Health and a case study in the pregnancy domain. It is a qualitative descriptive study on the preferences and expectations expressed by pregnant women regarding responsible AI and affective computing. A 41-items structured interview was distributed among 150 pregnant patients attending prenatal care at Hospital Virgen del Rocío and the Clinic Victoria Rey (Seville, Spain) during the months of October and November 2020. A substantial interest in intelligent pregnancy solutions among pregnant women has been revealed in this study. Participants with a lower level of interest reported privacy concerns and lack of trust towards AI solutions. Regarding affective computing based intelligent solutions specifically, most participants reported positively and no significant difference was found between women having a healthy or a high risk pregnancy on this matter. Our findings also suggest that a high demand of personalized intelligent solutions exists among participants. On the topic of sharing pregnancy data with the healthcare provider in favor of scientific research, pregnant women assisting public healthcare services were found to be more likely to share their data when the provider was a public healthcare system rather than a private entity. Pregnant women who are interested in using an AI pregnancy application share a strong idea that it needs to be responsible, trustworthy, useful, and safe. Likewise, we found that pregnant women would change their mind about their concerns and they would feel more confident if the intelligent solution gives explanations about the system decisions and recommendations, as XAI approach promotes. 1. Introduction Over the last decades, Artificial Intelligence (AI) has been generating a remarkable interest in a great number of fields and industries, including healthcare and medicine. In the context of medical research, prediction generally refers to a diagnosis, which is the probability that a certain condition is present but has not yet been detected. Particularly, in the medical field, AI research has been applied to the prediction of a wide range of health-related problems: diabetes [1–4], cardiovascular diseases [5], obesity [6], cancer [7–9], pregnancy related disorders [10], among many others. Prediction studies can also be identified into several categories: 1) risk factor studies, in which the aim to identify which predictors -age, stage of disease, biomarkerscontribute to the prediction of an outcome; 2) prediction models studies, in which the development, validation or improvement of prediction models are proposed to guide personalized patient care; and (3) prediction models impact studies, in which the effect of using a model to guide patient care versus not using it is evaluated [11]. Abbreviations: RAI, Responsible Artificial Intelligence; XAI, eXplainable Artificial Intelligence. * Corresponding author. E-mail address: [email protected] (M.C. Romero-Ternero). Contents lists available at ScienceDirect Information Fusion journal homepage: www.elsevier.com/locate/inffus https://doi.org/10.1016/j.inffus.2022.03.011 Received 13 December 2020; Received in revised form 17 December 2021; Accepted 18 March 2022 Information Fusion 83–84 (2022) 53–78 54 In the specific context of pregnancy, many AI algorithms and techniques have been applied for the prediction or prevention of disorders such as preeclampsia or gestational diabetes, and the predominant AI technique employed in the last decade is Machine Learning [10]. Most of the reviewed studies assume by default a traditional medical diagnosis model in which the predictive indicators are based solely on the biological factors of the patients (such as previous events, anatomy or physiology). However, the biopsychosocial diagnostic model [12] is increasingly predominant approach, where seeking the fusion of biological and behavioral data is key, both on a psychological and social level [13] (see Fig. 1). Studies that focus only on biological factors usually translate into the fact that the data used to develop such prediction models originates from one single source, typically medical records or medical instrumentation. Meanwhile, if the biopsychosocial diagnostic model is followed, the prediction models would be expected to work with heterogeneous data from different sources. Then, data would be collected from, not only medical records and medical instrumentation, but also smartphones, remote sensing devices, inputs from patients, or healthcare professionals, among others. We also derive from the biopsychosocial diagnostic model that a relevant data source is the person’s psychological data, which would enable the development of AI techniques to detect the emotional state. Ultimately, the variety of available data opens up possibilities for evaluating healthcare from a more holistic approach, but also arises questions regarding how to collect and consume that data: what types of data have to be collected, how often, from which the data source, and, of course, how to fuse all different types of data to obtain and improve health predictions. Due to the massive potentially disruptive and transformative impact AI can have on both a societal and individual el, questions regarding its possible negative consequences have been arisen. There is a need to ensure an ethical, ansparent, and accountable use of AI, which has led to the creation of the Responsible Artificial Intelligence (RAI) methodology. According to [14], AI approaches not only need to be well performing, but also trustworthy, transparent, interpretable, and explainable. They argued that if medical professionals are complemented by sophisticated AI sys tems and in some cases even overruled, the human experts must still have a chance, on demand, to understand and to retrace the machine decision process. [15] showed Explainable Artificial Intelligence (XAI) as a technique that can be used in the analysis and diagnosis of health data by AI-based systems and proposed an approach with the aim of achieving accountability, transparency, result tracing, and model improvement in the domain of healthcare. They used feature importance techniques for enabling explainability and interpretability of the model, and the underlying workflow from the perspective of medical practitioners. According to [16], demand for ethical AI is increasing and European guidelines and regulations have emerged around this issue. The European Commission (EU) has been closely watching AI and published the White Paper on Fig. 1. Heterogeneous data in biopsychosocial diagnostic model from (Novack et al., 2007). [13]. Abbreviations–PNI–Psychoneuroimmunology; PNE–- Psychoendocrinology; GI–Gastrointestinal; CV–Cardiovascular. A.M. Oprescu et al. Information Fusion 83–84 (2022) 53–78 55 Artificial Intelligence: a European approach to excellence and trust (2020) [17]. It contains measures proposed to promote and support the development and use of AI, but also identifies its main risks: risks for fundamental rights, including personal data and privacy protection and nondiscrimination and risks for safety and the effective functioning of the liability regime. More recently, the EU has published a proposal for a regulation laying down harmonized rules for placing AI on the European market (2021) [18]. This proposal includes prohibitions of certain AI practices, specific requirements for high-risk AI systems and obligations for operators of such systems, harmonized transparency rules for AI systems intended to interact with natural persons, emotion recognition systems and biometric categorisation systems, and AI systems used to generate or manipulate image, audio or video content and rules on market monitoring and surveillance. The Commission strongly supports a human-centric approach in their communication on Building Trust in Human Centric Artificial Intelligence (2019) [19]. Additionally, the High-Level Expert Group on AI presented Ethics Guidelines for Trustworthy AI (2019) [20]. Horizon 2020 Program published a guidance on how to complete an ethics self-assessment for research proposals (2019) [21]. Under these premises, it can be stated that there is a need of RAI (and XAI) applications for real-world environments in Health. According to Cluster 1 (Health) of Horizon Europe 2030 [22], European efforts are addressed “to deliver health care by contributing to safer, trusted, more effective and efficient, affordable and cost-effective tools, technologies and digital solutions for improved (personalized) health promotion and disease prevention, diagnosis, treatment and monitoring for better health outcomes and well-being, by integrating people in the design and decision-making, based on expected health outcomes and potential risks involved.” It will also contribute to an AI health-related industry in the EU. In a previous work in which we reviewed how AI has been applied to pregnancy health and well-being in the last decade, we found that Machine Learning (ML) is the most used AI technique [10]. Some of the principal findings of this work are that only a few studies used emotional and psychological data, and most of the studies on ML models lacked further efforts needed to embed them into devices or machines to work in. Additionally, apart from the transparent ML algorithms (linear/logistic regression, classification rules, decision trees, among others), there were no further efforts dedicated to the explainability of non-transparent algorithms. Similarly, there was an absence of scientific literature reporting or developing RAI approaches in the pregnancy healthcare domain. Our research line aims to respond to these gaps by seeking a standard methodology to design and deploy RAI- based solutions for personalizing pregnancy healthcare. Thus, we aim to achieve two fundamental objectives: 1) to prevent the risks of pregnancy and 2) to improve the pregnant woman’s experience along the process by using affective computing to detect emotional distress (in line with the biopshycosocial diagnostic model). Considering the aspects presented in this section and aiming to approach health in a holistic manner, a data collection methodology for RAI in Health is needed, which we are currently exploring. Briefly, the key contributions of the present study are the following: •A preliminary data collection methodology for RAI in Health is proposed. •The first step of this methodology is applied to a case study for pregnancy health. •From this case study, our findings show that RAI can help pregnant women trust an intelligent solution to prevent pregnancy risks. •Our findings evidence that pregnant women who are interested in using an AI pregnancy application share a strong idea that it needs to be responsible. The remainder of this article is organized as follows: In Section 2, we present the role that AI can play in pregnancy health and well-being and why intelligent solutions are needed. We also briefly introduce the project in which this work is framed. We cover the main disciplines involving this work in Section 3. A proposal for data collection methodology and the methods used to accomplish its first stage with a study case in pregnancy are described in Section 5. In Section 6, we present our findings in different themes designed in a structured interview for a prospective and qualitative study. In Section 7, the main results of the study from RAI and Affective Computing perspectives are discussed and the limitations are addressed. Finally, Section 8, provides a general interpretation of the results, as well as potential implications and next steps. 2. Context–AI and pregnancy As mentioned in the previous section, very few studies on AI applied to pregnancy health and well-being used emotional data to develop prediction models, however, there is a considerable amount of literature that illustrates the hypothesis that a pregnant woman’s emotional state is a contributor to adverse pregnancy conditions and outcomes, such as preeclampsia. Some adverse factors that have been specifically associated with preeclampsia are: emotional stress and hypertension, stressful events in life such as death or trauma, sleep disorders, altered circadian rhythm, overweight, hyperinsulinemia, recent miscarriages, and personal or family history of preeclampsia. Several authors associate anxiety and depression with pregnancy complications such as increased risk of suffering preeclampsia, giving birth through a c-section, low birth weight, or preterm birth. Therefore, scientific evidence suggests that anxiety and depression should be a factor to consider when evaluating a pregnant woman’s risk of developing complications during pregnancy. [23] concluded that correctly identifying women at risk of suffering from antenatal anxiety and depression would provide the opportunity to target those women who would benefit from preventive and supportive interventions. Pregnancy is a critical and complex event in a woman’s life, with an impact that can affect her psychological health. On the one hand, there are important physiological changes requiring an adaptation process that can be difficult to adjust to, physically and psychologically. On the other hand, the search for her own well-being as well as her fetus’s is a core factor that can affect her psychological health, since it may require learning and leading lifestyle changes (nutrition, fitness, sleep, work, among many others), as well as having her medical supervision. The fear or worry of developing a disease, especially one that can cause major complications for the mother or fetus, can also negatively impact the woman’s mental health. Although in this case psychological issues can be a consequence of other problems, they can also be a predisposing factor of pregnancy disorders. According to [24], emotional stress during pregnancy, sleep disorders and stressful life events were associated with higher risk of developing hypertensive diseases, such as preeclampsia. [25] concluded that anxiety and depression should be associated with risk for adverse pregnancy outcomes such as preeclampsia and non-vaginal deliveries. In a prospective population-based study on six hundred twenty-three pregnant women, [26] found that anxiety and depression in early pregnancy were associated with increased risk for preeclampsia. [27] also reported a link between hypertensive disorders and maternal mental health in a study in which one thousand three hundred seventy-one pregnant women participated. In a case-control study conducted on 150 pregnant women who suffered preeclampsia and 150 healthy pregnant women, [28] started with the hypothesis that there was a relationship between the incidence of preeclampsia and anxiety during pregnancy. They found that 26.7% of the women suffering preeclampsia and 10.7% of the control group had anxiety. They concluded that anxiety during pregnancy could be considered as a risk factor for preeclampsia. Although not directly related to depression or anxiety, [29] studied the effects that feelings of worry about the onset of poor health during early pregnancy had on the pregnant woman’s health. They found that A.M. Oprescu et al. Information Fusion 83–84 (2022) 53–78 56 feelings of worry during the first trimester can have a direct impact on the subsequent onset of adverse pregnancy outcomes such as preeclampsia. Authors in this study have also pointed out that all interventions and actions directed towards helping the pregnant woman manage her concerns about pregnancy-related health conditions can be beneficial not only for her but also for her infant. Therefore, the above-mentioned evidences suggest that the emotional state can have an impact not only on the incidence of hypertensive disorders such as preeclampsia, but can also lead to other adverse pregnancy outcomes. There is also evidence to believe that the impact of emotional distress during pregnancy can last far beyond giving birth, potentially having powerful long-term effects on both the mother and baby. According to [30], “even milder forms of maternal stress or anxiety during pregnancy affect the fetus causing possible long-term consequences for infant and child development”. Therefore, the authors “emphasize the importance to further investigate the influence of stress during pregnancy for infant and child development.” Regarding perinatal mental health, pregnant women have been identified as a vulnerable group and they are among those who are most worried and concerned about spreading or getting infected by SARS- CoV-2 [31]. Restriction of movement has led to pregnant women not attending antenatal care or having their appointments by telephone or through online services. Measures to restrict the virus have also led to impaired support from relatives and friends, an increase of financial difficulties and also an increase of the risk of intimate partner and domestic violence. These potential sources of maternal stress have been identified as risk factors for mental health problems during the COVID-19 pandemic [32]. Additional sources of stress include the challenges of homeschooling, remote work and overcrowded homes (Holmes et al., 2020) [33]. Poon et al. (2020) [34] found that during the ongoing COVID-19 pandemic, mental health problems during pregnancy have been associated with adverse maternal and infant outcomes including suicidal ideation and an inefficient mother-infant bonding, [35] found out an increased likelihood of depressive symptoms and anxiety among pregnant and non-pregnant women in the early postpartum period during the COVID-19 lockdown. The importance of surveillance for emerging threats to pregnant women and infants during times of crisis cannot be overemphasized [36]. It is important to proactively develop appropriate strategies to alleviate stress by screening, identifying and managing perinatal mental health disorders during the pandemic, without delay. For example, [37] proposed Internet-based screening tools, virtual online consultations/counseling and web-based psychological support and therapeutic interventions; [35] suggested that routine depression and anxiety screening should be considered in obstetrical settings in the wake of the current pandemic to ensure optimal perinatal and infant mental health, and [38] assessed the influence of the COVID-19 pandemic and subsequent physical distancing/isolation measures on the mental health and physical activity of pregnant and postpartum women and identified a substantial increase in the likelihood of maternal depression and anxiety during the COVID-19 pandemic. This highlights the strong need for heightened assessment and treatment of maternal mental health. [39] affirmed that strategies targeting maternal stress and isolation such as effective risk communication and the provision of psychological first aid may be particularly useful to prevent negative outcomes for women and their foetuses. Thus, from our understanding, all this is consistent with the mentioned idea of using data fusion based on the biopsychosocial model and including AI-based enabling technologies which consider emotional aspects. It is both suitable and necessary in helping prevent pregnancy risks and providing supportive care to patients that already suffer from it. AI is a computational field that has been consistently proving its potential. The last decade witnessed an important growth of AI in different disciplines, among which medicine can be found [40]. There is a vast amount of scientific literature describing how the many advantages AI can offer to healthcare services can significantly enhance patient’s health and well-being. [40], in their review of AI in the past, present, and future of healthcare, highlighting the importance of the availability of clinical data. It is a fact that both our capacity of generating and storing data has been rapidly growing, also in the field of medicine, and that this has played an important role in the development of AI applications for health and well-being. Health data comes under different formats: electronic medical records, information from experts, physical exams, images and scans from medical equipment, administrative information such as demographics, among many others. The availability of this kind of data eases and enhances the use of AI in the medical field and, in addition, non-availability can be a great barrier. As reported in a survey on AI and health, several advantages of AI in this field have been highlighted [40]: offering healthcare professionals recent information and updates on research findings, books, new techniques, and medical practices that could be used to inform proper patient care; providing insights into AI algorithms’ ability to ‘learn’ features from a large volume of healthcare data, that could result in assistance in clinical practice; reducing diagnostic and therapeutic errors, as well as extracting useful information from large patient populations to assist making real-time inferences for health risk alert and health outcome prediction. In a 2019 review article by [41], AI has been recognized to have an impact on three different levels. For clinicians, AI can assist medical diagnosis, predominantly via rapid, accurate image interpretation; for health systems, it can improve workflow and potentially reduce medical errors; and for patients, it can enable them to process their own data to promote health. 2.1. About this study This study is framed in a research project titled Human-centric and multimodal Emotional computing approach for Responsible Artificial Intelligence – A case study for preventing pregnancy risk, that is a result of the collaboration between researchers from the University of Seville, the University Hospital Virgen del Rocío, and Clinic Victoria Rey after identifying a potential research gap: AI techniques and affective computing (AC) can be used to improve the prevention of risks in pregnancy, and to give emotional support which helps in reducing the risk, respectively. This project presents a main research question: RQ: How to develop a standard-based and enhanced data collection method for researchers and industry within the health domain to better guide them in the process of undertaking responsible AI projects considering the domain’s specific challenges and unique characteristics?. From this, we put forward three research sub-questions: RSQ1: What are the existing algorithms, techniques and methods for guiding responsible AI projects in healthcare in order to support the development of a standardized process method for healthcare?, RSQ2: What are the main concepts and activities involved in medical responsible AI for constructing a standardized process method for healthcare? and RSQ3: How can the main concepts and activities be modeled into a standardized method for RAI projects in the healthcare?. In the current phase of the project, the present study specifically aims to propose a preliminary data collection methodology for RAI in Health, considering a holistic approach. From there, here we expose a case study applying the first step of the methodology to 1) to confirm the following hypothesis: "Responsible AI by design can help pregnant women trust an intelligent solution to prevent pregnancy risks, increasing adherence as a result" and 2) to identify which are the desirable characteristics of a pregnant-centered intelligent solution that is able to prevent pregnancy risks by providing emotional support. Specifically, after gathering and analyzing the information provided by pregnant women, we have been able to answer the following questions: What is the participants’ previous experience of smartphones and Internet usage? What functionalities are deemed interesting by the A.M. Oprescu et al. Information Fusion 83–84 (2022) 53–78 57 participants? What functionalities are deemed interesting by the physicians? Which are the aspects related to the user interface and user experience that participants find attractive and that would improve adherence? Are there any differences in the answers provided by patients that have a high risk pregnancy than the patients that are not in that group? If so, is there any correlation between the patients with a higher risk and a higher interest in a technological solution? Can a technological solution based on explainable Artificial Intelligence algorithms increase participants’ trust? What privacy and security related aspects are the participants aware of and which should be considered for this type of technological solution? Are there differences among the answers provided by participants from public/private care services? Are there differences among the answers provided by participants with assisted reproduction/natural conception? What ethical aspects should be taken into consideration in this type of technical solutions? Which are the obstacles and facilitators in adopting technological solutions for preventing risks during pregnancy? Which are the challenges in adopting an User- Centered Design (UCD) process for pregnant women? Are there any conclusions to be drawn that could help formulate different strategies for participatory design involving pregnant women for the next phase or our research? 3. Background As mentioned, including RAI-based technologies which consider emotional aspects may be appropriate to develop a pregnancy health solution implementing the biospychosocial diagnosis model. On the other hand, there are some ethical concerns with emotion recognition and biometric categorisation systems in the healthcare domain that should be solved by a RAI strategy. For this reason, this section includes a theoretical framework on RAI and AC. 3.1. Responsible Artificial Intelligence According to [42], RAI rests on three pillars of equal importance: 1) society in general must be prepared to take responsibility for the impact of AI (researchers and developers should be trained to be aware of their own responsibilities where it concerns the development of AI systems with direct impact on society and RAI is an issue of regulation and legislation), 2) RAI implies the need for mechanisms that enable AI systems themselves to reason about, and act according to, ethics and human values (it requires models and algorithms to represent and reason about, and take decisions based on, human values, and to justify their decisions according to their effect on those values), and 3) it is necessary to understand how different people work with and live with AI technologies across cultures to develop frameworks for RAI (AI must be understood as part of socio-technical relations). Under these principles, EU Guidelines [20] put forward a set of 7 key requirements that AI systems should meet to be deemed trustworthy: 1) Human agency and oversight (human-in-the-loop), 2) Technical Robustness and safety, 3) Privacy and data governance, 4) Transparency (and traceability), 5) Diversity, non-discrimination and fairness (no bias), 6) Societal and environmental well-being, and 7) Accountability (and algorithm auditability). From this perspective, in the next subsections, this study is focused on aspects related to Human-Centered Design, eXplainable AI, Privacy and Information Security by design. 3.1.1. Human-Centered Design Human-Centered Design (HCD) approach to system design and development aims to make interactive systems more usable by focusing on the use of the system; applying human factors, ergonomics, and usability knowledge and techniques [43]. Human-Centered Design tries to place the user at the center of the entire process of creating solutions, products and services. Usability is the extent to which a product can be used with effectiveness, efficiency, and satisfaction in a specified context of use [43]. While usability evaluation of traditional browsers from PC environments —desktop or laptop— and also touchscreen-based mobile has been widely studied [44], new IoT-based wearable devices present new usability challenges [45,46] and [47], in particular in Health [48–55] and with IoT [56,57], and in pregnancy domain [58]. In Human-Centered Design several aspects can be evaluated, some of them related to usability and others more related to the user experience. There are several usability evaluation techniques. Nielsen [59] developed the basis for the heuristic evaluation. A classification of usability techniques proposed in [60] is shown in Fig. 2. Over real systems or prototypes, the best alternatives are evaluations conducted by experts, also known as Inspection Methods, or evaluations involving users, which are divided into inquiry methods and testing methods depending on the methodology adopted. Predictive evaluation offers some predictions over the usability of a potential and not yet existent prototype, that is the case of the present study. Usability aims to create systems which are efficient, effective, safe, useful, easy to learn, easy to remember, and easy to wear. In this study, to ensure the usability of the final product, the final users (pregnant women and healthcare professionals) will be incorporated throughout the whole development process, which is iterative. Quantitative mea sures have been established at the beginning of the project, usability tests must be performed, and a multidisciplinary team has to be formed. On the other hand, User Experience (UX) is concerned with broader and more subjective aspects: satisfaction, fun, entertainment, motivation, aesthetics, creativity, or emotions. In the field of RAI, aspects related to human freedom, autonomy, and dignity are also considered in the design of intelligent solutions. 3.1.2. Explainable Artificial Intelligence Generally speaking, humans are reluctant of adopting methods that are not directly interpretable, tractable, and trustworthy [61]. An eXplainable AI (XAI) is an Artificial Intelligence that produces details or reasons to make its functioning clear or easy to understand to a given audience [62]. To this end, XAI borrows concepts from philosophy, cognitive sciences, and social psychology to yield a spectrum of methodological approaches that can provide explainable decisions, not only for technical profiles such as Data Scientists, but even for users without a strong background on AI. Therefore, XAI targets at bridging the gap between the complexity of the model to be explained, and the cognitive skills of the audience for which explainability is sought. The challenge is for ML systems to achieve the ability to explain their rationale, characterize their strengths and weaknesses, and convey an understanding of how they will behave in the future [63]. For this, new or modified ML techniques have to produce more explainable models (see Fig. 3). These models will be combined with state-of-the- art human-computer interface techniques capable of translating the models into understandable and useful explanation dialogues for the end user (see Fig. 4). Therefore, evidence suggests that the use of XAI in HCD can improve adherence, user trust, and user experience. our research line, we aim to work with explanation models and objective metrics in the pregnancy domain, where pregnant women and physicians make up the audience. As [62] stated, RAI extends further XAI by ensuring that other critical modeling aspects are taken into account when deploying AI-based systems in practice. These systems include not only algorithmic proposals, but also new procedures devoted to ensuring responsibility in the application and usage of AI models (including tools for accountability and data governance, methods to assess, and explain the impact of decisions made by AI models, or techniques to detect, counteract or mitigate the effect of bias on the model’s output). 3.1.3. Privacy by design Developing AI research requires, without prejudice to future regulations, to make strategic decisions that are guided a set of principles that are considered particularly relevant: 1) ensure a development of technology that takes into account fundamental rights not only as a A.M. Oprescu et al. Information Fusion 83–84 (2022) 53–78 58 limit, but also as a goal; 2) ensure that technology development processes take into account pre-existing legal framework; 3) being able to identify principles, values and rules that should be applied to technology without prejudice to its prior and even analogical nature; 4) ensure that the legal approach of AI achieves a balanced approach to technology, rather than purely reactive approaches focused on fear, prohibition and limitation; 5) being able to understand that the legal approach to the development of AI requires a coherent approach to the design processes. This obliges legal operators to identify methodologies and practical solutions. To do that, the experience of Privacy and Data Protection Law may be extremely useful. In the 1990s, with the increasing complexity and interconnectedness of information technologies, [64] developed pr ivacy by Design (PbD), a framework to proactively embed privacy directly into information technology, business practices, physical design, and networked infrastructures – making it the default. In October 2010, Privacy by Design was unanimously adopted as an international privacy standard and data protection authorities from around the world pledged to advance the Cavoukian’s 7 Foundational Principles of PbD in their respective jurisdictions. These funda mental principles of privacy by design in the context of their application to the healthcare sector are shown in Fig. 5. The EU embedded PbD into the rules of the General Data Protection Fig. 2. Classification of some usability evaluation techniques [60]. Fig. 3. Explainable Models (DARPA) [63]. A.M. Oprescu et al. Information Fusion 83–84 (2022) 53–78 59 Regulation (GDPR) [66]. The GDPR places phasis on transparency, fairness, and accountability (art. 5 GDPR). Moreover, under the GDPR, controllers and processors are required to implement “data protection by design and by default (art. 25 GDPR). The main goal of the requirement consists in providing ‘the necessary safeguards for the processing in order to meet the requirements of his regulation and protect the rights of data subjects’. To do that, the controllers or processors must implement a set of proceedings which, based on the capacities of the organization and the state of the art, will be able to correlate the nature of the processing (purposes, type of data, etc.) with the risks inherent therein and design appropriate regulatory compliance conditions. In addition, the GDPR has also enforced a new ‘data protection by default’ principle, which involves the data minimization principle. This principle is usually considered to operate as a constraint, which means that a large amount of data cannot be used. Well, this restrictive interpretation is a serious mistake. What the legislator provides for is to determine the volume, nature, and quality of data required for a particular processing activity. The Ethics Guidelines for Trustworthy Artificial Intelligence [20] underline the significant role of GDPR. But it is not enough to guarantee the right to data protection. AI ethics implies a design strategy that guarantees the rule of law by design. We could adopt the software development methodology recommended by the Norwegian Data Protection Authority (2018) [67] and translate it into an “AI design based on the rule of law” [68]. This authority proposes software design processes with differentiated phases: 1) Training (ethically and legally training the team); 2) Project requirements analysis (identify the legal requirements of the project); 3) Artificial Intelligence design process (risk Analysis); 4) Coding or programming (integrating principles into design and documenting compliance); 5) Test or verification; 6) Release (iterating from programming to market); and 7) Maintenance (maintaining the cycle throughout the life of an AI). This is a circular procedure intended not only at ensuring proper predesign, but also maintaining the standard of compliance as a commitment to continuous updating. Privacy by design and by default take privacy and data protection considerations into the whole developmental process from the start of the development and throughout the whole lifecycle to all types of sensitive information such as healthcare information [69–71]. In this study, the privacy issues concern to potential users are analyzed to keep from designing not only the normative complain but also to meet their expectations and thus increase their confidence in these types of solutions. 3.1.4. Information security by design An exhaustive review was carried out on security and privacy in the health field, showing at that time a significant concern regarding information security for the protection of personal data as sensitive as those in the health field [72]. However, a decade later, the study carried out by [10] shows that, in the field of AI applied to pregnancy, efforts have focused on the development of learning models without addressing safety-related issues, since only 23% of the studies analyzed in this area contemplate some aspect related to safety. This may be mainly due to the fact that the studies carried out do not finally obtain a product that can really be used by pregnant women and, therefore, the need to consider safety from the beginning, that is, from the design, is being obviated. Traditionally, much research has been focused on information systems solutions and based on the concept of containment where information is being kept within a certain perimeter and that it should be Fig. 4. Emphasis and scope of XAI research (DARPA) [63]. A.M. Oprescu et al. Information Fusion 83–84 (2022) 53–78 60 protected from threats from the outside world [73]. This implicit philosophy of the protection of information is based on the notion of containment and to create physical boundaries around assets, compartments or perimeters that need protection [74]. This perception of the ‘inside’ versus the ‘outside’ is becoming problematic as it is getting more difficult to draw a line between insiders and outsiders and it is not always clear where an organization’s boundaries lie. Employees are becoming mobile and are not bound to the geographical location to perform their jobs. Furthermore, many organizations outsource processes to third parties, sharing data with them and connecting their networks. However, information security management has changed the perception of risk, and nowadays it is generally agreed that risks should be analyzed from a wider context [74–78]. This is even more the case in modern health care which is perceived as a complex environment where technology, human risk factors, scope of services, and geographical scope cannot always be separated from each other. The variety within health care services, ranging from scanning, epidemiology, diagnosing, caring, insuring, online services, transportation or research makes it hard to define the boundaries of what is the health care organization as an entity [73]. Increasingly the socio-technical nature of information security is coming to the fore [79] and the human dimension to both information security practice and technology design is being recognized [80]. Socio-technical studies consider, amongst other things, the interaction between the technology that is constructed and the people who affect and are affected by the technology. Researchers have adopted several underlying theories from reference disciplines such as psychology and sociology to analyze information security risk management and economic theories to characterize investment decisions and information governance [72]. The socio-technical approach acknowledges the importance of ‘social’ or the ‘human’ factors in security management. In this approach, human risks are separated from the technical information security risks. Along with this socio-technical vision comes a different way to measure risks. Where the traditional perception was one of quantification and technology, the sociologists evaluate risk by subjective perception based on values, belief and opinion, which are influenced by factors such as history, culture, politics, law and religion. Risk is seen as subjective or perceived risk, rather than pure mathematics [73]. Fig. 6 shows the different research domains in healthcare information security [72]. Some examples of risk management frameworks based on privacy by design philosophy in healthcare are presented in [71]. In this study, the security issues concerned with potential users are analyzed to keep from designing not only normative complain but also to meet their expectations and thus increase their confidence in these types of solutions. 3.2. Affective computing Computational modeling of affective mechanisms (emotion and motivation in particular) has been an area of increasing interest in research on adaptive behavior, cognitive systems, and robotics in the last two decades. In the last 30 years, there has been a growing interest in the effect of emotions in cognitive sciences [81], especially in new areas of research such as affective neuroscience [82] and affective Fig. 5. Seven fundamental principles of privacy by design applied to the healthcare sector [65]. A.M. Oprescu et al. Information Fusion 83–84 (2022) 53–78 61 computing [83], driven in particular by a large body of neuroscientific insights into affective, motivational, and emotional mechanisms and their role in cognition [84] and [85]. More recently, we can find authors such as Moerland et al. [86], Zhou [87], and Zheming [88] who propose different cognitive-affective multi-agent architectures using the concept of emergence. Affective Computing (AC) is a concept defined by Picard (1997) [83]. She defined as a discipline which tries to get machines to be able to recognize emotions in humans and to be able to produce human emotional responses. Scientists in the fields of psychology and cognitive sciences do not find a consensus on what is and what is not emotion and how emotion differs from affect. For example, the classical view of emotions understands that there are primary emotions universal to every human being [89‒92] and, nevertheless, more current views understand emotions as something that is built throughout people’s lives and takes into account a large number of factors in their environment, including cultural ones [84] and [93]. Perhaps for this reason, in the scientific literature in the field of Computer Science Engineering and Artificial Intelligence, there is also no consensus on the most appropriate term. In any case, it is not about the machine feelings, but rather that we here understand affective or emotional computing as the set of devices, models, processes, and algorithms necessary for a machine to recognize the emotional state of a person with whom it interacts and can be intelligently programmed to act accordingly in that interaction. We think it is in this “act accordingly” that special care must be taken from the perspective of the RAI to avoid ethical problems. 4. DMM-HERA–Data management methodology in the context of heterogeneous environment and RAI applications in health AI offers many advantages to healthcare services and can significantly enhance the patient’s health and well-being. Jiang’s review of AI in the past, present, and future of healthcare [40] highlights the importance of the availability of clinical data obtained from different sources (medical records in different databases and wearable health monitors). The availability of this data, alongside advances in computing power, eases and enhances the use of AI in the medical field. Among its uses, we can find assistance in clinical practice reducing diagnostic and therapeutic errors, as well as extracting useful information from large patient populations to assist making real-time inferences for health risk alert and health outcome prediction. [41] positioned AI’s impact in health care on three different levels: AI can assist clinicians in medical diagnosis, via rapid, accurate image interpretation; it can improve health system workflows and potentially reduce medical errors; and it can enable patients to promote their own health. AI linked to the advancement of Information and Communications Technology is significantly changing the experience of users for whom technological solutions are designed [94,95]. The customization of technological solutions is possible thanks to the use of AI in its different variants, enabling significant advances in improving the quality of life of patients of various kinds [96,97]. In this context, because heterogeneous sources of data are present, we propose a methodology based on humancentered design, RAI principles, and Enterprise Architecture standards [98] for collecting data in Health domain. This methodology is detailed in the next section. 5. Methods This section describes the proposed data collection methodology and the methods used to accomplish its first stage with a study case in pregnancy, by elaborating a prospective and qualitative study. 5.1. Data collection methodology Four main elements must be considered to propose a standard-based and enhanced data collection method for researchers and industry within the health domain to better guide them in the process of RAI projects considering the domain’s specific challenges and unique characteristics: 1) patient data, 2) medical protocols, 3) health services workflows, and 4) stakeholders (see Fig. 7). The traditional patient data sources are medical records and medical instrumentals. However, IoT wearable devices heterogeneous data from the biospychosocial model are also considered here. In this multimodal data context, we opose a preliminary methodology that consists of the following steps: Fig. 6. Research domains in the healthcare information security [72]. A.M. Oprescu et al. Information Fusion 83–84 (2022) 53–78 68 Other functionalities: Participants were asked to provide via free text which other functionalities they would look for in the application. 44% (66) of participants shared their thoughts. The most frequently mentioned functionality was having a question and answer system and receiving information and advice on pregnancy: 31.8% (21). Secondly, 15.2% (10) of participants were interested in receiving information specifically about the baby (fetal state and wellbeing, weight, and development), adapted to their gestational age, and being able to see to listen to the baby. Moreover, the following functionalities have been pointed out by 10.6% (7) of participants: reminders, appointments, and agendas; 7.6% (5) of participants: advice and recommendations on how to live a healthy lifestyle; and 6.1% (4) of participants: having fetal photographs from ultrasound in the application. Some examples of verbatim comments are listed in Table 9. 6.2.2. Usability perspective Features for usability enhancement: Participants were asked to rate the features they would value the most in a pregnancy application from a usability perspective. As can be seen in Fig. 11, participants rated as the most important a secure, useful, and well-functioning application, followed by being easy to learn, creating a sense of satisfaction when using it, easy to remember, motivational and being able to make the person feel happy. Is is also important that the application is multiplatform. The aspects rated as less important are that the application is fun, entertaining, and aesthetically pleasing. 6.3. Responsible AI aspects This section describes the results regarding explainable AI, privacy, information security, and ethics. 6.3.1. Explainable AI Explainable recommendations: The intelligent solution would be able to provide recommendations based on the emotional state (see Section 6.5). The 68.7% (103) of respondents that were interested in receiving recommendations based on their emotional state were asked their opinion on receiving explanations from the system on why such recommendations are given. 85.2% (82) of participants agreed that they would trust the system more and that they would be reassured of its use, if they understood why they are receiving certain recommendations. 11.7% (12) did not consider it an important nor necessary aspect when using the system. We posed the same question to the 18% (27) of respondents that were not interested in receiving these recommendations based on the emotional state (see Section 6.5). 55.5% (15) would still not use the system and did not find explainability to be an important feature. Meanwhile, 25.9% (7) of participants considered this feature as a way to improve their trust in the system. 6.3.2. Privacy aspects General privacy concerns: 66.7% (100) of the respondents shared their privacy concerns. The biggest privacy concerns reported by participants were protection of personal data, fear of being heard or seen without them being aware of it, use of data that they have not authorized, and their information being seen by people other than their physician. Privacy policy reading: Participants were asked about how often they read the privacy policy of the services they used. 49.4% (74) of participants reported reading the privacy policy only when they were interested in the service. Few participants, 7.3% (11) indicated to always read the privacy policy, 14.6% (22) read it frequently and 24.6% (37) never read it. Privacy policy of social media: Participants were surveyed on their awareness of the privacy policies of social media applications. Their feedback was varied. Meanwhile 50% (75) of participants reported not only being aware, but also having changed their configuration to set their privacy settings to a more restrictive option, 32.6% (49) reported not being aware of the privacy policy. Permissions granted to mobile applications: Subsequently, participants were asked if they were aware of the missions they granted to the applications installed on their phones. 56% (84) of women reported being usually familiar with the permissions they granted. Half of them indicated changing the default permissions according to their privacy preferences. 6.3.3. Information security aspects Concerns on Information security of IT services: 74% (111) of the participants reported being concerned about the information security of the services they consume, of which 20.6% (31) even felt discouraged from using certain services due to this security concern. Only 18% (27) of respondents were not worried, of which 12% (18) answered that they trusted the service provider to guarantee data security. Security measures on owned smartphone device: Participants were surveyed on the safety precautions they implement for their own device. The most popular safety measure among the participants was screen locking with a password, pattern, PIN or biometric scan, followed by uninstalling unwanted or unused applications, performing periodical backups, avoiding installing applications from unknown sources, allowing automatic operating system updates and installing an Table 9 Desirable functionalities of the application reported by the participants functionality group Verbatim quotes. Functionality group Verbatim quotes Questions and Answers, information and advice “Advice and recommendations of healthy lifestyle at each stage. Advice for delivery preparation” “Being able to ask questions to the physician over the app if needed” “Physician’s feedback to the data collected by the app” “Information about the different stages of pregnancy” “Send pregnancy questions” “Information on vaccination” “It would offer advice. The app would adapt to each person” “Reliable answers to pregnancy questions” “All the possible information and recommendations to follow’’ “Common pregnancy discomfort and what to do” Information related to the baby “Ways to see or listen to the baby” “Control the baby” “Input and save the weight of the baby during pregnancy” “It would be interesting to have some sort of map or protocol according to each gestational week” “Being able to upload your own pictures of the baby and ultrasound sessions. Also a simulation of the baby in each gestational week” “Facts and information about the baby in its gestational age” Reminders, appointments and calendars “Appointment reminder and how to live a healthy lifestyle” “Check-up reminders. Hospitals and physicians nearby if I am out of town” “Calendar with appointments for ultrasound, blood tests, vaccination from the first visit” Nutrition and exercise “Healthy eating” “Foods I should not eat” “Diet and food recommendations” “Number of steps” “Exercises recommendations” Risk alert “Notify me if there is any risk in the pregnancy (that could be affecting the baby)” Application requirements “All should be under medical supervision” “Information has to be reliable” “I want to be able to get in touch with my physician” “App has to be transparent” “Easy to use” “Easy and intuitive” A.M. Oprescu et al. Information Fusion 83–84 (2022) 53–78 69 antivirus. Most participants, 94% (141), indicated having at least one of the said security measures, from which 1 participant indicated “Other measures”. 36.8% (52) of participants had 1 or 2 security measures in place, followed by 29.7% (42) who had 3 or 4 security measures. Almost a third of the participants (32%, 45) had applied 5 or 6 security measures and only 7.8% (11) applied 7 or 8 safety features. On average, participants who secured their devices had implemented roughly 3 (3.3) measures. Two-authentication factor as a security measure: On the one hand, 77.3% (116) of participants reported some level of agreement when asked about their thoughts on applications that used twoauthentication factor (having to provide a secondary password or security token). They thought that (a) the application was more trustworthy by implementing this additional layer of security: 35.3% (41) of participants, (b) it was a reasonable measure: 34.5% (40) of participants, and (c) found it to be adequate: 30.2% (35) of participants. On the other hand, 17.3% (26) of participants reported finding it annoying (77%, 20) or unnecessary (23%, 6). Trustworthiness of a pregnancy application depending on the provider: If it came up with the use of a pregnancy medical application on their mobile device, pregnant women were surveyed on the trustworthiness of the application regarding its provider. The potential providers would be the public health system, a private company working for the public health system, and a private company specialized in this type of service that held no relationship with the public health system. Overall, 86% (129) of the participants felt that the more reliable option is the public health system. 6.3.4. Ethics aspects Willingness to allow personal data to be used for scientific and research purposes: 87.3% (131) of pregnant respondents were willing to provide their pregnancy-related information for research purposes. 56.5% (74) would only agree if the data could not be associated with them (anonymized data). Willingness to participate in a research study as data donor: The number of pregnant women that would be willing to participate in a research study decreased by 6% in comparison with the previous question, resulting in 81.3% (122) of participants. In this case, a higher rate would only agree if the data were anonymized: 60.7% (74) of participants. 6.4. Affective computing aspects This section describes the results related to affective computing questions. Recommendations based on emotional state: Participants were informed about the existence of a computational field called Affective Computing, in which an intelligent solution can recognize the user’s emotional state and provide recommendations based on it with the purpose of assisting her when she is not feeling well. When asked if they would be interested in using this type of technology, 68.7% (103) responded “yes”. Participants that answered negatively (18%, 27) were questioned on the reasons from the perspective of privacy, ethics, fear of technology, trustworthiness, and skepticism. The main reasons disclosed were not knowing if they would trust the recommendations (trustworthiness: 48.1%, 13), privacy concerns (22.2%, 8) and not thinking it is possible (skepticism: 29.7%, 6). Willingness to use or wear a sensor, microphone or camera: A positive feedback has been provided by 45.3% (68) of the participants when asked if they would be willing to use an integrated sensor on their clothing or accessories for emotional recognition. The number of participants to respond positively decreased to 38.7% (58) when asked if they would be willing to use a wearable microphone (mobile phone’s integrated microphone or a built-in microphone in clothing or accessories). Finally, 43.3% (65) also reported positively on the use of the mobile phone’s built-in camera. Subsequently, unwilling participants were asked for reasons similarly as in the previous question. As can be seen in Table 10, in all three Fig. 11. Which aspects do you find most valuable in the application?. Table 10 Reasons behind negative responses to willingness of wearing or using a sensor, microphone or camera. Would not wear a sensor (n=53) Would not use a microphone (n=67) Would not use a camera (n=66) Privacy concerns 22 (41.5%) 45 (67.1%) 50 (75.8%) Would not trust the recommendations 18 (33.7%) 17 (25.3%) 14 (21.2%) Do not think it is possible 13 (24.5%) 12 (17.9%) 9 (13.6%) Ethical concerns 4 (7.5%) 6 (9%) 5 (7.5%) Fear of Technology 4 (7.5%) 4 (6%) 1 (1.5%) A.M. Oprescu et al. Information Fusion 83–84 (2022) 53–78 70 scenarios (wear or use a sensor, microphone, or camera), the main concern is privacy, followed by lack of trust and skepticism. Use of a camera and microphone raised the most privacy concerns. A relatively high number of participants did not respond (marked answer as “No answer”): 19.3% (29) for sensor use, 16.7% (25) for microphone use and 12.7% (19) for camera use. 7. Discussion As a result of the application of the first step of our methodology, we now have a better understanding of the needs and expectations, as well as the potential trust and adoption barriers that pregnant women have towards using a pregnancy AI-based solution. We have applied this step to qualitative research. As mentioned in the previous sections, we have focused our research and this qualitative study on two fundamental pillars: RAI (with special focus on User- Centered Design, eXplainable IA, Privacy, Information Security and Ethics) and Affective Computing. We proposed the following general hypotheses and secondary objectives: the general hypothesis states that “Pregnant women that consider the use of an intelligent solution during pregnancy, also think it should be designed and used in a responsible way” (interest in RAI). Subsequently, the three secondary objectives are presented. First, we sought to determine if pregnant women with higher risk of complications are more interested in AC than women with less risk. Second, we examined whether pregnant women demand personalized technological solutions, adapted to the health condition of the user. Third, we analyzed if participants attending public healthcare services have a higher rate of agreement on sharing data for research purposes if the provider is a public healthcare entity rather than a private entity. General hypothesis: To determine which participants are interested in RAI, we propose a score ranging from 0 to 15. This score is based on nine multiple answer questions in which we posed concerns about privacy, ethical implications, security, data anonymization, and mobile device security. The following score ranges reflect the participants interest in RAI: low level of interest (0-4): 35.4% of participants; medium level of interest (5-8): 56% of participants; high level of interest (9-12): 8.6% of participants; maximum level of interest (13-15): 0% of participants. The mean score was achieved at 5.33 points with a median of 5 (medium interest). The minimum score obtained from the analysis was 0 and the highest was 12 points, not reaching the maximum score in any case. The percentage of participants related to each interest level in RAI can be seen in Fig. 12. Secondly, we determined the level of demand of personalized technological solutions using seven questions. We have established a final scoring system consisting of a score ranging from 0 to 7 for each participant (0-1: no demand of technological solutions; 2-3: demand of technological solutions, however, it is not a priority in their pregnancy; 4-5: demand of technological solutions as an important aspect in their pregnancy; 6-7: high demand of technological solutions). The analysis did not reveal a significantly higher interest in RAI by the participants in the high demand group (medium score 5.15, t-test 1.679, p=0.181). Initially, no significant correlation was found between the level of demand of personalized intelligent solutions and aspects related to RAI (Pearson correlation coefficient: -0.091, p=0.268). Participants that were not interested in an intelligent solution that could offer recommendations were reevaluated after informing them that said recommendations would be science-based and reviewed by physicians. 40% of patients changed their minds, beginning to consider the solution to be useful. As a conclusion, although we have identified that participants showed a significant interest in IT solutions, they have also raised some concerns and unknowns on the ethical implications and, specially, on the potential invasion of their privacy, which may justify that some participants showed a moderate interest in RAI. On the other hand, the results show that 80% of participants are interested in a recommendation app (120) with high demand (score 6- 7). Among them, 18% (22) had less than 6 and 7% had less than 4. Significant differences were not found between women interested in a recommendation system and at once not interested in XAI (regardless of the cutoff used to define low interest in XAI). Although currently we do not have any more data to understand why there are more women interested in an application that gives recommendations on how to have a healthy lifestyle during pregnancy than in an intelligent solution that helps detecting pregnancy risks, we could hypothesize some explanations to this result. To the best of our knowledge, an intelligent solution Fig. 12. Interest in Responsible Artificial Intelligence (general hypothesis). A.M. Oprescu et al. Information Fusion 83–84 (2022) 53–78 71 that could detect pregnancy risks is not directly available for pregnant women to download from a smartphone application store. Most pregnancy applications focus on providing information on how to live a healthy pregnancy, along with information about the mother and baby. As there is a very high need of information for the mother-to-be, it is understandable that they would be interested in this type of application. In addition, we know from our questionnaire that 78% of the participants have already downloaded a pregnancy app, which we can assume is a healthy lifestyle recommendation type of app. We can derive from the other desirable functionalities that the participants reported (this can be seen in verbatim form in Table 9) that some of this functionalities match what would be expected in a healthy lifestyle recommendation app: “healthy eating”, “number of steps”, “Facts and information about the baby in its gestational age”. However, we can also see some desirable functionalities that could not be implemented in that type of application: “Physician’s feedback to the data collected by the app”. “Notify me if there is any risk in the pregnancy (that could be affecting the baby)”, “All should be under medical supervision”. Thus, we understand that the intelligent solution we design and implement should also include recommendations on how to live a healthy pregnancy. Additionally, early in pregnancy, healthcare professionals make an effort to always transmit to their patients that it is a natural process. Initially, new mothers-to-be may not consider themselves to have a high-risk pregnancy that would need special care or monitoring, unless their healthcare providers consider otherwise and inform them. For most pregnant women, the main concern is to stay healthy and enjoy their pregnancy, hence their more immediate interest in this type of application. This could manifest differently in pregnant women that have already been pregnant, especially depending on the outcome of the pregnancy and their health state and experience during the former pregnancy. However, at the moment, we do not have data to explore these possible cases. Secondary objective on RAI: “participants from the public healthcare service are more willing to share their data with a public research institution rather than a private one”. We defined a score ranging from 1 to 5, 1 being the lowest and 5 being the highest confidence. The mean degree of confidence for the public healthcare providers was of 4.63 and 4.08 for private companies specialized in healthcare, working for the public system. Lastly, the mean score for the private specialized in healthcare companies was 3.19. This results are illustrated in Fig. 13. Participants who expressed a higher level of trust in the public healthcare system, also considered more positively a ivate company that worked for the public system (Pearson: 0.348, p<0.001). As could be expected, they considered less positively a private company with no relation to the public system (Pearson: 0.180. p=0.036). In conclusion, pregnant women perceive as more trustworthy medical and health applications whose provider is the public healthcare system (4.63/5). We have found a direct correlation between women’s trust and different providers, and we have found that participants that trust more the public system, are also more likely to trust a private company, Secondary objective on affective/emotional computing: “pregnant women with high-risk pregnancy are more interested in AC than women with low-risk pregnancies”. The 126 participants that provided information on pregnancy type and responded to the affective computing questions are considered as the study population for this subanalysis. 79.37% of participants (100/126) were interested in AC solutions. For this subanalysis, we differentiated patients according to their pregnancy type, resulting in two groups: low-risk pregnancy group (53.17%, 67/126) and high-risk pregnancy group (46,83%, 59/126). First group: 76,12% (51/67) of women were interested in AC solutions, and in the second group, 83,05% (49/59) of women were interested, as is apparent in Fig. 14. The Pearson’s chi-square test with a value of 0.920 (p=337) indicated no significant difference between the two groups. Therefore, we can conclude that pregnant women are interested in AC solutions. Additionally, although patients in the high risk group expressed a higher interest, there is no statistical significance to support the fact that they are more interested in this solution than women with healthy pregnancies. (a) Interest in affective computing solutions by women carrying healthy pregnancies (b) Interest in affective computing solutions by women carrying high-risk pregnancies Secondary objective on human-centric design: “pregnant women demand personalized technological solutions”. For the analysis of the secondary objective, in which we sought to find if pregnant women demand personalized technological solutions (human-centric design), we established a score ranging from 0 to 7. Analogously to the score for the general hypothesis, and as can be seen in Fig. 15, the scoring system was established as follows: (0-1: no demand of technological solutions; 2-3: demand of technological solutions, however it is not a priority in their pregnancy; 4-5: demand of technological solutions as an important Fig. 13. Mean confidence level participants presented for each provider both with the intervention of the public system or without it. A.M. Oprescu et al. Information Fusion 83–84 (2022) 53–78 72 aspect in their pregnancy; 6-7: high demand of technological solutions). The questions used for the analysis and the percentage of affirmative responses are detailed in Table 11. The average score obtained was 6.23 points (high demand) and a median of 7 (highest demand). The highest demand was reported by 59.3% of participants, and the minimum score was 2 (see Fig. 15). Therefore, we have found that a high demand of technological solutions that offer customized care exists among pregnant women. This confirms the interest the potential users show towards new diagnostic and therapeutic techniques which can adapt to each individual. As we mentioned in the introduction, pregnancy is a delicate stage in a woman’s life which requires important health monitoring. Both mother and child can benefit from health promoting AI intelligent solutions, and not only from early detection and prevention of pregnancyrelated disorders, but also from the sensation of confidence, comfort, reassurance, and satisfaction that can be experienced resulting from the interaction with it. Almost all pregnant women found both aspects to be essential. Unsurprisingly, it has also been revealed that to achieve these benefits, pregnant women need to trust the system and find it reliable, both from a privacy point of view as well as from a technological point of view. We found some unexpected findings regarding privacy concerns, especially on the topic of Affective Computing the use of devices (sensors, cameras, or microphones) to collect emotional data. It stands out that meanwhile pregnant women are highly interested in receiving recommendations and risk notifications from an intelligent solution, hey are reluctant to provide certain data, especially emotional data. Understandably, pregnant women feel more comfortable and secure with the idea of an intelligent solution that manages data that cannot be immediately associated with themselves, such as medical parameters (blood glucose, blood pressure, results of blood tests, among others). This sense of security can be increased since they are aware of the data that the application is working with to provide recommendations, and this data is more tangible and factual than emotional data. Participants, Fig. 14. Results of secondary objective on affective/emotional computing. A.M. Oprescu et al. Information Fusion 83–84 (2022) 53–78 73 understandably, also seem to perceive emotional data as more personal and sensitive than medical data. We also think that the general population is used to employ mobile applications that manage this type of factual data, and they are already used with certain personal information being known and operated by a machine. It is noteworthy that the highest privacy concern expressed by participants referred to the fear of being watched or heard (use of camera or microphones) in moments they are unaware. We see in this finding an evident reason for concern and consequently the lack of trust participants can have in intelligent systems. Altogether, designing, developing, and using an intelligent solution in a responsible way make them feel more safety. These results are consistent with the perspective that the general population have on AI [120,121]. Our study reveals that RAI is essential in all steps of the process when designing an intelligent solution for pregnancy health and ell-being. This structured questionnaire has revealed even more interesting aspects that can be analyzed. As far as we know, participants who were reluctant towards the use of AC also showed a lack of trust in technological innovation (25.3%, women reported at least once “Would not trust the recommendations”, and 3.3%, 5 reported at least once “Fear of technology”). We think that social perception of science, innovation and technological advance could be improved by encouraging communication, dissemination and outreach of scientific activities in the academic and cultural scene, and training digital competences in formal education environment. In this sense, for example, European Commission promotes Digcomp framework [122,123]. This framework provides matured guidelines for training digital skills in citizens and it is being widely accepted by European governments, and around the world. Referring back to the questions we initially proposed in Section 2.1, the results show that the pregnant women that participated in this study were familiar with the use of smartphones and Internet. The most interesting functionalities for participants are: 1) receiving recommendations to maintain a healthy lifestyle, 2) sharing health data from the application with the physician, 3) notification of potential risks, 4) monitoring of health parameters such as blood pressure, blood glucose, weight, among others. Regarding the aspects related to the user interface and user experience that participants find attractive and that would improve adherence, participants found as most important that the system presents good functioning, which is secure and useful. Respecting if there are any differences in the answers provided by patients that have a high risk pregnancy than the patients that are not in that group, we have not found pregnant women in the high risk group to be more interested in AC solutions. On the question if a technological solution based on explainable Artificial Intelligence algorithms can increase participants’ trust, our analysis showed that 40% of the participants that were initially not interested in an intelligent solution changed their perception after being informed about RAI. On which are the challenges in adopting an user-centered design, it is interesting to study the wearable and environmental sensors that would be most suitable. Furthermore, we will take an anthropological and gender dimension approach to analyze how the daily life problems of Fig. 15. Results of secondary objective on human-centric design. Table 11 Responses to questions on functionality demand (Human-Centered Design). Question Affirmative percentage (%) 1) Would you find useful an application capable of detecting if there is a risk of 87.3% suffering a pregnancy related disorder or illness (i.e. preeclampsia)? 2) Although throughout our life we should try to live a healthy lifestyle, there are 94.7% certain aspects that are even more important to take care of during pregnancy (i.e. nutrition, health, stress, physical activity). Would you find it appealing to receive recommendations from the application regarding healthy lifestyle habits? 3) Certain information cannot be automatically obtained by the smartphone or 90.7% electronic device (i.e. your weight), and an additional device has to be used. Would you be willing to input this information manually into the application? 4) Would you find useful to receive reminders from the application? 89.3% 5) Are there any particular aspects you would like the application to monitor? In this 77.3% case, which are they? (i.e. blood glucose, weight, sleep / rest, blood pressure) 6) An application capable of monitoring aspects mentioned in the previous question 90% is also able to detect if they are not in the healthy range recommended by the physician (i.e. detection of poor quality of sleep). Would you want to receive notifications in such cases? 7) Would you find it interesting if the application showed you information from your 94% periodical checkups with your physician, such as blood works? A.M. Oprescu et al. Information Fusion 83–84 (2022) 53–78 74 pregnant women are and how they are related to the socioenvironmental health context. These results suggest that the use of XAI together with Human- Centered Design might improve adherence, user confidence, and user experience. There is a high demand for technological solutions by pregnant women adapted to their health situation. There is interest from potential users for new personalized diagnostic and therapeutic techniques based on affective computing that allow monitoring according to each patient. Pregnant women find it interesting to use applications based on Artificial Intelligence to improve their health outcomes. Even a good number of pregnant respondents were willing to provide their pregnancy-related information for research purposes, but are concerned about the confidentiality and privacy of their data. Regarding privacy, the development of regulatory compliance processes for a technology can never be static. The very nature of AI makes this principle a rule. We must understand that we are in an early phase of this technology, in which we have learned that the data analysis processes are very sensitive to bias. On the other hand, we begin to entrust basic decision-making processes to this technology, with legal or material consequences for its recipients. Proper compliance requires a state of ongoing monitoring and updating that is deployed at several levels: 1) learn from the operation of the technology itself. The results obtained, the errors of operation, incidents, and ultimately any verified or verifiable element should also be indexed and studied by legal support. And not only to prevent potential conflicts and responsibilities, but above all to improve compliance conditions; 2) deepen the design of compliance by proposing improvements when necessary; 3) accompany each phase or evolution of the product. AI faces a substantial challenge. To offer a logbook for an immediate approach, we need to be able to integrate the development of AI with an evolutionary interpretation of the regulatory framework. In this sense, certain necessary and essential efforts are required. The first of them assume the dynamics of the gestation of a project of a technological nature in its different phases. And, therefore, adapt our legal methodology to make it functional to the management model. It is not about becoming engineers, but about being able to work functionally with their methodologies. The second, to be able to descend to reality. The law applied to technology requires an additional understanding effort. The intensity of the requirement is very high and requires an open interaction in which receptivity and understanding are decisive. Third, an open and dynamic conception of regulatory compliance is necessary. AI projects will force the development of an enormous effort to systematically interpret the legal framework from its constitutional principles and values to the sectorial laws, from local to transnational. However, this openness is not only material, it is also methodological. The legal support effort must be permanent, never ends, accompanies AI throughout its life cycle. And this task is deployed in an emerging regulatory context and with rules under construction. Although this does not involve a blank check, either the industry or the researchers. The framework of fundamental rights and constitutional principles and values constitutes an insurmountable barrier. Similarly, the argument that an "analog" right is inapplicable is unacceptable. The regulatory program that incorporates our legal system defines objectives and principles that can be identified and applied to all human relations and their developments. To consider otherwise would be against any reason. It is precisely for this reason that legal professionals are clearly obliged to incorporate new methodologies to support the development of technology. We are obliged to build a society of digital transformation oriented to the common good from the values that incorporates the guarantee of dignity of the human being and the rights that are inherent to humans. Finally, this qualitative study has allowed us to identify some obstacles in adopting technological solutions for preventing risks during pregnancy, such as privacy and security concerns, lack of digital skills, and technology skepticism, but also we found some facilitators such the high interest and demand of intelligent solutions from pregnant women. Furthermore, it provides us very valuable information to face the second phase of our project. 7.1. Limitations Firstly, we have to consider the geographical limitation that is presented in this study. Although participants from both private and public healthcare systems have been interviewed, there is no geographic diversity since both the private clinic and the public hospital in which the interviews occurred in the same city. We acknowledge as another possible limitation not having collected information on race, marital status, employment, financial income, natural or assisted reproduction pregnancy. Another limitation that needs to be considered is that we have not included in our interview questions related to the current COVID-19 pandemic situation, since we think we would have collected significant knowledge for our study. 8. Conclusions In this paper, we presented a preliminary data collection methodology for RAI-based solutions in health. In our attempt to understand pregnancy health and risk prediction from a holistic approach, guided by the biopsychosocial model, we sought to explore the types of data, along with its sources, and characteristics from a multimodal data fusion perspective, and the privacy, legal and ethical implications, needed to develop a RAI-based solution. We also applied the first step of our methodology to study what pregnant women need and expect from an intelligent solution, and to identify potential trust and adoption barriers from a RAI perspective. We found out that the most interesting functionalities for pregnant women are receiving recommendations to maintain a healthy lifestyle, sharing health data from the application with the physician, notification of potential risks, and monitoring of health parameters such as blood pressure, blood glucose, weight, among others. Additionally, security, usefulness and good functioning are the aspects pregnant women consider are key in the solution. Furthermore, we identified that privacy concerns and trustworthiness are the most considerable limitations of the use of intelligent solution based on affective computing (potential trust and adoption barriers). Regarding the hypothesis of the case study ("Responsible AI by design can help pregnant women trust an intelligent solution to prevent pregnancy risks, increasing adherence as a result"), our findings evidence that pregnant women who are interested in using an AI pregnancy application share a strong idea that it needs to be responsible, trustworthy, useful, user-centered and safe. They were interested in personalized solutions to assist them by providing reliable and trustworthy information based on their own personal situation. This study showed that in order for the participants to use an AI pregnancy application, they need to be fully convinced that it can be trusted. From the qualitative study, we have confirmed this hypothesis and we have identified which are the desirable characteristics of a pregnantcentered intelligent solution that is able to prevent pregnancy risks through providing emotional support. Women with high risk pregnancies are more interested in an intelligent solution that could detect emotional problems than a woman with a low-risk pregnancy. An intelligent solution for pregnancy health, and wellbeing is deemed more trustworthy when the provider is a public healthcare system rather than a private company. Finally, the results found in this study provide us with the knowledge needed for the next phase of the project, in which we will validate the methodology by applying it in this case study of pregnancy using affective computing and wearable IoT. A.M. Oprescu et al. Information Fusion 83–84 (2022) 53–78 75 CRediT authorship contribution statement A.M. Oprescu: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Visualization, Writing – original draft. G. Mir´ o-Amarante: Formal analysis, Investigation, Writing – review & editing. L. García-Díaz: Formal analysis, Investigation, Writing – review & editing. V.E. Rey: Methodology, Formal analysis, Investigation, Writing – review & editing. A. Chimenea-Toscano: Formal analysis, Investigation, Writing – review & editing. R. Martínez-Mar- tínez: Formal analysis, Investigation, Writing – original draft. M.C. Romero-Ternero: Conceptualization, Formal analysis, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Visualization, Writing – original draft. Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Acknowledgment Authors would like to thank the participants in this study and Hospital Universitario Virgen del Rocío and Clinic Victoria Rey staff. This research did not receive yet any specific grants from funding agencies in the public, commercial, or not-for-profit sectors. Appendix Data protection Information clause addressed to interested parties in application of the principle of transparency regulated in the RGPD is available at htt ps://sic.us.es/sites/default/files/pd/ratproyectohera.pdf. Information about data treatment addressed to interested parties in application of the principle of transparency regulated in the RGPD is available at https://sic.us.es/sites/default/files/pd/cip.hera_.pdf. Appendix Structured interview Structured interview is available at http://grupos.us.es/pigma lion/HERAqualitativestudy. References [1] H. Yang, Y. Luo, X. Ren, M. Wu, X. He, B. Peng, K. Deng, D. Yan, H. Tang, H. Lin, Risk prediction of diabetes–Big data mining with fusion of multifarious physical examination indicators, Inf. Fus. 75 (2021) 140–149, https://doi.org/10.1016/j. inffus.2021.02.015. [2] J. Chaki, S.T. Ganesh, S.K. Cidham, S.A. Theertan, Machine learning and Artificial Intelligence based diabetes mellitus detection and self-management–A systematic review, J. King Saud Univ. (2020), https://doi.org/10.1016/j. jksuci.2020.06.013. [3] S. Ellahham, Artificial Intelligence–The future for diabetes, Am. J. Med. 133 (2020) 895–900, https://doi.org/10.1016/j. amjmed.2020.03.033. [4] I. Contreras, J. Vehi, Artificial Intelligence for diabetes management and decision support–Literature review, J. Med. Internet Res. 20 (2018) 1–21, https://doi.org/ 10.2196/10775. [5] K. Seetharam, Artificial Intelligence in cardiovascular medicine, Curr. Treat. Options Cardiovasc. Med. 21 (2019), https://doi.org/10.1007/s11936-019-0728- 1. [6] K.W. DeGregory, P. Kuiper, T. DeSilvio, J.D. Pleuss, R. Miller, J.W. Roginski, C. B. Fisher, D. Harness, S. Viswanath, S.B. Heymsfield, I. Dungan, D.M. Thomas, A review of machine learning in obesity, Obes. Rev. 19 (2018) 668–685, https:// doi.org/10.1111/obr.12667. [7] P. Jin, X. Ji, W. Kang, Y. Li, H. Liu, F. Ma, S. Ma, H. Hu, W. Li, Artificial Intelligence in gastric cancer–A systematic review, J. Cancer Res. Clin. Oncol. 146 (2020) 2339–2350, https://doi.org/10.1007/s00432-020-03304-9. [8] N. Houssami, G. Kirkpatrick-jones, N. Noguchi, I. Christoph, N. Houssami, G. Kirkpatrick-jones, N. Noguchi, Artificial Intelligence (AI) for the early detection of breast cancer–A scoping review to assess AIs potential in breast screening practice, Expert Rev. Med. Devices 16 (2019) 351–362, https://doi. org/10.1080/17434440.2019.1610387. [9] D.J.V. Booven, M. Kuchakulla, R. Pai, F.S. Frech, R. Ramasahayam, P. Reddy, M. Parmar, R. Ramasamy, H. Arora, A systematic review of Artificial Intelligence in prostate cancer, Res. Rep. Urol. 13 (2021) 31–39, https://doi.org/10.2147/ RRU.S268596. [10] A.M. Oprescu, G. Miro-Amarante, L. Garcia-Diaz, L.M. Beltran, V.E. Rey, M. Romero-Ternero, Artificial Intelligence in pregnancy–A scoping review, IEEE Access 8 (2020) 181450–181484, https://doi.org/10.1109/ access.2020.3028333. [11] R. Wolff, K. Moons, R. Riley, P. Whiting, M. Westwood, G. Collins, J. Reitsma, J. Kleijnen, S. Mallett, Probast–A tool to assess the risk of bias and applicability of prediction model studies, Ann. Intern. Med. (2019). https://doi.org/10.7326 /M18-1376. [12] G. Engel, The need for a new medical model–A challenge for biomedicine, Psychodyn. Psychiatry 40 (2012) 377–396, https://doi.org/10.1521/ pdps2012403377. [13] D. Novack, O. Cameron, E. Epel, R. Ader, S. Waldstein, S. Levenstein, Psychosomatic medicine–The scientific foundation of the biopsychosocial model, Acad. Psychiatry 31 (2007) 388–401, https://doi.org/10.1016/j. echu.2013.10.004. [14] Holzinger, A., Biemann, C., Pattichis, C.S., Kell, D.B., 2017. What do we need to build explainable AI systems for the medical domain? arXiv:1712.09923. [15] U. Pawar, S. Donna Rea, R. O’Reilly, Incorporating explainable Artificial Intelligence (xai) to aid understanding of machine learning in the healthcare domain. Irish Conference on Artificial Intelligence and Cognitive Science, Springer-Verlag, 2020, https://doi.org/10.13140/RG.2.2.24754.02246. [16] B. Goodman, S. Flaxman, European union regulations on algorithmic decisionmaking and a “right to explanation, AI Mag. 38 (2017) 50–57, https://doi.org/ 10.1609/aimag.v38i3.2741. [17] European Commission, 2020d. White Paper on Artificial Intelligence–A European approach to excellence and trust. URL: https://ec.europa.eu/info/publications/ white-paper-artificial-intelligence-european-approach-excellence-and-trust_en. (Accessed 20 November 2021). [18] European Commission, 2020c. Proposal for a regulation laying down harmonized rules on Artificial Intelligence. URL: https://eur-lex.europa.eu/legal-content/EN/ TXT/?uri=CELEX:52021PC0206. (Accessed 04 November 2020). [19] European Commission, 2019. Communication–Building trust in human centric Artificial Intelligence. URL: https://ec.europa.eu/digital-single-market/en/news/ communication-building-trust-human-centric-artificial-intelligence. (Accessed 20 November 2021). [20] European Commission, 2020a. Ethics guidelines for trustworthy AI. URL: htt ps://www.aepd.es/sites/default/files/2019-12/ai-ethics-guidelines.pdf. (Accessed 20 November 2021). [21] European Commission, Horizon 2020 programme - guidance–How to complete your ethics self-assessment. URL: https://ec.europa.eu/research/participants/ data/ref/h2020/grants_manual/hi/ethics/h2020_hi_ethics-self-assess_en.pdf. (Accessed 10 November 2020). [22] European Commission, 2020b. Horizon Europe strategic plan 2021-2024. URL: https://ec.europa.eu/commission/presscorner/detail/en/ip_21_1122, doi: 10.2777/083753. (Accessed 10 November 2020). [23] A. Biaggi, S. Conroy, S. Pawlby, C.M. Pariante, Identifying the women at risk of antenatal anxiety and depression–A systematic review, J. Affect. Disord. 191 (2016) 62–77, https://doi.org/10.1016/j.jad.2015.11.014. [24] B. Leeners, P. Neumaier-Wagner, S. Kuse, R. Stiller, W. Rath, Emotional stress and the risk to develop hypertensive diseases in pregnancy, Hypertens. Pregnancy 26 (2007) 211–226, https://doi.org/10.1080/10641950701274870. [25] G. Rejn¨ o, C. Lundholm, S. ¨ Oberg, P. Lichtenstein, H. Larsson, B. D’Onofrio, K. Larsson, S. Saltvedt, B.K. Brew, C. Almqvist, Maternal anxiety, depression and asthma and adverse pregnancy outcomes – A population based study, Sci. Rep. 9 (1) (2019). https://doi.org/10.1038/s41598-019-49508-z. [26] T. Kurki, Depression and anxiety in early pregnancy and risk for preeclampsia, Obstet. Gynecol. 95 (2000) 487–490, https://doi.org/10.1016/s0029-7844(99) 00602-x. [27] M.K. Thombre, N.M. Talge, C. Holzman, Association between pre-pregnancy depression/anxiety symptoms and hypertensive disorders of pregnancy, J. Women’s Health 24 (2015) 228–236, https://doi.org/10.1089/jwh.2014.4902. [28] M. Kordi, A. Vahed, F. Rezaee Talab, S.R. Mazloum, M. Lotfalizadeh, Anxiety during pregnancy and preeclampsia–A case - control study, J. Midwifery Reprod. Health 5 (1) (2017), https://doi.org/10.22038/jmrh.2016.7881. [29] T. Krishnamurti, A.L. Davis, H.N. Simhan, Worrying yourself sick? association between pre-eclampsia onset and health-related worry in pregnancy, Pregnancy Hypertens. 18 (2019) 55–57, https://doi.org/10.1016/j.preghy.2019.09.003. [30] A.L. Zietlow, N. Nonnenmacher, C. Reck, B. Ditzen, M. Müller, Emotional stress during pregnancy – associations with maternal anxiety disorders, infant cortisol reactivity, and mother–child interaction at pre-school age, Front. Psychol. 10 (2019), https://doi.org/10.3389/fpsyg.2019.02179. ISSN: 1664-1078. [31] S.K. Brooks, R.K. Webster, L.E. Smith, L. Woodland, S. Wessely, N. Greenberg, G. J. Rubin, The psychological impact of quarantine and how to reduce it–Rapid review of the evidence, Lancet N. Am. Ed. 395 (2020) 912–920, https://doi.org/ 10.1016/s0140-6736(20)30460-8. [32] Coronavirus (COVID-19) infection and pregnancy, 2020. Coronavirus (COVID-19) infection and pregnancy. URL: https://www.rcog.org.uk/coronavirus-pregnancy. [33] E.A. Holmes, R.C. O’Connor, V.H. Perry, I. Tracey, S. Wessely, L. Arseneault, C. Ballard, H. Christensen, R.C. Silver, I. Everall, T. Ford, A. John, T. Kabir, K. King, I. Madan, S. Michie, A.K. Przybylski, R. Shafran, A. Sweeney, C. M. Worthman, L. Yardley, K. Cowan, C. Cope, M. Hotopf, E. Bullmore, Multidisciplinary research priorities for the COVID-19 pandemic–A call for action for mental health science, Lancet Psychiatry 7 (2020) 547–560, https://doi.org/ 10.1016/s2215-0366(20)30168-1. A.M. Oprescu et al. Information Fusion 83–84 (2022) 53–78 76 [34] L. Poon, H. Yang, A. Kapur, N. Melamed, B. Dao, H. Divakar, H. Mcintyre, A. Kihara, D. Ayres-de Campos, E. Ferrazzi, G. Renzo, M. Hod, Global interim guidance on coronavirus disease 2019 (COVID-19) during pregnancy and puerperium from figo and allied partners–Information for healthcare professionals, Int. J. Gynaecol. Obstet. 149 (2020). https://doi.org/10.1002/ijg o.13156. [35] M. Ceulemans, T. Hompes, V. Foulon, Mental health status of pregnant and breastfeeding women during the COVID-19 pandemic–A call for action, Int. J. Gynecol. Obstet. 151 (2020) 146–147, https://doi.org/10.1002/ijgo.13295. [36] M.T. Frey, D. Meaney-Delman, V. Bowen, M.M. Yazdy, S.M. Watkins, P. G. Thorpe, M.A. Honein, Surveillance for emerging threats to pregnant women and infants, J. Women’s Health 28 (2019) 1031–1036, https://doi.org/10.1089/ jwh.2019.7943. [37] S.B. Thapa, A. Mainali, S.E. Schwank, G. Acharya, Maternal mental health in the time of the COVID-19 pandemic, Acta Obstet. Gynecol. Scand. 99 (2020) 817–818, https://doi.org/10.1111/aogs.13894. [38] M.H. Davenport, S. Meyer, V.L. Meah, M.C. Strynadka, R. Khurana, Moms are not OK–COVID-19 and maternal mental health, Front. Glob. Women’s Health 1 (2020), https://doi.org/10.3389/fgwh.2020.00001, 1. [39] Y. Wu, C. Zhang, H. Liu, C. Duan, C. Li, J. Fan, H. Li, L. Chen, H. Xu, X. Li, Y. Guo, Y. Wang, X. Li, J. Li, T. Zhang, Y. You, H. Li, S. Yang, X. Tao, Y. Xu, H. Lao, M. Wen, Y. Zhou, J. Wang, Y. Chen, D. Meng, J. Zhai, Y. Ye, Q. Zhong, X. Yang, D. Zhang, J. Zhang, X. Wu, W. Chen, C.L. Dennis, H. feng Huang, Perinatal depressive and anxiety symptoms of pregnant women during the coronavirus disease 2019 outbreak in China, Am. J. Obstet. Gynecol. 223 (2020) 240, https:// doi.org/10.1016/j.ajog.2020.05.009, e1–240.e9. [40] F. Jiang, Y. Jiang, H. Zhi, Y. Dong, H. Li, S. Ma, Y. Wang, Q. Dong, H. Shen, Y. Wang, Artificial intelligence in healthcare–Past, present and future, Stroke Vasc. Neurol. 2 (4) (2017) 230–243, https://doi.org/10.1136/svn-2017-000101, https://svn.bmj.com/content/2/4/230.full.pdf ISSN: 2059-8688. [41] E.J. Topol, High-performance medicine–The convergence of human and Artificial Intelligence, Nat. Med. 25 (2019) 44–56, https://doi.org/10.1038/s41591-018- 0300-7. [42] V. Dignum, Responsible Artificial Intelligence: Designing AI for human values, ITU Journal (1) (2017). http://dspace.daffodilvarsity.edu.bd:8080/bitstream/ha ndle/123456789/2181/itu2017-1.pdf?sequence=1&isAllowed=y. [43] International Organization for Standardization, 2018. Ergonomics of humansystem interaction — Part 11–Usability–Definitions and concepts. [44] R. Inostroza, C. Rusu, S. Roncagliolo, C. Jim´ enez, V. Rusu, Usability heuristics for touchscreen-based mobile devices, in: Proceedings of the 9th International Conference on Information Technology (ITNG ’12) 32, 2012, pp. 662–667. [45] R. Benbunan-Fich, Usability of wearables without affordances, in: 23rd Americas Conference on Information Systems, 2017, pp. 57–66. [46] J. Jones, C. Gouge, M. Crilley, Design principles for health wearables, Commun. Des. Q. Rev. 5 (2017), https://doi.org/10.1145/3131201.3131205. [47] N. Niknejad, W. Ismail, A. Mardani, H. Liao, I. Ghani, A comprehensive overview of smart wearables–The state of the art literature, recent advances, and future challenges, Eng. Appl. Artif. Intell. (90) (2020) 103529, https://doi.org/10.1016/ j.engappai.2020.103529. [48] M. Pavic, V. Klaas, G. Theile, J. Kraft, G. Tr¨ oster, M. Guckenberger, Feasibility and usability aspects of continuous remote monitoring of health status in palliative cancer patients using wearables, Oncology 98 (2020) 57–66. [49] S. Baumann, Evaluation of data usability generated by wearables & IoT-enabled home use medical devices via Telehealth to identify if blockchain can solve potential challenges, Iowa State University, 2020. https://dr.lib.iastate.edu/entiti es/publication/586375d1-f2e2-42a1-bf3f-e604a47e08b1. In press. [50] J. Oh, H. Kang, User engagement with smart wearables–Four defining factors and a process model, Mobile Media Commun. 98 (2020) 57–66, https://doi.org/ 10.1177/2050157920958440. [51] Z. Gu, J. Wei, Wearable services adoption study from a perspective of usability, in: G. Salvendy, J. We (Eds.), International Conference on Human-Computer Interaction 12216, Springer, 2020, pp. 16–22, https://doi.org/10.1007/978-3- 030-50350-5_2. [52] Nielsen, D., Shipon, D., 2020. Pilot evaluation of the usability and utility of a wearable technology to monitor and evaluate health condition. URL: https://jdc. jefferson.edu/si_dh_2022_phase1/8. [53] J. Khakurel, J. Porras, H. Melkas, B. Fu, A comprehensive framework of usability issues related to the wearable devices, in: S. Paiva, S. Paul (Eds.), Convergence of ICT and Smart Devices for Emerging Applications, Springer, 2020, pp. 21–26, https://doi.org/10.1007/978-3-030-41368-2_2. [54] M. Krey, Wearable device technology in healthcare—exploring constraining and enabling factors. Advances in Intelligent Systems and Computing, in: X.S. Yang, S. Sherrat, N. Dey, A. Joshi (Eds.), in: Fourth International Congress on Information and Communication Technology, 1041, Springer, Singapore, 2020, pp. 1–13. [55] H.R. Lim, H.S. Kim, R. Qazi, Y.T. Kwon, J.W. Jeong, W.H. Yeo, Advanced soft materials, sensor integrations, and applications of wearable flexible hybrid electronics in healthcare, energy, and environment, Adv. Mater. 32 (2020), 1901924, https://doi.org/10.1002/adma. 201901924. [56] N. Pathak, A. Mukherjee, S. Misra, Reconfigure and reuse–Interoperable wearables for healthcare iot, in: IEEE INFOCOM 2020- IEEE Conference on Computer Communications, 2020, pp. 20–29. [57] S. Khan, M. Alam, Wearable internet of things for personalized healthcare study of trends and latent research, arXiv preprint (2020) 21–26. https://arxiv.org/ftp/ arxiv/papers/2005/2005.06958.pdf. [58] L. Lu, J. Zhang, Y. Xie, F. Gao, S. Xu, X. Wu, Z. Ye, Wearable health devices in health care: Narrative systematic review, JMIR mHealth uHealth 8 (11) (2020), https://doi.org/10.2196/18907. https://pubmed.ncbi.nlm.nih.gov/33164904/. [59] Nielsen, J., 2020. 10 usability heuristics for user interface design. https://www. nngroup.com/articles/ten-usability-heuristics/. (Accessed 12 November 2020). [60] R. Y´ a˜ nez G´ omez, Ph.D. Thesis, Universidad de Sevilla, 2019. [61] J. Zhu, A. Liapis, S. Risi, R. Bidarra, G. Youngblood, Explainable ai for designers–A human-centered perspective on mixedinitiative co-creation, in: IEEE Conference on Computational Intelligence and Games (CIG), 2018, pp. 1–8, https://doi.org/10.1109/CIG.2018.8490433. [62] A. Barredo Arrieta, N. Díaz-Rodríguez, J. Del Ser, A. Bennetot, S. Tabik, A. Barbado, S. Garcia, S. Gil-Lopez, D. Molina, R. Benjamins, R. Chatila, F. Herrera, Explainable Artificial Intelligence (xai)–Concepts, taxonomies, opportunities and challenges toward responsible AI, Inf. Fus. 58 (2020) 82–115, https://doi.org/10.1016/j.inffus.2019.12.012. [63] Turek, M., 2016. Explainable Artificial Intelligence (XAI). URL: https://www. darpa.mil/program/explainable-artificial-intelligence. (Accessed 10 November 2020). [64] Cavoukian, A., 2012. Privacy by design origin and evolution. URL: https://www. ipc.on.ca/wp-content/uploads/resources/7foundationalprinciples.pdf. (Accessed 15 November 2021). [65] Cavoukian, A., 2010. Privacy by design–The 7 foundational principles–Implementation and mapping of fair information practices. [66] European Parliament, Regulation (EU) 2016/679 of the european parliament and of the council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing directive 95/46/EC (general data protection regulation). URL: https:// eur-lex.europa.eu/eli/reg/2016/679/oj. (Accessed on 12 November 2020). [67] Norwegian Data Protection Authority - Datatylsinet, Guide Software Development with Data Protection by Design and by Default. URL: https://www. datatilsynet.no/en/about-privacy/virksomhetenes-plikter/innebygd-personvern /data-protection-by-design-and-by-default/. (Accessed 12 November 2020). [68] R. Martínez Martínez, Intelig` encia artificial des del disseny. reptes i estrat` egies per al compliment normatiu, Rev. Catalana Dret Públic (2019) 64–81, https:// doi.org/10.2436/rcdp.i58.2019.3317. [69] G. Iachello, J. Hong, End-user privacy in human-computer interaction, Found. Trends Hum. Comput. Interact. (1) (2007) 1–137, https://doi.org/10.1561/ 1100000004. [70] L. Dang, M. Piran, D.H.K.M.H. Moon, A survey on internet of things and cloud computing for healthcare, Electronics 768 (7) (2019) 768, https://doi.org/ 10.3390/electronics8070768. [71] F.H. Semantha, S. Azam, K.C. Yeo, B. Shanmugam, A systematic literature review on privacy by design in the healthcare sector, Electronics (3) (2020) 452, https:// doi.org/10.3390/electronics9030452. [72] A. Appari, M.E. Johnson, Information security and privacy in healthcare–Current state of research, Int. J. Internet Enterp. Manag. 6 (2010) 279–314. [73] N. van Deursen, W.J. Buchanan, A. Duff, Monitoring information security risks within health care, Comput. Secur. 37 (2013) 31–45, https://doi.org/10.1016/j. cose.2013.04.005. [74] J. van den Hoven, M. Blaauw, W. Pieters, M. Warnier, Privacy and information technology, in: E.N. Zalta (Ed.), The Stanford Encyclopedia of Philosophy, Metaphysics Research Lab, Stanford University, 2020. https://plato.stanford. edu/archives/sum2020/entries/it-privacy/. [75] M. Gerber, R. von Solms, Management of risk in the information age, Comput. Secur. 24 (2005) 16–30, https://doi.org/10.1016/j.cose.2004.11.002. [76] E.D. Thompson, M.L. Kaarst-Brown, Sensitive information–A review and research agenda, J. Am. Soc. Inf. Sci. Technol. (3) (2005) 245–257, https://doi.org/ 10.1002/asi.20121. [77] P. Dourish, K. Anderson, Collective information practice: Exploring privacy and security as social and cultural phenomena, Hum. Comput. Interact. (2006) 319–342. http://www.douri.sh/publications/2006/DourishAnderson-InfoPra ctices-HCIJ.pdf. [78] G. Dhillon, J. Backhouse, Current directions in is security research–Towards socio-organizational perspectives, Inf. Syst. J. 11 (2001) 127–153. [79] M. Siponen, Information security standards focus on the existence of process, not its content, Commun. ACM 49 (2006) 97–100, https://doi.org/10.1145/ 1145287.1145316. [80] L. Coles-Kemp, Information security management–An entangled research challenge, Inf. Secur. Tech. Rep. 14 (2009) 181–185, https://doi.org/10.1016/j. istr.2010.04.005. [81] L. Pessoa, On the relationship between emotion and cognition, Nat. Rev. Neurosci. 9 (2008) 148–158, https://doi.org/10.1038/nrn2317. [82] J. Panksepp, Affective Neuroscience–The Foundations of Human and Animal Emotions. Affective Neuroscience – The Foundations of Human and Animal Emotions, Oxford University Press, New York, 1998. ISBN: 978-0195178050. [83] R. Picard, Affective Computing. Affective Computing, MIT Press, Cambridge, MA, USA, 1997. ISBN: 9780262661157. [84] A.R. Damasio, A Second Chance for Emotion. Series in Affective Science, Oxford University Press, New York, NY, US, 2000. ISBN 0-19-511888-X. [85] J.M. Latorre Postigo, Emoci´ on, Psicología M´ edica, Elsevier Health Sciences Spain, Spain, 2014. ISBN: 9788491136675. [86] T.M. Moerland, J. Broekens, C.M. Jonker, Emotion in reinforcement learning agents and robots–A survey, Mach. Learn. 107 (2018) 443–480, https://doi.org/ 10.1007/s10994-017-5666-0. A.M. Oprescu et al. Information Fusion 83–84 (2022) 53–78 77 [87] Q. Zhou, Multi-layer affective computing model based on emotional psychology, Electron. Commer. Res. 18 (2018) 109–124, https://doi.org/10.1007/s10660- 017-9265-8. [88] Z. Zheming, Ph.D. Thesis, Victoria University of Wellington, 2020. [89] J.R. Millenson, Principles of Behavioral Analysis. Principles of Behavioral Analysis, 2nd ed., Macmillan, New York, 1967. ISBN:0-02-381280-X. [90] P. Ekman, An argument for basic emotions, Cogn. Emotion 6 (1992) 169–200, https://doi.org/10.1080/02699939208411068. [91] C.E. Izard, Human Emotions, Springer, New York, NY, 1977. https://public.eb ookcentral.proquest.com/choice/publicfullrecord.aspx?p=5575935. [92] R. Plutchik, A general psychoevolutionary theory of emotion. Theories of Emotion, Elsevier, 1980, pp. 3–33. https://doi.org/10.1016/B978-0-12-558701- 3.50007-7. [93] B.L. Feldman. How Emotions are Made–The Secret Life of the Brain, Mariner Books, 2017. ISBN: 978-1509837526. [94] M. Huang, R. Rust, Artificial Intelligence in service, J. Serv. Res. 21 (2018) 155–172, https://doi.org/10.1177/1094670517752459. [95] D. Peters, N. Ahmadpour, R. Calvo, Tools for wellbeing-supportive design–Features, characteristics, and prototypes, Multimodal Technol. Interact. 4 (2020), https://doi.org/10.3390/mti4030040. [96] V. Cornet, R. Holden, Systematic review of smartphone-based passive sensing for health and wellbeing, J. Biomed. Inform. 77 (2018) 120–132, https://doi.org/ 10.1016/j.jbi.2017.12.008. [97] F. Thabtah, D. Peebles, J. Retzler, C. Hathurusingha, A review of dementia screening tools based on mobile application, Health Technol. 10 (2020) 1011–1022, https://doi.org/10.1007/s12553-020-00426-5. [98] M. Lankhorst, Enterprise Architecture at Work–Modelling, Communication and Analysis. Enterprise Architecture at Work–Modelling, Communication and Analysis, 4th ed., Springer, 2017. ISBN-13: 978-3642296505. [99] World Medical Association, World Medical Association Declaration of Helsinki ethical principles for medical research involving human subjects. doi:10.1001/ jama.2013.281053. (Accessed 10 November 2020). [100] A. Jobin, M. Ienca, E. Vayena, Artificial Intelligence–The global landscape of ethics guidelines, Nat. Mach. Intell. 1 (2019) 389–399. [101] D. Ryu, et al., Comprehensive pregnancy monitoring with a network of wireless, soft, and flexible sensors in high- and low-resource health settings, Proc. Natl. Acad. Sci. 118 (2021), https://doi.org/10.1073/pnas.2100466118. [102] J. Runkle, M. Sugg, D. Boase, S. Galvin, C. Coulson, Use of wearable sensors for pregnancy health and environmental monitoring–Descriptive findings from the perspective of patients and providers, Digit. Health 5 (2019), https://doi.org/ 10.1177/2055207619828220. [103] R.C. King, E. Villeneuve, R.J. White, R.S. Sherratt, W. Holderbaum, W.S. Harwin, Application of data fusion techniques and technologies for wearable health monitoring, Med. Eng. Phys. 42 (2017) 1–12, https://doi.org/10.1016/j. medengphy.2016.12.011. [104] R. Bokade, A. Navato, R. Ouyang, X. Jin, C.A. Chou, S. Ostadabbas, A.V. Mueller, A cross-disciplinary comparison of multimodal data fusion approaches and applications–Accelerating learning through trans-disciplinary information sharing, Expert Syst. Appl. 165 (2021), https://doi.org/10.1016/j. eswa.2020.113885. In press. [105] Kalamkar, S., Mary, G.A., 2020. Clinical data fusion and machine learning techniques for smart healthcare. 2020 International Conference on Industry 4.0 Technology, I4Tech 2020, 211–216doi:10.1109/I4Tech48345.2020.9102706. [106] E. Malacova, S. Tippaya, H.D. Bailey, K. Chai, B.M. Farrant, A.T. Gebremedhin, H. Leonard, M.L. Marinovich, N. Nassar, A. Phatak, C. Raynes-Greenow, A. K. Regan, A.W. Shand, C.C. Shepherd, R. Srinivasjois, G.A. Tessema, G. Pereira, Stillbirth risk prediction using machine learning for a large cohort of births from Western Australia, 1980–2015, Sci. Rep. 10 (2020) 1–8, https://doi.org/10.1038/ s41598-020-62210-9. [107] R. Chu, W. Chen, G. Song, S. Yao, L. Xie, L. Song, Y. Zhang, L. Chen, X. Zhang, Y. Ma, X. Luo, Y. Liu, P. Sun, S. Zhang, Y. Fang, T. Dong, Q. Zhang, J. Peng, L. Zhang, Y. Wei, W. Zhang, X. Su, X. Qiao, K. Song, X. Yang, B. Kong, Predicting the risk of adverse events in pregnant women with congenital heart disease, J. Am. Heart Assoc. 9 (2020), https://doi.org/10.1161/JAHA.120.016371. [108] A. Sheikhtaheri, M.R. Zarkesh, R. Moradi, F. Kermani, Prediction of neonatal deaths in NICUs–Development and validation of machine learning models, BMC Med. Inf. Decis. Making 21 (2021) 1–14, https://doi.org/10.1186/s12911-021- 01497-8. [109] A. Koivu, M. Sairanen, Predicting risk of stillbirth and preterm pregnancies with machine learning, Health Inf. Sci. Syst. 8 (2020) 1–12, https://doi.org/10.1007/ s13755-020-00105-9. [110] P.M. Iftikhar, M.V. Kuijpers, A. Khayyat, A. Iftikhar, M. DeGouvia De Sa, Artificial Intelligence–A new paradigm in obstetrics and gynecology research and clinical practice, Cureus 12 (2) (2020), https://doi.org/10.7759/cureus.7124. ISSN: 2168-8184. [111] L. Davidson, M.R. Boland, Towards deep phenotyping pregnancy–A systematic review on Artificial Intelligence and machine learning methods to improve pregnancy outcomes, Brief. Bioinf. 22 (2021) 1–29, https://doi.org/10.1093/bib/ bbaa369. [112] A. Joerin, M. Rauws, M.L. Ackerman, Psychological Artificial Intelligence service, Tess–Delivering on-demand support to patients and their caregivers–Technical report, Cureus 11 (2019) 1–7, https://doi.org/10.7759/cureus.3972. [113] C. Falala-Sechet, L. Antoine, I. Thiriez, C. Bungener, Owlie–A chatbot that provides emotional support for coping with psychological difficulties, in: IVA 2019 - Proceedings of the 19th ACM International Conference on Intelligent Virtual Agents, 2019, pp. 236–237, https://doi.org/10.1145/3308532.3329416. [114] Nielsen, R., Gurzawska, A., Brey, P., Principles and approaches in ethics assessment. ethical impact assessment and conventional impact assessment. deliverable 1.1. annex 1.a. in–Satori. ethical assessment of research and innovation–A comparative analysis of practices and institutions in the eu and selected other countries. URL: https://satoriproject.eu/media/1.a-Ethical-impact -assessmt-CIA.pdf. (Accessed 10 November 2020). [115] High-Level Expert Group on Artificial Intelligence, 2020. Assessment list for trustworthy Artificial Intelligence (ALTAI) for self-assessment. URL: https ://digital-strategy.ec.europa.eu/en/library/assessment-list-trustworthy-artificial -intelligence-altai-self-assessment. (Accessed 02 November 2020). [116] Open Group, 2018. The togaf standard, version 9.2 overview. URL: https://www. opengroup.org/togaf. (Accessed 05 November 2020). [117] Open Group, The Archimate Enterprise Architecture Modeling Language. The Archimate Enterprise Architecture Modeling Language, Open Group, 2019. http s://www.opengroup.org/archimate-forum/archimate-overview (Accessed 05 November 2020). [118] K. Kelley, B. Clark, V. Brown, J. Sitzia, Good practice in the conduct and reporting of survey research, Int. J. Qual. Health Care 15 (2003) 261–266, https://doi.org/ 10.1093/intqhc/mzg031. [119] Spanish LOPD 3/ 2018, Ley org´ anica 3/2018, de 5 de diciembre, de protecci´ on de datos personales y garantía de los derechos digitales. https://www.boe.es/bus car/pdf/2018/BOE-A-2018-16673-consolidado.pdf. (Accessed on 12 December 2020). [120] D. Tundrea. Artificial Intelligence is Getting Personal : A study on the Usage Motivations and Privacy Concerns of Intelligent Personal Assistants’ Users, Uppsala University, Media and Communication Studies, 2017. http://www.div a-portal.org/smash/record.jsf?pid=diva2%3A1161295&dswid=-6764. URN: urn: nbn:se:uu:diva-334967. [121] Y. Sun, J. Liu, J. Wang, Y. Cao, N. Kato, When machine learning meets privacy in 6g–A survey, IEEE Commun. Surv. Tutor. 22 (2020) 2694–2724, https://doi.org/ 10.1109/COMST.2020.3011561. [122] B. Brecko, A. Ferrari, The Digital Competence Framework for Consumers, EU Science Hub - European Commission, 2019. URL https://ec.europa.eu/jrc/en/ digcompconsumers. In press (Accessed 13 November 2020). [123] G.S. Carretero, V. Riina, P. Yves, DigComp 2.1–The Digital Competence Framework for Citizens with Eight Proficiency Levels and Examples of Use, Publications Office of the European Union, Luxembourg, 2017. https://data. europa.eu/doi/10.2760/38842. Andreea M. Oprescu obtained her bachelor’s and master’s degrees in Computer Science Engineering at the University of Seville (Spain). She started out her research career when she obtained a research collaboration grant, during which she acquired knowledge on research and review methodologies, as well as on Artificial Intelligence and affective computing techniques, especially for health and well-being. From 2018 to 2020, she worked as a Research and Development assistant at the Library of the University of Seville. She is currently pursuing her PhD studies in Computer Science at University of Seville, where she is continuing her work on responsible and explainable Artificial Intelligence and affective computing techniques applied specifically to pregnancy health and wellbeing. As from 2020, she is a collaborating professor in the Department of Electronic Technology at the University of Seville and she is part of the ICT150 Electronic Technology and Industrial Informatics group. G. Mir´ o-Amarante received a B.S. degree in Physics from the University of Seville, Spain, in 1994, and the Ph.D. degree in Physics from the Complutense University of Madrid, Spain, in 2000. From 2006, she is Associate Professor at the University of Seville and member of the ICT150 research group: Electronic Technology and Industrial Informatics. Her research activity begins at the National Institute of Aerospace Technique, Huelva, Spain, in 1995, and continues at the International Center for Theoretical Physics, Trieste, Italy, in 2002, with a Marie Curie Fellowship of the European Community. She is the author of more than 30 scientific articles and book chapters. Her research interests include data and signal processing, analysis and validation of communications and environment models. She has been member of several international scientific committees (IRI Task Force Activity group, Committee Meeting of COST 251), a reviewer of several scientific journals, and National Award of the Academy of Sciences of Cuba to the result of scientific research. A.M. Oprescu et al.