Ethical issues arising from the attitudes of Swedish radiologists and women towards the use of AI in mammography

1 Ethical issues arising from the attitudes of Swedish radiologists and women towards the use of AI in mammography Michele Farisco Centre for Research Ethics and Bioethics, Uppsala University, Uppsala, Sweden Bioethics Unit, Biogem, Biology and Molecular Genetics Research Institute, Ariano Irpino (AV), Italy. orcid.org/0000-0002-3298-7829 e-mail: michele.fa[email protected] Abstract The debate about the use of AI in medicine is very lively among different stakeholders, including academics, developers, patients associations, regulators, and general public, among others. This debate is fueled by the development of a number of AI technologies aimed at different medical applications, including diagnosis, prognosis, management of patients, and healthcare provision. This paper starts from previous research on the perception of the use of AI in mammography by Swedish radiologists and women. Against this background, the paper aims to identify the ethical issues that emerge from these two populations, with the final goal of elaborating a list of points and a map of ethical priorities that may possibly inform the discussion on the use of AI in medicine, including ethical and regulatory debates. Introduction The use of AI in medicine is a multifaceted issue, raising increasing interest from different perspectives, including regulatory, ethical, political, and sociological perspectives (Azer & Guerrero, 2023; Beam Andrew et al., 2023; Farhud & Zokaei, 2021; Lee, Bubeck, & Petro, 2023). The number of applications relying on or informed by AI are already impressive and its expansion very likely. The form of AI that is increasingly impacting medical practice is machine-learning (ML) technology, especially Deep-Learning (DL) applications. The range of potential medical applications of such technologies is wide: from automatic transcription of dictated medical advice to direct elaboration of medical notes starting from patients’ interviews and laboratory test results; from informing decisions about health insurance coverage to assisting in interpreting medical images and other patient-derived data; from analyzing and interpreting large research databases to assisting the diagnostic and prognostic processes (Beam Andrew et al., 2023). More specifically, medical applications of AI include the selection of patients for supplemental cancer screening (Liu et al., 2024; Salim et al., 2024), assisting the cancer diagnosis trough the interpretation of medical images (Yoon et al., 2023), using Large Language Models (LLM) to automatically generate medical documentation (Sánchez-Rosenberg et al., 2024), identifying and classifying orthopedic issues from plain radiographs (Akbarian et al., 2024; Magnéli et al., 2024), predicting the outcome of medical interventions, including cognitive behavioral therapy (Hentati Isacsson, Ben Abdesslem, Forsell, Boman, & Kaldo, 2024). The impact of AI on medicine is so significant that the number of relevant papers published is increasingly growing, and dedicated papers launched (e.g., the New England Medical Journal AI, https://ai.nejm.org/). 2 In order to maximize the positive impact of AI on medical practice, along with the technological optimization, it is key to reflect on the ethical issues arising from the use of AI in medicine. Such issues include equity, fairness, explainability, and generalizability of AI models, and more generally the implications of AI for personalized and precision medicine (e.g., assuring equal opportunities to get access to AI-based medical applications) (Shandhi & Dunn, 2022). Against this background, the present paper aims at contributing to this ethical reflection through the clarification of the conditions for an effective ethical analysis of the impact of AI on medicine. Since the development and the use of AI are very sensitive issues (because AI catalyzes several interests, expectations, hypes, concerns, and worries by several different stakeholders), a balanced ethical analysis requires two fundamental conditions: a clear method for identifying the issues arising from AI, including their different kinds and relative impact; an empirically-informed approach, which consists in starting from actual opinions of relevant stakeholders to eventually elaborate an ethical reflection on their basis. In fact, if not informed by empirical data revealing the state-of-the-art of medical practice, including the priorities and most urgent needs to address, ethical reflection risks to be detached from reality and eventually not effective (i.e., not really impacting ordinary practice). In what follows I describe a method that has been recently introduced for the identification of ethical issues arising from AI; then I summarize the results from two empirical studies on the opinions of radiologists and patients about the use of AI in radiology; finally, I identify the ethical issues emerging from these two groups of stakeholders through the application of the abovementioned method. While the elaboration of specific recommendations for handling the emerging issues is beyond the scope of this paper, its final goal is contributing to clarifying and mapping the priorities that ethical reflection and regulation should focus on. A method for the ethical analysis of AI There is a wide variety of approaches to AI ethics, and also the topics covered are very different (Coeckelbergh, 2020; Dignum, 2018). To illustrate, taking brain-inspired or neuro-AI as a case study, a recent paper introduced a two-tier method conceived to be heuristic in nature (i.e., to provide a framework that in principle is applicable also to other forms of AI in order to identify the ethical issues arising from them) (Farisco et al., 2024). This method relies on a preliminary distinction between fundamental/foundational and practical/applied ethical issues: both relevant values and principles and key notions like moral subject, moral reasoning, and moral action, among others, including their implications for decision making, are key to justify moral choices. Accordingly, two main kinds of ethical issues arise from AI: fundamental/foundational (i.e., concerning both the justification of AI and its impact on how we think about key moral notions) and practical/applied (i.e., concerning the impact and implications of AI on different sectors of our daily life). The first kind of issues involves theoretical analysis, while the second kind of issues involves mainly applied analysis, that is, the use of ethical theory to identify and address the practical issues related to the use of AI. More specifically, (Farisco et al., 2024) propose the practical/applied issues arising from AI to be organized in terms of (at least) the following main levels: - Operational, related to how AI works. For instance, because of limited training data, some AI applications may process personal data in a biased way, eventually providing not reliable information. - Instrumental, related to how people use AI. For instance, people may use AI tools for complementing and/or facilitating their decision-making process, rather than delegating the decision to AI altogether. 3 - Relational, related to how people see AI and to the resulting psychological and metaphysical human-AI relationship. For instance, especially, but not exclusively lay people may overrate the reliability of AI systems, inappropriately trusting their results. - Societal, related to the social and economic costs and consequences of the development and use of AI. For instance, AI may challenge the present job system, making some professional figures redundant or raising the need for re-allocating some workers in other fields. Also, (Farisco et al., 2024) identify two main categories of fundamental/foundational ethical issues: - Those related to goals, which refer to questions like: What is the driver of AI? What do we want to achieve by it? For instance, AI may be inspired by the goal to make human job easier and more productive, or rather by the goal of making more money through the commercialization of AI tools. - Those related to concepts, which include issues like implicit assumptions and biases about AI, and considerations about the historical, cultural, and societal contexts of AI. For instance, we may see AI just as a tool that complements human experience and competence, or rather as a reliable source of information that can replace human activity altogether. Results from radiologists and women As mentioned above, for maximizing the effectiveness and impact of the ethical reflection on AI, avoiding resulting a too abstract and disconnected from reality principles, it is imperative to start from the actual experience and opinions of involved stakeholders. Accordingly, I here summarize the results from two recently conducted studies, which involved readiologists and women during a clinical trial on the use of AI in mammographic radiology (ScreenTrustCAD, NCT04778670) at Capio S:t Görans Hospital, Sweden, between 2021 and 2023 (J. Viberg Johansson, Dembrower, Strand, & Grauman, 2024; Jennifer Viberg Johansson & Engström, 2024). (Jennifer Viberg Johansson & Engström, 2024) is an exploration with professionals, including 7 semistructured interviews, whose transcripts were analyzed using inductive thematic content analysis. Emerging results were grouped into three main categories: AI in society, AI and human interaction, and AI as a tool. As reported in Table 1, these are the main responses from radiologists about potential positive effects, uncertainties and ambivalences, and potential negative effects of using AI as a diagnostic tool in radiology. Among the potential positive effects, they report: - AI improves well-being (of both patients and professionals, e.g., creating a more attractive workplace, reducing queues, mitigating the workload) - AI increases efficiency (e.g., better diagnosis, multifactorial risk models, tailored approach) - AI reduces costs (e.g., one rather than two radiologists needed) - AI fits well in work routine - AI gives the radiologists an added reason to trust their own judgment (i.e., a sense of security) Among uncertainties/ambivalences they report: - The effect that AI will have on the relationship between professionals and patients - Disclose or not disclose the use of AI? Risk of “overinforming” vs need to be transparent - Distribution of responsibility when AI is used as a form of decision support 4 - Difficulty in interpreting/understanding AI results may be stressful or exciting Among potential negative effects they report: - If used by a professional alone, AI raises a bigger sense of responsibility for possible mistakes - AI may decrease the competence of radiologists if these are deprived of analyzing all the cases, including the obvious cancers - AI might generate false-positives, with more work in the second review and possible disproportionate worries by patients and excessive costs. The study with women included 16 semi-structured interviews, whose transcripts were analyzed using inductive thematic content analysis. As reported in table 1, these are the main responses from women about potential positive effects, uncertainties and ambivalences, and potential negative effects of using AI as a diagnostic tool in radiology. Among the potential positive effects, they report: - AI may improve the screening process in several ways - AI may be used as an independent reader of the mammograms to reduce workload for the breast radiologists as well as triaging patients in first-line care Among uncertainties/ambivalences they report: - Limited understanding of the underlying mechanisms of AI (i.e., how it works) - Prevailing skepticism regarding the current capabilities of AI (i.e., what it can actually do) - Raised concerns about the sustainability of AI’s effectiveness in the long run - Uncertainty regarding AI’s ability to detect all cancers (e.g., it may have difficulty with certain more uncommon conditions) - Uncertainty regarding the specific areas in which AI excels beyond the capabilities of the human brain. Among potential negative effects they report: - AI lacks the holistic perspective that humans have (e.g., thinking about consequences, conducting investigative work, and demonstrating greater imagination), as well as qualities like intuition, empathy, and contextual understanding - Using AI too much risks to lead to the decline of learning skill development for the radiologist. In addition to the above, the following ethically relevant points emerge from the two studies. The radiologists share a willingness to be involved in further development of AI diagnostic applications in radiology. Interviewed women request a thorough evaluation of AI diagnostic tools, transparency about when and how AI is used, the involvement of radiologists in the assessment of AI diagnostics tools, an effective communication regarding the role and limitations of AI. Also, from the interviewed women it emerges the need for evaluating the AI’s cancer detection performance compared to radiologists and the overall integration process. Participants also emphasize the necessity of control and monitoring to prevent AI from learning incorrect behaviors, recognizing the constant evolution of AI. 5 Significantly, women participating to the study declare to prefer increased worry, due to being recalled more often, over missed cancer cases. This point connects to the risk of generating false-positives and disproportionate worries by patients highlighted by radiologists. Furthermore, women declare to have less tolerance towards errors if made by AI rather than by humans. This point may be connected to the perceived increased sense of responsibility reported by some radiologists when using AI. Still about responsibility, some women believed that if AI were to make a mistake, it would be the responsibility of humans, as AI is viewed as a tool incapable of being held accountable for decisions. This point may be related to the distribution of responsibility expressed by radiologists. Yet overall, the opinions regarding the issue of responsibility vary among participants. Finally, women consistently declare their willingness to share their health data. Emerging ethical issues Tables 2 and 3 reports the results emerging from the application of the method for the ethical analysis of AI described above to both radiologists’ and women’s responses. Below is an analysis following the same order as previously described: first a focus on practical/applied ethical issues, then a focus on fundamental/foundational ethical issues. The text reports what the participants declared about the ethical issues arising from the use of AI in radiology. Therefore, I remain neutral regarding the reliability and the generalizability of the participants’ perceptions. Both points would require further analyses, including a comparison with previous studies and reviews. Nevertheless, these declared perceptions are relevant in themselves, because they are indicative of the attitudes of relevant stakeholders, which should inform an empirically-grounded ethical assessment. Practical/applied ethical issues At the operational level (i.e., with reference to how AI works), results from the studies above indicate two specific issues: limited understanding of how AI works and uncertainty regarding the specific areas in which AI excels. These operational points may have ethical implications, for instance because the radiologists may eventually get a wrong conclusion relying on results from AI whose limited reliability they are not able to recognize. Or radiologists may fail to realize for which specific application AI may be an excellent tool to use. At the instrumental level (i.e., with reference to how people use AI), results from the studies above indicate the following issues. Both radiologists and women agree in considering AI as complementary of human competence rather than a replacement of human professionals. Along this line, participants to the study think that AI may increase efficiency of medical practice, for instance making it possible for radiologists to get rid of too repetitive (e.g., procedural) tasks and to focus on most urgent medical needs instead. According to the interviewed people, this kind of use of AI as complementing/integrating human work is facilitated by the fact that AI fits well in work routine. On a negative side, participants tend to agree that AI may generate disproportionate worries by patients as well as excessive costs related to how it works, and it may also cause the decline of learning skill development for the radiologists that may eventually delegate some tasks and competencies to AI. The relational level (i.e., with reference to how people perceive AI and their relationship with it) is the one where the most ethical issues arise according to the participants. As already mentioned analyzing 6 the instrumental level, participants tend to agree that humans cannot be replaced by AI, which may rather support self-trust in medical staff. This, for instance, may derive from the fact that AI lacks the holistic perspective that humans typically have (e.g., thinking about consequences, conducting investigative work, and demonstrating greater imagination), as well as qualities like intuition, empathy, and contextual understanding. Along a similar line of thought, participants report skepticism towards current capabilities of AI as well as uncertainty regarding the actual AI diagnostic capability. There is also a shared sense of uncertainty about the need for disclosing the use of AI, with a conflict between the need for avoiding over-informing and the need for transparency. The topic of responsibility emerges as particularly controversial, with a shared sense of uncertainty about the distribution of responsibility when AI is used as decision support. Some respondents think that AI may raise a bigger sense of responsibility in professionals for possible mistakes. Others say that they would personally have less tolerance towards errors if made by AI rather than by humans, while others think that if AI were to make a mistake, it would still be the responsibility of humans, as AI is eventually viewed only as a tool. Still at the relational level participants think that radiologists should be involved in the development of AI, with a significant convergence among professionals and women. In fact, there is a convergence towards the need to evaluate the AI’s cancer detection performance compared to radiologists and the overall integration process, as well as the necessity of control and monitoring to prevent AI from learning incorrect behaviors, recognizing the never-ending evolution of AI. Finally, participants invoke an effective communication regarding the role and limitations of AI in cancer diagnosis. At the societal level (i.e., with reference to the impact of AI on society) the following issues emerge. As mentioned in relation to the instrumental level, AI may improve the well-being of professionals as well as of patients, for instance reducing the workload and helping in triaging patients in the first-line care. Also, AI may reduce costs with a possible positive impact on the quality of healthcare and the possibility to have access to it. On a negative note, participants report uncertainty about the impact of AI on the relationship between professionals and patients and concerns about the sustainability of AI’s effectiveness in the long run. Finally, the difficulty in interpreting/understanding AI, already outlined at the operational level, may be stressful or exciting, and, as reported at the operational level, AI may decrease the competence of radiologists with a negative impact on their professional relationship with patients. Fundamental/foundational ethical issues With reference to goals, there is a convergence among participants towards considering the improvement of healthcare provision as a fundamental goal of using AI in radiology. This improvement may happen at different levels, including financials, working condition, quality of diagnosis, and better patients´ management. Other reported goals for the use of AI in radiology is increasing the medical doctors´ self-confidence, involving professionals and patients in the assessment and development of AI, and improving and/or streamlining decision-making process in healthcare. With reference to concepts, participants tend to evaluate the impact of the diagnostic use of AI in relation to the following fundamental ethically salient concepts: transparency, understanding, uncertainty, human-technology relationship, clinicians-patients relationship, right to be informed, responsibility, trust vs skepticism, competence, holistic perspective, humanness, and human irreplaceability, supervision, control and monitoring of AI, privacy and confidentiality. Conclusion 7 The application of the two-tier ethical method described above resulted in the identification of some practical/applied and fundamental/foundational ethical issues. Among other things, radiologists and women interviewed significantly agree that AI is a potential useful tool, but as such it cannot replace human professionals. In fact, participants report that there are a number of potential positive effects deriving from the use of AI in radiology. In addition to these potential positive effects, participants report also a number of potential negative effects. Among these, a sense of uncertainty emerges as particularly salient. People declare to be uncertain about different aspects, including how AI works, how much it can actually improve medical diagnostics, what are its consequences on the relationship between doctors and patients, and how it impacts the attribution of responsibility in medical practice Both potential negative and positive effects of medical use of AI, as well as the uncertainty about it, deserve a specific analysis in the ethical and regulatory debates to eventually maximize the positive impact of AI on healthcare, which both radiologists and women interviewed indicate as the final goal of the diagnostic use of AI. Finally, also the fundamental ethical concepts highlighted above (e.g., transparency, understanding, uncertainty, and human irreplaceability) may serve as a useful reference for ethical and regulatory reflections on AI. 8 Table 1. Radiologists’ and women’s opinions about potential positive, negative and uncertain effects of the use of AI in radiology Shared view AI is an excellent complemental decision tool, but cannot replace medical doctors Potential positive effects of AI according to radiologists - improve well-being (of both patients and professionals, e.g. creating a more attractive workplace, reducing queues, mitigating the workload) - increase efficiency (e.g., better diagnosis, multifactorial risk models, tailored approach) - reduce costs (e.g., one rather than two radiologists needed) - AI fits well in work routine - AI gave the radiologists an added reason, a sense of security, to trust their own judgment women - There are several potential ways in which artificial intelligence software (AI) could improve the screening process - It can also be used as an independent reader of the mammograms to reduce workload for the breast radiologists as well as triaging patients in first-line care - Uncertainties/ambivalences on AI according to radiologists - the effect that AI will have on the relationship between professionals and patients - Disclose or not disclose the use of AI? Risk of “overinforming” vs need to be transparent - Distribution of responsibility when AI is used as a form of 9 decision support - Difficulty in interpreting/understanding AI results may be stressful or exciting women - limited understanding of the underlying mechanisms of AI - prevailing skepticism regarding the current capabilities of AI was evident among many participants - raised concerns about the sustainability of AI’s effectiveness in the long run - Some participants expressed uncertainty regarding AI’s ability to detect all cancers, and that they might have difficulty with certain more uncommon conditions - uncertainty regarding the specific areas in which AI excels beyond the capabilities of the human brain. Potential negative effects of AI according to radiologists - If used by a professional alone, AI raises a bigger sense of responsibility for possible mistakes - AI may decrease the competence of radiologists if these are deprived of analyzing all the cases, including the obvious cancers. - AI might generate falsepositive, with more work in the second review and possible disproportionate worries by patients and excessive costs. women - AI lacks the holistic perspective that humans have (e.g., thinking about consequences, conducting 16 screening in a clinical setting. Health Informatics Journal, 30(3), 14604582241275020. doi:10.1177/14604582241275020 Yoon, J. H., Strand, F., Baltzer, P. A. T., Conant, E. F., Gilbert, F. J., Lehman, C. D., . . . Mann, R. M. (2023). Standalone AI for Breast Cancer Detection at Screening Digital Mammography and Digital Breast Tomosynthesis: A Systematic Review and Meta-Analysis. Radiology, 307(5), e222639. doi:10.1148/radiol.222639