Flow-Volume Curves for Effective and Interpretable Artificial Intelligence in Respiratory Medicine

Thomas T. Kok; John Morales; Christophe Smeets; David Ruttens; Kristyna Sirka Kacafırkova; Dolores Blanco-Almazan; An Jacobs; Vojkan Mihajlovic; Femke Ongenae; Sofie Van Hoecke

doi:10.5281/zenodo.17435304

Flow-Volume Curves for Effective and Interpretable Artificial Intelligence in Respiratory Medicine

Thomas T. Kok; John Morales; Christophe Smeets; David Ruttens; Kristyna Sirka Kacafırkova; Dolores Blanco-Almazan; An Jacobs; Vojkan Mihajlovic; Femke Ongenae; Sofie Van Hoecke

Publisher: Zenodo
DOI: 10.5281/zenodo.17435304
Repository: zenodo.org
Language: en

Abstract

This study evaluates whether the use of flow-volume data can improve classification performance and interpretability in Artificial Intelligence (AI) models for respiratory medicine, and examines the impact of these explanations on model understanding. We assessed the classification performance of AI models trained on flow-volume and respiratory airflow data, using a case study of estimating breathing difficulty for COPD patients. Additionally, we evaluated model performance with varying dataset sizes, and conducted user tests with physicians to assess the impact of including explanations on the classification performance when receiving decision support from the AI model. The results showed that models trained on flow-volume data outperformed those trained on respiratory airflow data when the dataset was sufficiently large, with a minimum required number of 30 to 35 patients for estimating ease of breathing. Providing explanations alongside data and model predictions improved the classification performance of physicians in user tests, most notably with the explanations from flow-volume data, despite subjective evaluations rating the explanations below average in usefulness. In conclusion, flow-volume data can offer benefits for classification, conditioned by the size of the available dataset. Although there is relevant information present in the explanations that improves physician performance, further efforts are needed to win physicians' trust. Overall, the results highlight the potential of flow-volume data in respiratory AI applications.

Full text

Flow-Volume Curves for Effective and Interpretable Artificial Intelligence in Respiratory Medicine IDLAB –ELIS/INTEC Thomas T. Kok, John Morales, Christophe Smeets, David Ruttens, Kristyna Sirka Kacafírková, Dolores Blanco-Almazán, An Jacobs, Vojkan Mihajlović, Femke Ongenae, Sofie Van Hoecke Motivation Respiratory diseases, such as COPD and asthma, are a leading cause of mortality worldwide. Research into machine learning for respiratory diagnostics has shown promising results. In clinical settings, physicians primarily use flow-volume curves for diagnosis, instead of the respiratory flow signals used to train machine learning models. Can these flow-volume curves improve model performance and interpretability for clinicians? Contact [email protected] https://predict.idlab.ugent.be/ Case study: Ease of breathing estimation for COPD patients Dataset: 76 COPD patients with 6 different inspiratory loads applied (0% to 60%) as proxy for ease of breathing. Input: A baseline and an observation respiratory flow signal, for the same patient. Classification target: Whether the load applied to the observation signal was higher (+1), lower (-1), or the same (0). We designed an approach that uses differential flow-volume images as input for a pre-trained vision transformer model. We compared our approach to a baseline approach that uses the original respiratory flow signals as input. Our approach outperforms the baseline approach when enough data is available (around 30 to 35 patients). Can flow-volume curves improve model performance? Can flow-volume curves improve physicians’ understanding? We surveyed medical professionals using saliency-based explanations from both models. Participants were asked to classify each case and then rate the quality of the corresponding explanation. They were also asked for subjective evaluation Predictive performance accuracy of medical professionals consistently improved with the inclusion of explanations. This suggests that the explanations improved their understanding of the problem. Although the explanations improved accuracy, the results of the subjective evaluation were varied, indicating that further research is needed to improve trust.