Artificial intelligence to determine correct midsagittal plane in dynamic transperineal ultrasound

Depósito de investigación de la Universidad de Sevilla https://idus.us.es/ Esta es la versión aceptada del artículo publicado en: Journal of Clinical Ultrasound (2025) This is a accepted manuscript of a paper published in: 25 April 2025 DOI: 10.1002/jcu.24050 Copyright: © 2025 Wiley Periodicals LLC. El acceso a la versión publicada del artículo puede requerir la suscripción de la revista. Access to the published version may require subscription. “This is the peer reviewed version of the following article: 2025. Journal of Clinical Ultrasound , Martín-Morán M, Borrero-Gonzalez C, Fernández-Gomez A, FernándezPalacín F, Sainz-Bueno JA. Artificial Intelligence to Determine Correct Midsagittal Plane in Dynamic Transperineal Ultrasound. J Clin Ultrasound. https://doi.org/10.1002/jcu.24050 which has been published in final form at 10.1002/jcu.24050. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Use of Self-Archived Versions. This article may not be enhanced, enriched or otherwise transformed into a derivative work, without express permission from Wiley or by statutory rights under applicable legislation. Copyright notices must not be removed, obscured or modified. The article must be linked to Wiley’s version of record on Wiley Online Library and any embedding, framing or otherwise making available the article or pages thereof by third parties from platforms, services and websites other than Wiley Online Library must be prohibited." 1 Artificial intelligence to determine correct midsagittal plane in 1 dynamic transperineal ultrasound 2 3 Article type: Original Research 4 5 Authors: 6 José Antonio García-Mejido1, Juan Galán-Paez2, David Solis-Martín2, Marina Martín-7 Morán1, Carlota Borrero-Gonzalez1, Alfonso Fernández-Gomez1, Fernando Fernández-8 Palacín3 , José Antonio Sainz-Bueno1 9 10 Afiliation: 11 1Department of surgery, Faculty of Medicine, University of Seville, Spain. 12 2Department of computer science and artificial intelligence, Faculty of Mathematics, 13 University of Seville, Spain. 14 3 Department of statistics and operational research, University of Cadiz, Cadiz, Spain. 15 16 Corresponding author: 17 José Antonio García Mejido., Department of Obstetrics and Gynecology Valme 18 University Hospital, Seville, Spain. Department of surgery, Faculty of Medicine, 19 University of Seville, Spain. E-mail: [email protected]. Phone: +34955015385 20 21 Data availability statement: Data available on request from the authors. 22 23 Funding statement: None. 24 25 Conflict of interest disclosure： The authors have no conflicts of interest to disclose 26 Ethics of approval statement：The study was approved by Andalucia’s Board of 27 Biomedicine Ethics Committee, with code SICEIA-2024-001928. The study was 28 2 conducted in accordance with the Declaration of Helsinki. All patients gave their written 29 informed consent before starting the study. 30 31 Permission to reproduce material from other sources: None. 32 33 Clinical trial registration： None. 34 35 Word count: 3236 words including abstract but excluding references, tables, and 36 figures. 37 38 39 3 Artificial intelligence to determine correct midsagittal plane in 40 dynamic transperineal ultrasound 41 42 Abstract: 43 Purpose:To create and validate a machine learning(ML) model that allows identifying 44 the correct capture of the midsagittal plane in a dynamic ultrasound study, as well as 45 establishing its concordance with a senior explorer and a junior explorer. 46 47 Methods:Observational and prospective study with 90 patients without pelvic floor 48 pathology. Each patient was given an ultrasound video where the midsagittal plane of the 49 pelvic floor was recorded at rest and during the Valsalva maneuver.A segmentation model 50 was used that was trained on a previously published article, generating the segmentations 51 of the 90 new videos to create the model.The algorithm selected to build the model in this 52 project was XGBoost(Gradient Boosting).To obtain a tabular dataset on which to train 53 the model feature engineering was carried out on the raw segmentation data.The 54 concordance of the model, of a junior examiner and a senior examiner,with the expert 55 examiner was studied using the kappa index. 56 57 Results:The first 60 videos were used to train the model and the last 30 videos were 58 reserved for the test set.The model presented a kappa index 0.930(p<0.001) with very 59 good agreement for detection of the correct midsagittal plane. The junior explorer 60 presented a very good agreement (kappa index=0.930(p<0.001)).The senior explorer 61 presented a kappa index 0.789(p<0.001)(good agreement) for detection of the correct 62 midsagittal plane. 63 64 Conclusion: We have developed a model that allows determining the correct midsagittal 65 plane captured through dynamic transperineal ultrasound with a level of agreement 66 comparable to or greater than that of a junior or senior examiner, using expert examiner 67 assessment as the gold standard. 68 69 4 Keywords: Machine learning, pelvic floor, ultrasonography, gradient boosting, 70 XGBoost, artificial intelligence, levator ani muscle. 71 72 5 Introduction. 73 74 Pelvic floor ultrasound has represented a breakthrough in the study and diagnosis of 75 pelvic floor dysfunctions. One of the characteristics of transperineal pelvic floor 76 ultrasound is that it is standardized from the midsagittal plane [1] to two-dimensional 77 ultrasound. From the midsagittal plane we can simultaneously study the pubic symphysis, 78 the urethra, the urinary bladder, the vagina, the uterus, the anal canal, the rectum and the 79 levator ani muscle [1] and therefore most of the pathology that covers each compartment. 80 Correct capture of the midsagittal plane requires learning that sometimes depends on the 81 skills of the examiner. In addition, transperineal ultrasound is associated with manual 82 measurements, which involve a time consumption in the consultation and depend on the 83 experience of the examiner, which will directly influence the variations in the score [2]. 84 To these aspects we must add that pelvic floor ultrasound allows a dynamic study of the 85 different structures, making its study more complicated, especially in the case of pelvic 86 organ prolapse. In fact, it has been described that in order not to lose the midsagittal plane 87 during the Valsalva maneuver, the rotational movement of the transducer must be 88 avoided, preserving the original alignment of the pelvic floor in the ultrasound image [3]. 89 90 On the other hand, the development of artificial intelligence (AI) in the field of 91 urogynecology is progressively expanding, showing its usefulness in identifying different 92 structures of the pelvic floor [4-7]. AI allows a computer program to perform reasoning 93 processes similar to the human brain, with deep learning (DL) being a subcategory that 94 can recognize medical images, classify them, and detect objects [8]. Currently, CNN 95 (convolutional neural network) has been used in pelvic floor ultrasound for analysis of 96 the levator ani muscle [5, 8-10], for measurements of the levator hiatus [5,6,11,12], 97 measurement of the urogenital hiatus [13], the assessment of urodynamic stress 98 incontinence with ultrasound [4] and for the study of pelvic organ prolapse [14]. 99 However, all these studies are based on static images, and to obtain the ultrasound 100 diagnosis of the different pelvic floor dysfunctions we need a dynamic ultrasound study 101 [3]. Recently, it has been described that it is possible to apply deep learning to identify 102 the different pelvic floor organs in a dynamic ultrasound study in the correctly captured 103 midsagittal plane [15]. However, sometimes it is possible that the midsagittal plane is not 104 well defined, either due to the condition of the patient's tissues or due to the lack of 105 6 training of the examiner who performs the technique. These defects in the capture of the 106 image can lead to diagnostic errors and interpretation of the ultrasound. Based on these 107 aspects, we consider that AI can be of great help if it can help us define the correct 108 midsagittal plane for the ultrasound study of the pelvic floor. Therefore, our objective is 109 to create and validate a predictive model that allows us to identify the correct midsagittal 110 plane in a dynamic ultrasound study, as well as establish its concordance with a senior 111 examiner and a junior examiner. 112 113 114 Materials 115 116 An observational and prospective study was conducted with 90 patients. The included 117 patients had no pelvic floor pathology and were recruited consecutively in the general 118 gynecology clinic from May 1, 2024 to June 31, 2024. Patients with difficulty performing 119 the Valsalva maneuver or a history of pelvic floor dysfunction were excluded. The 120 following clinical parameters were collected for each patient: age, weight, height, body 121 mass index (BMI), parity, menopausal status, age at menopause. 122 123 The study was approved by Andalucia’s Board of Biomedicine Ethics Committee, with 124 code SICEIA-2024-001928. The study was conducted in accordance with the Declaration 125 of Helsinki. All patients gave their written informed consent before starting the study. 126 127 Ultrasound examination 128 All transperineal ultrasounds were performed by the same expert pelvic floor ultrasound 129 examiner using a Canon i700 Aplio® (Canon Medical Systems Corp., Tokyo, Japan) 130 ultrasound with a PVT-675MV 3D abdominal probe. Images were acquired following 131 guidelines previously established in the literature, with patients in dorsal lithotomy 132 position with hips flexed [1]. Prior to capturing and storing the video, the patient was 133 trained to correctly perform the Valsalva maneuver. Each patient was given an ultrasound 134 video capture showing the midsagittal plane of the pelvic floor at rest and the Valsalva 135 maneuver. The ultrasound videos were oriented by placing the cranioventral region on 136 the left and the dorsocaudal region on the right. The expert examiner captured 45 videos 137 7 of the correct midsagittal plane and 45 videos of the incorrect midsagittal plane. A correct 138 midsagittal plane was defined as one that included the view of the pubic symphysis, 139 urethra, bladder, vagina, uterus, anus, rectum and levator ani muscle (figure 1). An 140 incorrect midsagittal plane was defined when any of the previously described anatomical 141 structures was missing, either due to a displacement of the probe in the anteroposterior 142 axis (figure 2) or a rotation of the image (figure 3). 143 144 Algorithm 145 146 The aim of this work is to build a model a model that determines whether the ultrasound 147 was performed correctly for each specific organ. In order to create such model, the first 148 60 videos were used as training set, whereas the last 30 cases were reserved as test set. 149 150 For this project, a segmentation model trained on a previously published paper [15], that 151 aims to identify organ positions on images (frames) extracted from ultrasound videos, 152 was used to generate segmentations of the 90 new videos. In this stage, the video frames 153 are extracted and sent one by one to the mentioned segmentation model which will 154 generate a segmentation per image (figure 4). 155 156 From the segmentations, a set of features or statistics was extracted for each frame and 157 each segmented organ to capture the relative confidence in the identification of each 158 organ. The extracted features include the average confidence level of the full 159 segmentation (prediction mean), confidence variation of the full segmentation (prediction 160 deviation), maximum confidence level of the full segmentation (maximum prediction 161 value), and minimum confidence level of the full segmentation (minimum prediction 162 value). 163 164 Additionally, for each organ in each frame, these same features were calculated, but only 165 considering prediction values greater than 0.5. This approach allows capturing the 166 absolute confidence of the model in the identification of the organs, compared to the 167 general confidence in each frame. 168 169 8 If the values obtained for the total predictions are significantly lower than those calculated 170 for predictions greater than 0.5, it indicates that the confidence assigned to the pixels that 171 do not belong to the organ is very low, reflecting a high confidence of the model in the 172 identification of the organ. Conversely, if the values between both groups are similar, it 173 means that high levels of confidence have been assigned to pixels outside the organ, 174 indicating lower certainty of the model in the segmentation. 175 176 For each segmentation of each organ, its bounding box was calculated, i.e., the box that 177 contains the predicted segmentation. For each bounding box, the centroid, the maximum 178 and minimum coordinates on each axis, the width, height, and area were calculated. The 179 rationale for these features was that the model could determine whether the location, size, 180 and aspect ratio were consistent for that organ. 181 182 To build a tabular dataset in which each row represents a patient and an organ, labeled as 183 "correct" or "incorrect," it was necessary to aggregate the features calculated at the frame 184 level to the patient or ultrasound level, i.e., at the frame sequence level (figure 5). This 185 aggregation was performed by calculating the mean, standard deviation, maximum value, 186 and minimum value of the features obtained over a sequence of N frames. 187 188 As a result, a dataset was generated in which each row represents an organ and a sequence 189 of N frames, with a total of 68 features per row. This dataset was used to train a model 190 that determines whether the ultrasound was performed correctly for each specific organ. 191 N = 60 was set, resulting in a total of 1,816 rows for the 90 cases analyzed. After dividing 192 the set into training and test subsets, 1,184 rows were obtained for the training set and 193 632 for the test set. Figure 6 shows the distribution of the samples in both subsets. 194 195 The value of N = 60 was chosen instead of using the complete data sequence of each 196 ultrasound as a "data augmentation" technique, useful when there is little data available. 197 If only one row per ultrasound had been generated, the dataset would have 720 rows, 198 given that there are 90 patients and 8 organs per patient. However, by using windows of 199 N frames, multiple rows were generated per patient and organ, which increased the 200 dataset's size to 1,816 rows. 201 202 15 pelvic levator hiatus segmentation from ultrasound images. Eur J Radiol Open. 2022; 399 24(9):100412. doi: 10.1016/j.ejro.2022.100412. 400 12) Li X, Hong Y, Kong D, Zhang X. Automatic segmentation of levator hiatus from 401 ultrasound images using U-net with dense connections. Phys. Med. Biol. 2019, 64, 402 075015. 403 13) Noort FvD, Manzini C, van der Vaart CH, van Limbeek MAJ, Slump CH, Grob ATM. 404 Automatic identification and segmentation of slice of minimal hiatal dimensions in 405 transperineal ultrasound volumes. Ultrasound Obstet. Gynecol. 2022, 60, 570–576. 406 14) Duan L, Wang Y, Li J, Zhou N. Exploring the clinical diagnostic value of pelvic floor 407 ultrasound images. The Journal of Supercomputing. 2021, 77: 10699–720.38. 408 15) García-Mejido JA, Solis-Martín D, Martín-Morán M, Fernández-Conde C, 409 Fernández-Palacín F, Sainz-Bueno JA. Applicability of Deep Learning to Dynamically 410 Identify the Different Organs of the Pelvic Floor in the Midsagittal Plane. Int Urogynecol 411 J. 2024; 24. doi: 10.1007/s00192-024-05841-0. 412 16) Chen, T. & Guestrin, C., 2016. XGBoost: A Scalable Tree Boosting System. In 413 Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge 414 Discovery and Data Mining. KDD '16. New York, NY, USA: ACM, pp. 785–794. 415 Available at: http://doi.acm.org/10.1145/2939672.2939785. 416 17) Gradient Boosting: Friedman, J.H., 2001. Greedy function approximation: a gradient 417 boosting machine. Annals of statistics, pp.1189–1232. 418 18) Szentimrey Z, Ameri G, Hong CX, Cheung RYK, Ukwatta E, Eltahawi A. Automated 419 segmentation and measurement of the female pelvic floor from the mid-sagittal plane of 420 3D ultrasound volumes. Med Phys. 2023;50(10):6215-6227. 421 19) García-Mejido JA, Fernández-Palacín A, Bonomi-Barby MJ, De la Fuente Vaquero 422 P, Iglesias E, Sainz JA. Online learning for 3D/4D transperineal ultrasound of the pelvic 423 floor. J Matern Fetal Neonatal Med. 2020;33(16):2805-2811. 424 20) Muta M, Takahashi T, Tamai N, Suzuki M, Kawamoto A, Sanada H, Nakagami G. 425 Pelvic floor muscle contraction automatic evaluation algorithm for pelvic floor muscle 426 training biofeedback using self-performed ultrasound. BMC Womens Health. 427 2024;4;24(1):219. doi: 10.1186/s12905-024-03041-y.. 428 21) Ni D, Ji X, Gao Y, Cheng JZ, Wang H, Qin J, Lei B, Wang T, Wu G, Shen D. 429 Automatic Cystocele Severity Grading in Ultrasound by Spatio-Temporal Regression. 430 16 Med Image Comput Comput Assist Interv. 2016;9901:247-255. 431 22) Eisenberg VH, Chantarasorn V, Shek KL, Dietz HP. Does levator ani injury affect 432 cystocele type? Ultrasound Obstet Gynecol 2010;36(5):618-23. 433 23) Green TH Jr. Urinary stress incontinence: differential diagnosis, pathophysiology, 434 and management. Am J Obstet Gynecol 1975;122(3):368-400. 435 24) Chantarasorn V, Dietz HP. Diagnosis of cystocele type by clinical examination and 436 pelvic floor ultrasound. Ultrasound Obstet Gynecol 2012; 39 (6): 710-4. 437 25) García-Mejido JA, Ramos-Vega Z, Armijo-Sánchez A, Fernández-Palacín A, García-438 Jimenez R, Sainz JA. Differential diagnosis of middle compartment pelvic organ prolapse 439 with transperineal ultrasound. Int Urogynecol J 2021;32(8):2219-25. 440 26) García-Mejido JA, Ramos Vega Z, Armijo Sánchez A, Fernández-Palacín A, 441 Fernández CB, Sainz Bueno JA. Interobserver variability of ultrasound measurements for 442 the differential diagnosis of uterine prolapse and cervical elongation without uterine 443 prolapse. Int. Urogynecol. J. 2022;33(10):2825-31. 444 27) Dietz HP, Steensma AB. Posterior compartment prolapse on two-dimensional and 445 threedimensional pelvic floor ultrasound: the distinction between true rectocele, perineal 446 hypermotility and enterocele. Ultrasound Obstet Gynecol 2005; 26 (1): 73-7. 447 28) García-Mejido JA, Ramos-Vega Z, Fernández-Palacín A, Borrero C, Valdivia M, 448 Pelayo-Delgado I, Sainz-Bueno JA. Predictive Model for the Diagnosis of Uterine 449 Prolapse Based on Transperineal Ultrasound . Tomography. 2022;8(4):1716-25. 450 29) García-Mejido JA, Martín-Martinez A, González-Diaz E, Núñez-Matas MJ, 451 Fernández-Palacín A, Carballo-Rastrilla S, Fernández-Fernández C, García-Jimenez R, 452 Sainz-Bueno JA. Is It Possible to Diagnose Surgical Uterine Prolapse With Transperineal 453 Ultrasound? Multicenter Validation of Diagnostic Software. J Ultrasound Med. 2023; 8. 454 doi: 10.1002/jum.16303. 455 30) García Mejido JA, Ortega I, García-Jiménez R, Sainz-Bueno JA. Differential 456 diagnosis of posterior compartment prolapse using transperineal ultrasound Ultrasound 457 Obstet Gynecol. 2022;60(1):142-144. 458 459 17 Table 1: Characteristics of the patients included in the test set. 460 461 ! n: 30! 95% CI Age! 46.1±13.9! 40.9; 51.3! Weight! 68.9±10.1! 65.1; 72.6! Height! 161.5±5.9! 159.3; 163.7! BMI! 26.3±4.3! 24.6; 27.9! Parity! 1.5±1.1! 1.1; 1.9! Menopause! 9 (31.0%)! 17.1%; 49.4%! Menopause age! 53.7±1.5! 52.5: 54.8! 462 18 Table 2: Kappa index assessment of the predictive model for the correct identification 463 of the different pelvic floor organs using the expert examiner's examination as the gold 464 standard. 465 466 Predictive model (n:30) P value (McNemar) Kappa (p) Correct midsagittal plane 19 (63.3%) 1.0 0.930 (<0.001) Correct visualization of the pubis 27(90.0%) 1.0 1.0 (<0.001) Correct visualization of the urethra 23(76.7%) 1.0 1.0(<0.001) Correct visualization of the urinary bladder 29(96.7%) 1.0 1.0(<0.001) Correct visualization of the vagina 30(100%) --- --- (---) Correct visualization of the uterus 27(90.0%) 1.0 0.783(<0.001) Correct visualization of the anus 25(83.3%) 1.0 0.760(<0.001) Correct visualization of the rectum 26(86.7%) 1.0 0.516(<0.001) Correct visualization of the levator ani muscle 26(86.7%) 1.0 0.870(<0.001) 467 19 Table 3: Kappa index assessment of the junior examiner for the correct identification 468 of the different pelvic floor organs using the expert examiner's examination as the gold 469 standard. 470 471 Junior examiner (n:30) P value (McNemar) Kappa (p) Correct midsagittal plane 19(63.3%) 1.0 0.930(<0.001) Correct visualization of the pubis 30(100%) --- --- (---) Correct visualization of the urethra 24(80.0%) 1.0 0.902(<0.001) Correct visualization of the urinary bladder 28(93.3%) 1.0 -0.047(0.786) Correct visualization of the vagina 30(100%) --- --- (---) Correct visualization of the uterus 29(96.7%) 1.0 0.651 (<0.001) Correct visualization of the anus 26(86.7%) 1.0 0.609(<0.001) Correct visualization of the rectum 24(80.0%) 0.250 0.615(<0.001) Correct visualization of the levator ani muscle 24(80.0%) 1.0 0.667(<0.001) 472 20 Table 4: Kappa index assessment of the senior examiner for the correct identification 473 of the different pelvic floor organs using the expert examiner's examination as the gold 474 standard. 475 476 senior examiner (n:30) P value (McNemar) Kappa (p) Correct midsagittal plane 19(63.3%) 1.0 0.789(<0.001) Correct visualization of the pubis 28(93.3%) 1.0 0.348(0.051 ) Correct visualization of the urethra 24(80.0%) 1.0 0.902(<0.001) Correct visualization of the urinary bladder 30(100%) --- --- (---) Correct visualization of the vagina 30(100%) --- --- (---) Correct visualization of the uterus 24(80.0%) 0.125 0.444(0.003) Correct visualization of the anus 26(86.7%) 1.0 0.609(<0.001) Correct visualization of the rectum 25(83.3%) 0.625 0.429(0.014) Correct visualization of the levator ani muscle 24(80.0%) 1.0 0.667(<0.001) 477 478 21 Figure 1: Shows the correct midsagittal plane. Pubis (P), urethra (U), urinary bladder 479 (UB), vagina (V), uterus (U), anus (AN), rectum (R), levator ani muscle (L). 480 481 Figure 2: Incorrect midsagittal plane where the anus, rectum and levator ani muscle 482 are not visualized due to displacement in the sagittal axis. Pubis (P), urethra (U), 483 urinary bladder (UB), vagina (V), uterus (U). 484 485 22 Figure 3: Incorrect midsagittal plane where the uterus, anus, rectum and levator ani 486 muscle are not visualized due to a rotation of the image (C). Pubis (P), urethra (U), 487 urinary bladder (UB), vagina (V). 488 489 Figure 4: Extraction of frames from the video and sending to the segmentation model 490 that will generate segmentation by image. 491 492 23 Figure 5: Aggregation of computed features at the frame sequence level. 493 494 Figure 6: Train set distribution (A) Test set distriubution (B). 495 496 Figure 7: Hyperparameter optimization through grouped 5-fold cross-validation 497 498