Vision-Based Framework for Vessel–Instrument Distance Estimation in Robotic Prostatectomy

Vision-Based Framework for Vessel–Instrument Distance Estimation in Robotic Prostatectomy 1st Eleonora Pollini Department of Electronics, Information and Bioengineering Politecnico di Milano Milan, Italy [email protected] 2nd Laura Cruciani Department of Electronics, Information and Bioengineering Politecnico di Milano Milan, Italy [email protected] 3rd Elena De Momi Department of Electronics, Information and Bioengineering Politecnico di Milano Milan, Italy [email protected] Abstract—Radical prostatectomy is the gold standard treatment for localized prostate cancer, where surgical success depends on preserving vascular and nervous structures while ensuring oncological radicality. Quantifying the interactions between surgical instruments and vessels could provide surgeons with objective indicators of intraoperative safety. We present a vision-based framework for vessel–instrument distance estimation in robot-assisted radical prostatectomy, combining stereo 3D reconstruction, vessel and instrument segmentation, and Euclidean distance computation. Evaluation reported a Mean Absolute Error (MAE) of 2.83 ± 2.22 mm for 3D reconstruction, a mean Dice of 0.83 ± 0.07 for vessel segmentation, and 0.70 ± 0.18 for instrument segmentation. The framework achieved an average processing rate of 4 FPS. These results demonstrate the feasibility of quantifying vessel–instrument proximity during robotic surgery, providing a proof-of-concept toward quantitative safety metrics. Current limitations include sub–real-time performance and the lack of quantitative ground truth for distance validation, which will be addressed in future work. Index Terms—segmentation, 3D reconstruction, vesselinstrument distance estimation, surgical safety I. INTRODUCTION Radical prostatectomy is the standard treatment for localized prostate cancer. Despite the diffusion of robotic platforms, the procedure remains technically demanding, as success depends on the surgeon’s ability to preserve vascular and nerve structures. Intraoperative bleeding and vessel injury remain critical risks, directly related to the interaction between surgical instruments and vascular branches. Quantifying vessel–instrument proximity would provide an objective indicator of surgical safety, with applications in novice surgeons training, risk assessment and real-time assistance. Similar approaches exist in retinal surgery [1] and minimally invasive collision avoidance [2], but none in the Da Vinci Surgical System that we use. We propose a preliminary vision-based framework for vessel–instrument distance estimation from stereo-endoscopic videos of robot-assisted radical prostatectomy, including 3D reconstruction, segmentation of vessels and instruments, and Euclidean distance computation. This work was supported by Intuitive Clinical Research Grants for Roboticassisted Surgery in 2024-2025. The images are provided by the Department of Urology of European Institute of Oncology (IEO), IRCCS, Milan, Italy II. MATERIALS AND METHODS Figure 1 shows the proposed pipeline for vessel-instrument distance estimation. Stereo endoscopic data from the robotic platform are processed for segmentation and 3D reconstruction, yielding dense intraoperative point clouds. Stereo camera calibration and image pre-processing ensure epipolar alignment and distortion removal. Dense disparity maps are generated using the Hierarchical Stereo Matching Network (HSMNet) [3] and triangulated into 3D points. Concurrently, segmentation highlights the common iliac artery and surgical instruments. Since no dataset contains annotated masks of both, a Feature Pyramid Network (FPN) [4] was trained separately: vessel segmentation used a private dataset of 16,099 pelvic lymph node dissection frames (100 epochs, batch size 8, ResNet50 backbone), while instrument segmentation used the public EndoVis Instrumentation Dataset [5] (525 frames from four colorectal surgeries) with identical hyperparameters. Frames were resized to 256×160 pixels, augmented for generalization, and trained with a combined Dice and Binary Cross-Entropy loss. From the reconstructed point clouds, the Euclidean distance between instruments and vessels was computed to assess procedural safety. Instrument point clouds were refined via downsampling and clustering, and distances estimated using KDTree nearest-neighbor search. The 5th percentile of closest distances was then used as a robust measure, minimizing outlier influence. The SERV-CT dataset [6] with Mean Absolute Error (MAE), and average inference time per frame was used for 3D reconstruction evaluation: MAE =1 |N|X (u,v) |d(u, v)−d′(u, v)|.(1) Nis the number of predicted pixels, d(u, v)the predicted disparity, and d′(u, v)the ground truth. Segmentation performance was assessed using dice coefficient (2|A∩B| |A|+|B|), IoU (|A∩B| |A∪B|), precision (T P TP+F P ), recall (T P T P +F N ), and average inference time. Here, |A|and |B|represent activated pixels in predicted and ground truth masks. The segmentation model and the stereo disparity model were 2025 I-RIM Conference October 17-19, Rome, Italy ISBN: 9788894580570 10.5281/zenodo.17629758 129 Fig. 1. Overview of the proposed pipeline for vessel–instrument distance estimation. Intraoperative stereo images are processed via segmentation and stereo depth estimation to reconstruct dense point clouds of anatomy and instruments. The minimum vessel–instrument distance is computed as the 5th percentile of Euclidean distances and visualized in the surgical scene. Fig. 2. Qualitative evaluation of instrument–vessel distance (instrument: blue, vessel: red). (A) Direct contact yields a distance of 0.08 mm. (B) Retraction increases the distance to 7.52 mm. tested on an NVIDIA GeForce 4060 Ti GPU (16 GB VRAM) with an Intel i9 CPU. III. RESULTS 3D reconstruction achieved MAE 2.83 ±2.22 mm with inference time 29.34 ±3.00 ms, showing a good balance between accuracy and speed. For vessel segmentation, results were: precision 0.93 ±0.06, recall 0.75 ±0.10, dice 0.83 ±0.07, IoU 0.71 ±0.09, and inference time 2.45±2.31 ms, meeting real-time requirements. Instrument segmentation achieved precision 0.62±0.23, recall 0.87 ±0.07, dice 0.70 ±0.18, IoU 0.57 ±0.20, and inference time 10.14 ±0.17 ms. Figure 2 shows qualitative distance estimation results for two frames. In the absence of ground truth point clouds, quantitative evaluation was not possible. When instrument touches the vessel, the Euclidean distance approaches zero (0.08 mm) (A), while, when retracting, the distance increases (B), confirming consistency between spatial separation and distance estimation. The average computation time was 188.42 ±112.98 ms. The average processing time per frame was 0.24±0.05 s (≃4 FPS). IV. CONCLUSION We presented an autonomous framework for visualizing distances between surgical instruments and target vessels during pelvic lymph node dissection. The system is accurate and efficient, achieving 4 FPS and showing potential to enhance surgical safety by reducing operator variability. Future work includes developing a standardized dataset for joint instrument–vessel segmentation, collecting ground truths to validate our method using a sensor-based framework, and further reducing computation time to enable real-time distance estimation. Preoperative CT and PET could be leveraged to localize lymph nodes, and their integration with intraoperative data via registration would enable direct measurement of instrument–lymph node distances, beyond surrounding vessels. REFERENCES [1] M. Sommersperger, J. Weiss, M. Ali Nasseri, P. Gehlbach, I. Iordachita, and N. Navab, “Real-time tool to layer distance estimation for robotic subretinal injection using intraoperative 4d oct,” Biomedical Optics Express, vol. 12, no. 2, pp. 1085–1104, 2021. [2] L. Li, X. Li, B. Ouyang, H. Mo, H. Ren, and S. Yang, “Three-dimensional collision avoidance method for robot-assisted minimally invasive surgery,” Cyborg and Bionic Systems, vol. 4, p. 0042, 2023. [3] X. Cheng, Y. Zhong, M. Harandi, Y. Dai, X. Chang, H. Li, T. Drummond, and Z. Ge, “Hierarchical neural architecture search for deep stereo matching,” Advances in neural information processing systems, vol. 33, pp. 22 158–22 169, 2020. [4] A. Kirillov, K. He, R. Girshick, and P. Doll´ ar, “A unified architecture for instance and semantic segmentation (2017),” SF Banu et al./WEU-Net: A Weight Excitation U-Net for Lung Nodule Segmentation, vol. 356, 2017. [5] “Endovis 2015 surgical instrument segmentation and tracking,” 2015. [Online]. Available: https://endovissub-instrument.grand-challenge.org [6] P. Eddie”Edwards, D. Psychogyios, S. Speidel, L. Maier-Hein, and D. Stoyanov, “Serv-ct: A disparity dataset from ct for validation of endoscopic 3d reconstruction,” arXiv e-prints, pp. arXiv–2012, 2020. 130