Investigating Adversarial Image Attacks in a Sensor Fusion Framework Using a Scaled Autonomous Vehicle Testbed

Preprints, 2025 IFAC Modeling, Estimation and Control Conference October 5–8, 2025. Pittsburgh, PA, USA Investigating Adversarial Image Attacks in a Sensor Fusion Framework Using a Scaled Autonomous Vehicle Testbed Tahmid Hasan Sakib ∗Wesam Al Amiri ∗Abhijeet Solanki ∗ Syed Rafay Hasan ∗Terry N. Guo ∗∗ Syed Ali Asad Rizvi ∗ ∗Electrical and Computer Engineering, Tennessee Technological University, Cookeville, TN 38505, USA (e-mail: {tsakib42, walamiri, asolanki42, shasan, srizvi}@tntech.edu) ∗∗ Center for Manufacturing Research, Tennessee Technological University, Cookeville, TN 38505, USA (e-mail: [email protected]) Abstract: Sensor fusion plays a crucial role in ensuring robust perception and decision-making in autonomous vehicles (AVs). This work presents a lightweight sensor fusion framework that integrates 2D LiDAR with YOLOv5-based object detection to enable real-time, way-pointindependent navigation. The system is experimentally validated on a scaled autonomous vehicle platform, demonstrating consistent stop sign detection and reliable stopping behavior across different experimental runs, even under slight variations in detection distance and environmental conditions. Furthermore, the framework is evaluated under adversarial image attacks using perturbed stop signs, where it maintains functionality with minor reductions in detection range and stopping precision. The presented results are validated on the Quanser QCar platform, which confirm the effectiveness of sensor fusion for real-time AV navigation while highlighting key vulnerabilities to adversarial perturbations. Keywords: Autonomous vehicles, sensor fusion, object detection, LiDAR, YOLOv5, adversarial robustness, embedded perception 1. INTRODUCTION Autonomous vehicles (AVs) rely on a robust perception system to interpret their surroundings and make safe navigation decisions. A common approach involves sensor fusion between cameras and LiDAR, where visual data provides semantic understanding, and LiDAR contributes geometric precision for spatial localization Liu et al. (2022); Rappaport et al. (2015). While high-end AV platforms often deploy 3D LiDAR with heavy compute infrastructure, lightweight 2D LiDAR paired with realtime object detection offers a more accessible solution for embedded systems and academic testbeds Chen et al. (2019); Mounabhargav et al. (2024). Recent work has shown that vision-based models are vulnerable to physical adversarial attacks, where small perturbations to signs can cause missed detections and unsafe behavior Eykholt et al. (2018); Chen et al. (2018); Lu et al. (2017). However, most such evaluations are conducted in simulation environments, where sensor behavior may not fully reflect real-world imperfections, latency, or noise Dosovitskiy et al. (2017). To address these gaps, we develop a real-time sensor fusion testbed on the Quanser QCar platform that integrates 2D LiDAR and YOLOv5-based object detection to perform stop sign detection and control without relying on maps or way-points. The system runs entirely on a resourceconstrained Jetson TX2 and employs multithreaded synchronization across camera, LiDAR, and control modules. We evaluate the architecture and performance of this onestop navigation routine under both clean and physically perturbed stop signs. Results confirm that even in the presence of adversarial image-based attacks, sensor fusion enables reliable stopping behavior with only minor degradation in detection consistency. 2. METHODOLOGY The proposed testbed combines 2D LiDAR with YOLOv5based object detection in a multithreaded framework to enable real-time navigation on the Quanser QCar platform. The system architecture includes three parallel threads: one for object detection using the front CSI camera, one for processing LiDAR scans, and one for controlling steering and throttle commands based on fused sensor input. Figure 1 illustrates this architecture, highlighting how raw sensor data flows through the perception and control modules. This design supports modular debugging and allows future integration of additional sensors or behaviors. Each thread operates asynchronously, with global variables shared between them to minimize synchronization delays. The camera thread uses a YOLOv5 Nano model to detect stop signs and determine their position (left, center, or right) in the image frame. Simultaneously, the LiDAR thread filters angular slices corresponding to the camera’s field of view (28°to 143°), and calculates the closest object © 2025 the authors. Accepted by IFAC for publication under a Creative Commons License CC-BY-NC-ND 97 Paper available at: https://paperhost.org/proceedings/ifac/MECC25/files/0339.pdf Preprints, 2025 IFAC Modeling, Estimation and Control Conference October 5–8, 2025. Pittsburgh, PA, USA Fig. 1. Multithreaded sensor fusion architecture for stop sign detection and control. distance in the region corresponding to the bounding box. Fusion occurs in the control thread, which matches visual detections with LiDAR distances and initiates stop commands if a stop sign is detected within a threshold distance of 0.8 meters. Fig. 2. Experimental setup showing the Quanser QCar with front CSI camera and RP LiDAR A2 navigating toward a stop sign on the indoor track. To validate the system, a single-stop routine is executed where the vehicle drives forward, detects a stop sign, confirms its proximity using LiDAR, and applies braking. After a brief pause, it resumes driving. The experimental setup, shown in Figure 2, includes printed clean and adversarial stop signs placed along a fixed indoor track. This reactive routine operates without map-based localization or way-points, leveraging onboard sensor fusion to ensure lightweight autonomous control. 3. EXPERIMENTAL RESULTS The QCar platform was evaluated across four conditions: clean stop sign (baseline) and three adversarial variants (Lu2, Lu3, and Yang) from prior work. All tests followed the same single-stop routine on a fixed indoor path, with five to six repeated trials per condition. The vehicle’s throttle, braking threshold (0.8 m), and stop duration were held constant. For each run, we recorded the detection range, stop distance range, average detection distance, average stop position, and missed detections. Table 1. Detection and Stopping Metrics Under Normal and Adversarial Conditions Metric Baseline Lu2 Yang Lu3 Detection Dist. (m) 3.03–3.48 2.05–2.86 1.95–2.38 1.30–1.95 Stopping Dist. (m) 0.55–0.69 0.37–0.68 0.54–0.68 0.62–0.69 Average Detection (m) 3.12 2.43 2.20 1.60 Average Stop (m) 0.63 0.60 0.60 0.64 Missed Detections 0/5 0/6 0/6 0/6 All adversarial conditions led to later detections compared to baseline, reducing average detection distance by 22–49%. The Lu3 attack had the greatest impact, lowering it to 1.60 m. Despite visual perturbations, the system maintained consistent control behavior with safe stops across all trials, demonstrating the effectiveness of LiDARvalidated fusion. 4. CONCLUSION This work demonstrates a lightweight and reactive sensor fusion testbed that combines YOLOv5 object detection and LiDAR sensing for autonomous stop sign navigation. The system performs reliably across clean and adversarial conditions, showing graceful degradation in detection range under perturbation while consistently achieving safe stops. In future work, we aim to expand the testbed to a multi-stop routine with turning maneuvers, enabling analysis of consistency across sequential way-points. Additional research will explore the impacts of stronger imagebased adversarial attacks and develop detection or mitigation mechanisms to enhance perception robustness under real-world threat scenarios. REFERENCES Chen, C. et al. (2019). Multi-modal sensor fusion for vision-based autonomous vehicles. In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops (ICCVW). Chen, H. et al. (2018). Shapeshifter: Robust physical adversarial attack on faster r-cnn. In Proceedings of the European Conference on Computer Vision (ECCV). Dosovitskiy, A. et al. (2017). Carla: An open urban driving simulator. In Conference on Robot Learning (CoRL). Eykholt, K. et al. (2018). Robust physical-world attacks on deep learning models. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Liu, X. et al. (2022). A survey on sensor fusion in autonomous vehicles. IEEE Access. Lu, J. et al. (2017). Adversarial examples that fool detectors. In Advances in Neural Information Processing Systems (NeurIPS). Mounabhargav et al. (2024). Camera and lidar integration for lane-following and obstacle avoidance. In Proceedings of the ESCI. Rappaport, T.S. et al. (2015). Millimeter Wave Wireless Communications for 5G. Cambridge University Press. © 2025 the authors. Accepted by IFAC for publication under a Creative Commons License CC-BY-NC-ND 98 Paper available at: https://paperhost.org/proceedings/ifac/MECC25/files/0339.pdf