Semi-synthetic Dataset Generation of Thrown Objects Incidents for Safety Analysis in Sports Stadiums
Abstract
In sports stadiums, objects thrown from the stands (such as bottles or cups) pose safety risks to players and officials. Detecting such incidents automatically is challenging because these events are rare, objects are small and fast, and privacy regulations limit access to (labelled) footage. We introduce a methodology to generate semi-synthetic training data by integrating simulated thrown objects into real CCTV footage. Our approach first reconstructs a textured 3D model of an empty stadium from drone footage, establishing a consistent coordinate frame for all simulations. Fixed CCTV cameras are then localised within this model by matching their frames to the drone imagery using hierarchical localisation with learned local features, followed by pose refinement to ensure accurate geometric alignment. Next, physically plausible throws are simulated using a game engine (Unity) and rendered with the estimated CCTV intrinsics and extrinsics at native resolution and frame rate; only the object layer is composited onto the original CCTV frames, preserving real crowd motion, lighting, and compression characteristics. Automatic masks, IDs, and bounding boxes are generated for each frame. This results in a multi-camera, pixel-accurate dataset for stadium safety monitoring, suitable for training and benchmarking computer-vision methods for detecting and analyzing incidents. This methodology scales to diverse venues and object types, delivers realistic labelled data where real incidents are scarce, and paves the way for more robust, automated incident-analysis systems.