On the black-box explainability of object detection models for safe and trustworthy industrial applications

Results in Engineering 24 (2024) 103498 Available online 26 November 2024 2590-1230/© 2024 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY-NC license (http://creativecommons.org/licenses/bync/4.0/). Contents lists available at ScienceDirect Results in Engineering journal homepage: www.sciencedirect.com/journal/results-in-engineering Research paper On the black-box explainability of object detection models for safe and trustworthy industrial applications Alain Andres a,b,∗, Aitor Martinez-Serasa, Ibai Lañaa,b, Javier Del Ser a,c aTECNALIA, Basque Research and Technology Alliance (BRTA), Mikeletegi Pasealekua 2, Donostia-San Sebastian, 20009, Spain bUniversity of Deusto, 20012, Donostia-San Sebastián, Spain cUniversity of the Basque Country (UPV/EHU), Bilbao, 48013, Spain A R T I C L E I N F O A B S T R A C T Keywords: Explainable Artificial Intelligence Safe Artificial Intelligence Trustworthy Artificial Intelligence Object detection Single-stage object detection Industrial robotics In the realm of human-machine interaction, artificial intelligence has become a powerful tool for accelerating data modeling tasks. Object detection methods have achieved outstanding results and are widely used in critical domains like autonomous driving and video surveillance. However, their adoption in high-risk applications, where errors may cause severe consequences, remains limited. Explainable Artificial Intelligence methods aim to address this issue, but many existing techniques are model-specific and designed for classification tasks, making them less effective for object detection and difficult for non-specialists to interpret. In this work we focus on model-agnostic explainability methods for object detection models and propose D-MFPP, an extension of the Morphological Fragmental Perturbation Pyramid (MFPP) technique based on segmentation-based masks to generate explanations. Additionally, we introduce D-Deletion, a novel metric combining faithfulness and localization, adapted specifically to meet the unique demands of object detectors. We evaluate these methods on real-world industrial and robotic datasets, examining the influence of parameters such as the number of masks, model size, and image resolution on the quality of explanations. Our experiments use single-stage object detection models applied to two safety-critical robotic environments: i) a shared human-robot workspace where safety is of paramount importance, and ii) an assembly area of battery kits, where safety is critical due to the potential for damage among high-risk components. Our findings evince that D-Deletion effectively gauges the performance of explanations when multiple elements of the same class appear in a scene, while D-MFPP provides a promising alternative to D-RISE when fewer masks are used. 1. Introduction In recent years, Artificial Intelligence (AI) has emerged as a transformative force across various domains, especially in human-machine interaction, where it has enabled significant advancements in datadriven decision-making processes. Among these advances, object detection has become a key component, finding application in critical areas such as autonomous driving, security surveillance, industrial automation, and robotics [45,22]. State-of-the-art object detection models, including Faster-RCNN [28], DETR [6], and the YOLO series [37], have demonstrated impressive performance in identifying and localizing objects within images. Despite their success, the adoption of these models in highly sensitive environments remains limited, particularly in domains where errors could result in serious consequences such as injury, equipment damage, or operational failures. One of the primary * Corresponding author at: TECNALIA, Basque Research and Technology Alliance (BRTA), Mikeletegi Pasealekua 2, Donostia-San Sebastian, 20009, Spain. E-mail address: [email protected] (Alain Andres). reasons for this hesitancy is the black-box nature of object detectors implemented as Deep Learning models, which to date amount to the majority of proposals in the literature. The internal activations of these are not inherently interpretable, making it challenging for end-users to trust the predictions issued by object detectors, especially in high-risk environments operating in open-world environments such as autonomous vehicles and industrial robotics. In this context, the field of Explainable AI (XAI) [5] aims to enhance the interpretability of AI systems by their audience and ultimately, to enhance the user’s trust in the output of AI-based systems. Leaving aside the category of transparent AI models (which are inherently interpretable and do not require any explanations for a user to understand how they work), explainability methods in XAI can be broadly categorized into white-box and black-box approaches. White-box XAI methods require access to the internal workings of the model, such as weights, https://doi.org/10.1016/j.rineng.2024.103498 Received 24 October 2024; Received in revised form 14 November 2024; Accepted 21 November 2024 Results in Engineering 24 (2024) 103498 2 A. Andres, A. Martinez-Seras, I. Laña et al. activations, or gradients (e.g., Grad-CAM [34]). While these methods can provide powerful insights, they are often limited by their dependence on specific model architectures, making them difficult to generalize across different models and less accessible to users unfamiliar with AI research/tools. In contrast, black-box XAI methods treat the model as an opaque entity, providing explanations based solely on the model’s input-output behavior without requiring any access to its internal components. However, most black-box XAI methods are designed for classification tasks rather than for object detection [29,19,25,3]. While classification models produce a single label per image, object detection models must identify and localize multiple objects within an image. Therefore, they need to explain not only the class prediction for each detected object – what they detect– but also the spatial reasoning behind the bounding boxes that define the object’s location – where the object is positioned within the image. Balancing these dual aspects complicates the explanation process and requires more sophisticated techniques than those used for classification tasks. In this paper, we address the gap in XAI methods for object detection by focusing on model-agnostic, black-box XAI techniques. We propose and evaluate novel black-box XAI methods and XAI metrics that are specifically tailored for object detection models, without requiring access to internal model details. Our proposed methods are generalizable to object detection frameworks beyond those utilized in our experiments. Specifically, the contributions of this work can be summarized as follows: •We formally define a quantitative evaluation metric, D-Deletion, which extends the existing Deletion metric [4,25] proposed for classification tasks. This metric is adapted to handle the unique challenges of object detection, including localization (as seen in Fig. 4), which is of utmost importance when multiple instances of the same object appear in the same scene. •By using the similarity score of D-RISE [26], we analyze multiple mask generation methods’ performance and introduce D-MFPP, an extension of MFPP [42] originally developed for classification tasks. D-MFPP utilizes segmentation-based mask generation to improve explanations for object detection models. •We analyze the impact of key parameters, such as image dimensions and the model sizes within the YOLOv8 architecture utilized in our experiments, which can significantly influence the quality of the resulting explanations. •Last but not least, we facilitate the broader adoption of the developed techniques for object detection in real-world use cases by releasing the code publicly in a repository: https://github .com /aklein1995 /drise _ dmfpp _ddeletion. The remainder of this paper is structured as follows: in Section 2, we first review literature related to XAI for object detection. In Section 3, we provide the necessary background on object detection and XAI to familiarize the reader with the key concepts used in the definitions of D-RISE and Deletion. Next, Section 4presents the experimental setup, including datasets, object detection training configuration, employed XAI methods, and evaluation metrics. In this section we also introduce our proposed D-MFPP method and D-Deletion metric. We discuss our results in Section 5. Finally, Section 6concludes the paper with a summary of our key findings and directions for future research. 2. Related work Before proceeding with the materials and novel methods introduced in this work, we first pause briefly at XAI methods, focusing on those used for object detection tasks and put to practice in industrial applications: XAI methods. As stated in the introduction, XAI offers insights into the procedure followed by an AI-based system to elicit their outputs, enabling end-users to understand and eventually trust the decisions output by the AI-based system grounded on objective data [3]. To date, the majority of XAI methods are designed for models learned to address classification tasks. For instance, CAM-based methods like GradCAM [34], GradCAM++[7]and Integrated Gradients [36] quantify and attribute the pixel-wise importance of a given input according to the gradients with respect a target class. Moreover, making use of backpropagation, LRP [20] calculates the contribution that a neuron has with neurons in consecutive layers to get relevance scores. In contrast, perturbationbased techniques work by occluding certain parts of the input and analyzing its impact in the predictions. Within this type of techniques, LIME [29], approximates a NN with an interpretable model; SHAP [19]assigns importance values to each input feature based on Shapley values; RISE [25] generates saliency maps by probing the model with randomly masked versions of the input image; and MFPP [42] generates masks by dividing the input image into multi-scale superpixels. Nonetheless, none of them have been explicitly extended for object detection tasks –with the exception of RISE, which has been adapted for this purpose– although techniques like SHAP can also be utilized for regression problems. XAI methods for object detection models. In recent times, a scarcity of XAI approaches has been proposed to support the interpretability of complex object detection models. SODEx [33]is a method capable of explaining any object detection algorithm using classification explainers, demonstrating how LIME can be integrated within YOLOv4, a variant of the YOLO family of single-stage object detectors. Similarly, D-RISE [26] extends RISE’s mask generation technique by introducing a new similarity score that assesses both the localization and classification aspects of object detection models. More recently, D-CLOSE [38] enhances D-RISE by producing less noisy explanations. Along with other methodological improvements, D-CLOSE uses multiple levels of segmentation in the mask generation phase. Other approaches focusing on hierarchical masking have been proposed. Concretely, GSM-NH [41]evaluates the saliency maps at multiple levels based on the information of previous less fine-grained saliency maps, whereas BODEM [21]further extends this idea but focuses on an extreme black-box scenario where only object coordinates are available. XAI methods for industrial applications. Although XAI is increasingly important in industrial settings to ensure safety, reliability and compliance, the adoption of XAI for object detection methods in industrial use cases has been limited to date [17,15,9]. The vast majority of the works focus either on image classification, like [8]that utilizes GradCAM to interpret vibration signal images in the classification of bearing faults; time-series data, e.g. [35]that presents the implementation and explanations of a remaining life estimator model; or tabular data, as in [31]where SHAP is used to interpret and study the influence of soil and climate features on crop recommendations. Regarding XAI and object detection for industrial applications, we can find a few exemplary studies that expose the shortage of real-world use cases currently noted in this technological crossroads. In [23], various object detection models are evaluated for their effectiveness in detecting weld characteristics in radiography images, with an emphasis on explainability and deployment on edge devices to assist workers. In the same sense, [32]provides a comprehensive review and analysis of various XAI techniques applied to object detection tasks in computerized tomography imaging for medical purposes. Finally, [14] demonstrates how to integrate Grad-CAM into the YOLO architecture and performs experiments in both public and private datasets of vehicle front collision and rear-view cameras. 3. Background We now proceed by elaborating on key concepts needed to properly understand the details of the proposed D-MFPP technique and the DDeletion metric that lie at the core of this work. Concretely, we provide fundamentals for object detection models (Section 3.1) and XAI, with a focus on model-agnostic black-box methods to explain the predictions of object detection models (Section 3.2). Results in Engineering 24 (2024) 103498 3 A. Andres, A. Martinez-Seras, I. Laña et al. 3.1. Object detectors Object detectors are crucial components in computer vision tasks, capable of identifying and localizing objects within an image. They can be broadly categorized into single-stage and two-stage detectors. Single-stage detectors. They directly predict bounding boxes and class probabilities from input images in a single pass. Popular single-stage detectors, such as YOLO [37], SSD [18]and RetinaNet [30], treat object detection as a simple regression problem, straight from image pixels to bounding box coordinates and class probabilities. To this end, they produce a dense grid of bounding box proposals and class probabilities in one step. Specifically, YOLO [37] divides the input image into a grid and predicts bounding boxes and class probabilities for each grid cell. Although this efficiency is beneficial for real-time applications, it often comes at the cost of accuracy when compared to two-stage detectors Two-stage detectors. These models, among which Faster R-CNN [28] can be considered to be the most representative one, follow a more complex approach that divides the detection process into two stages. In the first stage, a Region Proposal Network (RPN) generates a set of candidate object proposals (bounding boxes) from the input image. In the second stage, these proposals are refined and classified into different object categories by a second network. This second stage typically involves a more complex network, such as a convolutional neural network (CNN), which performs classification and further refinement of the bounding box coordinates. This two-step process boosts accuracy by allowing for a more refined feature analysis, though it also slows down processing, making two-stage detectors less suited for applications that require high-speed performance. Most detector networks, including Faster R-CNN and YOLO, produce a large number of bounding box proposals which are subsequently refined using confidence thresholding and Non-Maximum Suppression (NMS) to produce a set of finally detected objects in the image. Each bounding box proposal 𝑑𝑖can be defined as follows: 𝐝𝑖=[𝐋𝑖,𝑂 𝑖,𝐏𝑖]=[(𝑥𝑖 1,𝑦 𝑖 1,𝑥 𝑖 2,𝑦 𝑖 2),𝑂 𝑖,(𝑝𝑖 1,…,𝑝 𝑖 𝐶)],(1) where 𝐋𝑖defines the bounding box corners (𝑥𝑖 1, 𝑦𝑖 1)and (𝑥𝑖 2, 𝑦𝑖 2); 𝑂𝑖∈ [0, 1] refers to the probability that bounding box 𝐿𝑖contains an object of any class; and 𝐏𝑖is a vector of probabilities (𝑝𝑖 1, … , 𝑝𝑖 𝐶)representing the probability that region 𝐋𝑖belongs to each of 𝐶classes. Unlike traditional classifiers, which assign a single class label to an entire image, object detectors must handle both classification and localization simultaneously. This dual task, predicting the class and precise location of each object, increases the complexity of making these models interpretable. 3.2. Explainable Artificial Intelligence (XAI) Despite the great performance exhibited by object detectors in manifold applications, their adoption in risk-sensitive scenarios is often hindered by a lack of trust and transparency by the user making decisions based on the detections issued by these models. As introduced previously, research on XAI produce techniques and methods that make the behavior and predictions of AI models understandable to humans without sacrificing performance [11]. To this end, multiple XAI techniques have been proposed, which can be classified into four broad categories [3]: •Scoop-based techniques focus on the extent of the explanation, providing either local explanations for specific predictions or global explanations for the overall model behavior. •Complexity-based methods consider the complexity of the model, with simpler, interpretable models offering intrinsic interpretability and more complex models requiring post-hoc explanations. •Model-based approaches distinguish between XAI methods that are specific to particular types of models, and those that are model-agnostic, capable of being applied to any model disregarding the specifics of their internals. •Methodology-based techniques are categorized by their methodological approach, such as backpropagation-based methods that trace input influences, or perturbation-based methods that alter inputs to observe changes in the output of the model. Given that object detectors are typically complex neural networks, they fall under the complexity-based category, thereby requiring post-hoc explainability methods to explain their decisions. Among the various methodology-based techniques, attribution methods are commonly used to estimate the relevance of each pixel in an image for the detection task. Attribution methods are particularly important for object detection, where both localization and classification need to be explained. Traditional attribution methods have been primarily developed for image classifiers [1], which produce a single categorical output, making them less suited for object detectors. Object detectors, unlike classifiers, generate multiple detection vectors that encode not only class probabilities, but also localization information and additional metrics, such as objectness scores (see Section 3.1). Furthermore, techniques like NMS and confidence threshold filtering, which are used to refine bounding box proposals, add complexities that require a deeper understanding of the model’s internal workings, complicating the use of certain XAI methods, such as gradient-based approaches. Therefore, we focus on model-agnostic black-box XAI approaches, which are designed to be architecture-independent, and do not depend at all on the specifics of the model under target. Among model-agnostic XAI methods, perturbation-based approaches are commonly used due to their simplicity and effectiveness in revealing which parts of the input are most influential for the model’s predictions. Perturbation-based techniques offer a direct way to assess how changes to the input image affect the model’s output. By systematically altering or masking parts of the input image (using masks to generate perturbed samples), these methods allow inferring the importance of different regions based on the model’s input-output behavior. The typical pipeline for perturbation-based XAI methods can be divided into three stages: (1) Data Preparation, (2) Model Assessment, and (3) Importance Computation. In the Data Preparation stage, masks are generated and applied to the image to create perturbed samples. The Model Assessment stage involves passing these perturbed images through the model to observe the changes in output. Finally, in the Importance Computation stage, the importance of each pixel is calculated by comparing the model’s outputs for the original and perturbed images. While the Model Assessment stage remains consistent across methods, with each perturbed image passed through the model, the Importance Computation varies depending on the XAI approach used. This can range from simple techniques like retraining a model (e.g., LIME) to more complex approaches. Since the effectiveness of these methods largely depends on how the perturbed images are generated, three mask generation algorithms are next described (Fig. 1): •Sliding Window: This method, which is similar to the Occlusion technique proposed in [43], systematically moves a window of fixed size across the image and sets the region within the window to a constant value (e.g., zero) to occlude that part of the image. By iteratively sliding the window across the entire image, we can assess the impact of each occluded region on the model’s output. The method requires specifying the window size, which determines the area of the image being occluded at each step, and the stride, which sets how much the window moves between iterations. •RISE: Randomized Input Sampling for Explanation (RISE) [25]involves sampling 𝑁binary masks of size ℎ ×𝑤, which are smaller than the original image size 𝐻×𝑊. Each element in the mask is independently set to 1 with probability 𝑝and to 0 with the remaining probability 1 −𝑝. These masks are then upsampled to size (ℎ+1)⋅𝐶𝐻×(𝑤+1)⋅𝐶𝑊using bilinear interpolation, where Results in Engineering 24 (2024) 103498 4 A. Andres, A. Martinez-Seras, I. Laña et al. Fig. 1. Example of three masks generated using Sliding Window (top), RISE (middle), and MFPP (bottom). MFPP masks are dependent on the image at the input of the model. In this case, we consider a sample from the battery assembly dataset detailed in Section 4. 𝐶𝐻×𝐶𝑊=⌊𝐻∕ℎ⌋×⌊𝑊∕𝑤⌋. The upsampled masks are cropped to the original image size 𝐻×𝑊with uniformly random offsets ranging from (0, 0) to (𝐶𝐻, 𝐶𝑊). This method creates a diverse set of masks that cover different parts of the image, allowing for a comprehensive evaluation of the importance of various regions. •MFPP: The so-called Morphological Fragmental Perturbation Pyramid (MFPP) [42]method divides the input image into multi-scale fragments and perturbs them randomly. In this sense, it is similar to RISE, but instead of perturbing elements of the generated masks with dimension ℎ ×𝑤, MFPP defines regions according to segmentations at different scales. Depending on the number of defined fragments, the regions would be more fine-grained yet more time-consuming. The segments are dependent on each image, requiring the creation of new masks for every image. 4. Materials and methods This section describes the industrial robotics use cases in what refers to the datasets (Section 4.1), object detection model (Section 4.2), XAI methods (Section 4.3) and the explanation quality metrics (Section 4.4) considered in our work. The novel XAI technique and quality metrics proposed in this manuscript are also described in Section 4.3. 4.1. Industrial robotics datasets under consideration The datasets used in this manuscript have been collected during the course of the ULTIMATE project, https://ultimate -project .eu/, which features two distinct real robotics use cases [16]. The first dataset, from PIAP https://piap .lukasiewicz .gov .pl/, involves a collaborative workspace where a human and a robotic arm work together. The second dataset, provided by Robotnik https://robotnik .eu/, focuses on a battery assembly area, where a robotic arm assembles components for a battery kit.1 Dataset 1: Human-Robot Dataset. This dataset consists of 96 images captured from three different cameras, as exemplified in Fig. 2, with 32 images taken from each camera. The dataset includes two object classes: human and gripper. Importantly, each image in this dataset contains 1While the datasets contain a relatively small number of images, this data shortage is typically encountered in real-world industrial scenarios subject to data availability constraints. Nevertheless, in the use cases under considerations the contextual and scene variability is minimal, yielding short-tailed distributions of the objects to be detected. Therefore, the small datasets described in the paper sufficiently capture the relevant features for the specific object detection tasks addressed by the models. Fig. 2. Dataset 1 (Human-Robot collaboration): Data are captured from cameras located in 3 different positions. All the images belonging to this dataset contain the faces blur to preserver anonymity. Fig. 3. Dataset 2 (Battery Assembly kit): The setup where a robotic arm would assemble the kit based a bird-eye view of the table where all component are expected to be; (left) a theoretical setup; (right) an actual sample. only a single object of each class, meaning a maximum of one human and one gripper per image. To ensure a diverse and representative sample, we applied feature extraction using ResNet [12]to obtain embeddings for the entire dataset. The dimensionality of these embeddings was reduced using Principal Component Analysis (PCA), followed by Kmeans clustering (with 𝑘 =8clusters). From each cluster, four images were randomly selected, resulting in a final subset. The data were split into three sets: 72 images for training (75%), 6 for validation (6.25%), and 18 for testing (18.75%). To maintain consistency, we applied the same partitioning to the data from each camera. This resulted in 24 images for training, 2 for validation, and 6 for testing from each camera. Dataset 2: Battery Assembly Dataset. This dataset consists of 7 images, all captured from a bird’s-eye (top-down) view, showing a robotic arm assembling a battery kit, as shown in Fig. 3. The dataset includes five distinct object types: individual battery,bms_a,bms_b,battery holder, and unknown object. In contrast to the Human-Robot Dataset, each image in the Battery Assembly Dataset may contain multiple objects of the same class, such as several individual batteries in a single scene. It is worth noting that XAI techniques can be applied to any type of data. When applied to training data, they help reveal what the model has learned to focus on during training. When applied to test data, they provide insight into how well the model generalizes to new, unseen examples. For the Human-Robot Dataset, XAI explanations were applied exclusively to the test images, allowing us to assess the model’s behavior on unseen data. However, for the Battery Assembly Dataset, given the limited number of images (only 7), XAI explanations were applied to the entire dataset. 4.2. Object detection model: YOLOv8 Among the possible object detector models, we selected one of the state-of-the-art options, YOLOv8, due to its numerous advancements over previous versions and its robust performance in object detection tasks [37]. YOLOv8 [27] integrates a novel combination of Feature Pyramid Network (FPN) and Path Aggregation Network (PAN) architectures, enhancing its ability to detect objects at various scales and resolutions. Results in Engineering 24 (2024) 103498 5 A. Andres, A. Martinez-Seras, I. Laña et al. The FPN gradually reduces the spatial resolution of the input image while increasing feature channels, facilitating multi-scale object detection. The PAN architecture further aggregates features from different levels through skip connections, improving the detection of objects with diverse sizes and shapes. Additionally, YOLOv8 introduces an anchorfree detection mechanism that directly predicts the center of an object (instead of the offset from a known anchor box), reducing the number of box proposals and speeding-up the post-processing. Furthermore, it was trained with larger and more diverse datasets including the popular COCO dataset, improving its performance across a wider range of images. YOLOv8 was developed and released by Ultralytics, and although the model and its weights are open-source, most users are expected to utilize the Ultralytics framework for its enhanced usability. However, unlike previous YOLO releases where the probability for each class per predicted box was accessible, in YOLOv8, the Ultralytics API outputs only the probability for the class with the highest confidence in each box.2Consequently, by default, YOLOv8 outputs: 𝐝𝑖=[𝐋𝑖,𝑂 𝑖,𝐶 𝑖]=[(𝑥𝑖 1,𝑦 𝑖 1,𝑥 𝑖 2,𝑦 𝑖 2),𝑂 𝑖,𝐶 𝑖],(2) where 𝐋𝑖=(𝑥𝑖 1, 𝑦𝑖 1, 𝑥𝑖 2, 𝑦𝑖 2)represents the coordinates of the bounding box, 𝑂𝑖denotes the objectness score, and 𝐶𝑖corresponds to the predicted class label for the object within the bounding box, which differs with respect to the outputs shown in Expression (1). 4.3. Explainability methods We evaluate four popular methods for generating visual explanations of black-box models: LIME, RISE, D-RISE, and D-MFPP. The first two methods, LIME and RISE,3were originally developed for image classifiers but can be adapted to object detectors. However, they primarily focus on explaining classification aspects and are not capable of addressing localization characteristics. In contrast, D-RISE is one of the first XAI methods specifically designed for object detectors, providing explanations that encompass both classification and localization. Additionally, we extend the existing MFPP method (originally tailored for classifiers) into a version suitable for object detection, which we refer to as D-MFPP. In what follows we briefly describe them, flowing into a description of the proposed D-MFPP approach: •LIME was originally designed to explain the predictions of any classifier by approximating it locally with an interpretable model. To explain the prediction for an input image 𝐼, LIME fits an interpretable model 𝑔(e.g., a linear model) to approximate the behavior of the black-box model 𝑓locally around 𝐼. The similarity between the original image and the perturbed samples is measured using a kernel function 𝜋𝐼(𝑧). When image explanations are targeted, LIME groups contiguous pixels into superpixels based on similar features they represent. This approach allows LIME to measure the importance of regions in the image rather than individual pixels, making the explanations more interpretable. •As introduced in the previous section, RISE [25]was originally designed for deep neural networks that take images as input and output a class probability (e.g., a classifier like ResNet-50). It generates saliency maps that indicate the importance of each pixel by applying randomly generated binary masks 𝑀𝑖to the input image 𝐼and observing the changes in the model’s output 𝑓(𝐼⊙𝑀 𝑖). In RISE, 𝑁binary masks 𝑀𝑖∈{0, 1}ℎ×𝑤are generated (as explained in Section 3.2). 2https://github .com /ultralytics /ultralytics /issues /2863% https://github .com /ultralytics /ultralytics /issues /4908. 3These XAI methods have been chosen due to their perturbation-based nature, which aligns closely with the methodology followed by the XAI methods DRISE and D-MFPP proposed in this work. Both D-RISE and D-MFPP generate explanations through perturbations. These masks are then applied to the input image 𝐼to generate masked images 𝐼′ 𝑖=𝐼⊙𝑀 𝑖, where ⊙denotes element-wise multiplication. The model is evaluated on each masked image 𝐼′ 𝑖to obtain the outputs 𝑓(𝐼⊙𝑀 𝑖). The importance score for each pixel (𝑥, 𝑦)is then calculated as the weighted sum of the outputs: 𝑆𝐼,𝑓 (𝑥, 𝑦)= 1 𝑁 𝑁 ∑ 𝑖=1 𝑓(𝐼⊙𝑀 𝑖)⋅𝑀𝑖(𝑥, 𝑦)(3) where the weights 𝑀𝑖(𝑥, 𝑦)represent the value of mask 𝑖at pixel (𝑥, 𝑦). The intuition behind RISE is that 𝑓(𝐼⊙𝑀 𝑖)would be high when pixels preserved by mask 𝑀𝑖are important. Although this is true when having infinite diverse masks, in practice RISE calculates each pixel’s importance empirically by Monte Carlo sampling. Therefore, RISE largely depends on the number of masks (𝑁) and how they are generated (i.e., is sensitive to the selected probability 𝑝and resolution 𝑠). 4.3.1. D-RISE and proposed D-MFPP approach Unlike the other two approaches originally designed for classifiers that measure solely classification aspects, D-RISE (Detector Randomized Input Sampling for Explanation) [26]was designed to explain both the classification and localization of a detection. In this sense, D-RISE extends RISE by producing saliency maps specifically for object detectors. As previously seen in Section 3.1, the output given by an object detector differs from the probability vector given by a classifier, obtaining localization information 𝐿𝑖, an objectness score 𝑂𝑖and the probability of classifying each bounding box to any of the considered classes 𝑃𝑖. As a consequence, Expression (3)used by RISE is replaced in D-RISE with a new similarity score, given by: 𝑆𝐼,𝑓 (𝐝𝑡,𝐝𝑗)=𝑠𝐿(𝐝𝑡,𝐝𝑗)⋅𝑠𝑃(𝐝𝑡,𝐝𝑗)⋅𝑠𝑂(𝐝𝑡,𝐝𝑗),(4) where 𝑠𝐿=𝐼𝑜𝑈(𝐋𝑡, 𝐋𝑗), 𝑠𝑃=𝐏𝑡⋅𝐏𝑗∕(||𝐏𝑡|| ⋅||𝐏𝑗||), and 𝑠𝑂=𝑂𝑗. In this formulation, 𝑠𝐿represents the spatial proximity of the bounding boxes encoded by the target detection 𝐝𝑡and the proposal 𝐝𝑗, measured using the Intersection over Union (IoU); the term 𝑠𝑃evaluates the similarity between the class probabilities of the target detection and the proposal using cosine similarity; and 𝑠𝑂incorporates the objectness score of the proposal 𝑂𝑗. It is important to note that for a detection target 𝐝𝑡there would potentially be more than one detection proposals 𝐝𝑗. Therefore, we would have multiple 𝑆𝐼,𝑓(𝐝𝑡, 𝐝𝑗). As explained in D-RISE, the explanations consider only the detection with maximal score for each mask: 𝑆𝐼,𝑓 (𝐝𝑡,𝑓(𝑀𝑖⊙𝐼))=max 𝐝𝑗∈𝑓(𝑀𝑖⊙𝐼)𝑆𝐼,𝑓(𝐝𝑡,𝐝𝑗).(5) Given the YOLOv8 outputs explained in Section 4.2, which do not provide the class probability vector 𝑃𝑖without modifying its architecture (an approach we want to avoid within the scope of this paper), we must adapt the similarity score to only consider 𝑠𝐿and 𝑠𝑂. Consequently, the modified similarity score can be expressed as: 𝑆𝐼,𝑓 (𝐝𝑡,𝐝𝑗)=𝑠𝐿(𝐝𝑡,𝐝𝑗)⋅𝑠𝑂(𝐝𝑡,𝐝𝑗)=𝐼𝑜𝑈(𝐋𝑡,𝐋𝑗)⋅𝑂𝑗.(6) This adjustment allows still utilizing D-RISE effectively for generating saliency maps with the default YOLOv8 model, focusing on the spatial and objectness aspects of detections, while maintaining the integrity of the model’s original architecture. Similarly, we can adopt this similarity score but apply it with a different mask generation process. The MFPP method introduced in Section 3.2, originally designed for classification tasks, can be extended by applying Equation (6), resulting in D-MFPP. To the best of our knowledge, no previous work has proposed this variant of MFPP for object detection tasks. Results in Engineering 24 (2024) 103498 6 A. Andres, A. Martinez-Seras, I. Laña et al. 4.4. Metrics Evaluating the performance of attribution-based explainability methods for image data involves assessing how well the generated relevance heatmaps highlight important regions of the input image that contribute to the model’s decision. Generally, according to [13], explanation quality metrics can be grouped into six categories based on their logical similarity: faithfulness, robustness, localization, complexity, randomization, and axiomatic metrics. In this study, we focus on two of these categories that are particularly relevant to object detection: localization (Section 4.4.1) and faithfulness (Section 4.4.2). 4.4.1. Localization Localization metrics evaluate whether the explainable evidence is centered around a region of interest (RoI) defined by a bounding box, segmentation mask, or a cell within a grid. These metrics aim to verify if the saliency maps correctly highlight the areas in the image that contain the object of interest. Among them, our experiments will consider: •Pointing Game (PG), which is a human evaluation metric introduced in [44]. If the highest saliency point lies inside the human-annotated bounding box of an object, it is counted as a hit. The PG accuracy is given by: PG =#𝐻𝑖𝑡𝑠 #𝐻𝑖𝑡𝑠+#𝑀𝑖𝑠𝑠𝑒𝑠 ,(7) which is averaged over all categories in the dataset. •Energy-based Pointing Game (EBPG) [39], which measures the proportion of activations within the given bounding box relative to the whole activation in the image. It assesses how much of the model’s activation energy is concentrated within the predefined region of interest. Formally: EBPG =∑(𝑥,𝑦)∈bbox 𝑆𝐼,𝑓 (𝑥, 𝑦) ∑(𝑥,𝑦)∈bbox 𝑆𝐼,𝑓 (𝑥, 𝑦)+∑(𝑥,𝑦)∉bbox 𝑆𝐼,𝑓(𝑥, 𝑦),(8) where 𝑆𝐼,𝑓 (𝑥, 𝑦)represent the saliency score at pixel (𝑥, 𝑦), ∑(𝑥,𝑦)∈bbox 𝑆𝐼,𝑓 (𝑥, 𝑦)represents the sum of activation values within the bounding box, and ∑(𝑥,𝑦)∉bbox 𝑆𝐼,𝑓(𝑥, 𝑦)represents the sum of activation values outside the bounding box. 4.4.2. Faithfulness Metrics accounting for faithfulness quantify to what extent explanations follow the predictive behavior of the model, asserting that more important features play a larger role in model outcomes. These metrics focus on understanding the causal relationship between input features and the model’s output by systematically altering the features and observing the changes in predictions. Among them: •Deletion: Inspired by the work by [4], the Deletion metric was proposed in RISE [25]. This metric measures a decrease in the probability of the predicted class as more and more important pixels are removed, where the importance is obtained from the saliency map. A sharp drop, and thus a low Area Under the probability Curve (AUC, as a function of the fraction of removed pixels), indicates a good explanation. Given the importance score for each pixel calculated by any XAI method, 𝑆𝐼,𝑓, we can formulate the Deletion metric as: Deletion(I,S,c) =AUC({𝑃𝑟(𝑓(𝐼⊙𝑀 𝑘)=𝑐)}𝐾 𝑘=1),(9) where 𝐼is the original image, 𝑀𝑘represent a mask with the 𝑘-th most important pixels removed sorted by 𝑆𝐼,𝑓 , 𝑃𝑟(𝑓(𝐼⊙𝑀 𝑘)=𝑐) represents the probability of model 𝑓predicting that the bounding box belongs to class 𝑐, and AUC(⋅) computes the area under the curve for the 𝐾predictions. •Minimum Subset: It follows the same logic as Deletion, but instead of determining the AUC, it considers the required number of pixels that make the prediction to change [10]. Given the importance score for each pixel (𝑆𝐼,𝑓 ), Min-Subset is defined as the smallest subset of pixels that needs to be removed to change the model’s prediction. Mathematically: Min-Subset(𝐼,𝑆,𝑐)= min {𝑘∈{1,2…,𝐾}∶ 𝑓(𝐼⊙𝑀 𝑘)≠𝑓(𝐼)},(10) where 𝑓(𝐼⊙𝑀 𝑘)represents the class label assigned by the model 𝑓 after passing the image 𝐼with the top 𝑘most important pixels removed, and 𝑓(𝐼)is the class label predicted for the original image. 4.4.3. Proposed D-deletion and D-minimal subset metrics Originally, Deletion was designed for classifiers. However, with object detectors, multiple detections in a single image can occur. Although D-RISE stated the necessity to adapt this metric for object detectors [26], no formal definition can be found in the literature. Therefore, considering the importance of this issue in real use cases, we formally re-define Equation (9)in two manners: 1. Deletion. Measures the explanation given the target class label 𝐶𝑡(regardless if there is more than one element for a class) and iteratively removes the top 𝑘most important pixels: Deletion(𝐼,𝑆,𝐶𝑡)= 𝐴𝑈𝐶 ⎛⎜⎜⎝{max 𝐝𝑘 𝑗[𝑂𝑘 𝑗 ⋅I{𝐶𝑘 𝑗=𝐶𝑡}]}𝐾 𝑘=1⎞⎟⎟⎠ .(11) The model 𝑓(⋅)takes as input the masked image 𝐼⊙𝑀 𝑘and outputs a set of bounding box proposals 𝐝𝑘 𝑗=[𝐋𝑘 𝑗, 𝑂𝑘 𝑗, 𝐶𝑘 𝑗]. The indicator function I{𝐶𝑘 𝑗=𝐶𝑡}equals 1 if the predicted class 𝐶𝑘 𝑗matches the target class 𝐶𝑡, and 0 otherwise. The term max𝐝𝑘 𝑗[𝑂𝑘 𝑗 ⋅I{𝐶𝑘 𝑗=𝐶𝑡}] selects the maximum objectness score 𝑂𝑘 𝑗for the bounding boxes where the predicted class matches the target class. The AUC is then computed over the set of prediction scores for the 𝐾steps, where at each step the most important pixels are progressively removed. 2. D-Deletion. While the standard Deletion metric evaluates the impact of pixel removal on a class prediction, it lacks the ability to account for spatial localization, which is essential in object detection tasks where multiple instances of the same class can appear. D-Deletion addresses this limitation by focusing on a specific target bounding box 𝐝𝑡, considering both the class information, 𝐶𝑡, and IoU between the target and other detected proposals, 𝐝𝑘 𝑗. This ensures that the metric not only measures faithfulness but also takes localization into account, providing more precise explanations in situations where different objects of the same class coexist. Mathematically is expressed as: D-Deletion(𝐼,𝑆,𝐶𝑡)= 𝐴𝑈𝐶 ⎛⎜⎜⎝{max 𝐝𝑘 𝑗[𝑂𝑘 𝑗 ⋅I{𝐶𝑘 𝑗=𝐶𝑡}⋅I{𝐼𝑜𝑈(𝐝𝑡,𝐝𝑘 𝑗)>𝛾}]}𝐾 𝑘=1⎞⎟⎟⎠ (12) where 𝛾is a threshold. As a consequence, when multiple elements of the same class are in an image, D-Deletion will only consider those proposals 𝐝𝑘 𝑗predicted by the model that have a predefined IoU with the target bounding box 𝐝𝑡. The difference between Deletion and D-Deletion is illustrated in Fig. 4. This figure highlights how D-Deletion distinguishes between different objects of the same class by incorporating localization information, leading to more refined and accurate explanations (↓AUC in the Figure’s last row) when multiple objects of the same class are detected in an image. For the sake of clarity, we provide the pseudocode of Deletion in Algorithm 1, where the main difference with respect to D-Deletion are lines 10 to 12. Results in Engineering 24 (2024) 103498 7 A. Andres, A. Martinez-Seras, I. Laña et al. Fig. 4. Illustration of a collaborative workspace featuring two humans and a robotic arm. The first row shows the original image. The second row displays the image with the 10% most important pixels removed for each human, as identified by an XAI method. In the third row, the Deletion metric curve, which only considers class type, shows a high probability score even when the primary human is largely occluded by the other person. The fourth row presents the DDeletion metric curve, which incorporates a localization component, providing a more accurate measure of explanation importance by considering the positions of entities within the image. A lower area under the curve indicates a better explanation. Lastly, akin to how Min-Subset is related to Deletion, D-Min-Subset is associated with D-Deletion. Consequently, D-Min-Subset considers both the class type and the IoU to determine the number of pixels required to make the prediction to change: D-Min-Subset(𝐼,𝑆,𝐶𝑡)= Algorithm 1 Deletion Metric’s Pseudocode for Object Detector. Require: Image 𝐼, saliency map 𝑆𝐼,𝑓 , number of steps 𝐾, target detection 𝐝𝑡, target class 𝐶𝑡 1: Initialize 𝑆←[] 2: for 𝑘 =1to 𝐾do 3: 𝑀𝑘←𝑆𝐼,𝑓 removing the top 𝑘most important pixels 4: Apply mask 𝑀𝑘to image 𝐼 5: Forward pass through the model 𝑓and obtain the bounding box proposals 𝐝𝑗=[𝐋𝑗, 𝑂𝑗, 𝐶𝑗] =𝑓(𝐼⊙𝑀 𝑘) 6: Initialize list of proposals: proposals ←[] 7: for each bounding box 𝐝𝑗predicted by the model 𝑓do 8: if 𝐶𝑗=𝐶𝑡then 9: proposals.append(𝑂𝑗) % For D-Deletion 10: if 𝐼𝑜𝑈(𝐝𝑡, 𝐝𝑗) >𝛾 then 11: proposals.append(𝑂𝑗) 12: end if 13: else 14: proposals.append(0) 15: end if 16: end for 17: Insert the maximum score within the deletion buffer: 𝑆←𝑆∪ max(proposals) 18: end for 19: Calculate the Deletion metric as the area under the curve: 𝐷=AUC(𝑆) 20: return Deletion score 𝐷 min {𝑘∈{1,2…,𝐾}∶ 𝐶𝑘 𝑗≠𝐶𝑡or 𝐼𝑜𝑈(𝐝𝑡,𝐝𝑘 𝑗)<𝛾},(13) where 𝐶𝑘 𝑗represents the predicted class label for detection 𝑗when passing the masked image 𝐼⊙𝑀 𝑘through the model 𝑓, with 𝐝𝑘 𝑗=𝑓(𝐼⊙𝑀 𝑘) being the set of detections after removing the top 𝑘most important pixels. In this context, D-Min-Subset depends on two conditions: (1) the class probability labels 𝐶𝑘 𝑗for the predicted bounding box 𝐝𝑘 𝑗must no longer match the target class 𝐶𝑡, or (2) the IoU between the target bounding box 𝐝𝑡and the predicted bounding box 𝐝𝑘 𝑗falls below the threshold 𝛾. The minimum 𝑘is identified as the step where either of these conditions is first met. 5. Experiments and results Contrarily to most studies in the XAI literature that primarily focus on benchmark datasets, our research work focuses on assessing the explainability of object detectors in real-world industrial data. In this context, to evaluate the effectiveness of explanations, we formulate four key research questions to answer them with empirical evidence: •RQ1: Which XAI method provides the most reliable and insightful explanations for object detection models? •RQ2: Does the D-Deletion metric enhance the trustworthiness of XAI outputs when multiple objects of the same class are present in the image? •RQ3: How does the mask generation process influence the quality of explanations, particularly when using similarity scores for object detection? How does D-MFPP behave? •RQ4: Do different image dimensions impact the explanations generated by XAI methods? Do models of varying sizes (large, medium, small, nano) focus on different regions of the image in their explanations? Next, we outline the hyperparameters used across our experiments to ensure consistency in training and evaluation. For both datasets, models were trained using the YOLOv8 architecture for a total of 100 epochs. The image size (imgsize) was set to the largest dimension of the input image (e.g., 720 × 1280 ←←→ 1280), and data augmentation techniques such as random horizontal flipping and color jitter were applied. For consistency, the default Ultralytics settings were used wherever applicable. In Results in Engineering 24 (2024) 103498 8 A. Andres, A. Martinez-Seras, I. Laña et al. Table 1 Quantitative metrics of LIME, RISE and D-RISE for the Human-Robot dataset. The table presents the performance of each XAI technique in terms of classification (Deletion, D-Deletion, Min-Subset, D-Min-Subset) and localization metrics (PG and EBPG), with scores reported for each class (Human,Gripper) and the overall average. Lower values are better for metrics marked with ↓, while higher values are better for those marked with ↑.Bold values indicate the best average scores across all objects, highlighting the best-performing XAI method for each metric. Values highlighted in gray represent the best scores for each object category (Human or Gripper) and should be interpreted vertically. XAI Method Object Deletion (↓) D-Deletion (↓)Min-Subset(↓) D-Min-Subset (↓)PG(↑)EBPG(↑) LIME Human 0.0759 0.0632 4.2703 4.2703 1.0000 34.984 Gripper 0.4688 0.0324 1.3859 1.3859 1.0000 2.2270 Average 0.2723 0.0478 2.8281 2.8281 1.0000 18.6060 RISE Human 0.1827 0.1241 9.0108 9.0108 0.7500 19.0824 Gripper 0.2637 0.0060 0.6355 0.63 1.0000 1.0542 Average 0.2232 0.0651 4.8232 4.8232 0.875 10.0683 D-RISE Human 0.1255 0.0818 5.7335 5.7335 0.8750 20.4061 Gripper 0.2777 0.0059 0.6091 0.6091 1.0000 1.0815 Average 0.2016 0.0438 3.1713 3.1713 0.9375 10.7438 Fig. 5. Heatmaps obtained by applying RISE (left) and D-RISE (right) for the detection of a human in the Human-Robot Dataset. the case of LIME, we use the baseline implementation of [29], where we adopt the SLIC segmentation algorithm [2](with 100 segments) and generated 1000 samples to assess the quality of the produced explanations. For RISE and D-RISE, we employed 5000 masks with a probability of 0.25 and a resolution of 16 ×16to produce the saliency maps. Lastly, for all object detection predictions, a confidence threshold of 0.7 was set to determine the validity of each detection. In what follows we present and discuss on the results obtained to answer each of the RQ formulated above: RQ1: Comparison between XAI methods In the Human-Robot dataset, the comparison between LIME, RISE, and D-RISE, as shown in Table 1, reveals distinct strengths across different metrics (Section 4.4). LIME performs good in terms of localization, with higher PG and EBPG scores (100% and 18.60%, respectively) compared to D-RISE (93.75% and 10.74%). This indicates that LIME generates more localized saliency maps, focusing closely on the bounding boxes of detected objects. However, this superior performance is partly due to the size of the object being analyzed. LIME’s superpixel generation is better suited for larger objects (e.g., human), as larger regions of the image can be grouped effectively into meaningful segments, leading to higher localization scores. This advantage also applies to classification, where larger objects allow LIME to better preserve relevant features for detection. Conversely, for smaller objects (e.g., gripper), LIME struggles when compared to the other methods, as reflected by its worse performance metrics in those cases. In contrast, RISE and D-RISE are less sensitive to object size, making them more robust across different object scales, which is evident in their better performance on smaller objects like the gripper. They achieve Deletion scores of 0.2636 and 0.2777, respectively, compared to LIME’s 0.4688. When considering the overall performance across classes, RISE, with a Deletion score of 0.2232 and D-Deletion of 0.0651, shows improvement over LIME in classification-related tasks but still lags behind D-RISE, which achieves the lowest Deletion (0.2016) and D-Deletion (0.0438) scores. Although D-RISE offers the best balance between classification and localization, the difference between RISE and D-RISE is Fig. 6. Explanations of a scene using different stride and window size configurations when using D-Sliding Window (combination of mask generation explained in Section 3.2 and Equation (6)). (Left) Stride of 16; (Right) Stride of 8; (Top) Window size of 32; (Bottom) Window size of 64. minimal in this dataset, where each image contains only a single object per class. As a result, as shown in Fig. 5, their heatmaps are very similar to each other, both highlighting the human head. However, D-RISE eliminates less relevant areas more effectively. In the results obtained over the Battery Assembly dataset (Table 2), a similar pattern can be noticed. LIME excels at localization with an average EBPG of 16.03%, while RISE and D-RISE perform better in retaining key classification features. Since this dataset includes multiple objects of the same class (e.g., multiple batteries), both LIME and RISE, which are not designed to handle multiple detections of the same class, expose severe limitations. RISE, with a D-Deletion score of 0.1474, preserves key features better than LIME, but is outperformed by D-RISE, which achieves a score of 0.0344. D-RISE also shows the highest PG score (97%), performing significantly better than LIME (76.85%) and RISE (66.95%). Overall, when dealing with datasets containing only one object per class, the differences between LIME, RISE, and D-RISE are relatively small in quantitative terms. However, when multiple objects of the same class appear in a given input image, D-RISE clearly dominates over the rest of techniques. As illustrated in Fig. 7, D-RISE generates coherent heatmaps for each detected object in the Battery Assembly dataset, whereas LIME and RISE provide a global saliency map for the entire class. By combining the individual saliency maps from D-RISE, a more accurate and object-specific explanation can be produced. This also highlights the limitations of LIME and RISE when applied to mul- Results in Engineering 24 (2024) 103498 9 A. Andres, A. Martinez-Seras, I. Laña et al. Table 2 Quantitative metrics of LIME, RISE and D-RISE for the Battery Assembly dataset. The table presents the performance of each XAI technique in terms of classification (Deletion, D-Deletion, Min-Subset, D-Min-Subset) and localization metrics (PG and EBPG), with scores reported for each object (indiv batt, bms a, bms b, unknown object, batt holder) and the overall average. Lower values are better for metrics marked with ↓, while higher values are better for those marked with ↑.Bold values indicate the best average scores across all objects, highlighting the best-performing XAI method for each metric. Gray-highlighted values represent the best scores for each object category (indiv batt, bms a, bms b, unknown object or batt holder) and should be interpreted vertically. XAI Method Object Deletion (↓) D-Deletion (↓)Min-Subset(↓) D-Min-Subset (↓)PG(↑)EBPG(↑) LIME indiv batt 0.7549 0.2806 44.3412 14.6756 0.3188 2.5347 bms a 0.0184 0.0181 0.8784 0.8784 1.0000 30.7573 bms b 0.0125 0.0125 0.8321 0.8321 1.0000 16.5440 unknown object 0.0624 0.0245 2.0342 2.0342 1.0000 20.1071 batt holder 0.1498 0.0849 6.8115 4.3766 0.5238 10.2087 Average 0.1996 0.0841 10.9795 4.5594 0.7685 16.0304 RISE indiv batt 0.7659 0.4359 81.2344 32.0482 0.0144 1.2558 bms a 0.0190 0.0190 1.9417 1.9417 1.0000 3.0409 bms b 0.0217 0.0217 2.1266 2.1266 1.0000 2.4561 unknown object 0.5333 0.2008 3.8372 3.8372 1.0000 5.5780 batt holder 0.0902 0.0595 7.9519 5.5632 0.3333 3.3681 Average 0.2860 0.1474 19.4184 9.1034 0.6695 3.1398 D-RISE indiv batt 0.6214 0.0311 35.7678 2.7725 1.0000 2.0546 bms a 0.0181 0.0181 1.9880 1.9880 1.0000 3.2955 bms b 0.0128 0.0116 1.3407 1.3407 1.0000 2.8876 unknown object 0.0879 0.0485 3.7448 4.2071 0.8571 7.6831 batt holder 0.4839 0.0626 21.7291 5.1009 1.0000 4.9635 Average 0.2448 0.0344 12.9141 3.0819 0.9714 4.1768 tiple objects, as their global saliency maps do not differentiate between individual instances. RQ2: D-deletion metric for scenes with multiple objects of the same class As a secondary observation in the experiments of RQ1, the DDeletion metric is specifically designed to overcome the limitations of traditional deletion metrics, particularly when multiple objects of the same class are present in an image. In the Battery Assembly dataset, where several instances of the same class (e.g., indiv batt) appear, D-Deletion demonstrates clear advantages. By inspecting Table 2, RISE, while performing reasonably well with an average Deletion score of 0.2860, it still obtains a relatively high D-Deletion score of 0.1474, suggesting that it struggles to differentiate between the contributions of individual objects. In contrast, D-RISE, which obtains an average Deletion score of 0.2448, outperforms RISE with a D-Deletion score of 0.03444. This highlights D-RISE’s ability to isolate and preserve key features for each object, providing more trustworthy, object-specific explanations rather than broad, class-level insights. The Min-Subset and D-Min-Subset metrics, which measure the minimal proportion of pixels needed to disrupt a detection, reinforce these findings. In the Human-Robot dataset, Table 1, where only one object per class appears, the differences between Deletion and D-Deletion scores are minor, and the MinSubset and D-Min-Subset values are close to each other. However, in the Battery Assembly dataset, where the differences between Deletion and D-Deletion are more substantial and multiple objects of the same class co-occur in the same image, the Min-Subset (12.9141) and D-Min-Subset (3.0819) values also diverge significantly. RQ3: Influence of the mask generation strategy When comparing XAI approaches for object detection tasks configured with different mask generation techniques, the results in Tables 3 and 4initially suggest that D-Sliding Window performs the best in almost all metrics. However, as noted in the captions, this is only in cases where explanations were provided. For the Human-Robot dataset (Table 3), regardless of the window size and stride, D-Sliding Window failed to provide explanations for larger objects, such as humans, and only provided meaningful explanations for smaller objects like gripper instances. Similarly, in the Battery Assembly dataset (Table 4), D-Sliding Window struggled with large objects when using a smaller window size (w=32), which led to higher scores in classification metrics. Even with Fig. 7. Heatmaps generated in a scene of the Battery Assembly dataset for two target classes: individual battery (top row) and battery holder (bottom row). The first and second columns shows the saliency maps obtained using LIME and RISE, independent of the number of objects of the same class in the image. Columns 3,4 and 5 display heatmaps generated using D-RISE for three different individual elements of the same class.