Ergodic tactile estimation of defects

Ergodic tactile estimation of defects Jorge G. P´ erez Fuentevilla1,3, Matteo Saveriano 1,3, Luigi Palopoli2,3, Daniele Fontanelli1,3 Abstract—In this work, an algorithm for estimating the position and Gaussian-like shape of an unknown number of defects on a given flat surface is developed. A natural behavior for exploring/exploiting possible defects using the ergodic control theory for trajectory planning has been achieved. Informative tactile measurements along with the end-effector/sensor position related have been used to provide a posterior belief using the Gaussian Mixture Model technique, and a Kullback-Leibler divergence-dependent constraint has been designed to enhance the accuracy of the estimation. Index Terms—Defect detection, Gaussian Mixture Model, Ergodic control, Tactile exploration. I. INTRODUCTION The surface inspection problem has gained popularity in the last years, mainly by the manufacturing industry. Defects lead to bad quality products or unsafe scenarios, and the usual practice is to have workers doing the inspection manually; however, this is costly, time-consuming, and the performance is subjective. Thus, there exists a particular interest in automating this process. The majority of works use visual approaches to deal with this problem. Visual inspection has some characteristics: using vision one can cover larger areas in a relatively short time, but the accuracy is often poor because of different noisy factors. On the other hand, there is tactile inspection, which has, in general, a higher accuracy but the covered area is lower and it takes longer than the visual approach [1]. In this work, we aim to investigate a tactile solution, keeping always an ergodic behavior. Ergodic control is a recent and exciting line of research, which provides a natural exploration/exploitation trade-off in the solution trajectory taking a probability map as reference [2], [3]. We use ergodic control as trajectory planner for the search task since we want to expend more time in more probable regions of space where defects can be, and spend less time visiting areas where defects are unlikely. Ergodic trajectory planning has already proven to work much better than alternative algorithms for exploration and target localization tasks, see [4]. Another crucial factor in our work is the use of Gaussian Mixture Models (GMM), which along with a posterior covariance Kullback-Leibler (KL) divergence dependent constraint, we use as density estimator for the posterior belief of the position and shape of defects. We acknowledge the support of the MUR PNRR project FAIR - Future AI Research (PE00000013), the project INVERSE (GA n. 101136067), and the project MAGICIAN (GA n. 101120731). 1Department of Industrial Engineering, Universit` a degli studi di Trento, Trento, Italy. 2Department of Information Engineering and Computer Science, Universit` a di Trento, Trento, Italy. 3Interdepartmental Robotics Labs (IDRA), University of Trento. Fig. 1. Overall workflow. II. PROBLEM STATEMENT We deal with the problem of having defects in a given bidimensional space Ω = Ll 1, Lu 1×Ll 2, Lu 2⊂R2where Lu i, Ll iand Li=Lu i−Ll iare the upper bounds, lower bounds and lengths of the i-th dimension, respectively. The purpose is to estimate the position and the shape of them, relying on tactile measurements and maintaining a dynamic ergodic behavior. Let us make the next assumptions: •Defects are static. •Defects has a gaussian-like shape. •The overall space occupied per defect on the search space is known. The general workflow is defined as follows: we have a real Probability Density Function (PDF), i.e., a probability map Φ(x)∈Ωthat represents the real defects on the task space x=x1x2⊺∈Ω, which we don’t know and we want to estimate. First, we define a prior probability map ˆ Φ(x)∈Ω that represents our belief of defects in the search space; this belief is used as a reference for planning the ergodic trajectory of the robot; then we execute the motion and collect data for computing the next reference PDF. We call this sequence of steps one iteration, and we repeat until convergence, as represented in Fig. 1. III. ERGODIC CONTROL The motion of the system is planned in order to achieve ergodicity, i.e., the trajectory of the system is computed such that the time spent in a region of the search space is proportional to the utility of that region. The goal is to minimize the distance from ergodicity between the time average trajectory and the spatial reference PDF that represents possible defects —and where we could find more useful measurements. For that, we use the classic Spectral Multi-scale Coverage (SMC) approach described in [2]. 2025 I-RIM Conference October 17-19, Rome, Italy ISBN: 9788894580570 10.5281/zenodo.17629732 107 Given the robot state trajectories q(t), control actions uq(t), dynamics ˙ q(t)=f(q,uq)and end-effector/sensor trajectories Xe(t)=g(q(t)) given by the kinematic equations. We consider the dynamics of the end-effector/sensor as a secondorder system ¨ Xe(t)=u(t)with u(t)the control actions in task space, and define the formulation of the ergodic trajectory optimization problem as the following minimization problem over state and control trajectories Xe(t),u(t)with a given time horizon tf: min Xe(t),u(t)γε ˆ Φ (x),Xe(t)+Ztf t0 u⊺(τ)Ru(τ)dτ, (1) where γ∈Rand R∈Rn×nare design parameters that penalize ergodic cost εˆ Φ (x),Xe(t)and control efforts u(t), respectively. IV. PDF ESTIMATION We start the whole process by defining a first belief of the defect locations on a 2-D search space. Then, the ergodic control is executed to get the optimal trajectory —in terms of ergodicity and control efforts— and collect measurements, which are used to update the reference PDF ˆ Φ(x)via the GMM algorithm and a KL divergence-dependent constraint. The algorithm is split into two stages, namely: the exploration stage, where the KL divergence-dependent variation constraint is used to determine how many possible defects we have in Ωand collect enough information about every defect to have an idea of the position and the shape; and the exploitation stage, where we focus on exploiting the defects we already discovered to provide a better estimate until a metric threshold that gives an idea of the overall space occupied by every estimated component has been achieved. The objective here is to find the posterior probability map ˆ Φ(x)with ndpossible defects that serve as reference for the trajectory planner (ergodic control) in the next iteration (m+1) as: ˆ Φm+1(x) = nd X i=1 iˆπNiˆµ, iˆ Σ.(2) Where iˆµis the estimated position of the i-th defect, iˆ Σis the estimated shape, and iˆπare the weights on every Gaussian. We consider the termination condition for the i-th defect: iˆ ζ(m+1) ≤Γζ,(3) where iˆ ζ(m+1) stands for the sum of the semi-axis lengths of the i-th component in ˆ Φm+1(x). Once the i-th Gaussian satisfies (3), we save it as a “found” defect and delete it from the posterior PDF, in order to avoid wasting time visiting that area in the next iterations. The algorithm continues until every defect satisfies (3), or until a maximum number of iterations has been reached. Fig. 2. Evolution of the estimation: In the top-left chart is shown the real defects represented as a probability map which we don’t know and we want to estimate Φ(x); the next charts show the trajectory of the robot (solid black line) given the estimated PDF ˆ Φm(x)for iteration m, notice that we start iteration 1 with a Uniform distribution, red ellipsoids represent the real and unknown defects. Fig. 3. Final result after 5 iterations of 10 seconds each. V. SIMULATION RESULTS AND CONCLUSIONS The estimation process across iterations is shown in Fig. 2; here, one can see that by iteration 4 one of the defects has already reached the condition (3) and therefore that Gaussian component is no longer considered in iteration 5. In this last iteration, the remaining two defects also reach the termination condition. The final result is shown in Fig. 3. The proposed algorithm has been tested for different numbers of defects; furthermore, false positives are mitigated due to its probabilistic nature and double-checking behavior. However, in real-world implementations, the modeling of the tactile sensor and the pre-processing of the data are fundamental aspects for the algorithm to work and should be done carefully. Future work should explore the extension of the problem to 3-D surfaces and consider its conjunction with a visual approach. REFERENCES [1] A. Agarwal, A. Ajith, C. Wen, V. Stryzheus, B. Miller, M. Chen, M. K. Johnson, J. L. Susa Rincon, J. Rosca, and W. Yuan, “Robotic defect inspection with visual and tactile perception for large-scale components,” in 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2023, pp. 10 110–10 116. [2] G. Mathew and I. Mezi´ c, “Metrics for ergodicity and design of ergodic dynamics for multi-agent systems,” Physica D: Nonlinear Phenomena, vol. 240, no. 4, pp. 432–442, 2011. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S016727891000285X [3] L. M. Miller and T. D. Murphey, “Trajectory optimization for continuous ergodic exploration,” in 2013 American Control Conference, 2013, pp. 4196–4201. [4] L. M. Miller, Y. Silverman, M. A. MacIver, and T. D. Murphey, “Ergodic exploration of distributed information,” IEEE Transactions on Robotics, vol. 32, no. 1, pp. 36–52, 2016. 108