Automated image-based analysis unveils acute effects due to sub-lethal pesticide doses exposure

Automated image-based analysis unveils acute effects due to sub-lethal pesticide doses exposure* Gianluca Manduca1, Valeria Zeni2, Sara Moccia1, Giovanni Benelli2, Angelo Canale2, Cesare Stefanini1and Donato Romano1 Abstract— Pesticides are still abused in modern agriculture. The effects of their exposure to even sub-lethal doses can be detrimental to ecosystem stability and human health. This work aims to validate the use of machine learning techniques for recognizing motor abnormalities and to assess any effect post-exposure to a minimal dosage of these substances on a model organism, gaining insights into potential risks for human health. The test subject was the Mediterranean fruit fly, Ceratitis capitata (Wiedemann) (Diptera: Tephritidae), exposed to food contaminated with the LC30 of Carlina acaulis essential oil. A deep learning approach enabled the pose estimation within an arena. Statistical analysis highlighted the most significant features between treated and untreated groups. Based on this analysis, two learning-based algorithms, Random Forest (RF) and XGBoost were employed. The results were compared through different metrics. RF algorithm generated a model capable of distinguishing treated subjects with an area under the receiver operating characteristic curve of 0.75 and an accuracy of 0.71. Through an image-based analysis, this study revealed acute effects due to minimal pesticide doses. So, even small amounts of these biocides drifted far from distribution areas may negatively affect the environment and humans. I. INTRODUCTION Poor management of ecosystems can have feedback effects on animal and human health. EcoHealth and One Health are paradigms that investigate the health of living beings in the context in which they live, promoting interdisciplinary research [1], [2]. Pesticides represent an important amount of chemical pollutants that can affect the ecosystem, nontarget organisms, and human health [3]. The impact of pesticides on animals and humans shows effects on the vestibular and auditory systems [4], [5], but also possible risks for the immune system [6]. In addition to biochemical and neurophysiological analyses, motor analysis is also a valid approach to assess the effects of pesticide exposure [7], [8]. Studying the toxicological impact of pesticides on insects makes it possible to assess potential risks *This research was carried out in the framework of the EU H2020 FETOPEN Project ‘Robocoenosis - ROBOts in cooperation with a bioCOENOSIS’ [899520]. Funders had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. 1Gianluca Manduca, Sara Moccia, Cesare Stefanini, and Donato Romano are with The BioRobotics Institute and The Department of Excellence in Robotics and AI, Scuola Superiore Sant’Anna, Viale R. Piaggio 34, 56025 Pontedera (PI), Italy, {gianluca.manduca, sara.moccia, cesare.stefanini, donato.romano}@santannapisa.it 2Valeria Zeni, Giovanni Benelli, and Angelo Canale are with The Department of Agriculture, Food and Environment, University of Pisa, Via del Borghetto 80, 856124 Pisa (PI), Italy, [email protected], {giovanni.benelli, angelo.canale}@unipi.it to human health as physiological mechanisms that underlie pesticide toxicity are often similar with humans [9]. Insects thus represent a model system, accessible and simple. Different approaches are used for data collection and analysis to investigate motor abnormalities due to pesticide exposure. Radio Frequency Identification (RFID) tags are a valuable approach for tracking model organisms due to the possibility of analysing a large number of test subjects simultaneously [10]. However, physical marker systems can potentially alter subject behaviour [11]. Automatic videotracking systems are an alternative to physical markers. These cost-effective and versatile solutions were widely used in this context [12]–[14]. However, motor analysis is limited to the study of the trajectory travelled. Another alternative is the pose estimation, i.e., the geometric configuration of the different parts of the test subject’s body, which can provide a complex and detailed motor analysis. Deep learning algorithms have made a significant contribution in terms of accuracy and process automation for pose estimation purposes [11]. However, they are still not used in this context. After data collection, statistical analysis is the most commonly used approach to identify significant data from motor analysis that can testify to the effects of pesticides [7], [10], [12], [13]. Alternatively, mathematical and fractal methods are helpful tools [14]. Machine learning (ML) techniques have been employed in this context for genetics analysis [15] or for crop pest classification [16]. Here we investigated whether ML techniques can detect any motor effects from exposure to low pesticide doses on a model organism, the Mediterranean fruit fly (medfly), Ceratitis capitata (Wiedemann) (Diptera: Tephritidae). Besides being a polyphagous fruit pest, the medfly is a model organism for biology and behavioural research. To the best of our knowledge, the impact of pesticide exposure on the locomotion behaviour of this organism has been poorly investigated. Following recent research achievements, herein we exposed C. capitata adults with a food contaminated by a LC30 of Carlina acaulis essential oil (EO), a bioactive insecticide of botanical origin [17], [18]. Medflies were isolated into an arena. Motor disorders were analysed from video recordings of treated and untreated specimens. We exploited an online software based on deep learning for pose estimation. The data collected from video analysis were used to train and test Random Forest (RF) and XGBoost algorithms to recognise treated subjects. The performance of learning-based classifiers allowed us to quantify and interpret toxicological effects after pesticide exposure (Fig. 1). 979-8-3503-2447-1/23/$31.00 ©2023 IEEE 2023 45th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) | 979-8-3503-2447-1/23/$31.00 ©2023 IEEE | DOI: 10.1109/EMBC40787.2023.10340800 Authorized licensed use limited to: Scuola Superio Sant'Anna di Pisa. Downloaded on December 17,2025 at 13:10:09 UTC from IEEE Xplore. Restrictions apply. Fig. 1. Workflow of the proposed approach. II. METHODS A. Insect Rearing and Treatments Medflies used for experiments were reared as described by Canale & Benelli [19]. Mass-rearing conditions were 25 ± 1 °C, 45 % R.H., and 16:8 (L:D). Adult medflies (both sexes) used in bioassays were 10–12 days old. We aimed to determine whether a low dose of Carlina acaulis EO altered the locomotory and grooming behavior of medfly adults. Following Benelli et al. [17], we administered the EO LC30 dose (716 ppm) and the corresponding control through ingestion. The EO was formulated in a mucilagineous solution containing sucrose and hydrolized protein. B. Experimental and Video Recording Procedures Behavioural studies were carried out in a circular arena (Ø = 5 cm). A video camera (12MP, ƒ/2.8) was placed over the arena to record medfly locomotory and grooming activities. To better investigate the non-aerial behaviours, including walking and grooming, medflies were not allowed to fly. To do this, the height of the arena was adapted to the size of the tested insect. Three circular rings (Ø = 5 mm) with different heights (h= 2.7, 3.5, and 4.5 mm) were fast-prototyped in Anycubic 3D Printing UV Sensitive Resin. Videos were recorded at 30 fps with a resolution of 1920x1080 pixels. A total of 152 videos with a duration of 5 minutes were realised. Before testing, medflies were acclimated in a Petri dish (Ø = 5 cm) for 1 minute. Treated insects were kept separate from control ones before the experiment. The overall videos were balanced between treated and control specimens and by gender. All recordings occurred between 11:00 and 17:00 to reduce the effect of circadian rhythms, at 25 ± 1°C. C. Pose Estimation We performed a semi-automatic annotation of the insect’s pose at frame level using DeepLabCut (DLC), an online software for pose estimation based on deep learning [11]. The procedure and parameters’ choice followed the guidelines in literature [20]. A total of 1157 images were extracted from five videos to train a convolutional neural network (CNN) to recognize key points (KPs) on the insect’s body. The training set images were labeled manually via the user graphic interface provided by the DLC software. Five KPs were annotated: the head, thorax, abdomen, and two wings (Fig. 1). The number of images was balanced by the size and gender of the insect. We used a pre-trained ResNet50 backbone using ImageNet weights and we fine-tuned the network on 600,000 iterations of the training set, with the batch size set to one. We used the pre-trained model to annotate an additional 1,366,843 images from 152 videos. D. Features Extrapolation Various temporal data concerning the medflies’ movement with possible behavioural relevance were obtained by video analysis. We investigated the insect’s distance travelled, speed, positive acceleration, and deceleration through the medfly thorax tracking. The times when medfly stopped were analysed (stop duration). The overall time the insect was motionless was also considered (overall motionless duration) as the sum of the various stop durations. The wings’ elongation was analysed during tests. As the insect size varies among individuals, the elongation was normalised to the maximum extension measured during the experiment. The skeleton joining the thorax and abdomen points allowed the insect’s orientation and extrapolation of the rotation angles. The angles travelled clockwise and counterclockwise were considered separately, and the case in which the insect rotated on itself when stationary was distinguished from when it was in motion. Temporal data were then converted into features considering the median, variance, maximum absolute value, and number of peaks. The data obtained from DLC were processed, and features were then extrapolated using Matlab (MathWorks Inc., MA) software. E. Statistical Analysis A Shapiro-Wilk test was used for each feature to verify for normal distribution (p= 0.01). For trials with two conditions (e.g., LC30–exposed flies vs. control), statistical significance was established using the t-test for normally distributed features and the Wilcoxon test for non-normally distributed ones. Statistical analyses were performed using JMP Pro 16 (SAS) software. The threshold was set at p= 0.05. F. Classifier Based on the statistical analysis results, we selected the features to train RF and XGBoost models to unveil pesticide effects [21], [22]. The choice of standard treebased classifiers allows the results to be interpretable. We used grid search 4-fold cross-validation for hyperparameters tuning. The hyperparameters tuned were the number of tree Authorized licensed use limited to: Scuola Superio Sant'Anna di Pisa. Downloaded on December 17,2025 at 13:10:09 UTC from IEEE Xplore. Restrictions apply. Fig. 2. ROC curves for (a) Random Forest and (b) XGBoost. Feature importance for Random Forest is shown in (c). estimators and the maximum tree depth with a grid-search space of [20, 50, 100, 150, 200, 250, 300, 350, 400, 450, 500] and [3–7], respectively. We iterated the process with a 4-fold cross-validation for robust testing. In each iteration, 114 subjects were used for training and 38 for testing. All the analyses were performed using Scikit-learn in Python. G. Performance Assessment For assessing classification performance and compare the algorithms, we considered precision, recall, accuracy, and F1-score. The resulting receiver operating characteristic (ROC) curves and the relative area under the curve (AUC) were also evaluated. The features’ importance during algorithms’ training was assessed in terms of the mean decrease in impurity. III. RESULTS Statistical analysis revealed 19 features with significance between treated and control specimens (Table I). The same features were then used to train and test the classifiers. Table II compares RF and XGBoost results in terms of precision, recall, accuracy and F1-score. As a 4-fold crossvalidation was used for testing the models, we expressed the four results in terms of median and inter-quartile range. Fig. 2(a) and 2(b) show the resulting ROC curves. From the classifiers comparison, it can be seen that RF performs better in distinguishing the two groups with an area under the mean ROC curve of 0.75 and an overall model accuracy of 0.71. Fig. 2(c) thus shows the feature relevance obtained with RF. IV. DISCUSSION This work proposes an automated image-based approach to detect and evaluate the acute effects of exposure to sublethal doses of a pesticide on adult medflies, selected as a model organism to assess potential risks to non-target species, including humans. Several studies have evaluated the effects of pesticides on insects through biochemical and neurophysiological analyses [23]–[25]. In our study, motor displays of medfly individuals treated with sub-lethal doses of C. acaulis EO were TABLE I RELEVANT FEATURES FROM STATISTICAL ANALYSIS. Feature Number χ2p<0.05 Overall motionless duration [s] 1 5.0362 0.0248 Variance [s] 2 5.8257 0.0158 Stop duration Maximum [s] 3 8.1665 0.0043 Median [mm/s] 4 6.9224 0.0085 Speed Number of peaks [] 5 4.7890 0.0286 Acceleration Number of peaks [] 6 7.1453 0.0075 Median [mm/s2]7 4.4583 0.0347 Deceleration Number of peaks [] 8 4.8124 0.0283 Left wing normalized elongation (Ll)Median [0-1] 9 10.0196 0.0015 Median [0-1] 10 8.8213 0.0030 Right wing normalized elongation (Lr)Variance [0-1] 11 12.2549 0.0005 Median [0-1] 12 6.3901 0.0115 Variance [0-1] 13 3.9596 0.0466 Module of the left and right wing normalized elongation difference (|Ll−Lr|)Number of peaks [] 14 5.7693 0.0163 Median [rad] 15 6.1320 0.0133 Clockwise Number of peaks [] 16 6.3647 0.0116 Median [rad] 17 4.5365 0.0332 Angle of rotation on itself when stationary Counterclockwise Number of peaks [] 18 6.2993 0.0121 Angle of rotation when travelling Clockwise Number of peaks [] 19 5.3414 0.0208 TABLE II CLASSIFICATION RESULTS FOR THE CONSIDERED ML CLASSIFIERS. MEDIAN (INTER-QUARTILE RANGE)ARE REPORTED. Algorithm Precision Recall Accuracy F1-score Random Forest 0.70 (0.02) 0.74 (0.00) 0.71 (0.01) 0.72 (0.01) XGBoost 0.63 (0.05) 0.66 (0.08) 0.65 (0.06) 0.66 (0.07) analysed through ML techniques. Behavioural alterations due to pesticide treatment have been previously investigated in bees through RFID technology and microchip-equipped specimens [10]. In [12], an automated video-tracking system, EthoVisionXT, was used to monitor the activity of the treated specimens. The presence of food and the interaction between test subjects was considered during the tests. The temporal data obtained were analysed using statistics. Three variables were considered: distance travelled, time in food zone, and interaction time. The same software and methodology were applied in [13], where motor abnormalities were studied in larval amphibians exposed to a pesticide. Tenorio et al. [14] used a video-tracking approach with Image J 1.49v software to investigate the locomotion of shrimps by applying mathematical and fractal methods. Authorized licensed use limited to: Scuola Superio Sant'Anna di Pisa. Downloaded on December 17,2025 at 13:10:09 UTC from IEEE Xplore. Restrictions apply. Herein, we proposed a different, image-based deep learning approach for the pose estimation of medflies. This strategy is less invasive than physical markers [11]. The specimens were isolated during the experiments, and no external factors were considered. The effects on insect locomotion we investigated were difficult to detect by a human operator in a traditional toxicological study. The pose estimation enabled a detailed and complex motor analysis. Statistical analysis revealed 19 significant features. The most significant features concern wings elongation. Based on the relevant features as proposed in [21], ML algorithms made it possible to highlight motor abnormalities due to the administration of pesticides. The performances of two decision tree classifiers were compared using different metrics. RF algorithm better distinguished treated specimens with an overall model accuracy of 0.71. Classification results showed toxicological effects even in the short period of time analysed. The treebased approach allowed the interpretation of the results. The relevance of the features used during the algorithm’s training confirmed the statistical analysis results, emphasising the importance of wing movement in distinguishing treated from control specimens. Overall, this strategy based on automated image analysis revealed effects due to minimal doses of a pesticide. The integration of statistical analysis and machine learners allows to quantify effects and interpret results. This work is intended as a warning against the excessive use of pesticides, showing acute effects even at low doses. Future work will integrate classification techniques, based on different strategies, for a broader comparison. Sex differences in response to this minimal pesticide dosage will also be investigated. REFERENCES [1] S. Harrison, L. Kivuti-Bitok, A. Macmillan, and P. Priest, “Ecohealth and one health: A theory-focused review in response to calls for convergence,” Environment International, vol. 132, p. 105058, 2019. [2] G. Benelli and M. F. Duggan, “Management of arthropod vector data– social and ecological dynamics facing the one health perspective,” Acta Tropica, vol. 182, pp. 80–91, 2018. [3] I. Mahmood, S. R. Imadi, K. Shazadi, A. Gul, and K. R. Hakeem, “Effects of pesticides on environment,” in Plant, Soil and Microbes. Springer, 2016, pp. 253–269. [4] L. A. Cogo, V. A. V. d. Santos Filha, A. d. A. B. Murashima, M. A. Hyppolito, and A. F. d. Silveira, “Morphological analysis of the vestibular system of guinea pigs poisoned by organophosphate,” Brazilian Journal of Otorhinolaryngology, vol. 82, pp. 11–16, 2016. [5] J. Mac Crawford, J. A. Hoppin, M. C. Alavanja, A. Blair, D. P. Sandler, and F. Kamel, “Hearing loss among licensed pesticide applicators in the agricultural health study running title: hearing loss among licensed pesticide applicators,” Journal of Occupational and Environmental Medicine/American College of Occupational and Environmental Medicine, vol. 50, no. 7, p. 817, 2008. [6] E. Corsini, J. Liesivuori, T. Vergieva, H. Van Loveren, and C. Colosio, “Effects of pesticide exposure on the human immune system,” Human & Experimental Toxicology, vol. 27, no. 9, pp. 671–680, 2008. [7] S. Tosi and J. Nieh, “A common neonicotinoid pesticide, thiamethoxam, alters honey bee activity, motor functions, and movement to light,” Scientific Reports, vol. 7, no. 1, pp. 1–13, 2017. [8] B. G´ omez-Gim´ enez, V. Felipo, A. Cabrera-Pastor, A. Agust´ ı, V. Hern´ andez-Rabaza, and M. Llansola, “Developmental exposure to pesticides alters motor activity and coordination in rats: sex differences and underlying mechanisms,” Neurotoxicity Research, vol. 33, no. 2, pp. 247–258, 2018. [9] M. J. Klowden, Physiological systems in insects. Academic Press, 2013. [10] A. Decourtye, J. Devillers, P. Aupinel, F. Brun, C. Bagnis, J. Fourrier, and M. Gauthier, “Honeybee tracking with microchips: a new methodology to measure the effects of pesticides,” Ecotoxicology, vol. 20, no. 2, pp. 429–437, 2011. [11] A. Mathis, P. Mamidanna, K. M. Cury, T. Abe, V. N. Murthy, M. W. Mathis, and M. Bethge, “Deeplabcut: markerless pose estimation of user-defined body parts with deep learning,” Nature Neuroscience, vol. 21, no. 9, pp. 1281–1289, 2018. [12] B. S. Teeters, R. M. Johnson, M. D. Ellis, and B. D. Siegfried, “Using video-tracking to assess sublethal effects of pesticides on honey bees (apis mellifera l.),” Environmental Toxicology and Chemistry, vol. 31, no. 6, pp. 1349–1354, 2012. [13] M. Deno¨ el, S. Libon, P. Kestemont, C. Brasseur, J.-F. Focant, and E. De Pauw, “Effects of a sublethal pesticide exposure on locomotor behavior: a video-tracking analysis in larval amphibians,” Chemosphere, vol. 90, no. 3, pp. 945–951, 2013. [14] B. M. Tenorio, E. A. da Silva Filho, G. S. M. Neiva, V. A. da Silva, F. d. C. A. M. Tenorio, T. d. J. da Silva, E. C. S. e Silva, and R. de Albuquerque Nogueira, “Can fractal methods applied to video tracking detect the effects of deltamethrin pesticide or mercury on the locomotion behavior of shrimps?” Ecotoxicology and Environmental Safety, vol. 142, pp. 243–249, 2017. [15] J. S. Tomiazzi, M. A. Judai, G. A. Nai, D. R. Pereira, P. A. Antunes, and A. P. A. Favareto, “Evaluation of genotoxic effects in brazilian agricultural workers exposed to pesticides and cigarette smoke using machine-learning algorithms,” Environmental Science and Pollution Research, vol. 25, no. 2, pp. 1259–1269, 2018. [16] K. Thenmozhi and U. S. Reddy, “Crop pest classification based on deep convolutional neural network and transfer learning,” Computers and Electronics in Agriculture, vol. 164, p. 104906, 2019. [17] G. Benelli, R. Rizzo, V. Zeni, A. Govigli, A. Samkov´ a, M. Sinacori, G. L. Verde, R. Pavela, L. Cappellacci, R. Petrelli et al., “Carlina acaulis and Trachyspermum ammi essential oils formulated in protein baits are highly toxic and reduce aggressiveness in the medfly, Ceratitis capitata,” Industrial Crops and Products, vol. 161, p. 113191, 2021. [18] E. Spinozzi, M. Ferrati, L. Cappellacci, A. Caselli, D. R. Perinelli, G. Bonacucina, F. Maggi, M. Strzemski, R. Petrelli, R. Pavela et al., “Carlina acaulis l.(asteraceae): biology, phytochemistry, and application as a promising source of effective green insecticides and acaricides,” Industrial Crops and Products, vol. 192, p. 116076, 2023. [19] A. Canale and G. Benelli, “Impact of mass-rearing on the host seeking behaviour and parasitism by the fruit fly parasitoid Psyttalia concolor (sz´ epligeti)(hymenoptera: Braconidae),” Journal of Pest Science, vol. 85, no. 1, pp. 65–74, 2012. [20] T. Nath, A. Mathis, A. C. Chen, A. Patel, M. Bethge, and M. W. Mathis, “Using deeplabcut for 3d markerless pose estimation across species and behaviors,” Nature Protocols, vol. 14, no. 7, pp. 2152– 2176, 2019. [21] E. Ambrosini, M. Caielli, M. Milis, C. Loizou, D. Azzolino, S. Damanti, L. Bertagnoli, M. Cesari, S. Moccia, M. Cid et al., “Automatic speech analysis to early detect functional cognitive decline in elderly population,” in 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, 2019, pp. 212–216. [22] M. Salati, L. Migliorelli, S. Moccia, M. Andolfi, A. Roncon, G. M. Guiducci, F. Xium` e, M. Tiberi, E. Frontoni, and M. Refai, “A machine learning approach for postoperative outcome prediction: Surgical data science application in a thoracic surgery setting,” World Journal of Surgery, vol. 45, no. 5, pp. 1585–1594, 2021. [23] C. Papaefthimiou and G. Theophilidis, “The cardiotoxic action of the pyrethroid insecticide deltamethrin, the azole fungicide prochloraz, and their synergy on the semi-isolated heart of the bee Apis mellifera macedonica,” Pesticide Biochemistry and Physiology, vol. 69, no. 2, pp. 77–91, 2001. [24] E. Pilling, K. Bromleychallenor, C. Walker, and P. Jepson, “Mechanism of synergism between the pyrethroid insecticide λ-cyhalothrin and the imidazole fungicide prochloraz, in the honeybee (Apis mellifera l.),” Pesticide Biochemistry and Physiology, vol. 51, no. 1, pp. 1–11, 1995. [25] S. Rumpf, F. Hetzel, and C. Frampton, “Lacewings (neuroptera: Hemerobiidae and chrysopidae) and integrated pest management: enzyme activity as biomarker of sublethal insecticide exposure,” Journal of Economic Entomology, vol. 90, no. 1, pp. 102–108, 1997. Authorized licensed use limited to: Scuola Superio Sant'Anna di Pisa. Downloaded on December 17,2025 at 13:10:09 UTC from IEEE Xplore. Restrictions apply.