StoManager1: Automated, High-throughput Tool to Measure Leaf Stomata Using Convolutional Neural Networks

Copyright © Jiaxin Wang. For questions and requests, please email: [email protected]; [email protected] Manager1StoMa Automated, High-throughput Tool to Measure Leaf Stomata Using Convolutional Neural Networks MANUAL StoManager1 v.1.0.0 Wang, J., Renninger, H. J., Ma, Q., & Jin, S. (2024). Measuring stomatal and guard cell metrics for plant physiology and growth using StoManager1. Plant Physiology, 195(1), 378-394. https://doi.org/10.1093/plphys/kiae049 StoManager1: Automated, High-throughput Tool to Measure Leaf Stomata Using Convolutional Neural Networks What is StoManager1? What can StoManager1 do? How to use it? It is a tool created to automatically detect, count, measure, and analyze plant leaf stomatal metrics using geometrical, mathematical, and deep learning algorithms. Measured stomatal metrics include commonly used and newly developed, such as stomatal area, guard cell area, guard cell length, width, ratio of stomatal pore/guard cell area, and stomatal arrangement indices (aggregation, evenness, and divergence). The main function of StoManager1 is for “stomata” and “whole_stomata” detection, segmentation, measurement, and analysis using a trained YOLOv8-seg-x model. However, using empirical algorithms, StoManager1 also integrates bounding box-based YOLOv3 models for detection, counting, and stomatal metrics inference. StoManager1 can be used as a pre-trained automatic stomatal labeling tool using its generated labels. StoManager1 can be used as a YOLOv8 model trainer by using its integrated module-- model_training_in_app. exe. StoManager1 can be used through Command line-GUI (https://github.com/JiaxinWang123/StoManager1) and a standalone Windows executable app (https://zenodo.org/doi/10.5281/zenodo.7686022). R G B StoManager1 workflow Load weights & configration Image path Normalize file names Draw bounding boxes& masks Calculate detected stomatal metrics Output confidence, coordinates of boxes & masks & stomatal metrics Group statistics Image resolution Confidence threshold Pooling (down sampling feature inputs) Convolutional layer 2 Convolutional layer 1 Input Pooling ... Flatten Sigmoid a b c d e f g (Xi, Yi)(Xi, Yi) Segement and obtain the border coordinates Step 1 (0,0) Xi Y Y Xi (0,0) List all the vertices in clockwise or counterclockwise order Step 2 Find the edges and draw a min-area-rotated rectangle Step 3 width normal bounding box minimum-arearotated rectangle length Find the centroids of stomata and whole_stomata to match and get the guard cells Step 5 Step 4 Calculate the angle of the min-area-rotated rectangle >90° =90° <90° =180° guard cell width guard cell length Before using, please read below notes: -- Note: to change any input parameters such as resolution, confidence threshold, and training hyperparameters, you need to clean the input editline and just type the numeric values. -- How to determine your image’s resolution in pixels/0.1mm? You can determine it by measuring the total pixels between 0.1 mm on a Microscope Stage Calibration Slide under the microscope with same configuration and export settings (exactly same as how you measure and export your images). Once you get your measure of the Microscope Stage Calibration Slide, you can count the pixels in ImageJ or other software. Euphorbia celastroides Vernicia fordii Catha edulis Euonymus americanus Tripterygium wilfordii Celtis laevigata Amelanchier arborea Crataegus viridis Aronia arbutifolia Osteomeles anthyllidifolia Gynostemma pentaphyllum Betula papyrifera Carpinus caroliniana Morella cerifera Carya glabra Glycyrrhiza glabra Erythrina sandwicensis Senna italica Zanthoxylum americanum Pilocarpus jaborandi Bergera koenigii Citrus hystrix Aesculus parviflora Edgeworthia chrysantha Hibiscus clayi Alyogyne huegelii Hibiscadelphus distans Eucalyptus globulus Heimia salicifolia Fothergilla major Itea virginica Digitalis purpurea Myoporum sandwicense Diplacus aurantiacus Salvia guaranitica Melissa officinalis Pycnanthemum muticum Prunella vulgaris Andrographis paniculata Jasminum sambac Olea europaea Fraxinus pennsylvanica Bonamia menziesii Atropa belladonna Psydrax odorata Cephalanthus occidentalis Morinda citrifolia Catharanthus roseus Asclepias tuberosa Stephanotis floribunda Artemisia annua Arnica chamissonis Lipochaeta succulenta Delissea rhytidosperma Centella asiatica Eleutherococcus senticosus Pittosporum hosmeri Viburnum acerifolium Ilex cornuta Sideroxylon lycioides Zenobia pulverulenta Kalmia latifolia Stewartia malacodendron Cornus alba Camptotheca acuminata Charpentiera elliptica Heliconia angusta Carex platyphylla Vanilla planifolia Ruscus aculeatus Cordyline fruticosa Amorphophallus titanum Acorus gramineus Magnolia virginiana Persea borbonia Sassafras albidum Lindera benzoin Laurus nobilis Cinnamomum camphora Peperomia tetraphylla Piper auritum Warburgia salutaris Illicium floridanum Pinus strobus Athyrium filix−femina Sadleria cyatheoides Polystichum acrostichoides 20 -- StoManager1 is not limited to tree leaves, and we have tested it on other non-woody species, and it worked well. However, if you find it does not work well for your image, you can use the training module to train a model for your own data set. The detailed steps of preparing, training, and replacing model weights have been described in this manual. -- The current standalone software can only run on Windows platform, but you can use the source Python code if you want to run it on Mac. Using StoManager1 for stomatal metrics measuring: Step 1: Define the Input folder for images to be measured. Step 2: Check or uncheck the YOLOv8-seg-x model. If checked, the YOLOv8-seg-x model will be used, and more stomatal metrics will be measured based on segmentation results. If unchecked, the bounding-box model and empirical algorithms will be used to estimate stomatal metrics (fewer stomatal metrics will be measured compared with the segment model). Step 3: Define parameters for stomatal metrics measurement based on your image resolution and detection confidence threshold. Step 4: Press the “Start Process” button to start the primary function—detection, segmentation, and measurement. Step 5: Press the “Statistical Analysis” button to start the analysis. Step 6: Preview your exported results. Analyzed results will be saved to your output folder and subfolders, depending on which model you used. If you checked the YOLOv8-seg-x model, your analyzed results can be found in “YOUR_OUTPUT_PATH/Predict-output/Output_csv.” Under the Output_csv folder, you will find a CSV file and folder named using your image file name. If you are unchecked the YOLOv8-seg-x model, your analyzed results can be found in “YOUR_OUTPUT_ PATH”. Under YOUR_OUTPUT_PATH, you will find one CSV, JPG, and TXT file, all named using your image file name. Supported image formats including ‘jpg’, ‘png’, ‘tif’, and ‘jpeg’. Using StoManager1 for your own model training using your custom dataset: You need to get your image dataset and corresponding object labels to train YOLO models. Some excellent image labeling tools are available for free, such as LabelImg (https://github.com/HumanSignal/labelImg) and Roboflow online platform (https://roboflow.com/), which you can use to label your image. I strongly recommend Roboflow since you can manage your image dataset easily, such as splitting training, validation, and testing datasets. Once you get your labeled image dataset, you will create data.yaml file for dataloader. If you use Roboflow, it will automatically create data.yaml file. You might need to edit the relative path to the absolute path to make sure StoManager1 can find your dataset. For example: Change: train: ../train/images val: ../valid/images test: ../test/images to: train: D:/YOLOV8/leaf_stomata.v8i.yolov8/train/images val: D:/YOLOV8/leaf_stomata.v8i.yolov8/valid/images test: D:/YOLOV8/leaf_stomata.v8i.yolov8/test/images Now, you can start your own model! To start the model trainer, click “Training” menu and “Train YOLOv8-seg-x” option in the main window. Then you will see the training window like below: Then you can select your “data.yaml” file and “model_training_in_app.exe” file. Please note that “model_training_in_app.exe” file can be downloaded from the Zenodo page ((https://zenodo.org/doi/10.5281/zenodo.7686022), and it must work with its _internal folder, which means you cannot delete _internal folder. Then, you can define your training parameters. You can leave them as default if you don’t know what to set. Otherwise, you can consult YOLOv8 model training documentation by clicking the “Train YOLOv8-seg-x model” option under the “Training” menu in the Trainer window. Note that batch size cannot be set too high if your computer does not have a powerful GPU or you are training a large image dataset with a single image with over 100 objects. Once you set all parameters, you can press the “Start Training” button to start your training. If it is your first training, you must have internet access to download pre-trained weights-- “YOLOv8-seg-x.pt”, or you can manually download and save it into where the “model_training_in_app.exe” file is located. You can monitor the training process. The training will automatically stop once the best fitting model is obtained based on the precision, recall, mAP@50-95, and/or training loss. You can find more information about the YOLOv8 model training metrics online. Your training results will be saved in the folder of your StoManager1 app under the “runs/segment/train” subfolder, and you can find the training weights in the “weights” folder and other training and validation metrics plots in pdf and results data in CSV. You may encounter a Traceback error like the one below (it’s because “tqdm” is not working correctly under subprocess; it’s function is to estimate the training time for each epoch) if you stop the training manually, and you can ignore it since it will not affect your training process and results. Once you have done your measurement, you can use the following code to extract the label files and rename them as same with the image files. Once you have finished training your custom model, you can find, copy, and replace the “best.pt” file in the StoManager1 folder. Note, please make a copy of the original “best.pt” file, so that you can reuse it later. Then you can open StoManager1 and test how your custom model performs on your own dataset. Your labels will be saved in your “Output_csv” path, and you can then import those labels and corresponding images into Roboflow for label quality check and modification as needed. This process is beneficial and can save you a lot of time if you have particular stomatal images and need to label a large dataset. I hope you will find StoManager1 helpful for your research, and please let me know if you have any questions or requests regarding StoManager1. ♥ Using StoManager1 as an automated labeling tool. import os import os.path path = “C:\Users\YOUR_PATH\F1_training_data\Output\Predict_output\Output_csv” for dirpath, dirnames, filenames in os.walk(path): for filename in [f for f in filenames if f.endswith(“.txt”)]: labelFile = os.path.join(dirpath, filename) dir = os.path.dirname(os.path.dirname(labelFile)) ## dir of dir of file ## once you’re at the directory level you want, with the desired directory as the final path node: dirname1 = os.path.basename(dir) os.rename(labelFile, f”{dirname1}.txt”)