Accessible AI Enhances Monitoring of Coral Seeding Devices in Reef Restoration

1 Monitoring Coral Outplant Devices in TagLab The following protocol details a workflow for monitoring Coral Seeding Devices (CSDs) within orthoimages. The protocol leverages computer vison AI and provides instructions for the creation and use of automatic classifiers trained to detect CSDs. Once detected, the location of each CSD can be revisited in future orthoimages to enable monitoring of the growth, condition and survival of any coral colonies growing from CSDs. The protocol is conducted in the free and open-source annotation software TagLab and is dependent on the prior construction of orthoimages of CSD deployment sites. The specific instructions and settings documented in the protocol are valid for TagLab version v2025-8-27. Future updates to TagLab may necessitate adaptation of some steps of the protocol by users. The latest information about TagLab and its features can be found on the TagLab webpage: https://taglab.isti.cnr.it/docs#learning. This protocol was developed collaboratively by researchers with backgrounds in reef ecology, reef restoration and AI application at the Australian Institute of Marine Science (AIMS), University College London, Newcastle University, and the Palau International Coral Reef Center. 2 Images are provided to help to illustrate some instructions. icons indicate where an image is provided and clicking on an icon will navigate to the relevant image. All images are also labelled and available within the Images section at the end of the document. Contents: Graphical Overview of the protocol 3 Preparation & field considerations 4 1. Importing orthoimages to TagLab 5 2. Creating training data 7 3. Training a classifier 9 4. Running a classifier 10 5. QA/QC and exporting annotations 11 6. Aligning timeseries orthoimages 13 7. Importing annotations into future orthoimages 15 8. Updating past annotations 16 9. Monitoring CSDs through time 18 Images 19 3 Graphical overview of a protocol detailing how classifiers can be used to monitor deployments of coral seeding devices (CSDs). Extensive step-by-step instructions detailing how every step in this protocol can be conducted within the annotation software TabLab are provided below. 4 Preparation & field considerations A1 A CSD that is distinctive and easy to see immediately after deployment will make annotation of training data easier and likely increase classifier performance A2 Maintain consistency with preparation and deployment of CSDs. The more consistent the appearance of CSDs across orthoimages, the better the classifier performance is likely to be A3 Record the number of CSDs deployed in each area/orthoimage, where possible. This will later help to provide an indication of classifier recall and precision A4 Collect images of all outplant sites soon after CSD deployment (< 1 week; T1), then use these to make a T1 orthoimage for each site. - Various protocols have been published detailing how to use photogrammetry to create orthoimages of coral reefs, and several are stored here: https://www.lai-network.org/how-todocuments - Where possible, use the same orthoimage construction methods across all timepoints (T1, T2+). Before generating T2+ orthoimages, orientate the point cloud to be as similar to the T1 point cloud orientation as possible so that each orthoimage is a similar orientation through time (this will help alignment in Stage 7) A5 Download and install TagLab - Files and installation instructions can be accessed via the TagLab GitHub wiki: https://github.com/cnr-isti-vclab/TagLab/wiki/Install-TagLab. Toor et al (2025) provides further installation guidance 5 This monitoring protocol is conducted within the image annotation software TagLab. Once an orthoimage has been made, it must be exported from the photogrammetry software used to make it, then imported into TagLab. There is a maximum size for orthoimages that can be imported into TagLab (32767 pixels x 32767 pixels), so orthoimage size must be considered prior to export. The orthoimage’s pixel size should also be noted during export because this value is required during import into TagLab. The steps below provide guidance on orthoimage export from Agisoft Metashape (a photogrammetry software commonly used to construct reef orthoimages), and instructions for importing orthoimages into TagLab. 1. Importing orthoimages to TagLab 1.1 Export orthoimages of deployment sites from the photogrammetry software as a .tif file (e.g within Agisoft Metashape: File > Export > Export Orthomosaic > Export JPEG/TIFF/PNG… > Export > Save as type: Tiff/GEO TIFF (*.tif)). 1.2 If an orthoimage is larger than 32767 x 32767 pixels (the maximum import size for TagLab), either: 1.2a Crop the orthoimage into multiple smaller sections (then create separate projects for each crop) (e.g. within Agisoft Metashape: open Ortho > Draw polygon or Draw Rectangle > place points at the corners of the area you want to crop, creating a polygon. Left click polygon > right click polygon > Set boundary type > Outer Boundary. File > Export > Export Orthomosaic > Export JPEG/TIFF/PNG… > check that ‘Total size (pix)’ now complies with TagLab requirements > Export > Save as type: Tiff/GEO TIFF (*.tif)) 1.2b Or reduce orthoimage resolution (e.g. within Agisoft Metashape: File > Export > Export Orthomosaic > Export JPEG/TIFF/PNG… > reduce the ‘Pixel size (m)’ value > check that ‘Total size (pix)’ now complies with TagLab requirements > Export > Save as type: Tiff/GEO TIFF (*.tif) - n.b. reducing resolution will reduce clarity of orthoimages and potentially reduce capability to reliably identify features 1.3 Record the pixel size of all orthoimages at export (e.g. within Agisoft Metashape: record the ‘Pixel size (m)’ value reported in the ‘Export Orthomosaic’ window). This will be needed in Step 1.4 when importing orthoimages to TagLab 1.4 Open TagLab, then import an orthoimage and provide metadata (Project > Add New Map… > Apply) - Metadata Map name: provide a name for the orthoimage within the project (e.g. site_T1_2025) 6 RGB Image: provide the path to the orthoimage Depth Image: leave blank (unless also using DEMs in addition to orthoimages) Acquisition Date: the date the images were collected Pixel size (mm): orthoimage resolution - Ensure the (correct) pixel size is provided because it is vital to accurately measuring features within TagLab and it assists classifier implementation 1.5 Save the project in TagLab (File > Save Project) using a logical and informative naming convention 1.6 Repeat Steps 1.4 & 1.5 to create projects for all orthoimages containing CSDs 7 To create a classifier, an AI model must be provided with some examples of what it is expected to detect. These examples (‘training data’) are all of the information that a classifier is given so it is critical that care is given to creating high-quality, accurate training data. The total number of CSDs to include as training data should be determined by the user and is project-dependent. Previous work has created high-performing classifiers using training data containing 10 – 180 annotated CSDs. Training data is created by manually annotating all CSDs within a subsection of the orthoimage(s). This entails first selecting an area of the orthoimage to use, before segmenting (i.e. drawing the perimeter around) all CSDs then assigning a label (e.g. ‘CSD’) to each of the resulting segments/polygons. A list of desired label names (a ‘dictionary’) must be uploaded to the TagLab project. TagLab also has AI-assisted tools that drastically accelerate the manual annotation of features. Once features have been annotated, the subsection of orthoimage must be exported to prepare it in an appropriate format for training data. The steps below provide instructions for creating training data within TagLab, including setting recommendations and procedures for creating training data from single or multiple orthoimages. 2. Creating training data 2.1 In TagLab, open a project containing an orthoimage you want to use for classifier training (File > Open Project) 2.2 Load or create a dictionary containing your CSD as a class. Either: 2.2a Add your CSD class to the pre-existing dictionary; e.g. the default Scripps dictionary (Project > Labels Dictionary Editor… > RGB: square > select a colour* > OK > input your desired Label Name > Add > Save) 2.2b Or create a new dictionary containing your CSD as a class (Project > Labels Dictionary Editor… > New > provide a Dictionary name > RGB: square > select a colour* > OK > input your desired Label Name > Add > Save > dictionaries (within TagLab-main) > Save 2.2c Load an existing dictionary (Project > Labels Dictionary Editor… > Load > select a dictionary (via: TagLab-main > dictionaries) > OK) - *must select a colour that has not been assigned to another class in the dictionary - The library must eventually also contain any taxa that will be annotated (e.g. the coral taxa with which CSDs were seeded). These can be added to the dictionary at this stage, or later. 2.3 Identify the area of the orthoimage that you want to use as training data. The whole orthoimage can be used, if desired - Training data constitutes all of the information that the classifier will have about the appearance of CSDs in the given data so carefully select which areas are used 8 - Ideal attributes of the area used as training data: Contains the approximate number of CSDs desired CSDs are evenly distributed across the full extent of the area CSD appearance (e.g. colouration, orientation) is representative of appearance across the full dataset 2.4 Manually find and annotate all CSDs within this area, using TagLab’s tools (e.g. ‘Positive/negative clicks segmentation’) to accelerate annotation - Ensure all CSDs within the area are annotated. Any unannotated CSDs here could reduce classifier performance because unannotated CSDs are functionally labelled as orthoimage ‘background’, confusing the classifier during training. - Details of all TagLab annotation tools are provided on the TagLab webpage: https://taglab.isti.cnr.it/docs#learning 2.5 Export training set (File > Export > Export New Training Dataset) - Click the square icon next to Area to export: > draw a rectangle around the intended training area. If using the whole orthoimage as training data, leave as default. - Recommended settings: Dataset folder: path to folder where training data should be saved Dataset split: ‘Uniform-vertical’ Target pixel size: the pixel size of the orthoimage providing training data* Data format: ‘Tiles’ Show exported tiles: Yes** - *See Step 2.6 if compiling a training set from multiple orthoimages **If enabled, Show exported tiles saves an image (“tiles_cutted”) in the Taglab_main folder that shows how CSDs within the Area to export are partitioned between training, validation and testing data 2.6 To use multiple areas (e.g. from multiple orthoimages) in the training of a single classifier, repeat Steps 2.1 - 2.5 for each orthoimage you want to use and ensure the Dataset folder path is the same for all. This compiles training data from multiple projects into that specified folder. - The Target pixel size must be the same across all training data. Where orthoimages have different resolutions, input the mean pixel size of all orthoimages contributing to training data - To use multiple areas from the same orthoimage, rename the orthoimage within TagLab (Project > Maps Editor > edit > change ‘Map name’ > Apply > Close) before each export. This avoids training sets getting written over within the Dataset folder. 9 Once training data has been created, it must be presented to the AI network to enable training of the classifier. The interface for doing this in TagLab is straightforward and requires simply selecting the ‘Train your Network’ function and inputting values for four settings. Human involvement in training ends after it has been initiated, after which there will be a period of heavy computing. The length of this period depends on computational power, the size of the area used as training data and the training settings used. Training can take anywhere from a few hours to a several days to complete. The steps below provide instructions on how to initiate classifier training in TagLab and suggest settings to use. 3. Training a classifier 3.1 Initiate classifier training within TagLab (Train > Train your network > Train) - Recommended settings: Network name: provide a logical and informative name for the classifier Dataset folder: path to where the training data was exported to Number of epochs: 60 Learning Rate: 0.00005 L2 Regularization: 0.0005 Batch Size: 6 - The time required for classifier training is dependent on computational power, the size of the ‘Area to export’ and the training settings used; training can take anywhere between a few hours to a few days. 3.2 Inspect then save the Test Results and Training Graphs - Note that the Test Results values provided after training may not correlate with true classifier performance across whole orthoimage datasets so these should be interpreted with caution 3.3 Save the classifier (Confirm) 16 Once previous CSD annotations have been transferred into a new orthoimage, it is possible to inspect each location and investigate the fate of each CSD (e.g. some may have been dislodged/removed, whilst some may have yielded a coral colony). To record the data, measure any colonies growing from CSDs and enable continued monitoring through future time-points, annotations can be updated to reflect the development of CSDs. For example, if a CSD is absent in the new orthoimage the annotation label can be updated from ‘CSD’ to ‘Gone’, or if a colony is present the label can be updated from ‘CSD’ to ‘Acropora’. Segments can also be updated to reflect the boundary of any colonies growing from CSDs to enable measurement of these colonies. The steps below provide instructions for inspecting CSD locations and updating annotations in TagLab. Note that different monitoring approaches are possible, and different projects may require adjustments to the steps suggested here. 8. Updating CSD annotations to reflect changes between timepoints 8.1 Within a project containing a T2 orthoimage that has T1 annotations overlaid on it, toggle to the T2 orthoimage (via the ‘Map:’ drop-down) 8.2 Add new classes (e.g. ‘CSD Gone’, ‘CSD Coral’, ‘Acropora digitifera’) to the dictionary so annotations can be reassigned to a new class, where necessary (Project > Labels Dictionary Editor… > RGB: square > select a colour that is not yet used in the dictionary > OK > input your desired Label Name > Add > Save) 8.3 Visually inspect the first annotation and check if the CSD is still present (Zoom in to a scale at which a CSD can be clearly seen > left click on the first annotation row in the ‘Data table’ to centre the screen on that annotation) 8.3a If still present, proceed to Step 8.4 8.3b If the CSD is absent, change the annotation label to indicate that it is no longer in situ (Left click on the annotation, then double left click on the desired new class in the Dictionary (e.g. ‘CSD Gone’)) 8.4 Visually inspect the CSD for coral colonies 8.4a If no colony is visible, proceed to Step 8.5 8.4b If a colony is visible, update the annotation of the CSD to an annotation of the colony (Left click on the annotation to select it, select the ‘Positive/negative clicks segmentation’ tool and use it to segment the colony (instead of the CSD). Left click on the updated annotation, then double left click on the desired new class in the Dictionary (e.g. ‘CSD Coral’ or ‘Acropora digitifera’) to update the label) - Ensure that the annotation perimeter follows the perimeter of the colony as closely as possible because colony area is calculated from this annotation 17 8.5 Navigate to the next annotation (left click on the next annotation row in the ‘Data table’) and repeat Steps 8.3 and 8.4 8.6 Repeat Step 8.5 until all annotations have been checked and updated, where required 8.7 Save the project with the new T2 annotations (File > Save Project) 8.8 Export the T2 annotations as a csv table (File > Export > Export Annotations As Data Table) - Before export, order the data table by ascending Id number (this can help during analysis; see Step 9.2) (Left click on Id column name in the ‘Data table’) 8.9 Export the T2 annotations as a ‘Labeled Image’ ready for import to the orthoimage from the next timepoint (File > Export > Export Regions As Labeled Image) - In the ‘Labels’ panel (top right), ensure that an eye icon (rather than an ‘x’) is visible next to every ‘Class’ since only visible classes will be included in the export - Annotations can also be exported as a shapefile if preferred, but use of shapefiles is not covered here 18 The preceding stages detail how CSDs can be monitored between two time-points (the first and the second). Stages after use of the classifier (i.e. Stages 6, 7 and 8) can be repeated through additional time-points to enable monitoring of CSDs indefinitely. Updated annotations can be exported as data tables at each time-point and compiled to create a dataset reporting CSD fate, and colony fate and growth, providing data informing on the success of CSDs. The steps below provide some advice for compiling data tables to enable monitoring of CSDs through many time-points. 9. Monitoring CSDs through time 9.1 Import and update annotations through as many timepoints as required (Repeat Stages 6, 7 & 8), and export a data table for each site in each year 9.2 Collate data tables from different years and sites (e.g. in Microsoft Excel or R Studio) to compile a multi-year multi-site dataset for analysis of CSD fate and colony fate and growth - This methodology identifies individual CSDs/colonies across data tables through time by matching the unique ‘Id’ that TagLab assigns to each annotation. During Stage 8, it is critical that close attention is paid to ensuring that CSD/colony Id does not change between years - The same Id numbers will be used in different TagLab projects. To enable differentiation of Ids from different projects, prepend site names to Id numbers (e.g. SiteName_ + Id) before tables from multiple projects are collated - Add a column (e.g. Site_name) to each table to distinguish between sites once table are collated - Add a column (e.g. TimePoint) to each table to distinguish between time-points once table are collated 19 Images Back to 1.4 20 Back to 2.2 21 Back to 2.5 22 Back to 3.1 23 Back to 4.3 24 Back to 5.1 25 Back to 6.6