Pixel Labeler: A Pixel-Wise Annotation Tool for Active Learning Music Recognition

Pixel Labeler: A Pixel-wise Annotation Tool for Active Learning Music Recognition Carlos Pérez-Sancho[0000→0003→1099→7417],Alejandro Galan-Cuenca[0000→0002→5799→6270],JuanP. Martinez-Esteso[0009→0006→3246→3781],FranciscoJ. Castellanos[0000→0001→9949→5522],andAntonioJavier Gallego[0000→0003→3148→6886] University Institute for Computing Research, University of Alicante, Spain {carlos.perez, a.galan, juan.martinez11}@ua.es, {fcastellanos, jgallego}@dlsi.ua.es Abstract. Pixel-wise image annotation is a critical but demanding task, particularly in domains where precise labeling is essential for training supervised models. In this work, we present a novel annotation tool designed to assist with Optical Music Recognition (OMR) tasks and to support human-in-the-loop workflows within Active Learning (AL) cycles. The tool focuses on maximizing annotation e!ciency and user experience by combining a lightweight interface, usability-driven design, and support for fine-grained, per-pixel semantic labeling. Preliminary results suggest that the proposed interface can reduce the cognitive and operational burden on annotators, o"ering a viable solution for tasks requiring dense annotations. Feedback from domain users, as well as informal metrics such as annotation time and perceived usability, support the relevance of the design. Overall, this work contributes a flexible and user-centered annotation interface that lays the groundwork for more efficient and e"ective integration of human input in OMR and AL-based image segmentation workflows. Keywords: Optical Music Recognition (OMR) ·Active Learning ·Dataset annotation. 1Introduction Optical Music Recognition (OMR) plays a fundamental role in preserving and providing digital access to historical musical heritage [5]. By converting fragile physical scores—both printed and handwritten—into machine-readable formats, OMR enables searchable digital libraries and facilitates large-scale musicological research. However, early music documents often su!er from severe visual degradations, such as irregular layouts, faded ink, yellowed paper, and damaged or distorted symbols, resulting from aging and manual or printing inconsistencies [12]. These conditions pose significant challenges for automatic recognition, especially for machine learning methods that depend on large volumes of highquality labeled data. Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 711 C. Pérez-Sancho et al. Active Learning (AL) emerges as a promising strategy. AL enables models to be trained using a reduced set of labeled instances, selecting the most informative samples to be labeled by a human expert. This is particularly beneficial in the context of OMR, where full annotation of high-resolution images of complex scores is time-consuming and expensive, as AL enables human annotators to focus on the most challenging areas of a score. Moreover, because recognition errors are common in degraded historical sources, interactive systems involving a human-in-the-loop become highly valuable. These systems allow for incremental improvement through expert feedback, contrasting with traditional OMR pipelines that are static and non-adaptive [4]. Yet, e!ective integration of AL into OMR workflows demands specialized annotation tools. Pixel-level annotation is particularly useful for complex tasks such as layout analysis or semantic segmentation, where bounding boxes may fall short—for instance, when symbols overlap or have irregular shapes that cannot be precisely enclosed. Music scores, due to their heterogeneity, dense notation, and close symbol proximity, require precise annotation capabilities tailored to domain-specific challenges. The lack of dedicated tools for fine-grained annotation of music documents remains a bottleneck in generating training data necessary to advance OMR technologies. To address this gap, we present a novel annotation tool designed for pixellevel labeling of music documents, with a particular focus on tasks in OMR and AL pipelines. The tool is built to be both accessible to domain experts such as musicologists—who may not have technical expertise—and precise enough to support high-quality annotation. Our approach aims to bridge the gap between machine learning systems and expert annotators, enabling e"cient dataset creation and continuous model refinement with minimal manual e!ort. 2StateoftheArt AL has emerged as an e!ective strategy for reducing annotation costs by directing human e!ort toward the most informative and uncertain samples. This approach has proven valuable across diverse domains where dense annotations are costly. In historical document processing, Narag et al. [8] demonstrated how selective expert interventions, guided by model uncertainty, can e!ectively restore severely degraded architectural drawings with minimal manual input. Similarly, Budd et al. [3] emphasized the role of AL in medical image analysis, where careful sample selection significantly lowers labeling e!ort while maintaining clinical accuracy. In the context of semantic segmentation, Kasarla et al. [7] introduced aregion-basedALframeworkthatprioritizessuperpixelsforannotation,achieving high performance with only a fraction of the pixel labels. Together, these studies underscore AL’s potential to optimize annotation e"ciency in visual. Nonetheless, e!ectively incorporating AL into OMR workflows requires more than selective sampling. It also demands annotation tools that support pixellevel precision and enable seamless integration into iterative, human-in-the-loop processes. This is particularly critical in the context of early music documents, Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 712 A Pixel-wise Annotation Tool for Active Learning Music Recognition where dense layouts, irregular symbol shapes, and visual degradation challenge conventional annotation methods. Existing tools vary greatly in their design, goals, and capabilities. Some are general-purpose image editors repurposed for annotation; others are built for computer vision tasks like object detection or semantic segmentation. A smaller subset focuses on model-assisted correction [9,10,1], which aligns closely with AL workflows. Each type o!ers distinct advantages and limitations depending on the level of granularity required and the nature of the target documents. To clarify these di!erences, Table 1 summarizes the representative tools across these categories, highlighting their features and limitations. Despite the variety of tools, none of them fully meets the requirements for dense, pixel-level annotation of music documents. General-purpose editors o!er precision but lack structured workflows and class management. Region-based platforms provide semantic labeling but fall short in granularity and control. Pixel-wise tools, while more precise, often do not support multilayer annotations or iterative correction workflows. Finally, correction-focused systems are tightly coupled to specific pipelines and lack flexibility for arbitrary pixel editing. Table 1: Comparison of annotation tools based on functionality, granularity, and suitability for music document labeling. Type \Tool Main features Limitations General-purpose image editing Photoshop, GIMP, Pixelmator Precise pixel editing, layer support No class tracking; lacks data export and structured annotation; not suitable for large-scale workflows Picozu Web-based, layer support, basic drawing tools Low performance on large files; no annotation features or export formats Object/region-based annotation LabelImg, makesense.ai, LabelMe Open-source; supports bounding boxes and polygons; standard export formats (COCO, VOC, YOLO) No pixel-level support; unsuitable for fine-grained segmentation CVAT Web-based, collaborative, supports tracking, automation plugins Region-focused; lacks tools for pixel correction or dense symbol handling VIA (VGG Image Annotator) Lightweight browser tool for polygons, points, rectangles; export to JSON/CSV No advanced editing or segmentation refinement Pixel-wise annotation tools Supervisely Pixel-level tools (brush, fill, eraser); supports object labeling No multilayer editing; not tailored for iterative correction Correction-focused tools MuRET [9] Complete OMR pipeline; symbol-level annotation; integrated AL loop No free-form pixel tools (brush/fill); focused on region/symbol annotations Pixel.js [10] Web-based; pixel-level correction via layer interface; designed for score segmentation Limited multilabel control; postsegmentation correction only WebGT [1] Web tool for historical layout annotation; supports rectangles; collaborative mode No fine-grained pixel editing or multilabel overlay support Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 713 C. Pérez-Sancho et al. 3PixelLabeler We developed a custom pixel-level annotation tool (Pixel Labeler) in Python, designed to address the challenges of precisely labeling historical music scores, while remaining flexible enough for other types of music notation or imagery. These documents often su!er from visual degradation, such as paper yellowing, ink fading, stains, and other noise artifacts that hinder automated segmentation methods. Therefore, manual pixel-level annotation remains crucial in this context, and the proposed tool has been designed to e!ectively meet these specific requirements. Pixel Labeler is released as open-source software1,encouraging transparency, reproducibility, and community-driven improvement. The first step when using the tool is to configure a list of independent layers that will represent all the types of information to be annotated—such as sta! lines, text, musical notes, or even decorative ornaments. This setup is done by editing a simple configuration text file that specifies the name and display color of each layer. The file is reusable across images, so it only needs to be defined once. The annotation process is thus organized into a set of customizable layers, enabling a clear separation of di!erent semantic elements within the score. This design supports focused annotation and facilitates subsequent analysis. Pixel Labeler provides three complementary annotation methods: – Freehand pen, with adjustable brush size, for direct drawing. – Magic wand selector, which segments connected regions based on a useradjustable threshold. The selected region is displayed in real time, and the threshold can be dynamically increased or decreased using keyboard shortcuts. The tool also includes built-in expansion and reduction operations to refine the selection. By default, it applies an automatic smoothing step that combines an initial expansion followed by a contraction, e!ectively filling small gaps often caused by stains, faded ink, or degraded paper texture. – Bucket fill, for rapid labeling of uniform areas (with threshold control). One key feature of Pixel Labeler is its default behavior of protecting annotations across layers. Editing operations avoid modifying pixels already labeled in other layers, reducing the risk of accidental overwrites. This helps maintain consistency in multi-layer annotation workflows, while still allowing users to disable the safeguard when overwriting is needed. The interface is optimized for annotator productivity, including intuitive keyboard shortcuts for fast switching between tools and layers, undo actions, zoom/pan controls, and adjusting tool size or threshold values. The live visual feedback and multi-layer awareness enhance the user experience when working with complex manuscripts. Figure 1 shows an example of the annotation process using the tool on a music score image. It illustrates the high pixel-level precision of the interface and its multi-layer annotation capabilities. To further assist annotators, it includes a progress bar that displays the percentage of the image that has been annotated. Additionally, users can enable a visualization mode 1https://github.com/cperezs/pixel-level-annotator Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 714 A Pixel-wise Annotation Tool for Active Learning Music Recognition that highlights all unannotated pixels, making it easy to identify and complete any missing regions without resorting to manual pixel-spotting. Fig.1: User interface of the annotation tool, showing multi-layer control, tool selection, and real-time visual feedback. Each annotation session results in a set of binary images, one per layer, where pixels belonging to the corresponding semantic class are set to one and the rest to zero. These images serve as segmentation masks and can be directly used for training or evaluating machine learning models (see Figure 2). Additionally, a metadata file is generated, containing information such as the time spent annotating each layer. This temporal information enables further research on annotator behavior and task complexity. For example, it can be used to identify particularly challenging layers, analyze the e"ciency of di!erent tools or annotation strategies, or model the annotation cost for AL prioritization. 3.1 Interoperability and Integration The annotation tool is designed for interoperability and integrates easily into broader annotation workflows through a lightweight HTTP-based communication mechanism, exchanging data with other applications using standard web requests. It launches a local service that accepts requests containing: (1) the image to be annotated, (2) the definition of annotation layers, and (3) a callback URL to which the completed annotations are sent upon marking the image as Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 715 C. Pérez-Sancho et al. (a) Original (b) Background (c) Sta"(d) Notes (e) Lyrics Fig.2: Pixel-level annotation outputs. From left to right: original image and binary masks for background, sta!lines, notes, and lyrics layers. annotated. Although implemented in Python and run locally, this setup allows seamless integration with web-based annotation systems, enabling dynamic task dispatch and transparent coordination in collaborative or large-scale projects. 4Evaluation To assess the e!ectiveness of the proposed annotation tool, an evaluation was conducted with five expert human annotators (H1–H5). Each participant labeled the same image set using both Pixel Labeler and GIMP. This dual annotation enabled direct comparison across three dimensions: annotation time, interand intra-annotator agreement, and usability. GIMP was chosen as the baseline due to its overlap in pixel-level editing capabilities. Most annotation platforms focus on objector region-based labeling and lack support for free-form pixel editing. GIMP, in contrast, o!ers essential tools like brush, eraser, layers, and zoom, allowing users to manually create or refine segmentation masks. While not an annotation tool per se, its flexibility and widespread use make it a practical benchmark. However, obtaining usable mask outputs for training or evaluation requires exporting each layer individually as an image, which adds an extra step compared to the direct export functionality of the proposed tool. As a free, open-source, and cross-platform application, it also provides a neutral baseline for evaluating usability and e"ciency. To ensure consistency across tools, all annotators were given a common set of guidelines on how to approach the annotation task. 4.1 Dataset The evaluation was conducted using three publicly available datasets containing historical music manuscripts, each annotated at the pixel level with four semantic layers: staff,notes,text,andbackground. Rather than conducting large-scale training experiments, our focus is on evaluating the annotation process itself, as a step toward integrating the proposed tool into an AL loop in future work. To emulate realistic usage within such a loop, where fragment selection is guided by model-based or heuristic criteria, we Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 716 A Pixel-wise Annotation Tool for Active Learning Music Recognition extracted 256 →256 pixel patches from each dataset. A total of 15 patches (5 per dataset) were selected for annotation, resulting in 983,040 pixels labeled per annotator and tool. Full images are typically not used in this context because historical musical documents are large, heterogeneous, and often contain substantial regions that do not require annotation (e.g., margins or blank areas). Focusing on representative patches enables a more e"cient and targeted evaluation, while better reflecting the iterative nature of AL, where only small but informative regions are selected for labeling at each step. The datasets span a range of notational styles and historical contexts, o!ering arepresentativesubsetofthevisualandsemanticvariabilityencounteredin historical document annotation and serving as a suitable context for evaluating annotation workflows under controlled but realistic conditions: –Einsiedeln:9high-resolutionscansofneumaticnotationfromtheEinsiedeln, Stiftsbibliothek, Codex 611(89) (1314).2 –Salzinnes:10imagesfromtheSalzinnes Antiphonal (CDM-Hsmu M2149.14), also in neumatic notation, accessible via the Cantus Ultimus platform.3 –Capitan [6]: 10 double-page images of mensural notation from the 17th–18th centuries, sourced from the Cathedral of Our Lady of the Pillar,Zaragoza, Spain.4 4.2 Annotation Time Table 2 summarizes the average annotation time per layer and dataset, including standard deviation, for both tools. Overall, the results show that the proposed tool notably reduces annotation time, especially for complex layers such as staff and notes.Onsimplerlayerslikebackground,bothtoolsyieldcomparable times. Therefore, these outcomes suggest that Pixel Labeler o!ers a more e"cient and consistent annotation experience. As shown in Table 3, Pixel Labeler achieved an average time saving of 2:19 (mm:ss) per image, reducing the mean annotation time from 10:31 to 8:12—a 22% improvement in e"ciency across annotators. The largest gains were seen in the staff and lyrics layers, where the structured content benefits most from the tool’s tailored interaction mechanisms. While some annotators experienced longer times for the simpler background layer, this was o!set by notable savings in the more demanding layers. An indepth analysis of total annotation times reveals that using Pixel Labeler reduced the overall annotation e!ort from 47,321 seconds (↑13h 9min) to 36,872 seconds (↑10h 15min). 4.3 Agreement To evaluate annotation consistency, we measured both inter-annotator and intraannotator agreement across all annotated layers, comparing results obtained 2http://www.e-codices.unifr.ch/en/sbe/0611/ 3https://cantus.simssa.ca/manuscript/123723/ 4RISM Code “E-Zac” available at https://rism.info. Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 717 C. Pérez-Sancho et al. Table 2: Average annotation time per patch (mm:ss±std) for each layer and total, across three datasets, annotated by each human annotator Hn. The left columns report annotation times using GIMP, while the right columns use the proposed Pixel Labeler tool. Times correspond to the average of five annotated patches per dataset and layer. Bold values indicate the fastest annotation times per case. GIMP Pixel Labeler Ann. Layer Capitan Einsiedeln Salzinnes Avg. Capitan Einsiedeln Salzinnes Avg. H1 Bg. 00:41±00:15 00:28±00:13 00:22±00:04 00:31±00:10 01:00±00:50 02:02±01:18 00:30±00:23 01:10±00:47 Staff 03:50±01:08 02:20±00:29 03:08±01:49 03:06±00:45 02:23±01:30 03:02±00:44 02:15±00:41 02:33±00:25 Notes 02:41±00:36 02:42±01:10 02:50±00:57 02:44±00:05 02:07±01:16 03:31±02:06 01:15±00:54 02:18±01:09 Lyrics 02:48±00:43 04:03±01:45 03:16±01:00 03:22±00:38 01:33±00:39 02:09±01:29 01:00±00:36 01:34±00:35 Total 10:00±01:11 09:32±01:58 09:36±02:09 09:43±00:15 07:04±01:03 10:44±03:38 05:00±01:04 07:36±02:54 H2 Bg. 00:37±00:06 00:39±00:09 00:41±00:07 00:39±00:02 00:16±00:05 00:18±00:07 00:21±00:24 00:19±00:03 Staff 04:12±00:51 02:36±01:25 01:55±00:41 02:54±01:10 04:31±02:17 03:21±02:28 01:35±00:59 03:09±01:29 Notes 02:56±00:36 03:58±01:25 02:50±00:44 03:15±00:38 01:59±01:13 01:31±01:15 01:40±00:47 01:43±00:14 Lyrics 03:33±00:38 03:20±01:29 03:56±01:18 03:36±00:18 01:03±00:36 00:59±00:40 01:28±00:51 01:10±00:15 Total 11:17±01:05 10:34±02:26 09:21±02:22 10:24±00:59 07:49±02:52 06:10±03:27 05:04±01:28 06:21±01:23 H3 Bg. 00:38±00:06 00:33±00:06 00:40±00:08 00:37±00:04 01:17±00:41 00:56±00:27 00:47±00:25 01:00±00:15 Staff 03:56±01:04 03:03±01:17 02:09±00:51 03:03±00:53 03:41±01:22 01:37±00:38 00:47±00:25 02:02±01:30 Notes 02:41±00:23 02:36±01:00 02:18±00:26 02:32±00:12 03:01±00:36 02:09±00:33 02:08±00:49 02:26±00:30 Lyrics 02:31±00:24 02:51±01:02 03:07±01:05 02:50±00:18 02:09±00:21 00:23±00:04 01:47±01:13 01:26±00:56 Total 09:47±00:57 09:03±02:01 08:15±01:43 09:01±00:46 10:07±01:41 05:05±01:10 05:29±02:26 06:54±02:48 H4 Bg. 00:53±00:21 00:31±00:15 00:36±00:09 00:40±00:12 02:26±01:12 01:50±00:38 00:53±00:23 01:43±00:47 Staff 08:00±02:26 03:30±00:29 02:10±00:27 04:33±03:03 03:21±00:55 03:19±01:54 04:12±02:37 03:37±00:30 Notes 04:05±01:59 03:12±01:13 03:26±00:41 03:34±00:27 01:47±02:44 01:57±01:30 03:58±00:53 02:34±01:13 Lyrics 02:18±00:23 03:45±00:33 04:03±00:52 03:22±00:56 04:51±00:28 01:49±01:29 02:20±01:31 03:00±01:37 Total 15:15±04:37 10:58±01:33 10:16±00:28 12:09±02:42 12:25±03:38 08:55±04:40 11:24±03:21 10:55±01:48 H5 Bg. 00:41±00:17 00:27±00:06 00:23±00:05 00:30±00:09 02:36±01:32 00:22±00:15 00:20±00:27 01:06±01:18 Staff 06:48±01:53 02:58±00:44 03:32±00:36 04:26±02:04 05:27±00:51 02:11±00:20 00:48±00:32 02:49±02:23 Notes 02:46±00:08 03:09±01:14 03:34±00:29 03:10±00:24 03:51±00:45 03:33±00:55 02:39±00:37 03:21±00:37 Lyrics 03:10±00:31 03:35±00:42 02:48±00:57 03:11±00:24 02:16±01:13 01:32±01:06 02:03±00:30 01:57±00:22 Total 13:25±02:00 10:09±02:19 10:17±01:04 11:17±01:51 14:10±02:04 07:38±01:22 05:51±01:03 09:13±04:23 Avg. Bg. 00:42±00:07 00:31±00:05 00:32±00:09 00:35±00:05 01:31±00:59 01:06±00:49 00:34±00:15 01:04±00:30 Staff 05:21±01:55 02:53±00:27 02:35±00:43 03:36±00:49 03:53±01:10 02:42±00:46 01:55±01:25 02:50±00:36 Notes 03:02±00:36 03:07±00:33 03:00±00:31 03:03±00:25 02:33±00:52 02:32±00:56 02:20±01:03 02:28±00:35 Lyrics 02:52±00:30 03:31±00:27 03:26±00:32 03:16±00:17 02:22±01:28 01:23±00:42 01:44±00:31 01:49±00:43 Total 11:57±01:54 10:03±01:21 09:33±01:17 10:31±01:23 10:19±00:59 07:42±00:49 06:33±00:45 08:12±00:47 with Pixel Labeler and GIMP. Agreement was computed as the mean pixel-wise overlap percentage across the corresponding annotations. As shown in Table 4, Pixel Labeler achieved slightly higher inter-annotator agreement (96.66%±2.28) than GIMP (95.67%±3.22), consistently outperforming it across all annotation layers. The largest improvement was observed in the background layer, with a 3.22 percentage point gain. Notable gains were also seen in the notes layer. In addition, Pixel Labeler exhibited lower standard deviations in most cases, indicating more consistent annotation behavior among users. These results suggest that the tool’s interaction mechanisms help reduce manual variability, increase detail refinement, and promote more uniform annotations. Intra-annotator agreement, measured by comparing annotations made by the same user with GIMP and Pixel Labeler, averaged 96.81%±2.88 across all layers. Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 718 A Pixel-wise Annotation Tool for Active Learning Music Recognition Table 3: Annotation time saved (mm:ss±std) when using Pixel Labeler, in comparison with GIMP, shown per annotator Hn,foreachannotationlayerand in total. Values represent the average di!erence over 15 patches (5 per dataset: Capitan,Einsiedeln,andSalzinnes). Positive values (in green) indicate time savings, while negative values (in red) indicate longer annotation times. Time saved per layer Bg. Staff Notes Lyrics Total saved H1–00:40±00:47 +00:33±00:23 +00:26±00:31 +01:48±00:30 + 02:07±01:42 H2+00:20±00:01 –00:15±00:15 +01:32±00:49 +02:26±00:05 + 04:03±00:31 H3–00:23±00:16 +01:01±00:40 +00:06±00:08 +01:24±01:03 + 02:08±01:51 H4–01:03±00:40 +00:56±02:14 +01:00±00:53 +00:22±00:26 + 01:15±00:51 H5–00:36±01:04 +01:37±01:00 –00:11±00:22 +01:14±00:43 +02:04±01:51 Avg. –00:36±00:24 +00:52±00:39 +00:39±00:05 + 01:27±00:51 +02:19±00:41 Table 4: Inter-annotator and intra-annotator agreement (%) by annotation layer and tool. Values indicate mean agreement with standard deviation. ↑Statistically significant di!erence (p<0.05). Inter-annotator agreement Layer Pixel Labeler GIMP Intra-Annotator Background 94.18 ±3.45 90.96 ±8.75 94.24 ±2.83 Staff 97.84 ±1.09 97.80 ±0.88 97.67 ±1.53 Notes →97.88 ±1.28 97.24 ±1.54 98.18 ±1.29 Lyrics 96.74 ±3.62 96.66 ±2.17 97.16 ±2.88 Average 96.66 ±2.98 95.67 ±5.27 96.81 ±2.88 This ↑3% discrepancy, mostly due to di!erences in boundary precision, highlights the inherent variability in manual annotation. Notably, the inter-annotator agreement with Pixel Labeler approaches this level, underscoring its potential to promote consistent results across users. To assess the significance of agreement di!erences, we performed a Wilcoxon test on agreement values. Although the average agreement was 0.99 percentage points higher with Pixel Labeler, the di!erence was not statistically significant (p=0.071). This suggests a trend toward improved consistency, though more data is needed to confirm it. Layer-wise comparisons showed that the most significant improvement occurred in the notes layer, where agreement with Pixel Labeler was higher by 0.64 percentage points (p=0.022). The background layer also showed a mean gain of 3.21 points; however, this result was not statistically significant (p=0.454)and therefore should be interpreted with caution. For the staff and lyrics layers, we did not find statistically significant evidence of di!erences. These findings suggest that Pixel Labeler may provide measurable benefits particularly in complex, high-density regions such as musical notes, where symbols are often small, Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 719