Challenges in Basso Continuo Performance-to-Score Alignment

Challenges in Basso Continuo Performance-to-Score Alignment Suhit Chiruthapudi1[0009000592136478],Adam Štefunko2,1[0009000132019983],JanHajičjr. 2[000000029207567X],and Carlos Eduardo Cancino-Chacón1[0000000157707005] 1Institute of Computational Perception, Johannes Kepler University, Linz, Austria {suhit.chiruthapudi,carlos.cancino_chacon}@jku.at 2Charles University, Faculty of Mathematics and Physics, Institute of Formal and Applied Linguistics, Prague, Czech Republic {stefunko,hajicj}@ufal.mff.cuni.cz Abstract. Basso continuo is a Baroque practice of creating harmonic improvisations above a given bass line. In addition to its historical heritage, it is also a living improvisation genre performed by most keyboard players in historically informed performance. However, present-day basso continuo performance has remained understudied empirically, mostly due to lack of data. One key enabler for such research is an automatic system that aligns basso continuo performances to the written bass line in the score. The task of basso continuo alignment, however, presents challenges that are distinct from those of standard performance-to-score alignment tasks, due to the underspecified notation of basso continuo. We analyze the specifics of basso continuo performance-to-score alignment through a dataset of basso continuo performances that we had previously introduced, based on manually annotated performance-to-score alignments of a subset of the pieces. From our analysis, we provide a compilation of challenges specific to basso continuo performance which need to be addressed during the alignment process. Keywords: Basso Continuo ·Historically Informed Performance · Performance-to-Score Alignment ·Improvisation. 1Introduction Basso continuo is the musical accompaniment typical for the Baroque period. A continuo performance involves a keyboard instrument (lutes and melodic instruments such as cellos is beyond our scope for now) player performing a notated bass line from the score and improvising upper voices above it, based on the bass line and the performer’s knowledge of appropriate harmonic and stylistic All rights remain with the authors under the Creative Commons Attribution 4.0 International License (CC BY 4.0). Proc. of the 17th Int. Symposium on Computer Music Multidisciplinary Research, London, United Kingdom, 2025 Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 924 S. Chiruthapudi et al. 12345 Timeline (s) 0 Performance Score Fig. 1. Illustrating the relationship between a continuo realization (top) and the notated bass line in the score (bottom). Each score note corresponds to the realization notes within its segment (bordered in red). Blue lines indicate duration of realization notes (as in a piano roll). Dotted lines indicate which notes in the performance correspond to the bass notes in the score. patterns. Numerical figures and accidentals that imply expected harmonies are occasionally written above the given bass note [26]. This improvisation, called continuo realization,generallyconsistsofthreevoicesabovethebassline,but the artist is free to choose the textures of the realization as long as they adhere to harmonic constraints implied by the bass line, as shown in Figure 1. Historical performance practices of basso continuo have been studied extensively by musicologists and performers themselves [2,16,8]. However, continuo is still a living practice today [8], and is a required skill of almost every harpsichordist or organist of the successful Historically Informed Performance (HIP) movement of the 20th and 21st centuries [2]. Despite its importance for the early music community, basso continuo practice of today has been largely overlooked by empirical research, mostly due to lack of suitable materials that could be empirically analyzed. Drawing on a tradition of researching performance, digital musicology and music information retrieval are uniquely positioned to provide appropriate methods to study how performers realize basso continuo today and how it relates to what we know of historical practice. Aligning the written bass line — the score — with the continuo performance is the first step that must be done in order to study continuo performance. This links the contemporary practice, which has not yet been a subject of research, to the historical materials (scores), the context within which the contemporary practice must be placed. Performance to score alignment is a well-known task in music information retrieval, but basso continuo presents an atypical performance-to-score alignment problem. First of all, only approx. 25 % of notes Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 925 Challenges in Basso Continuo Performance-to-Score Alignment in the recording are present in the score (compared to — ideally — 100 % in a regular performance-to-score alignment), so the usual 1-to-1 paradigm (with some tolerance e.g. for ornaments) may not apply. However, this is not the only phenomenon in continuo that requires expanding the paradigm of alignment approaches. The first dataset of basso continuo performance, the Aligned Continuo Realization Dataset (ACoRD), has recently been released, together with baseline alignment results [27]. Included with this dataset are ground truth alignments between performances and the scores (that contain only the bass line), both for the notated bass line itself and for all performance notes. The contribution of this paper is the identification and systematic analysis of phenomena that make aligning performed continuo realizations to reference scores problematic, but which nevertheless have to be addressed for empirical continuo research to proceed. In the ACoRD manual annotations, we observed recurring situations in which the correct alignment is ambiguous or not clearly defined. We provide detailed descriptions and frequency estimates of these alignment challenges, based on our annotations. On the basis of this analysis, we conclude by proposing the term “weakly notated music” for continuo and other structurally similar traditions, such as jazz performed from lead sheets or popular music just from lyrics and chord signs, as a step towards systematically addressing the challenges these musical traditions pose. Supplementary materials can be found online.3 2Materials:basso continuo data and baseline alignments The first materials documenting the contemporary practice of basso continuo were collected in the Aligned Continuo Realization Dataset (ACoRD) [27]. The dataset comprises 175 pilot MIDI recordings of continuo realizations, totaling 6 hours and over 66,000 note events. It captures basso continuo performances of 7 professionals and students playing basso continuo realizations of 5 different pieces. The goal of the dataset is not to present perfect basso continuo realizations according to any music-theoretical source, but to observe the “living” practice, including real-world problems that affect the overall quality of the realization (such as performance mistakes). The dataset thus captures a variety of deviations from a theoretically ideal basso continuo performance. The fact that 72 % of the performance notes in the dataset are not present in the score, but improvised by the performer. Our previous work [27] explored the ability of the state-of-the-art performanceto-score alignment methods to handle basso continuo data and proposed a twostep approach for aligning of basso continuo performance that leverages how existing symbolic alignment methods work. This alignment system was based on the fact that every basso continuo realization is supposed to contain the exact bass notes that are captured in the score, and then realization notes that are 3https://basso-continuo-alignment-challenges.github.io/supplementarymaterial. github.io/ Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 926 S. Chiruthapudi et al. improvised by the performer. In the first step (called bass alignment), the performance bass line notes were aligned with the score. In the second step (called realization alignment), all the other performance notes were aligned (Figure 1). Four state-of-the-art symbolic performance-to-score alignment methods were used for the first step: DualDTWNoteMatcher [18], TheGlueNoteMatcher [20], Pitch-IOI HMM [17] and Merged-output HMM [12]. Two performance preprocessing methods were also introduced, their task being to filter the performance notes to those that are likely to be the bass notes; thus helping the alignment algorithms thereafter. The first step therefore becomes a case of “regular performance-to-score alignment with many insertions” and is handled to a high degree of success by existing symbolic performance-to-score alignment algorithms (F1-score up to 97.9 % [27]). The second step is then reduced to many small-scale alignments of segments of performance notes delimited by the aligned bass notes. This is done using a position-based algorithm, in which every performance note that was not yet aligned (i.e. the realization notes) is grouped with a performance bass note that has already been aligned to a score note in the first step. This grouping is done based on the temporal co-occurrence of the realization note and the performance bass note. This realization note is then aligned with the score note to which the bass note was aligned in the first step. In order to evaluate this two-step architecture, manually annotated groundtruth data reflecting both steps (bass alignment annotation on 35 performances, and realization alignment annotation on 15 performances in the dataset) were created, and included in the ACoRD dataset. Already this annotation process uncovered a number of phenomena that show how alignment in this setting is more than just a version of the symbolic music alignment task with a lot of insertions on the performance side. This is reflected in experiment results. The two-step alignment algorithm’s performance reaches an F-score of 94.4 %: while this number seems high, it means there are several misaligned notes on average in every bar [27], which is too noisy already for the basic application of spotting obvious errors in realizations. Continuo alignment thus presents an issue beyond trivial extensions of the 1-to-1 paradigm.4 3RelatedWork Computational research on basso continuo has largely centered on generating realizations from musical scores, primarily through algorithmic methods [15,14], decision trees [24], and machine learning techniques [6]. This line of research is closely related to the broader domain of AI-driven music generation, where approaches based on convolutional neural networks [5] and transformers [23] have achieved state-of-the-art results. However, existing systems do not address the 4Notably, among other “weakly notated” traditions such az jazz lead sheets, continuo is still easy: the performance is supposed to contain the notated bass line, while the accompaniment of a jazz standard rarely doubles the written melody. Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 927 Challenges in Basso Continuo Performance-to-Score Alignment Table 1. Estimated absolute counts of performance notes involved in different types of ambiguous situations and their frequency of occurrence (as percentages) relative to the total number of score notes in the case of performance bass note alignment (526 notes) and performance notes in the case of performance realization note alignment (1836 notes) in five representative examples from the dataset. Ambiguity type Number of notes % of occurrence Performance Bass Notes Alignment Pitch Substitutions 6/526 1.1% Repetitions 21 / 526 4.0% Ornaments 11 / 526 2.1% Skips 29 / 526 5.5% Performance Realization Notes Alignment Bass note to realization note 211 / 1836 11.5% Sustained Realization Notes 571 / 1836 31.1% specific challenges and intricacies of modeling human-performed basso continuo improvisations by aligning them to the continuo score. Performance-to-score alignment involves synchronizing a musical performance with its corresponding score [9,19]. In the symbolic domain, it refers to matching notes in a symbolic performance (e.g., MIDI) to those in a symbolic score (e.g., MusicXML or MEI). Probabilistic approaches to alignment, such as hidden Markov models (HMMs), treat score positions as latent variables and model performance notes as observations [12,21,1]. Dynamic programming techniques, particularly dynamic time warping (DTW), align sequences by minimizing cumulative cost and, though originally used for audio alignment [9,25,10], have been effectively adapted for symbolic alignment [19,18]. Symbolic algorithms tend to have high accuracy typically when the performance aligns closely with afullynotatedscore,asiscommoninWesternclassicalpianomusic[19,12]. While the harmonic language of basso continuo is clearly defined within the Western classical tradition [2], existing datasets—such as the Bach chorales [3] or GiantMIDI [7]—represent aspects of Baroque music and, by extension, certain ideals of continuo realization. However, these datasets fall far short of representing the full diversity and inherent variability found in live continuo practice. The PiJAMA dataset of jazz piano [4] represents a music tradition that is improvised and not fully notated, but it does not contain alignment to any scores. 4 Challenges in Continuo Alignment These results prompted a closer look at the misaligned notes. The analysis of these misalignments with respect to the alignment annotations that were done manually unearthed a variety of challenging situations in identifying the appropriate continuo performance-to-score alignment in both of these steps. Statistics of all such ambiguous situations are provided in Table 1, and we discuss these challenges in this section. Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 928 S. Chiruthapudi et al. b) d) a) c) Performance Performance Score Score Fig. 2. The four case studies of challenges in performance bass note alignment as detailed in section 4.1. The thin blue lines represent the alignment computed by an existing automatic alignment system.The score notes that are difficult to align with the performance are circled in blue, while the potential corresponding notes in the performance are highlighted in red and green. 4.1 Performance Bass Notes Alignment We first perform one-to-one alignment of the notated bass line to the performance. This follows from the premise that the notes in the continuo score should as a rule be preserved in performance [2], implying that each score note should correspond to a bass note in the performance. Any additional notes played in the performance are interpreted as realizations of the underlying bass note. The following subsections discuss challenging cases encountered during the process of manually annotating the alignment of just the bass line in the performance. Although these cases are seemingly infrequent (see Table 1) during the bass alignment step, the subsequent realization notes alignment step is built upon the correct alignment of the performance bass notes in this step, and therefore requires careful consideration. Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 929 Challenges in Basso Continuo Performance-to-Score Alignment Pitch Substitutions Pitch substitutions/errors are a challenge common in performance-to-score alignment with fully notated scores as well, but continuo alignment presents additional complexity: the correct alignment of realization notes in the second step of the alignment process depends on correct identification and alignment of performance bass notes in the first step. Figure 2a5 illustrates a situation where it is unclear whether a note with a pitch error but correct timing (C]4, circled in green) in the performance should be aligned with its corresponding score note (C4, circled in blue), or if the score note should be marked as a deletion. Furthermore, the performer seemingly corrected the pitch error by playing the correct note on a subsequent beat (C4, circled in red), which was detected by the automatic alignment system. Note Repetitions Situations involving repeated consecutive bass notes in the performance that correspond to a single score note are challenging for one-to-one alignment because determining the most appropriate match can be ambiguous. Such cases seem to occur approximately once in every 26 score notes. Figure 2b shows a case where the same bass note F]3 was played twice in the performance, both times corresponding to one score note: F]3. Although the automatic alignment process aligns the score note (blue) with the second performance note (red), the first F]3performancenote(green)mayhavebeen the correct choice, as it is the first occurrence of the bass note, even though the second occurrence of the bass note has a longer duration. On the other hand, Figure 2c shows a case where it would be more apt to ignore the first occurrence of the bass note F]3(red)andalignthesecondoccurrenceofthesamenote (green) with the score note (blue), owing to the long duration of the second occurrence of the performance note, as well as the fact that all the improvised realization notes above fall within the duration of the second performance bass note. Finally, Figure 2d shows a case where both occurrences of the bass note in the performance are extremely short. It seems appropriate here that the second occurrence of the performance bass note F]2(green)isalignedtothescorenote (blue) instead of the first occurrence (red) for two reasons: it is slightly longer duration, and the second occurrence falls better in tempo, as indicated by the relative parallelism of the alignment connectors of the preceding and following notes. Ornaments One is also expected to face challenges when the bass line is ornamented by the performer, for example with a trill that is not written in the basso continuo score. The ambiguity in ornament alignment lies in several aspects. It is necessary to determine whether a single bass note within a trill is to be aligned to the score note, or if the entire trill needs to be considered as a single entity that is aligned to the score note. Moreover, ornamentation is not always notated in a basso continuo score, which can make it challenging to distinguish an orna5Screenshots captured from the web alignment tool, Parangonada [19]. Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 930 S. Chiruthapudi et al. ... ... ... Performance Score Fig. 3. Illustration of an example where performer error resulting in a skipped line in performance creates challenges in the first step of alignment. The image consists of screenshots of two sections in the piece’s timeline that are separated by some time in between (this gap is marked by ‘...’ in the image). The notes circled in red mark the end of a line of music in the piece, following which the subsequent line of music was skipped in the performance, resulting in the subsequent score notes remaining unaligned or incorrectly aligned. The blue and green lines with their circled notes represent the ideal alignment, which was not achieved by the current state-of-the-art alignment algorithm. ment from a mistake or improvisation in the bass line.6Ornament alignment is an open problem even in strongly notated scores [11,20]. Note Skips Some performers mistakenly skip notes or even entire passages during a performance. While current state-of-the-art alignment algorithms are known to handle skips in the performance [20,11,13], the presence of the improvised realization notes complicate the automatic alignment process, resulting in incorrect alignments between the performance and score. An example related to a performance with two long skipped passages is shown in Figure 3. The performance and score notes D3 (highlighted with red circles) indicate the conclusion of a musical line within the piece. However, during the performance, the line of music that was supposed to follow this point was inadvertently skipped. As a result, the corresponding notes in the score that should have aligned with the performance became misaligned or improperly synchronized. In contrast, the blue (corresponding to the note D3) and green (corresponding to the note F]3) lines, along with their circled notes, illustrate the correct or intended alignment between the performance and the score—an alignment that the current state-ofthe-art algorithm failed to achieve. 6Duration is not a sufficient indicator: slow ornaments like appoggiatura are common in baroque music. Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 931 Challenges in Basso Continuo Performance-to-Score Alignment 4.2 Performance Realization Notes Alignment a) b) Performance Score Fig. 4. Examples that illustrate two different challenges a) and b) faced in the manual alignment annotation of the realization notes to the score. See Section 4.2 for details. Following the manual annotation of the bass note alignment, a many-to-many structure was adopted for annotating the alignment between score notes and performance realization notes. This approach allows multiple realization notes to be linked to a single score note. Often, a single realization note must be aligned with multiple score notes as well when its duration extends beyond the duration of its first associated bass note and across subsequent performance bass notes. This is especially relevant to tied notes on the organ, prepared dissonances, etc. Table 1 shows that a majority of continuo alignment challenges are of this type. Bass notes as realizations. Situations where a performance bass note itself stretches in duration partially or completely over the length of the next performance bass note(s), occur more than once in every ten notes (see Table 1. In such cases the overlap between these notes helps determine whether the first note becomes reinterpreted as a realization note of the following bass note(s). As illustrated in Figure 4a, in such cases, the performance bass note (green) is aligned with its own score note (blue), and it is also aligned as a realization note with the next score note (yellow); i.e., the score note corresponding to the next performance bass note (red). Sustained realization notes. Achallengingcasethatoccursalmostonceevery third note during the process of realization note alignment is when a realization Proc. of the 17th International Symposium on CMMR, London, UK, Nov. 3-7, 2025 932