Real-time Generation of Percussive Rhythms Using Descriptors

Master in Sound and Music Computing Universitat Pompeu Fabra Real-time Generation of Percussive Rhythms Using Descriptors Alexandre Vilanova Supervisor: Daniel Gómez Co-Supervisor: Sergi Jordà August 2025 Contents 1 Introduction 1 2 State of the Art 3 2.1 RhythmPerception............................. 3 2.2 RhythmSpaces............................... 5 2.3 Variational Autoencoders in Generative Music . . . . . . . . . . . . . . 6 3 Methods 8 3.1 ProblemSetup ............................... 8 3.2 Variational Autoencoder Model . . . . . . . . . . . . . . . . . . . . . . 9 3.2.1 ModelArchitecture............................. 9 3.2.2 TrainingProcedure............................. 10 3.2.3 Latent Space Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 3.3 Descriptor-based Model . . . . . . . . . . . . . . . . . . . . . . . . . . 11 3.3.1 RhythmDescriptors ............................ 11 3.3.2 Discrete descriptor precision . . . . . . . . . . . . . . . . . . . . . . . . 15 3.3.3 Neural Network Implementation . . . . . . . . . . . . . . . . . . . . . . 16 3.3.4 Dimensionality Reduction . . . . . . . . . . . . . . . . . . . . . . . . . 17 3.3.5 ModelEvaluation.............................. 18 3.4 Comparing VAE and descriptor-based approaches . . . . . . . . . . . . 19 3.5 Smoothness Experiment . . . . . . . . . . . . . . . . . . . . . . . . . . 21 3.5.1 ExperimentDesign............................. 22 3.5.2 Implementation............................... 24 3.6 User Experience Experiment . . . . . . . . . . . . . . . . . . . . . . . . 24 3.6.1 Implementation............................... 25 3.6.2 Structure .................................. 25 3.6.3 Data Collection and Storage . . . . . . . . . . . . . . . . . . . . . . . . 26 4 Results 31 4.1 Dimensionality Reduction . . . . . . . . . . . . . . . . . . . . . . . . . 31 4.2 Smoothness Experiment . . . . . . . . . . . . . . . . . . . . . . . . . . 34 4.3 User Experience Experiment . . . . . . . . . . . . . . . . . . . . . . . . 36 5 Discussion 40 5.1 VAE vs. Descriptor-Based Approaches . . . . . . . . . . . . . . . . . . 40 5.2 DescriptorSelection ............................ 41 5.3 SmoothnessAnalysis............................ 41 5.4 User Experience Experiment . . . . . . . . . . . . . . . . . . . . . . . . 42 5.4.1 Validation of the Descriptor-Based Approach . . . . . . . . . . . . . . . 42 5.4.2 User Background and Performance . . . . . . . . . . . . . . . . . . . . 43 5.4.3 Descriptor Interpretability . . . . . . . . . . . . . . . . . . . . . . . . . 43 5.5 Qualitative User Feedback Analysis . . . . . . . . . . . . . . . . . . . . 44 5.6 Methodological Considerations and Limitations . . . . . . . . . . . . . 45 6 Conclusions 46 7 Future Work 48 List of Figures 50 List of Tables 51 A Source code and demo 55 Acknowledgement I would like to express my sincere gratitude to my supervisor, Daniel Gómez, for his invaluable guidance, unwavering support, and contagious enthusiasm throughout this work. His thoughtful insights and continuous encouragement have played a crucial role in guiding the direction and enhancing the quality of this research, and his mentorship has made this journey both rewarding and inspiring. Abstract A fundamental challenge in computational music generation lies in developing control interfaces that provide intuitive, musically meaningful interactions with generative systems. This thesis addresses this challenge specifically for rhythmic generation, focusing on the development of a system capable of generating 16-step monophonic rhythmic patterns in real time using musically intuitive controls. Our method uses perceptually grounded rhythmic descriptors as an expressive, intuitive control space. A neural network is trained on all possible binary 16-step monophonic patterns, learning to map from descriptor space back to rhythmic patterns. We compare this descriptor-based approach to a variational autoencoder model and find the former more effective for usability and expressive control. An interactive interface is developed for exploration and testing, followed by quantitative and qualitative experiments evaluating the smoothness and user intuitiveness of the system. Findings show that the descriptor-based model aligns well with listener perception, balancing usability with expressive flexibility. While limited to monophonic rhythms, the system establishes descriptors as a strong foundation for extending interactive rhythm generation to polyphonic and more complex domains. Keywords: rhythm generation, descriptor engineering, variational autoencoders, generative music, symbolic music, real-time interaction. Chapter 1 Introduction One of the central challenges in computational music generation is designing control interfaces that allow users to interact with generative systems in ways that feel both intuitive and musically meaningful. This thesis focuses on this challenge for rhythm, developing a system capable of producing 16-step monophonic rhythmic patterns in real time, guided by controls that are easy for musicians to understand and use. Our main objective is to develop a musically meaningful method for generating 16-step monophonic rhythms in real time. To achieve this goal, we investigate two contrasting approaches: one based on abstract latent space representations learned through variational autoencoders (VAEs), and another grounded in explicit rhythm descriptors corresponding to established concepts in music perception theory. The first approach employs a VAE architecture that learns compact latent representations from rhythmic patterns without incorporating explicit musical knowledge. Following established work in VAE-based symbolic music generation (Brunner et al. 2018; Roberts et al. 2018; Vigliensoni et al. 2022), this method provides smooth interpolation capabilities through continuous latent spaces. The second approach directly maps rhythm descriptors to monophonic rhythm patterns using a feedforward neural network. These descriptors, grounded in music 1 Chapter 3 Methods 3.1 Problem Setup We frame the rhythm generation task as a mapping from a low-dimensional control space to 16-step binary monophonic rhythmic patterns. Each pattern consists of a sequence of binary values indicating silence (0) or onset (1), resulting in a total of 216 = 65,536 possible combinations. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Figure 1: Visualization of a 16-step rhythm pattern. In Figure 1, you can see an example of how a 16-step monophonic binary rhythm pattern looks like, onsets are represented as circles at steps 1, 5, 7, 9, and 13. At the core of our approach lies the question: how can we design a control space that is both compact and musically meaningful? To explore this, we investigate two contrasting strategies. The first strategy takes a purely data-driven approach: a VAE learns an abstract latent space from rhythmic patterns without incorporating any explicit musical priors. This latent space is compact and supports smooth interpolation between patterns, making it attractive for generative applications. However, because its dimensions are 8 3.2. Variational Autoencoder Model 9 not explicitly tied to perceptual or musical concepts, it is initially unclear whether this space will support intuitive or controllable user interaction. The second strategy leverages a set of perceptually grounded rhythm descriptors— such as syncopation, onset density, and balance—as an interpretable control space. Informed by rhythm perception literature, these descriptors allow users to directly influence musically relevant attributes during generation. By comparing the data-driven latent space learned by the VAE with the perceptually grounded descriptor-based control space, we aim to evaluate the trade-offs between generative expressiveness, interpretability, and user control in interactive rhythm generation. 3.2 Variational Autoencoder Model We train a VAE model to learn a compact latent space from binary rhythm patterns without relying on predefined musical features. The model aims to discover a low-dimensional, continuous representation that enables smooth interpolation and generation of rhythm patterns. Figure 2: Overview of the Variational Autoencoder model pipeline. 3.2.1 Model Architecture The autoencoder consists of two main components: an encoder, which maps the 16step binary rhythm patterns into a compact latent space that captures their essential features, and a decoder, which reconstructs the original patterns from this latent representation while preserving the key rhythmic structure. The following sections provide details on how these two layers have been configured: 10 Chapter 3. Methods Encoder •Fully connected layer with 64 neurons and ReLU activation. •Fully connected layer with dlatent neurons (no activation). Decoder •Fully connected layer with 64 neurons and ReLU activation. •Fully connected layer with 16 output neurons followed by a sigmoid activation to map outputs into [0,1] range. For our experiments, we set the latent space dimensionality to d= 5. This value is chosen with a knob-based interface in mind, as five knobs are considered an appropriate number to provide sufficient expressive control while maintaining a minimal and intuitive interface. 3.2.2 Training Procedure We train the autoencoder using all possible 16-step binary rhythm patterns (216 = 65,536 patterns), split into training (70%) and test (30%) sets. The model is optimized using the Binary Cross-Entropy (BCE) loss function, which is suitable for binary data reconstruction. The Adam optimizer is employed with a learning rate of 0.001. •Loss function: Binary Cross-Entropy Loss. •Optimizer: Adam. •Epochs: 400. •Batch size: 32. We monitor both reconstruction loss and binary reconstruction accuracy, where accuracy is computed as the proportion of correctly reconstructed onset positions after applying a 0.5 threshold on the decoder outputs. 3.3. Descriptor-based Model 11 3.2.3 Latent Space Sampling To assess the generative capability of the model, we sample random points from the latent space and decode them into rhythm patterns. While binary patterns are obtained by applying a threshold of 0.5 to the decoder outputs, we can also experiment with directly using the continuous outputs in the range [0,1] to represent note velocities. This approach allows the model to generate rhythms with dynamic intensity, capturing expressive variations beyond simple binary events and providing a richer representation of rhythmic nuances. 3.3 Descriptor-based Model We train a neural network model that maps perceptual rhythm descriptors to 16step monophonic rhythm patterns. Each descriptor represents a specific structural or perceptual property of rhythm, based on findings from rhythm perception literature. Figure 3: Overview of the Descriptor-based model pipeline. 3.3.1 Rhythm Descriptors A rhythm descriptor is a function that takes a monophonic rhythm pattern as input and returns a numerical value quantifying a specific structural or perceptual property of that rhythm. Examples include measures of onset density, syncopation, evenness, and balance. To ensure consistency and comparability across features, all descriptors are normalized to lie in the range [0,1], with 0 representing the minimum expression of a feature and 1 the maximum. This normalization allows the descriptors to be directly used as inputs to computational models, such as neural networks, without the need for additional scaling and ensures that each feature contributes proportionally during learning. 12 Chapter 3. Methods Onset count Number of active steps in a rhythmic pattern, represented by the number of onsets in a binary sequence. It provides a basic sense of rhythmic density or how many beats are played within the fixed number of steps. onsets = {i∈ {1,2,...,16} | patterni>0}nOnsets = |onsets| dOnsetCount =1 16 ·nOnsets Start Position of the first onset in the rhythm. It serves as a reference point for timing and can influence the perceived groove or alignment of the rhythm within a measure. dStart =1 16 ·min{i|patterni>0} Center Center of mass of the rhythm across 16 steps. It shows how the weight of the rhythm is distributed in time, helping to identify whether the rhythm feels front-heavy, backheavy, or balanced. dCenter =1 16 ·1 nOnsets · 16 X i=1 i·(patterni>0) Syncopation Measures how much a rhythm deviates from a regular metrical pattern. It quantifies the displacement of accents to weaker beats, providing insight into rhythmic complexity and tension (Huron 2006). w= [5,1,2,1,3,1,2,1,4,1,2,1,3,1,2,1] 3.3. Descriptor-based Model 13 si= max 0,(vi−v(i+1) mod 16)·(w(i+1) mod 16 −wi) dSyncopation =1 30 15 X i=0 si+ 15 This formula builds on the metrical hierarchy weights proposed by Lerdahl and Jackendoff (1983), later summarized in Toussaint (2013). In this hierarchy, beats at different positions within a 16-step pattern are assigned weights: strong downbeats (5), mid-points (4), half-beats (3), quarter subdivisions (2), and off-beats (1). This hierarchy is reflected in the weight vector wabove. The syncopation score sifollows the principle that rhythmic tension arises when an onset precedes silence or a weaker onset, but leads into a metrically stronger position. This is conceptually related to Longuet-Higgins and Lee (1984) and its later formalization by Fitch and Rosenfeld (2007), where syncopation is quantified as the difference in metrical weight between an onset and a subsequent rest at a stronger beat. The normalization factor ( 1 30) and offset (+15) ensure that syncopation values fall within a consistent range across 16-step rhythmic patterns. Syncopation awareness Refines the basic syncopation metric by factoring in perceptual salience. Each onset is weighted according to its perceived importance or noticeability to human listeners, yielding a measure that aligns more closely with musical perception (Gómez-Marín, Jordà, and Herrera 2015). a= [8,8,8,8,1,1,1,1,4,4,4,4,2,2,2,2] dSyncopationAwareness =1 115 15 X i=0 si·ai+ 65 14 Chapter 3. Methods Evenness As described in Milne and Herff (2020), and related to the concept of density, the evenness of a rhythm reflects the regularity of interonset intervals1. Rhythms with lower variance in interonset spacing are considered more even, while higher variance indicates irregularity. Our formulation adapts the geometric approach to rhythmic evenness introduced by Milne and Dean (2016), projecting onset positions onto the unit circle and comparing them against an ideal uniform distribution. dEvenness =1 nOnsets nOnsets−1 X k=0  cos2πk nOnsets −2π·onsetsk 16 +2π·onsets0 16  Balance Complementary to evenness, balance measures the symmetry of onset distribution. It is defined as the proximity of the rhythm’s center of mass to the center of the unit circle2. A rhythm with high balance is spread more symmetrically around the cycle, contributing to greater perceptual stability. Together, evenness and balance describe complementary aspects of rhythmic distribution: one quantifies uniformity of spacing, the other symmetry around the cycle. dBalance = 1 −1 nOnsets v u u u u u u u u t nOnsets−1 X k=0 cos 2πonsetsk 16 !2 + nOnsets−1 X k=0 sin 2πonsetsk 16 !2 Syness Combined metric that incorporates both syncopation and the number of onsets. It captures the interplay between rhythmic complexity and density, offering a nuanced view of groove and structure. 1An interonset interval is the time between two consecutive pulses (Milne and Herff 2020). 2The unit circle is a visualization of a rhythmic cycle where steps are placed around a circle, like positions on a clock (Milne and Herff 2020). 3.3. Descriptor-based Model 15 dSyness =1 0.633 ·dSyncopationAwareness nOnsets The normalization constant 1 0.633 corresponds to the maximum attainable value of the measure across all 16-step patterns, ensuring that dSyness ∈[0,1] 3.3.2 Discrete descriptor precision When using analog controls such as knobs, it is possible to obtain highly continuous input values. However, in the case of a digital user interface, we cannot assume access to continuous input signals. Moreover, in the context of our project, maintaining MIDI compatibility is an important requirement. To account for this, when generating the descriptor dataset we restricted values to the precision of the MIDI range (0–127). This inevitably introduces some redundancy in the data (e.g., multiple descriptor inputs may correspond to the same pattern). Input Type Dimensions Repeated Unique (%) MIDI 8 292 99.6 5 9742 85.1 Float 8 41 99.9 5 3855 94.1 Table 1: Precision using MIDI and floats to define descriptors. Table 1 summarizes the impact of using discrete MIDI values versus continuous floating-point values for defining descriptors. When restricting inputs to the MIDI resolution (0–127), a higher number of repeated patterns emerges, particularly in the 5-dimensional case, where over 9,000 repetitions occur and only 85.1% of patterns are uniquely identified. In contrast, using floating-point values greatly reduces redundancy, with nearly all patterns uniquely identified in both the 5 and 8-dimensional cases. These results highlight the trade-off between maintaining MIDI compatibility and achieving finer precision in the descriptor space. In the context of our experiment, we use the dataset in the MIDI range. 16 Chapter 3. Methods 3.3.3 Neural Network Implementation The model is a feedforward neural network trained to regress from rhythm descriptors to rhythm patterns. The input layer takes ndescriptor values, and the output layer produces a 16-dimensional vector of logits, which are passed through a sigmoid function to obtain onset probabilities for each of the 16 steps in the rhythm pattern. Again, these probabilities can be interpreted in two ways: (1) by applying a threshold (e.g. 0.5) to produce binary onset predictions indicating the presence or absence of an onset, or (2) by directly mapping the probabilities to velocity values, yielding a continuous representation of onset strength at each step. Architecture •Input layer:ndescriptor values. •Hidden layers: Four fully connected layers with 16, 32, 64, and 32 neurons respectively, each followed by a ReLU activation function. •Output layer: 16 neurons (no activation; sigmoid is applied externally to interpret the outputs as probabilities). The model is trained using the BCEWithLogitsLoss loss function, and optimized with the Adam optimizer. Training is performed for 200 epochs with mini-batches of size 32. Evaluation Methods To assess the quality of the model’s output, we employ two distinct evaluation strategies that reflect different aspects of rhythmic similarity: •Pattern-based accuracy: the predicted output is thresholded at 0.5 to obtain a binary pattern, which is then directly compared to the ground-truth binary rhythm pattern. Accuracy is computed as the proportion of correctly predicted onset positions across all steps and examples. 3.3. Descriptor-based Model 17 Score =1 16 16 X i=1 1[ˆpi=pi] •Descriptor-based accuracy: the predicted binary rhythm patterns are postprocessed to compute their descriptors (using the same feature extraction method as the input). Accuracy is computed based on the similarity (e.g., inverse normalized error) between the predicted descriptors and the original input descriptors. This version prioritizes perceptual or structural similarity over exact pattern match. Score = 1 −1 8 8 X j=1 |ˆ di−di| Further experiments will be conducted to determine which of the two evaluation approaches is more suitable for our use case. 3.3.4 Dimensionality Reduction To investigate the effectiveness of a reduced descriptor space for rhythm pattern generation, we conduct experiments using dimensionality reduction by systematically removing features from the full descriptor set. The motivation behind this process is to enable a more usable and compact control space—particularly relevant in interactive or hardware-based systems, such as knob-based interfaces, where having too many dimensions can hinder intuitive control. A smaller, well-chosen set of descriptors would allow for more expressive yet manageable manipulation of rhythm generation, enhancing both user experience and creative flexibility. This experiment is conducted using only the pattern-based accuracy. Specifically, we employ a leave-k-out approach, where k∈ {1,2,3}. In each case, k descriptors are excluded from the input feature set, and the model is retrained and evaluated using the remaining n−kdescriptors. This process allows us to evaluate how much each descriptor (or group of descriptors) contributes to the performance of the model, and whether a smaller subset of descriptors can still preserve predictive 24 Chapter 3. Methods Figure 6: Visualization of a movement in the descriptor space. 3.5.2 Implementation The experiment is implemented in a Python notebook that automatically generates a large number of random movements across the three distance categories, computes the above metrics, and saves the generated pattern sequences. We extend the Pure Data patch to support the simulation of these movements in the descriptor space. As shown in Figure 4, the two orange sliders allow users to navigate through the set of pre-generated movements and explore the interpolations interactively. The index slider selects the specific movement to be simulated, while the step slider controls the interpolation position along the movement. As the step slider is adjusted, the corresponding descriptor values smoothly interpolate from the start to the end point, and users can listen in real time to the gradual transformation of the generated rhythmic pattern. 3.6 User Experience Experiment To evaluate the usability and effectiveness of the system, we develop a fully webbased study where participants attempt to replicate reference rhythmic patterns using descriptor sliders. 3.6. User Experience Experiment 25 3.6.1 Implementation A web-based format is chosen to maximize accessibility and scalability, allowing a broader and more diverse pool of participants to take part in the study remotely. This approach also simplifies deployment and reduces setup time, enabling efficient collection of larger amounts of data across different devices and environments. Since the original interactive system is developed as a Pure Data patch, we reimplement it for the web. The webapp is built using the following stack: •React: for building the interactive user interface.3 •Tone.js: for sequencing and playing samples.4 •ONNX Runtime: to execute our final PyTorch model within the browser.5 This architecture ensures that pattern generation and audio playback are executed locally within the browser, minimizing latency and ensuring a smooth user experience. 3.6.2 Structure The experiment consists of the following stages: •Background questionnaire: participants provide demographic information, including age range and musical experience (years of study, performance, and percussion-specific experience). •Eight exercises: in each exercise, participants listen to a target rhythmic pattern (Pattern A) and attempt to replicate it using the provided sliders, which control rhythmic descriptors such as onset_count, start, center, syncopation, and balance. After submitting their solution (Pattern B), they rate 3https://react.dev 4https://tonejs.github.io 5https://onnxruntime.ai 26 Chapter 3. Methods the perceived similarity between the two patterns. The two patterns are randomized but they are always within a medium distance (as defined in the smoothness experiment). •Final feedback: participants offer their overall impressions regarding the user interface, the clarity of the task, and the difficulty of controlling each descriptor, along with open-ended comments about confusing aspects, liked features and suggestions for improvement. 3.6.3 Data Collection and Storage Throughout the study, a range of objective and subjective data points are collected for each participant and stored privately in Google Sheets using the App Scripts API. Some of these fields are directly submitted by the participants, while others derive from processing the experimental data (e.g., objective and parametric similarity metrics or elapsed time). No personal or identifiable information is collected, and all stored data is processed with the informed consent of the participants. The collected data includes: •Background information: –Participant ID –Age range (18-25, 26-35, 36-45, 46-55, 56+) –Years of musical study (0-4+ scale) –Years of musical performance (0-4+ scale) –Years spent performing percussion (0-4+ scale) •Exercise data: –Exercise number (1-8) –Target pattern (16-step pattern) –Submitted pattern (16-step pattern) 3.6. User Experience Experiment 27 –Target descriptors (5 descriptor values) –Initial descriptors (5 descriptor values) –Final descriptors (5 descriptor values) –Elapsed time (trial duration in seconds) –Subjective similarity (0-5 scale) –Objective similarity (euclidean distance between patterns)6 –Parametric similarity (euclidean distance between descriptors)7 •Final feedback: –Ratings of interface intuitiveness and difficulty per descriptor (0–5 scale). –Confusing aspects of the interface (free text) –Liked aspects of the interface (free text) –Feedback, suggestions, or ideas (free text) Figures 7 to 11 illustrate key stages of the user study, including the introduction, background questionnaire, exercise interface, and feedback questions. 6Defined by the euclidean distance between the presented pattern and the one submitted by the user. 7Computed with the euclidean distance between the initial set of descriptors and the final one defined by the user. 28 Chapter 3. Methods Figure 7: User experience experiment: intro and background questionnaire. Figure 8: User experience experiment: familiarity step. 3.6. User Experience Experiment 29 Figure 9: User experience experiment: exercise interaction interface. Figure 10: User experience experiment: exercise feedback form. 30 Chapter 3. Methods Figure 11: User experience experiment: final feedback questionnaire. Chapter 4 Results 4.1 Dimensionality Reduction We systematically performed a series of leave-k-out experiments, where we removed one, two, or three descriptors at a time and measured the classification accuracy. Tables 2, 3, and 4 summarize the results for the leave-one-out (L1O), leave-twoout (L2O), and leave-three-out (L3O) tests, respectively. Each row in these tables corresponds to one experimental configuration, listing the descriptors used and the resulting classification accuracy. Id Acc. (%) Descriptors 1 79.13 onset_count, start, center, syncopation, syncopation_awareness, evenness, balance 2 77.68 onset_count, start, center, syncopation, syncopation_awareness, evenness, syness 3 78.83 onset_count, start, center, syncopation, syncopation_awareness, balance, syness 4 78.38 onset_count, start, center, syncopation, evenness, balance, syness 5 74.92 onset_count, start, center, syncopation_awareness, evenness, balance, syness 6 76.22 onset_count, start, syncopation, syncopation_awareness, evenness, balance, syness 7 76.06 onset_count, center, syncopation, syncopation_awareness, evenness, balance, syness 8 78.94 start, center, syncopation, syncopation_awareness, evenness, balance, syness Table 2: Leave-one-out (L1O) experiment results. 31 32 Chapter 4. Results Id Acc. (%) Descriptors 1 77.66 onset_count, start, center, syncopation, syncopation_awareness, evenness 2 79.16 onset_count, start, center, syncopation, syncopation_awareness, balance 3 77.14 onset_count, start, center, syncopation, syncopation_awareness, syness 4 74.29 onset_count, start, center, syncopation, evenness, balance 5 77.05 onset_count, start, center, syncopation, evenness, syness 6 78.20 onset_count, start, center, syncopation, balance, syness 7 74.68 onset_count, start, center, syncopation_awareness, evenness, balance 8 73.59 onset_count, start, center, syncopation_awareness, evenness, syness 9 74.66 onset_count, start, center, syncopation_awareness, balance, syness 10 74.67 onset_count, start, center, evenness, balance, syness 11 76.11 onset_count, start, syncopation, syncopation_awareness, evenness, balance 12 74.87 onset_count, start, syncopation, syncopation_awareness, evenness, syness 13 74.42 onset_count, start, syncopation, syncopation_awareness, balance, syness 14 75.54 onset_count, start, syncopation, evenness, balance, syness 15 71.90 onset_count, start, syncopation_awareness, evenness, balance, syness 16 76.55 onset_count, center, syncopation, syncopation_awareness, evenness, balance 17 74.31 onset_count, center, syncopation, syncopation_awareness, evenness, syness 18 73.03 onset_count, center, syncopation, syncopation_awareness, balance, syness 19 75.48 onset_count, center, syncopation, evenness, balance, syness 20 72.46 onset_count, center, syncopation_awareness, evenness, balance, syness 21 70.21 onset_count, syncopation, syncopation_awareness, evenness, balance, syness 22 77.04 start, center, syncopation, syncopation_awareness, evenness, balance 23 77.68 start, center, syncopation, syncopation_awareness, evenness, syness 24 78.47 start, center, syncopation, syncopation_awareness, balance, syness 25 75.00 start, center, syncopation, evenness, balance, syness 26 74.79 start, center, syncopation_awareness, evenness, balance, syness 27 76.27 start, syncopation, syncopation_awareness, evenness, balance, syness 28 76.15 center, syncopation, syncopation_awareness, evenness, balance, syness Table 3: Leave-two-out (L2O) experiment results. 4.1. Dimensionality Reduction 33 Id Acc. (%) Descriptors 1 77.32 onset_count, start, center, syncopation, syncopation_awareness 2 73.02 onset_count, start, center, syncopation, evenness 3 74.05 onset_count, start, center, syncopation, balance 4 76.58 onset_count, start, center, syncopation, syness 5 73.56 onset_count, start, center, syncopation_awareness, evenness 6 74.58 onset_count, start, center, syncopation_awareness, balance 7 72.91 onset_count, start, center, syncopation_awareness, syness 8 71.16 onset_count, start, center, evenness, balance 9 73.58 onset_count, start, center, evenness, syness 10 74.47 onset_count, start, center, balance, syness 11 74.91 onset_count, start, syncopation, syncopation_awareness, evenness 12 74.31 onset_count, start, syncopation, syncopation_awareness, balance 13 73.65 onset_count, start, syncopation, syncopation_awareness, syness 14 70.78 onset_count, start, syncopation, evenness, balance 15 74.30 onset_count, start, syncopation, evenness, syness 16 73.62 onset_count, start, syncopation, balance, syness 17 71.92 onset_count, start, syncopation_awareness, evenness, balance 18 70.57 onset_count, start, syncopation_awareness, evenness, syness 19 69.94 onset_count, start, syncopation_awareness, balance, syness 20 71.77 onset_count, start, evenness, balance, syness 21 74.77 onset_count, center, syncopation, syncopation_awareness, evenness 22 73.06 onset_count, center, syncopation, syncopation_awareness, balance 23 71.99 onset_count, center, syncopation, syncopation_awareness, syness 24 71.77 onset_count, center, syncopation, evenness, balance 25 74.19 onset_count, center, syncopation, evenness, syness 26 72.63 onset_count, center, syncopation, balance, syness 27 72.59 onset_count, center, syncopation_awareness, evenness, balance 28 70.39 onset_count, center, syncopation_awareness, evenness, syness 29 68.58 onset_count, center, syncopation_awareness, balance, syness 30 72.28 onset_count, center, evenness, balance, syness 31 70.11 onset_count, syncopation, syncopation_awareness, evenness, balance 32 69.34 onset_count, syncopation, syncopation_awareness, evenness, syness 33 68.40 onset_count, syncopation, syncopation_awareness, balance, syness 34 69.74 onset_count, syncopation, evenness, balance, syness 35 65.77 onset_count, syncopation_awareness, evenness, balance, syness 36 75.22 start, center, syncopation, syncopation_awareness, evenness 37 75.36 start, center, syncopation, syncopation_awareness, balance 38 77.15 start, center, syncopation, syncopation_awareness, syness 39 72.73 start, center, syncopation, evenness, balance 40 73.57 start, center, syncopation, evenness, syness 41 74.08 start, center, syncopation, balance, syness 42 73.14 start, center, syncopation_awareness, evenness, balance 43 73.56 start, center, syncopation_awareness, evenness, syness 44 74.48 start, center, syncopation_awareness, balance, syness 45 72.84 start, center, evenness, balance, syness 46 72.86 start, syncopation, syncopation_awareness, evenness, balance 47 74.77 start, syncopation, syncopation_awareness, evenness, syness 48 74.26 start, syncopation, syncopation_awareness, balance, syness 49 71.08 start, syncopation, evenness, balance, syness 50 71.99 start, syncopation_awareness, evenness, balance, syness 51 73.75 center, syncopation, syncopation_awareness, evenness, balance 52 74.48 center, syncopation, syncopation_awareness, evenness, syness 53 72.84 center, syncopation, syncopation_awareness, balance, syness 54 72.33 center, syncopation, evenness, balance, syness 55 72.25 center, syncopation_awareness, evenness, balance, syness 56 70.04 syncopation, syncopation_awareness, evenness, balance, syness Table 4: Leave-three-out (L3O) experiment results. Chapter 5 Discussion In this section, we analyze and interpret the data collected in the results, examining the performance, usability, and perceptual relevance of the different rhythm generation approaches. By discussing both quantitative and qualitative findings, we aim to highlight the strengths, limitations, and practical implications of our descriptor-based rhythm generation approach. 5.1 VAE vs. Descriptor-Based Approaches The preliminary comparison between VAE and descriptor-based models validates the central hypothesis of this research. While the VAE approach demonstrated smooth latent space interpolation, it failed to provide meaningful user control due to abstract latent dimensions that made it impossible to predict or intentionally influence specific rhythmic properties. The descriptor-based model, enabled purposeful exploration and predictable outcomes. This trade-off between perfect smoothness and interpretable control favors interactive applications prioritizing user agency. The success reinforces the value of incorporating domain knowledge from music perception research rather than relying solely on data-driven feature learning. 40 5.2. Descriptor Selection 41 5.2 Descriptor Selection The leave-k-out experiments (Tables 2, 3, and 4) revealed important insights about descriptor redundancy and the minimal feature set required for effective rhythm generation. The results demonstrate that a reduced set of five descriptors (onset_count, start, center, syncopation and balance) maintains strong predictive performance while significantly simplifying the control space. The transition from eight to five descriptors resulted in only a modest decrease in pattern-based accuracy (from 78.35% to 74.05%, as seen in Table 4) while maintaining high descriptor-based accuracy (92.24%). This finding is particularly significant for practical applications, as it suggests that three descriptors (syncopation awareness, evenness, and syness) contribute relatively little unique information beyond what is captured by the core five features. The superior performance of descriptor-based accuracy (92.24%) compared to patternbased accuracy (74.05%) indicates that the model successfully learns to preserve the perceptual and structural properties of rhythms even when exact pattern reconstruction is imperfect. This aligns with music cognition research suggesting that human rhythm perception is more tolerant of surface-level variations when underlying structural relationships are maintained. 5.3 Smoothness Analysis The smoothness experiment (Figure 12) provided evidence for model selection between the pattern-based and descriptor-based training approaches. The ANOVA results in Table 5 confirm that the descriptor-based error metric produces significantly smoother transitions across all movement categories. The post-hoc Tukey’s HSD analysis results in Table 6 reveal a clear hierarchy in smoothness across movement magnitudes. Small movements consistently show the smoothest behavior, followed by medium movements, with large movements exhibiting the most variability. This hierarchy suggests that users can expect more predictable rhythmic transitions when making subtle descriptor adjustments compared to dramatic changes. 42 Chapter 5. Discussion The choice of KL divergence and Euclidean distance as complementary smoothness metrics proves valuable. KL divergence captures probabilistic differences in the model’s output distributions, reflecting the uncertainty and gradation in rhythm generation, while Euclidean distance in descriptor space measures how well the model preserves the intended control relationships. The concordance between these metrics strengthens confidence in the smoothness findings. 5.4 User Experience Experiment 5.4.1 Validation of the Descriptor-Based Approach The correlation between objective and subjective similarity measures (Figures 14 and 15) validates the use of Euclidean distance for rhythm pattern comparison and confirms that our computational measures align with human judgment. This alignment demonstrates that participants’ perceptual assessments of rhythmic similarity correspond meaningfully with algorithmic distance calculations, supporting the fundamental assumption that geometric relationships in descriptor space reflect musical relationships as perceived by listeners. Beyond validating our similarity metric, this correlation provides evidence for the effectiveness of our chosen descriptors as a control space. The fact that users’ subjective evaluations consistently relate to computational measures suggests that the five-dimensional descriptor space captures perceptually relevant aspects of rhythmic structure. This perceptual grounding distinguishes our approach from abstract latent representations, where such alignment between computational and human similarity judgments cannot be assumed. The statistical significance of this relationship establishes a foundation for automated evaluation of rhythm generation quality, enabling future systems to optimize for human-perceived similarity. Moreover, it validates the descriptor-based methodology as a bridge between computational representation and musical cognition, supporting the integration of music perception research with machine learning approaches for interactive music systems. 5.4. User Experience Experiment 43 5.4.2 User Background and Performance Table 7 summarizes the correlations between participant background, elapsed time doing the exercises, and the three types of similarity. The results reveal a weak but statistically significant negative correlation between elapsed time and objective similarity, suggesting that participants who spent more time on each exercise tended to perform slightly worse. This may indicate that longer interaction did not consistently support learning, possibly due to ineffective exploration, task difficulty, or frustration. Musical study experience also showed a weak negative correlation with subjective similarity ratings, revealing a paradox: more musically trained participants rated their reproductions as less similar to the targets, despite no significant difference in objective performance. This may reflect higher critical standards among experienced musicians or suggest that musical training shapes expectations about rhythm control that differ from those supported by the descriptor-based interface. By contrast, no significant correlations were found between age range, music performance experience, percussion performance experience, and any of the similarity measures. This suggests that the descriptor-based interface may be equally accessible to both specialists and non-specialists across age groups and experiential backgrounds. Nonetheless, a larger and more diverse sample of participants is warranted to obtain more robust insights. 5.4.3 Descriptor Interpretability User feedback revealed varying levels of descriptor intuitiveness (Figures 16 and 17): start and onset_count were most accessible, while syncopation and balance proved challenging. Despite syncopation’s theoretical grounding in music theory, users found it difficult to control in practice, indicating a gap between theoretical validity and practical usability. Balance, defined as symmetry of onset distribution around the unit circle, received low intuitiveness ratings. While mathematically well-defined, this geometric mea- 44 Chapter 5. Discussion sure may be too abstract for immediate musical comprehension without additional interface support or training. 5.5 Qualitative User Feedback Analysis In addition to quantitative measures, a qualitative analysis of user feedback provided rich insights into participants’ experiences with the descriptor-based rhythm interface. Tables 8, 9, and 10 summarize these findings. Table 8 highlights aspects of the system that users appreciated, such as the clarity of the interface, immediate auditory and visual feedback, and the enjoyment of exploratory learning. Participants generally found the start and onset_count parameters intuitive. Conversely, they reported difficulties with less tangible descriptors like syncopation and balance, reflecting a gap between theoretical relevance and practical interpretability. Participants also expressed a need for explanatory text or guidance, suggesting that some aspects of the interface were initially opaque. Table 9 distills recurring patterns in usability. Notably, users valued immediate feedback and hands-on experimentation, which facilitated engagement and learning. However, the complexity of certain sliders and parameter interactions sometimes hindered understanding, highlighting the importance of interface design that balances expressive control with accessibility. Finally, Table 10 presents specific feature requests. Users proposed contextual tooltips, tutorial systems, and visual aids to better map slider movements to rhythmic outcomes. Additional suggestions included adaptive interfaces tailored to user expertise and expanded control over musical parameters such as swing, instrument selection, and sound textures. These requests indicate directions for improving both learnability and creative flexibility in future systems. 5.6. Methodological Considerations and Limitations 45 5.6 Methodological Considerations and Limitations This research operates within several methodological constraints that shape both its contributions and applicability. The focus on 16-step monophonic patterns provides computational tractability and enables systematic analysis of the complete pattern space, but necessarily limits direct application to multiple genres of music that feature variable time signatures. However, this constraint serves the research goals of establishing fundamental principles for descriptor-based control that can inform more complex systems. The user study with 12 participants provides initial insights into system usability, but the sample size and participant composition limit the generalizability of findings. Notably, few participants had extensive percussion experience (Figure 13), which may have influenced interpretability ratings for rhythm-specific descriptors. The pattern replication task enables quantitative evaluation and objective performance metrics. While this may not fully reveal creative potential, it establishes baseline performance characteristics that inform system design, prioritizing rigorous evaluation of core principles over immediate practical deployment. Chapter 6 Conclusions The central goal of this research was to create a rhythm generation system where users can intuitively shape rhythmic patterns by controlling musically relevant properties. This chapter summarizes how we achieved this objective through systematic comparison of generation approaches, optimization of the control interface, and validation of the system’s effectiveness for musical interaction. We approached this problem by comparing two generation methods: variational autoencoders (VAEs) and descriptor-based neural networks. While the VAE approach is mathematically elegant and offers smooth latent space interpolation, it proved unsuitable for musical interaction due to its abstract, unintuitive control dimensions. In contrast, the descriptor-based approach—leveraging rhythm features grounded in music perception research—provided a more musically meaningful and engaging interface for users. Initially, we trained the neural network using 8 rhythm descriptors, however, in order to optimize the descriptor-based system for live use, we systematically reduced the control space from eight to five descriptors through leave-k-out experiments. This minimal set (onset_count, start position, center, syncopation, and balance) maintained a decent pattern accuracy while significantly simplifying the interface. We defined a smoothness metric to evaluate model performance, quantifying how 46 47 gradual changes in descriptor values correspond to smooth rhythmic transitions. The metric combined KL divergence and Euclidean distance in descriptor space to capture both probabilistic and geometric aspects of rhythm change. To systematically assess behavior, we introduced the concept of a movement, categorized into three types—small, medium, and large—based on the magnitude of descriptor adjustments. Using this metric across movement types, we identified the model that produced the most continuous and predictable rhythmic transformations. The user experience experiment, conducted with 12 participants, validated the system’s usability and effectiveness. Participants were able to successfully manipulate rhythmic properties using the descriptor sliders, demonstrating intuitive control over the generated rhythms. Moreover, the observed correlation between objective similarity metrics and participants’ subjective similarity ratings confirmed that our computational approach aligns closely with human rhythm perception, supporting the perceptual relevance of the chosen descriptors. While constrained to 16-step monophonic patterns and limited by our participant sample, this work demonstrates that perceptually grounded descriptors can bridge computational rhythm generation with intuitive musical control, establishing a foundation for real-time interactive rhythm systems. Chapter 7 Future Work A major direction for future research is the development of a pipeline that extends descriptor-controlled monophonic generation into polyphonic drum patterns. This next step would build on previous research, such as the tapping studies by Clark (2023)—which demonstrated how monophonic rhythmic input can be mapped to polyphonic outputs while preserving perceptual structure—and the dualized drum patterns dataset by Haki et al. (2023). By integrating our descriptor-based monophonic generator with polyphonic expansion techniques, users could design simple rhythmic skeletons through intuitive descriptor control and transform them into rich, multi-voice drum arrangements in real time. This approach would require further investigation into how descriptor relationships—such as syncopation and balance—translate to polyphonic structures and whether additional descriptors would be needed. Current experiments in this thesis relied on the full 16-step binary pattern dataset, which, while comprehensive, can introduce redundancy and noise in the resulting patterns. Future work could explore intelligent dataset reduction strategies to improve efficiency while preserving musical diversity, such as clustering rhythmically similar patterns or identifying archetypal rhythm families. Additionally, it would be interesting to explore extending the system to generate variable-length patterns beyond the 16-step constraint—which would require developing descriptor formula48 49 tions that generalize across different time signatures and pattern lengths. Finally, it would be valuable to replicate the user experience experiment with a larger and more diverse participant pool to obtain more robust insights into usability and perceptual alignment. Future studies could also consider an audio-only version of the experiment, removing the visual representation of patterns, to assess how well participants can manipulate and perceive rhythmic structures based purely on sound. Additionally, given that users in the current study were generally able to match target patterns effectively, presenting exercises with larger descriptor distances could help evaluate the system’s behavior over a broader range of rhythmic variations and further challenge participants’ control and perceptual sensitivity.