Neural Engine Sound Synthesis with Physics-Informed Inductive Biases and Differentiable Signal Processing

Master thesis on Sound and Music Computing Universitat Pompeu Fabra Neural Engine Sound Synthesis with Physics-Informed Inductive Biases and Differentiable Signal Processing Robin Doerfler Supervisor: Lonce Wyse August 2025 Acknowledgements First and foremost, I would like to thank my advisor, Lonce Wyse, for his thoughtful supervision and guidance throughout this project. Our discussions have not only shaped this work but have also been genuinely enjoyable and inspiring. I extend my gratitude to Xavier Serra and Esteban Gutiérrez for being on my defense committee and taking the time to read through and evaluate this work. I truly appreciate your expertise and valuable feedback. Special thanks to Anmol and Satya for being amazing friends and wonderful company throughout this program. You made everything so much more enjoyable, and I’m sure our friendship will continue far beyond this academic chapter. I am sincerely grateful to my company for trusting me to pursue this degree while working full-time. Balancing both commitments wasn’t always easy, but their flexibility and support made it possible. Thanks to my family and friends for all their encouragement and unwavering belief in me along the way. Your support meant the world to me, especially during the more challenging moments. Finally, my deepest gratitude goes to my partner Beste for your patience and understanding throughout this entire process. Thank you for your unconditional support and for celebrating every milestone, big and small, with me. It has been quite a journey, and I am grateful to everyone who helped make it possible. Abstract Engine sound synthesis is increasingly important in automotive audio and interactive media, yet presents unique challenges for neural audio generation that distinguish it from musical audio paradigms. Unlike sustained musical tones, where periodic oscillations exist inherently in the acoustic vibration, engine sounds emerge from sequential combustion events that generate sharp pressure transients recurring at rates from 600 to over 8000 RPM. This creates acoustic phenomena exhibiting significant inharmonicity, extremely low fundamental frequencies—down to 5 Hz—and rapid temporal sequences with intervals below 2 milliseconds, demanding approaches that can model both precision in timing and complexity in timbral evolution, beyond conventional musical audio assumptions. While existing differentiable digital signal processing (DDSP) methods have demonstrated success across various audio synthesis tasks, they often rely on generic synthesis modules that do not explicitly recognize or incorporate the acoustic principles and physical mechanisms underlying engine sounds. This thesis presents a novel approach to engine sound synthesis through systematic integration of physics-informed inductive biases within the entire differentiable synthesis pipeline. It proposes the Procedural Engines Model (PRCE), a deep learning architecture that combines time-varying embeddings of RPM and torque parameters – including their temporal derivatives – and derived conditioning signals – throttle position and deceleration fuel cutoff (DFCO) – with specialized model heads for physics-informed parameter conversion driving two custom differentiable synthesizer configurations that incorporate domain-specific acoustic principles. To guide learning toward accurate engine timbre reproduction, a custom loss function is introduced that prioritizes spectral energy near engine-order harmonics, drawing inspiration from Campbell diagrams commonly used in noise, vibration, and harshness (NVH) analysis. Engine sounds present a fundamental duality: while in reality a sum of structured noise-like pressure pulses, they manifest as distinctly harmonic acoustic phenomena. This motivates two complementary synthesis strategies that provide contrasting optimization pathways toward the same acoustic target: direct spectral-temporal reconstruction that implicitly reflects the underlying pulse structure, and explicit pulse sequence modeling through acoustic simulation of individual combustion events, their temporal alignment and exhaust system propagation. The PRCE framework implements both perspectives as two configurations. The Harmonic-Plus-Noise (HPN) variant employs modified harmonic synthesis with systematic inharmonicity and temporal-spectral structuring of noise components to model observable acoustic characteristics. The Pulse-Train-Resonator (PTR) configuration directly models physical–acoustic phenomena by composing combustion pulses aligned to engine firing patterns and propagating them through differentiable resonator networks simulating exhaust acoustics. Evaluation on procedurally generated engine sound datasets totaling 2.5 hours across varied operating conditions reveals complementary strengths between synthesis approaches. PTR achieves modestly superior validation performance (5.7% improvement in total loss) and demonstrates more consistent training-validation transfer, while HPN shows greater flexibility across diverse engine configurations and robustness to harmonic irregularities. Both variants successfully capture authentic engine acoustic behaviors despite distinct synthesis strategies and their audible signatures. This research demonstrates systematic integration of physics-informed inductive biases into differentiable synthesis architectures, providing a methodological framework applicable to physically-constrained audio generation beyond automotive contexts. The work reveals that domain-specific biases produce distinct acoustic signatures that influence both optimization strategies and perceptual outcomes. To support future research, we openly publish the Procedural Engines Dataset, a comprehensive collection of procedurally generated engine audio with time-aligned control annotations and the complete PRCE model pipeline. Keywords: Engine Sound Synthesis, Differentiable Signal Processing, PhysicsInformed Neural Networks, Inductive Biases, Neural Audio Synthesis Contents Acknowledgements Abstract List of Figures List of Tables 1 Introduction 1 1.1 Motivation and Research Question . . . . . . . . . . . . . . . . . . . . 1 1.1.1 Motivation ............................... 1 1.1.2 ResearchQuestion ........................... 3 1.2 Objectives and Rationale . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.2.1 Research Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.2.2 Rationale ................................ 6 1.3 StructureoftheThesis........................... 7 2 Background - State of the Art 10 2.1 Applications of Engine Sound Synthesis . . . . . . . . . . . . . . . . . . 10 2.2 Engine Acoustics Fundamentals . . . . . . . . . . . . . . . . . . . . . . 11 2.3 Traditional Synthesis Methods . . . . . . . . . . . . . . . . . . . . . . . 14 2.3.1 Sample-Based Approaches . . . . . . . . . . . . . . . . . . . . . . . 14 2.3.2 Procedural Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.3.3 Limitations and Gaps . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.4 Neural Synthesis: DDSP . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.4.1 DDSPFoundations........................... 19 2.4.2 DDSP in Engine Sound Synthesis . . . . . . . . . . . . . . . . . . . 19 2.4.3 Related DDSP Developments . . . . . . . . . . . . . . . . . . . . . 22 2.4.4 Engine Sound Datasets . . . . . . . . . . . . . . . . . . . . . . . . . 25 2.5 Research Landscape and Gaps . . . . . . . . . . . . . . . . . . . . . . . 25 3 Procedural Engine Sounds Dataset 26 3.1 Dataset Generation Methodology . . . . . . . . . . . . . . . . . . . . . 27 3.1.1 Feature Extraction from Real Engine Data . . . . . . . . . . . . . . 27 3.1.2 Real-Time Synthesis Architecture . . . . . . . . . . . . . . . . . . . 28 3.1.3 Dataset Augmentation Strategy . . . . . . . . . . . . . . . . . . . . 29 3.1.4 Data Format and Synchronization . . . . . . . . . . . . . . . . . . 30 3.2 Dataset Validation and Analysis . . . . . . . . . . . . . . . . . . . . . . 31 3.2.1 Representativeness and Realism Assessment . . . . . . . . . . . . . 31 3.2.2 Statistical Distribution and Variability Analysis . . . . . . . . . . . 32 3.3 Applications and Research Implications . . . . . . . . . . . . . . . . . . 34 3.3.1 Automotive Audio Processing Applications . . . . . . . . . . . . . . 34 3.3.2 Parameter Estimation and Inverse Modeling . . . . . . . . . . . . . 34 4 Procedural Engines Model (PRCE) 36 4.1 Architecture Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 4.2 Input Feature Engineering . . . . . . . . . . . . . . . . . . . . . . . . . 38 4.3 Conditioning Signal Design . . . . . . . . . . . . . . . . . . . . . . . . 40 4.3.1 DesignRationale............................ 40 4.3.2 Harmonic Frequency Scaling . . . . . . . . . . . . . . . . . . . . . . 40 4.3.3 Virtual Throttle and DFCO Derivation . . . . . . . . . . . . . . . . 41 5 Synthesizers and Model Heads 42 5.1 Differentiable Synthesis Methodology . . . . . . . . . . . . . . . . . . . 42 5.1.1 Bridging Traditional Methods and Neural Synthesis . . . . . . . . . 42 5.1.2 The Impulsive Origin of Engine Harmonicity . . . . . . . . . . . . . 43 5.1.3 Dual Synthesis Strategy . . . . . . . . . . . . . . . . . . . . . . . . 44 5.2 Harmonic-Plus-Noise Engine Sound Synthesis . . . . . . . . . . . . . . 44 5.2.1 Physically-Informed Synthesis Components . . . . . . . . . . . . . 45 5.2.2 Neural Parameter Control . . . . . . . . . . . . . . . . . . . . . . . 48 5.2.3 Complete Algorithm Formulation . . . . . . . . . . . . . . . . . . . 50 5.3 Pulse-Train-Resonator Engine Sound Synthesis . . . . . . . . . . . . . 51 5.3.1 Pulse Wave Composition . . . . . . . . . . . . . . . . . . . . . . . . 51 5.3.2 Firing Order Sequencer . . . . . . . . . . . . . . . . . . . . . . . . 57 5.3.3 Neural Parameter Control . . . . . . . . . . . . . . . . . . . . . . . 59 5.3.4 Differentiable Karplus-Strong Resonator . . . . . . . . . . . . . . . 61 6 Loss Functions 71 6.1 Multi-Resolution STFT Loss . . . . . . . . . . . . . . . . . . . . . . . . 71 6.2 HarmonicLoss ............................... 73 7 Training 75 7.1 ModelArchitecture............................. 75 7.2 Training and Validation Data . . . . . . . . . . . . . . . . . . . . . . . 75 7.3 DataProcessing............................... 76 7.4 OptimizationSetup............................. 77 7.5 Parameter Initialization . . . . . . . . . . . . . . . . . . . . . . . . . . 77 8 Results 78 8.1 Quantitative Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 8.1.1 Convergence and Performance . . . . . . . . . . . . . . . . . . . . . 78 8.1.2 Loss Component Analysis . . . . . . . . . . . . . . . . . . . . . . . 81 8.1.3 Generalization Across Datasets . . . . . . . . . . . . . . . . . . . . 81 8.2 Subjective Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 8.2.1 Overall Reproduction Quality . . . . . . . . . . . . . . . . . . . . . 84 8.2.2 Audible Signatures of Inductive Biases . . . . . . . . . . . . . . . . 84 8.2.3 Dynamic Response to Control Parameters . . . . . . . . . . . . . . 86 8.2.4 Noise and Artifacts . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 9 Conclusions and Future Work 88 9.1 Summary of Contributions . . . . . . . . . . . . . . . . . . . . . . . . . 88 9.2 KeyFindings ................................ 89 9.3 Implications................................. 89 9.4 FutureWork................................. 90 DataandCodeAvailability ............................ 92 Appendices 93 A Harmonic Deviation Analysis 93 B Supplementary Training Results 96 Bibliography 98 List of Figures 1 AudioandControlData ......................... 30 2 DatasetComparison ........................... 32 3 Procedural Engines Model overview . . . . . . . . . . . . . . . . . . . 36 4 HPNSignalflow.............................. 45 5 Pulse shapes for different parameter settings . . . . . . . . . . . . . . 56 6 Karplus-Strong Algorithm . . . . . . . . . . . . . . . . . . . . . . . . 62 7 Harmonic Mask applied to predicted and target spectral frame . . . . 74 8 Convergence Comparison for Total Loss . . . . . . . . . . . . . . . . . 79 9 Convergence Comparison for Harmonic Loss . . . . . . . . . . . . . . 82 10 Convergence Comparison for Multi-Resolution STFT Loss . . . . . . 83 11 Group Engine Order Harmonic Deviations . . . . . . . . . . . . . . . 94 12 Single Continuous Engine Order Harmonic Deviations . . . . . . . . . 95 List of Tables 1 DatasetStructure............................. 32 2 RPM and Torque Distributions . . . . . . . . . . . . . . . . . . . . . 33 3 Validation Loss Comparison . . . . . . . . . . . . . . . . . . . . . . . 80 4 Training Loss Comparison . . . . . . . . . . . . . . . . . . . . . . . . 80 5 Validation Loss at Early Stopping . . . . . . . . . . . . . . . . . . . . 96 6 Training Loss at Early Stopping . . . . . . . . . . . . . . . . . . . . . 97 instrument paradigms and may not effectively capture the continuous mechanical processes governing engine acoustics. The development of dual synthesis approaches addresses this limitation by providing both intuitive harmonic-based modeling and physically-grounded pressure-wave simulation within the same theoretical framework. The core methodological challenge lies in translating insights about engine sound acoustics into differentiable synthesis components that can guide the learning process toward plausible and interpretable solutions through fundamentally different computational pathways. The comparative evaluation of acoustic characteristics versus physical processes modeling approaches provides valuable insights into the relationship between physical understanding and synthesis architecture design. By developing both a synthetic dataset and dual physically-informed synthesis architectures, this work addresses the data bottleneck while making the primary contribution of demonstrating how systematic integration of physical knowledge can enhance neural audio synthesis performance through multiple perspectives, extending beyond traditional music-centric applications and providing a methodological framework applicable to other complex acoustic modeling tasks. 1.3 Structure of the Thesis This thesis is structured across nine chapters, progressing from theoretical foundations through methodological development to empirical validation. It follows a systematic approach: establishing the research context and identifying gaps in existing approaches, developing the necessary data infrastructure, designing dual synthesis architectures based on physics-informed principles, and finally evaluating their comparative performance. An overview of each chapter is provided below. 7 Chapter 1 – Introduction: Establishes the research motivation by identifying the unique challenges of engine sound synthesis within neural audio generation. Introduces the dual modeling paradigm, formulates the central research questions, and outlines the specific objectives for investigating physics-informed synthesis architectures. Chapter 2 – Background - State of the Art: Provides comprehensive coverage of engine acoustics fundamentals, traditional synthesis methods, and recent developments in neural audio synthesis with particular focus on DDSP approaches. Reviews existing applications, identifies limitations in current methodologies, and establishes the theoretical foundation for physics-informed synthesis. Concludes by highlighting research gaps that motivate the dual modeling approach. Chapter 3 – Procedural Engine Sounds Dataset: Details the development of a comprehensive synthetic dataset designed to support controlled investigation of both synthesis paradigms. Describes the real-time synthesis architecture used for data generation, feature extraction methodologies from real engine recordings, and validation procedures ensuring acoustic realism. Establishes the data foundation necessary for robust comparative evaluation of the proposed models. Chapter 4 – Procedural Engines Model (PRCE): Presents the core neural architecture shared by both synthesis approaches, including input feature engineering strategies and conditioning signal design. Introduces virtual throttle and DFCO signal derivation that enable physics-informed parameter control. Establishes the common foundation upon which both HPN and PTR synthesis variants are built. Chapter 5 – Synthesizers and Model Heads: Develops the theoretical framework for differentiable synthesis methodology and presents detailed implementations of both modeling approaches. The Harmonic-Plus-Noise (HPN) configuration implements direct spectral-temporal modeling through systematic domain-specific adaptations such as learnable inharmonicity and RPM-synchronized, temporally structured noise components. The Pulse-Train-Resonator (PTR) configuration realizes 8 pulse sequence modeling through acoustic pressure pulse simulation and exhaust system resonance modeling. Establishes the acoustic duality as the theoretical bridge between these approaches. Chapter 6 – Loss Functions: Introduces domain-specific loss function design that guides learning toward physically plausible solutions. Develops multi-resolution STFT loss for broad spectral coverage and harmonic loss components that prioritize engine-order harmonic content, drawing inspiration from NVH analysis principles used in automotive engineering. Chapter 7 – Training: Documents the training methodology including hyperparameter selection, initialization strategies, and data preparation procedures. Establishes consistent training protocols that enable fair comparison between HPN and PTR approaches while accounting for their different computational requirements and convergence characteristics. Chapter 8 – Results: Presents comprehensive comparative evaluation of both synthesis approaches across multiple dimensions: training dynamics and convergence behavior, reconstruction quality metrics, loss decomposition, generalization performance across different engine configurations, and subjective assessment of perceptual quality. Analyzes the practical manifestation of the dual modeling paradigm regarding objective metrics and beyond. Chapter 9 – Conclusions and Future Work: Summarizes findings from the comparative investigation, discusses the broader implications for physics-informed neural engine sound synthesis, and identifies opportunities for extending this methodology. Outlines future research directions and provides access information for the open-sourced code, models, and datasets developed in this work. 9 Chapter 2 Background - State of the Art This thesis investigates the synthesis of engine sounds through neural synthesis methodologies. This chapter provides a foundational review of the core concepts underlying this research area. It begins with an examination of engine sound synthesis application domains, followed by a review of traditional signal processing and synthesis methodologies for engine sound analysis and synthesis. An overview of contemporary neural synthesis approaches is then presented, with particular focus on differentiable signal processing techniques demonstrating promising efficacy in this field. Finally, the availability of relevant training datasets is assessed, and a summary is provided that highlights the current state of the art and identifies existing research gaps. 2.1 Applications of Engine Sound Synthesis Engine sound synthesis finds significant applications across multiple industries, with the automotive sector being the primary domain for implementation. The sound of internal combustion engines has evolved into a key perceptual aspect of vehicle performance and brand identity: Chang et al. [1] related brand positioning attributes to certain sonic characteristics of combustion engine sounds. Moon et al. [2] further investigated distinct inter-harmonic relationships and their relation to perceptional aspects. Melman et al. [3] demonstrated that vehicle sound modifications influ10 ences driving behavior, thereby serving even beyond the mere auditory experience. Schramm et al. [4] built on this concept by adaptively modifying engine sounds, matching predicted driver engagement and studying their interplay. As illustrated by Belschner and Bodden [5], the artificial modification and enhancement of engine sounds by Active Sound Design departments represents a decades-old procedure, which remains relevant in contemporary applications, including electric vehicles. This is supported by the work of Küppers et al. [6], suggesting that the application of driving sounds and their perceptual benefits remain desired, even in the context of electrified mobility, where traditional engine sounds are absent. Beyond automotive applications, real-time generation of engine noises represents a widespread application in the Game Audio industry Collins et al. [7], where adaptive synthesis techniques maintain immersion in interactive environments. 2.2 Engine Acoustics Fundamentals Engine sounds constitute a complex acoustic phenomenon with distinctive characteristics that differ significantly from musical instruments. Understanding these physical principles is crucial for developing effective synthesis methods aligned with the domain-specific characteristics of engine acoustics, as comprehensively reviewed by Jones [8]. Combustion engines produce propulsion by compressing and igniting an air-fuel mixture in cylinders. The resulting expansion of gases applies force to the piston, which transfers energy through the crankshaft to the drivetrain, ultimately generating rotational motion that propels the vehicle. The fundamental components of this acoustic phenomenon arise from alternating pressure fluctuations, driven by the sequential, periodic processes of air intake, compression, combustion, and exhaust within the engine’s combustion chambers. These pressure waves propagate through the complex geometry of the exhaust system, where they undergo interference, compression, resonance, and fluctuations in pressure, temperature, and air density, resulting in a highly complex and nonlinear acoustic phenomenon, as noted by Balistreri [9]. 11 What is commonly referred to as engine sound primarily originates from exhaust pressure pulses rather than direct combustion noise. While intake systems and combustion processes also contribute acoustic signatures, exhaust pressure pulses dominate the perceived sound character. Combustion generates broadband pulsating noise as a function of engine speed, load, and fuel input, and primarily acts as an exciter signal. Load and throttle position significantly influence acoustic characteristics by modulating backpressure, combustion intensity, and the amplitude of pressure waves. The characteristic tonal qualities arise from the lower-frequency exhaust pulses that occur at specific points in the engine’s rotation cycle. These pulses produce distinctive harmonic patterns, referred to in domain literature as engine orders, which are determined by the engine’s configuration, cylinder count, and firing sequence. In a four-stroke engine, the most common combustion engine, each piston produces one exhaust pulse for every two crankshaft rotations, corresponding to the 0.5th order (one pulse every other revolution). The fundamental frequency of an engine sound is given by the simple relationship f0[Hz] = N·RPM 120 ,(2.1) where Nis the number of cylinders, and RPM is the engine rotational speed in revolutions per minute. The path these pressure pulses travel significantly transforms the final acoustic output. Header design acts as an acoustic filter that dramatically alters the timing and interaction of these pressure waves: •Equal-length headers preserve the original timing sequence of pulses. •Unequal-length headers scramble pulse timing, creating distinctive acoustic signatures. •Manifold shape, diameter changes, and junction geometry further modify the sound through reflection and interference patterns. 12 These acoustic transformations effectively function as a complex, time-variant resonator system shaping the initial pressure pulses of distinct firing sequences, resulting in a complex, partially harmonic signal with both static and rhythmically varying noise components. The synthesis of such signals presents several unique challenges: 1. Dynamic Spectral Evolution: Unlike musical instruments with relatively stable harmonic structures, engine harmonics shift dramatically with changes in performance parameters, creating complex, continuously evolving timbres. 2. Stochastic Components: These are intrinsically tied to the engine revolution cycles and firing sequence. They occur as periodic pressure pulses, spanning from distinct audible rhythmic patterns at low engine speeds to dense, high-frequency pulse trains that blend into being perceived as periodic signals rather than distinct pulses. 3. Non-Linear Acoustic Interactions: Pressure waves interact through constructive and destructive interference in the exhaust system, creating emergent acoustic phenomena difficult to model with simple additive or subtractive synthesis techniques. These physical characteristics and signal analytical observations directly inform the design considerations for neural synthesis methods discussed in subsequent chapters. The complex interaction between excitation signals (combustion events) and acoustic filtering (exhaust system resonance) suggests that neural architectures modeling this excitation-resonator relationship may prove more effective than general-purpose harmonic-plus-noise models. The acoustic signatures of engines thus represent fascinating study subjects at the intersection of physics, signal processing, and perceptual acoustics, demanding computational approaches that can accurately model both their physical principles and perceptual characteristics. 13 2.3 Traditional Synthesis Methods Engine sound synthesis has evolved through several methodological approaches, broadly categorized into sample-based methods and procedural synthesis techniques. This chapter reviews these traditional approaches, examining their strengths, limitations, and applications before introducing neural synthesis methods as the focus of this thesis. 2.3.1 Sample-Based Approaches Sample-based approaches rely on recorded audio segments that are processed and reassembled according to control parameters. These methods can be further categorized into loopor wavetable-based methods, concatenative synthesis, and granular synthesis: Loop Playback and Wavetable Synthesis Loop-based methods, as described by Koivisto [10], operate by recording engine noise at a few static engine speeds. These recordings undergo editing to obtain seamless periodic audio loops that avoid discontinuities at their boundaries. They can be as small as wavetables of a few hundred samples, merely spanning a single period of the fundamental harmonic oscillation, as shown, for example, by Heitbrink and Cable [11]. The processed samples are then implemented in audio sampling engines. To achieve intermediate engine speeds between the recorded samples, the system either under-samples or over-samples the audio playback, effectively modulating the speed and pitch of the original sample within an acceptable range of pitch shift. This approach offers computational efficiency and a low memory footprint, but can produce acoustic artifacts when pitch modulation exceeds natural-sounding limits. Moreover, it provides limited flexibility beyond the recorded data and lacks the natural variance inherent in physical systems. 14 Concatenative Synthesis Concatenative synthesis strategies, in contrast, rely on recordings that continuously span the entire engine speed range, ensuring comprehensive coverage for synthesis at arbitrary speeds. As demonstrated by Jagla et al. [12], target engine speed noise is recreated by dynamically selecting and extracting sound segments from the original recordings and concatenating them using overlap-add methods. This approach efficiently reconstructs engine noises across all operating speeds, providing a computationally feasible solution for interactive simulations and virtual environments. More sophisticated approaches further decompose recordings into component parts. Chen et al. [13] and Li et al. [14] propose methods that separate low-frequency harmonics from high-frequency narrow-band amplitude-modulated signals. This decomposition enables frame-wise resynthesis, eliminates the need for phase-accurate sample alignment via Griffin-Lim overlap-add, and forms a hybrid approach that combines component-based resynthesis with sample-based techniques. Overall, concatenative methods differ in flexibility, real-time capability, and sonic realism, often relying primarily on engine speed as the control parameter. Granular Synthesis Granular synthesis methods, widely used in the game industry as noted by MacGregor [15], split audio examples into small units called “grains.” Different sound qualities and timbres are achieved by varying the waveform, envelope, duration, and density of these grains. This approach can produce more natural-sounding results through grain-level randomization, including small position shifts, pitch variations, gain adjustments, and modifications to grain size. While granular synthesis offers greater flexibility and the potential for more natural results, its grain-level real-time processing typically makes it more computationally demanding than other sample-based playback methods. It also presents several practical challenges: meticulous parameter tuning is required due to the numerous interrelated settings, which can be time-consuming and unpre15 dictable; precise alignment of grains with engine events, such as piston firings, is difficult and may result in timing artifacts or loss of rhythmic coherence; and the setup is labor-intensive, involving careful selection and cleanup of source recordings to avoid unwanted noise and ensure consistent grain material. 2.3.2 Procedural Synthesis In contrast to sample-based methods, procedural synthesis generates engine sounds directly through signal-processing techniques, primarily using additive, subtractive, and physically informed synthesis methods. Additive Synthesis Additive synthesis of engine harmonics has emerged as the predominant method within the automotive industry, as discussed by Bodden and Belschner [16] and Boussard et al. [17]. This approach is particularly suitable in applications where the actual engine noise is present but requires timbral modification through the addition and enhancement of single engine harmonics. The amplitudes of these harmonics are modulated as a function of engine parameters, including RPM, load, and gear, providing an efficient means to precisely modify the harmonic timbre of the original engine noise. While effective, the harmonic-based approach represents an idealized implementation that often neglects stochastic components and reduces the harmonics to a manageable subset, as the number of tunable parameters grows rapidly with each harmonic, requiring amplitude and frequency definitions across all combinations of engine performance parameters. Spectral Synthesis Cao et al. [18] evaluate engine order synthesis beyond the presence of a physical engine, specifically in the context of electrified vehicles. They extract magnitude spectra from engine recordings and synthesize new signals using the inverse shorttime Fourier transform combined with overlap-add. 16 challenge, Hayes et al. [31] proposed a complex exponential surrogate approach that replaces direct frequency optimization with optimization of complex parameters. By leveraging PyTorch’s Wirtinger derivatives for complex-valued parameters, this method enables gradient-based frequency estimation that avoids the local minima typical of direct sinusoidal frequency optimization. Stochastic Synthesis Barahona-Rios and Collins introduced NoiseBandNet [32], a model that addresses the time–frequency trade-off of spectral noise synthesis while simplifying the overall architecture. The system predicts time-varying amplitude envelopes for each noise band at frame rate, then upsamples them linearly to the audio sampling rate before applying them to a precomputed filtered noise bank. This design achieves high resolution in both time and frequency, closely aligning with our objective of a synthesizer operating directly at audio rate for all predicted parameters. Interestingly, they also found that providing MFCCs as input features was not essential, and that even non-deterministic, perceptually motivated control signals – manually drawn according to human intuition – were sufficient to guide the model during training and inference. This suggests that physically meaningful parameters, such as engine torque, can provide reliably structured information for learning acoustic patterns, even if they are not strict mathematical audio descriptors like loudness or f0. Resonator Modeling The original DDSP framework used learned impulse responses applied via convolution for reverberation in musical applications. For engine sound synthesis, however, reverberation is less relevant; instead, resonators – capturing the dynamic acoustic interplay within exhaust pipes – play a central role. Resonators are typically described as infinite impulse response (IIR) filters: systems whose outputs depend not only on current excitation but also on past states. Their recursive nature makes them efficient, but it also complicates gradient-based opti23 mization, as discussed in Chapter 5.3.4. Recent work has proposed several strategies for implementing differentiable IIR filters and resonators: Recurrent implementations. A direct approach is to unroll the recursion and implement IIR filters as recurrent layers [33, 34, 35]. While exact, this method becomes prohibitively slow when scaling to many filters operating at audio sample rate. Frequency-domain methods. An alternative is to avoid time-domain recursion entirely by evaluating the filter’s transfer function on a frequency grid. This ”frequencysampling trick” converts coefficients into a differentiable frequency response, which is then applied in the spectral domain. Notable examples include Nercessian et al. [36], who modeled audio distortion effects with interpretable IIR systems, and Diaz et al. [37], who applied the method to differentiable modal synthesis of rigidbody resonances. Data-driven regression. Colonel et al. [38] inverted the problem by training a neural network (IIRNet) to map target magnitude responses directly to cascaded biquad coefficients. This amortized approach bypasses iterative optimization and yields coefficients that can be used in a standard time-domain implementation. Direct differentiable IIRs. Yu and Fazekas [39, 40] introduced differentiable all-pole filters, successfully applying them as LPC vocoders and to effects such as phasers, subtractive synthesizers, and compressors. Waveguide-inspired resonators. Among the most elegant historical frameworks for modeling resonant systems such as tubes, membranes, and strings are waveguide formulations, with the Karplus–Strong algorithm, originally proposed by Karplus and Strong [41] and extended by Smith [42], being a particularly influential approximation. However, differentiable waveguide-inspired resonators remain largely unexplored, despite their potential for efficient and interpretable modeling of physical resonance phenomena. In summary, a variety of promising strategies exist for making resonant systems differentiable, ranging from recurrent formulations and frequency-domain approx24 imations to data-driven regression and direct IIR designs. Yet, waveguide-based approaches – long valued in physical modeling synthesis – have not been fully leveraged in the differentiable domain, representing a compelling direction for this work. 2.4.4 Engine Sound Datasets Datasets for work on the aforementioned neural approaches on engine sound synthesis (2.4.2) were obtained by the authors and are, despite not being available, reported to be limited in size and annotation. Existing engine sound collections focus primarily on classification tasks and anomalous sound detection rather than synthesis, with audio typically provided with insufficient quality and annotations describing vehicle classes or machine health conditions rather than parameters relevant to real-time sound generation [43, 44, 45, 46, 47]. These datasets lack the audio quality, standardization, and detailed timealigned annotations necessary for developing advanced generative models. Based on the promising applications of deep learning in this field, it is assumed that one of the main bottlenecks hindering research may be the lack of available data required for training of generative models. 2.5 Research Landscape and Gaps The field of engine sound synthesis encompasses a wide range of approaches, yet the state-of-the-art remains unclear. Perceptually high-quality methods are often insufficiently documented, while more theoretical contributions provide limited audio examples for evaluation. The most rigorous research originates from adjacent disciplines, such as noise–vibration–harshness (NVH) and acoustics, though some advanced techniques likely remain undisclosed due to corporate interests in gaming and automotive industries. A major obstacle to progress, particularly in adapting contemporary methods, is the lack of suitable and publicly available datasets for training. 25 Chapter 3 Procedural Engine Sounds Dataset The acquisition and preparation of comprehensive datasets for engine sound analysis present several critical challenges that significantly impact research progress in this field. Obtaining high-quality recordings is inherently difficult due to the need for expensive and complex setups, including specialized measurement equipment, dynamometers, and acoustically treated chambers. However, even with proper equipment, real-world recordings are often contaminated by confounding noise sources, such as dynamometer interference, vehicle aggregates, and other mechanical components that are not part of the actual engine sound signature. Since self-supervised generative neural network based approaches require vast quantities of standardized and annotated training data, this represents a significant bottleneck for research advancement in this domain. To address these limitations, this work adopts a synthetically generated dataset – an established methodology in audio research [48, 49, 50] – which enables precise parametric control over data complexity, elimination of confounding noise, unlimited scalability with consistent quality, and inherent access to ground truth labels. By leveraging traditional engine sound modeling techniques and analyzing a limited set of real recordings, fundamental acoustic principles can be extracted and implemented in a parametric synthesizer capable of generating extensive datasets suitable for neural audio synthesis model training and other applications beyond this thesis. 26 To enable the research conducted in this thesis, a dedicated dataset of engine recordings was created and made openly available as the Procedural Engine Sounds Dataset [51]. It comprises synchronized engine audio and time-aligned RPM and torque measurements, serving as the foundation for the experiments presented herein. 3.1 Dataset Generation Methodology This section documents the dataset used throughout the thesis. While the dataset creation is not the primary research contribution of this work, a detailed account of its generation, structure, and preprocessing is provided to ensure transparency and to establish the trustworthiness of the experimental results. 3.1.1 Feature Extraction from Real Engine Data The parametric foundation of the synthesis system derives from systematic analysis of engine recordings obtained from various vehicle and engine configurations. To identify and quantify underlying harmonic deviation and magnitude patterns, a dedicated spectral analysis and harmonic extraction pipeline was implemented, processing approximately 2.12 GB of raw multi-channel engine data spanning complete RPM and torque operating ranges. The audio content, sampled at 16 kHz, was segmented into frames of 65,536 samples (4.096-second chunks). Frames exhibiting strong RPM variance or zero RPM conditions (engine off) were excluded from the analysis. To ensure spectral stability, each audio frame underwent RPM-based resampling such that the pitch envelope remained continuous at the frame’s mean fundamental frequency (f0). This preprocessing eliminates pitch variations within frames, preventing harmonics from drifting between frequency bins and causing spectral artifacts. To minimize spectral leakage, an adaptive FFT size calculation was implemented on a frame-by-frame basis, ensuring that FFT bins consistently align with expected harmonic frequencies based on each frame’s fundamental frequency. Spectral centroid analysis (center of mass calculation) was employed around each 27 harmonic position, with non-overlapping regions between adjacent harmonics, as it demonstrated reduced susceptibility to peak detection errors compared to conventional peak picking ([25]). Harmonic deviations were quantified by calculating the ratio between the ideal harmonic frequency and the detected harmonic centroid frequency, with deviation values computed and stored for all processed frames. This analysis extracts comprehensive parameter sets that capture the essential acoustic fingerprint of each engine configuration, including harmonic magnitude distributions, inharmonic deviations, and their dependencies on operating conditions. 3.1.2 Real-Time Synthesis Architecture The dataset generation employs a comprehensive Max/MSP-based real-time synthesis system integrating three complementary synthesis approaches: additive, subtractive, and resonator-based processing. Additive Harmonic Synthesis: The core architecture uses 128 independent sine wave oscillators with dynamically controllable amplitude and frequency parameters driven by incoming RPM and torque data streams. The synthesizer operates through lookup tables storing the engine-specific harmonic characteristics extracted in the feature analysis stage. These stored parameter values are interpolated in real–time, ensuring smooth transitions across the operational parameter space while maintaining acoustically realistic harmonic signatures derived from authentic recordings. Stochastic Component Synthesis: While additive synthesis captures the harmonic foundation, broadband spectral characteristics require stochastic modeling. Filtered noise modulated by the time-domain signal of the summed harmonics, combined with a distinct low-frequency oscillation linked to engine speed, captures the inherent turbulence and mechanical noise of combustion events absent in pure sinusoidal synthesis. Resonator-Based Acoustic Modeling: The combined additive and stochastic signals are processed through a complex resonator bank consisting of eight parallel Karplus-Strong-inspired delay-line networks. These produce non-linear transfor28 mations modeling the interference patterns, comb-filtering effects, and resonances occurring as engine pressure pulses propagate through exhaust system geometry. This resonator architecture transforms the otherwise synthetic output into acoustically convincing engine sounds with realistic spatial and timbral complexity, while maintaining parametric control for systematic dataset augmentation. 3.1.3 Dataset Augmentation Strategy While the feature extraction process yields authentic engine-specific parameters, the limited scope of real-world recordings constrains dataset diversity. To generate comprehensive training datasets spanning broader operational ranges and timbral variations, a systematic augmentation methodology is employed. Cross-Configuration Recombination: The synthesis system enables application of control data trajectories from any engine configuration to the harmonic parameter sets of any other configuration. This cross–recombination expands control data available for each subset with physically consistent operating scenarios not present in the original recordings, substantially expanding the dataset while maintaining physical consistency. Parametric Variation: Beyond recombination, targeted parameter modifications augment timbral diversity. This includes systematic variation of stochastic noise shaping and intensity to model different exhaust configurations or microphone positions, adjustment of resonator characteristics to simulate acoustic propagation differences, and controlled modification of harmonic magnitude distributions within physically realistic bounds. Augmentation Scale: This methodology enables generation of extensive datasets from each engine configuration—transforming limited original recordings into thousands of synthesized samples covering comprehensive operational ranges, and diverse timbral characteristics. The result is a dataset sufficient for robust machine learning training while maintaining grounding in authentic acoustic characteristics extracted from real vehicles, with physically accurate control parameter trajectories. 29 3.1.4 Data Format and Synchronization To ensure realistic training data that captures authentic engine behavior, the synthesis system operates under control of actual vehicle performance data recorded during real driving scenarios. We record synthesized stereo audio alongside corresponding control parameters in a synchronized four-channel format sampled at 48 kHz, ensuring perfect temporal alignment between acoustic content and performance parameters throughout the dataset, as shown in Figure 1. Figure 1: Stereo audio channels with time-aligned control parameters: RPM and torque (Nm). Control parameters – rotational speed as revolutions per minute (RPM) and engine torque in Newton-Meters (Nm) – are normalized to the range [-1, 1] using predefined scaling boundaries of 10,000 RPM and 1,000 Nm respectively. At a bit depth of 16 bit, the resolution for RPM and torque is determined by the quantization step size ∆ = 2 216 =2 65536 ≈3.0518 ×10−5. Given the normalization scaling factors of 10−4 for RPM and 10−3for torque, the real-world parameter resolution becomes ∆RPM = ∆ 10−4≈0.305 RPM and ∆Nm =∆ 10−3≈0.0305 Nm, providing sufficient granularity for accurate reconstruction of engine operating conditions from the encoded audio data. 30 3.2 Dataset Validation and Analysis 3.2.1 Representativeness and Realism Assessment This dataset, though entirely synthetically generated, preserves representative realism through its grounding in authentic engine behavior. The synthesis process reproduces essential acoustic characteristics of real engines – including engine order magnitude structures, harmonic deviations, and performance parameter dependencies – while providing the controlled, noise-free environment necessary for systematic machine learning research. As a result, the dataset combines the analytical advantages of synthetic generation with the acoustic authenticity derived from real-world engine characteristics and operational dynamics. To validate this representativeness, a thorough analysis of the harmonic structure at each RPM and torque combination was conducted for both real-world data and the procedurally generated dataset, comparing magnitudes and harmonic deviations of engine orders. Figure 2 shows the engine order magnitudes as functions of RPM (top row) and Torque (bottom row), comparing the real recordings displayed on the left with the corresponding synthetic engine sound set on the right. The comparison highlights similarity across overall trends, harmonic distributions between engine orders, and performance parameter dependencies. Characteristics specific to the engine type (in this case V8) – such as a dominant 4th engine order – are observable in both datasets, suggesting that the synthetic dataset contains realistic features with generalization potential. Beyond demonstrating similarity, the plots also reveal differences in timbre and how empirical modification of synthesis parameters exceeding the extracted features transformed the engine sound beyond the limited available data points while maintaining cohesiveness and plausible acoustic behavior. 31 Figure 2: Comparison of engine order magnitudes between real recordings (left) and synthetic engine sounds (right) as functions of RPM (top) and torque (bottom) 3.2.2 Statistical Distribution and Variability Analysis The dataset comprises 5,935 audio files representing 19.01 hours of recorded data (24.47 GB total). The data is organized into eight sets: Full Sets (A-D) containing 3,068 files and Large Sets (E-H) containing 2,867 files, as detailed in Table 1. Category Files Duration (hrs) Size (GB) Full Sets (A-D) 3,068 9.83 12.65 Large Sets (E-H) 2,867 9.18 11.82 Total 5,935 19.01 24.47 Table 1: Dataset organization and structure The stochastic nature of the data is evident in the rotational speed and torque distributions, which exhibit significant variability characteristic of real-world operational conditions. RPM values demonstrate a wide operational range from 0.0 to 7,007.45 RPM, with a mean of 3,170.53 RPM and standard deviation of 1,713.87 RPM. The distribution shows a relatively symmetric pattern around the median value of 2,971.19 RPM, with the interquartile range spanning from 1,654.97 RPM 32 To provide physics-informed inductive biases that capture dynamic engine behaviors, we augment the control signals with first and second-order temporal differences— standard automotive engineering parameters used to characterize engine behavior: Engine speed deltas: •First-order difference (angular velocity): RPM(t)−RPM(t−h): captures rotational acceleration and deceleration phases •Second-order difference (angular jerk): ∆RPM(t)−∆RPM(t−h): identifies sudden rotational dynamics from gear shifts, clutch engagement, or load transitions Torque deltas: •First-order difference (torque rate): Nm(t)−Nm(t−h): distinguishes between steady-state and transient loading conditions •Second-order difference (torque jerk): ∆Nm(t)−∆Nm(t−h): captures abrupt mechanical events (gear-shifts, torque holes) that manifest as distinctive acoustic signatures We compute direct differences rather than normalized derivatives (df dt ≈f(t)−f(t−h) h) as they represent the same temporal dynamics while preserving comparable magnitude and scale of the input data. To obtain input features matching the model frame rate, we subsequently downsample the control signal sequences by segmenting them into frames and averaging each feature over the time dimension within each frame. All input features are standardized using the training-set mean and standard deviation to ensure comparable magnitudes across features. The same transformation, based on the training-set statistics, is applied to the validation and test sets to ensure unbiased evaluation and consistent inference. 39 4.3 Conditioning Signal Design Following parameter prediction by the model heads and upsampling to audio sample rate, synthesis parameters undergo conditioning that enforces physically-motivated behavior. This approach shifts the model’s learning task from absolute parameter prediction to relative parameter adjustment, where assignment to engine operating states is handled deterministically. 4.3.1 Design Rationale Conditioning enforces physical constraints without requiring the model to learn these relationships from data alone. This approach addresses a fundamental optimization challenge: certain parameter relationships are not directly detectable from spectral characteristics and would require specialized loss functions to learn correctly. For example, noise modulation depth optimization suffers from phase ambiguity between output and target signals-spectral losses provide inconsistent gradients as modulations sometimes coincide and sometimes oppose, preventing clear optimization pathways. 4.3.2 Harmonic Frequency Scaling Predicted harmonic ratios are converted to absolute frequencies through fundamental frequency scaling derived from RPM control signals: fk(t) = f0(t)·rk(t),(4.1) where f0(t) = RPM(t)×10000 60 reverses the normalization of the embedded RPM values and converts revolutions per minute to Hz, and rk(t)represents the predicted harmonic ratios that already incorporate 0.5 intervals (0.5, 1.0, 1.5, ...) corresponding to engine orders for four-stroke engines. 40 4.3.3 Virtual Throttle and DFCO Derivation Engine operating mode conditioning derives virtual throttle and deceleration fuel cutoff (DFCO) factors from torque measurements to condition noise gains according to distinct acoustic characteristics: throttlevirtual(t) = max(torque(t), ϵ)0.7,(4.2) DFCOfactor(t) = max(−torque(t), ϵ).(4.3) The virtual throttle derivation addresses the non-linear relationship between torque measurements and acoustic output. While measured torque reflects external factors (traction, friction, torque vectoring), throttle position directly governs intake airflow and butterfly valve dynamics that determine acoustic characteristics. The power law scaling (0.7) captures the non-linear acoustic response to throttle opening, particularly for intake noise and combustion pulse intensity, while maintaining minimum gains (ϵ) during idle operation. Negative torque conditions activate the DFCO factor, emphasizing turbulent flow noise characteristics of deceleration fuel cutoff scenarios where combustion ceases but airflow turbulence continues to dominate the acoustic signature. This conditioning enables the synthesis to automatically distinguish between propulsion-dominated and turbulence-dominated operating modes without explicit training on these distinct acoustic regimes. 41 Chapter 5 Synthesizers and Model Heads 5.1 Differentiable Synthesis Methodology Having established a neural network architecture capable of learning mappings between engine performance parameters and synthesis parameters, we now turn to the core challenge: defining differentiable synthesis modules that can generate acoustically realistic engine sounds while remaining amenable to gradient-based optimization. 5.1.1 Bridging Traditional Methods and Neural Synthesis Given our requirement for fully differentiable and continuous synthesis functions, sample-based approaches from the traditional methods reviewed in Section 2.3 can be excluded due to their inherent lack of differentiability and limited generalization capabilities. Examining the remaining procedural methods reveals a fundamental philosophical divide in how engine sounds are approached. Most traditional methods focus primarily on recreating the acoustic characteristics observed in recorded engine sounds. These methods adjust synthesis parameters to achieve perceptual similarity to target recordings, using general-purpose audio synthesis techniques to match spectral content, temporal envelopes, and timbral qualities. 42 In contrast, a smaller subset of methods attempts to model the underlying physical processes and their acoustic outputs that generate engine sounds in the first place. These approaches simulate combustion events, pressure wave propagation, exhaust resonances, and mechanical vibrations, with the expectation that accurate physicsinformed modeling will naturally produce realistic acoustic output. This fundamental distinction between recreating acoustic characteristics versus modeling underlying processes becomes particularly relevant when translated into neural synthesis architectures, where both perspectives can be explored systematically within a unified framework. 5.1.2 The Impulsive Origin of Engine Harmonicity Unlike resonant instruments where harmonic oscillations exist inherently in the physical vibration, engine sounds derive their harmonic structure from periodic repetition of broadband impulse events. Each combustion event generates acoustically complex, noise-like pressure waves; the perceived harmonicity emerges from temporal regularity at engine-order frequencies. This creates a fundamental duality: engine sounds are both at once: a sequence of inherently noise-like impulsive events at their source, yet also a distinctly harmonic acoustic structure. This duality suggests two complementary modeling paradigms that directly parallel the philosophical divide in traditional methods: we can either model the perceptual acoustic qualities directly, or we can synthesize the individual pressure pulses and allow the complete acoustic characteristics to emerge from their shapes, temporal organization and propagation. For these impulsive–periodic sources: 1. Forward modeling: Accurate modeling of individual pressure pulses and their temporal sequences should yield correct spectral and perceptual characteristics 2. Inverse modeling: Precise acoustic spectrum reconstruction should implicitly capture underlying pulse structure 43 5.1.3 Dual Synthesis Strategy This duality motivates implementing both modeling perspectives as differentiable synthesis modules: 1. Harmonic-plus-noise synthesis: Models spectral characteristics through adaptive harmonic generation with physically-coupled, temporally structured noise components 2. Pulse-train-resonator synthesis: Models temporal causality through parameterized pulse sequences and exhaust system propagation Both approaches incorporate physical constraints while allowing neural networks to discover optimal parameter mappings. This dual approach provides contrasting optimization pathways toward the same acoustic target, enabling systematic comparison of spectral-structure versus impulse-source modeling paradigms. In the following sections, we detail the implementation of each synthesis module, beginning with the harmonic-plus-noise approach due to its established theoretical foundation and intuitive parameter space, which also provides crucial building blocks for the pulse-train-resonator synthesis method. 5.2 Harmonic-Plus-Noise Engine Sound Synthesis The first configuration of the PRCE model – HPN – employs a harmonics-plus-noise based synthesis approach, directly modeling spectral and temporal dynamics while adapting the general method to the specific acoustic characteristics of engine sounds. An overview of the information flow through the components detailed in the following sections is provided in Figure 4: 44 Engine Parameters (RPM, Torque, Derivatives) Neural Network (Latent Embeddings xt) Harmonic Head Noise Head Complex Oscillators Hk(t) Noise Components N∗(t) Synthesized Audio y(t) Figure 4: System architecture showing information flow from engine control parameters through model heads to synthesis components. Components in bold are unique to the HPN variant. 5.2.1 Physically-Informed Synthesis Components Our harmonic-plus-noise synthesis architecture implements multiple physically-informed inductive biases that constrain the synthesis toward acoustically plausible engine sounds while maintaining full differentiability for gradient-based optimization. Systematic Inharmonicity via Gaussian Harmonic Bending One of the key limitations of standard harmonic synthesis for engine sound modeling lies in its assumption of perfect harmonic relationships. Real engine sounds exhibit systematic deviations from pure harmonic content due to complex acoustic and mechanical interactions. However, unlike musical instruments where inharmonicity has been extensively studied and modeled [52, 53, 54], there exists no scientifically rigorous physical model for inharmonicity in engine sounds comparable to established theories for stiff strings in pianos or other musical instruments. Given the absence of established theoretical frameworks, we adopted an empirical approach based on systematic analysis of real engine recordings. During dataset creation and analysis (Section 3.2), we conducted detailed spectral analysis to extract 45 actual harmonic deviations from recorded engine sounds. This analysis revealed that observed deviations can reach ±4-7%, which is quite substantial given that human perception thresholds for frequency shifts fall below 1% [55]. Despite the absence of obvious universal trends (unlike the increasing inharmonicity related to stiffness of piano strings), several interesting observations emerged. Engine orders tend to exhibit group-like behavior in response to specific RPM and torque situations, while others show more constant deviations across wider RPM ranges. These findings are detailed in Appendix A. Rather than learning arbitrary per-harmonic offsets that risk mere dataset memorization, we propose a time-varying Gaussian modulation of harmonic frequencies. For engine orders spaced at intervals of 0.5 (corresponding to four-stroke engine harmonics), we define fbent k(t) = fideal k(t)·"1 + G X g=1 ∆g(t) exp −(k−µg(t))2 2σg(t)2#,(5.1) where fideal k(t) = (k×0.5) ×f0(t)represents the ideal engine order frequencies, k∈ {1,2, ..., K}indexes the harmonic array, and Glearnable Gaussians with parameters {∆g(t), µg(t), σg(t)}enable systematic control over inharmonic patterns. The frequency deviation factors ∆g(t)are constrained to [−0.08,0.08] to provide sufficient range for observed dataset variations with exploration headroom for future datasets. Complex Harmonic Oscillators Because frequency and phase are now time-varying optimization parameters rather than fixed values, sinusoidal parameter estimation faces a well-known challenge: the gradient landscape of spectral loss functions with respect to frequency is highly nonconvex, hindering convergence [30]. To address this, we adopt a complex exponential representation for the harmonic oscillators, as proposed by [31], which improves 46 gradient flow during frequency optimization: Hk(t) = Ak(t)·real ejϕk(t)=Ak(t)·cos(ϕk(t)),(5.2) where the phase ϕk(t)is computed through cumulative integration of the predicted angular frequency: ϕk(t) = Zt 0 ωk(τ)dτ +ϕk,0=Zt 0 2πfbent k(τ)dτ +ϕk,0.(5.3) In discrete time, this integration is implemented as cumulative summation over time steps. Random initial phases ϕk,0∼ U(0,2π)are used to avoid constructive interference artifacts. Pressure Wave-Coupled Noise Synthesis The core physical inductive bias of our approach lies in the multiplicative coupling between harmonic and noise components, reflecting the acoustic reality that engine noise originates from the same combustion-generated pressure waves that produce the harmonic spectrum. We implement three distinct noise models, each with specific physical motivation: Noise Bursts model sharp pressure pulses from individual combustion events: Nbursts(t) = gpulse(t)· B X b=1 αb(t)ηb(t)!· M X m=1 wm(t)|˜ Hm(t)|pm(t),(5.4) where ˜ Hm(t)is converted from sinusoidal to triangular waveforms using an arcsin transformation to better match the impulsive nature of real combustion events (as observed in time-domain analysis of recorded engine data): ˜ Hm(t) = 2 πarcsin(0.99 ·Hm(t)).(5.5) The safety factor of 0.99 prevents numerical instability at the arcsin domain boundaries where d dx arcsin(x) = 1 √1−x2→ ∞ as |x|→1. 47 Turbulence Distortion captures aerodynamic turbulence effects coupled to the overall pressure wave: Nturbulence(t) = γturb ·[gpulse(t)+gflow(t)] ·Htotal(t)· B X b=1 αb(t)ηb(t)!,(5.6) where Htotal(t) = PK k=1 Hk(t)denotes the sum of all harmonic components. Flow Noise represents deceleration fuel cutoff and airflow phenomena without direct harmonic coupling: Nflow(t) = γflow ·gflow(t)· B X b=1 αb(t)ηb(t)!.(5.7) Here, Brepresents the number of ERB-filtered noise bands and Mdenotes the number of low-order harmonic oscillations used for modulator pulse signal generation. The variables ηb(t)are pre-computed ERB-filtered noise bands with time-varying amplitudes αb(t),wm(t)are weights balancing contributions of harmonics, and pm(t) are shaping exponents transforming the harmonic modulator oscillations ˜ Hm(t)to pulses with varying sharpness. The empirically-determined mixing coefficients are set to γturb = 0.7and γflow = 0.3based on perceptual evaluation of plausible magnitude ratios. As described in Section 4.3.3, the different noise components are activated according to specific throttle and engine operation states: gpulse(t) = [0.75 ·σ(pulse_gain(t)) + 0.25] ·throttlevirtual(t),(5.8) gflow(t) = [0.75 ·σ(flow_gain(t)) + 0.25] ·DFCOfactor(t).(5.9) 5.2.2 Neural Parameter Control Let xtdenote the latent embedding vectors from the final hidden layer of our neural network, which encode time-varying engine control parameters (RPM, torque) and their temporal derivatives. These embeddings are processed by specialized neural 48 where ν < 1is a learnable parameter describing the depth and curvature of the bending, constrained by a scaled sigmoid function (Equation 5.37). Because the exponent is smaller than 1, the phase advances faster at the beginning of the pulse and slows down toward the end. When mapped to angular phase and applied to the derivative-of-cosine sum, this results in: d dt "K X k=1 akcos k2π ϕmod(t)#=− K X k=1 akk2π ϕmod(t)′sin k2π ϕmod(t).(5.24) Intuitively, this "phase bending" compresses the pulse in time at its onset, producing a higher instantaneous frequency that corresponds to the rapid release of hot combustion gases. The pulse then gradually stretches toward its end, lowering the pitch as the gas cools and the speed of sound normalizes along the exhaust manifold. This approach allows the synthesized pulse train to more accurately reproduce the natural pitch envelopes, enriching timbre and producing the characteristic "grunting" of low-speed operation. The effect of this phase bending is shown in the bottom-left panel of Figure 5. Exponential Amplitude Envelope Real exhaust valve events produce finitetime pressure transitions, characterized by a rapid pressure release from the combustion chamber into the exhaust pipe, resulting in a sharp rise followed by a slower decay. To capture this behavior, we apply an exponential rise-and-fall envelope to the derivative pulse trains, E(t) = (1 −exp(−αt)) exp(−βt),(5.25) where αand βare learnable coefficients controlling the rise and decay slopes of each pulse. This formulation allows the waveform to reproduce the asymmetrical, gradient-driven pressure variations observed in real exhaust manifolds, while preserving the bipolar characteristic of the derivative pulse. The effect of this amplitude envelope is depicted in the top-right plot of Figure 5. 55 Combined Bipolar Pulse Train The final physically-informed pulse train, incorporating both the thermodynamically-inspired frequency modulation and the exponential amplitude envelope, can be written as: P(t)=E(t)· K X k=1 ak−sin k2π ϕmod(t),(5.26) where E(t)is the learnable exponential rise-and-fall envelope, ϕmod(t) = ϕnorm(t)ν is the exponentially bent normalized phase, and akare the normalized harmonic amplitude distributions derived as in the HPN model (Equation 5.10), incorporating a learned natural harmonic decay factor λ. This factor has a significant influence on the resulting pulse shape, as illustrated in top-left plot of Figure 5. Figure 5: Pulse shapes for different parameter settings. The four subplots show: (top-left) the base derivative-of-cosine pulse with varying numbers of harmonic decay factors λi; (top-right) pulses shaped by exponential envelopes (αi, βi); (bottom-left) phase-modulated pulses with factors fm,i; and (bottom-right) P(t)the combined effect of envelope and modulation. Multiple parameter settings are shown per subplot, with curves distinguished by gradients. 56 Notice that in Equation 5.26 we omitted the normalization by angular frequency k·ω, thereby avoiding the implicit emphasis on higher harmonics in the summation. Since each individual harmonic component contributes equally to shaping the resulting pulse wave, they are learned as a normalized distribution across all harmonics (ak). However, the k·ωscaling creates unequal gradient magnitudes during backpropagation, with higher harmonics receiving disproportionately large gradient updates and dominating the learning process. Noise Components Turbulence (Nturbulence(t)) and flow noise (Nflow(t)) are integrated into the pulse train model using the same methodology as the HPN variant, detailed in Section 5.2.1. However, the noise bursts (Nbursts(t)) are no longer coupled to amplitude-modulating oscillators. Instead, they are shaped using the same exponential rise-and-decay mechanics employed for pulse generation in Section 5.3.1, with independently predicted shaping coefficients. Complete Pulse Train Signal With the noise components in place, we arrive at our final pulse train signal: y(t) = P(t) + Nbursts(t)+Nturbulence(t)+Nflow(t).(5.27) 5.3.2 Firing Order Sequencer So far, we have been constructing only the periodic pressure pulse of a single cylinder. In order to create the rhythm and harmonic spectrum of a complete multi-cylinder engine, we need to sequence each of the cylinder pulse trains according to their physical arrangement and the engine’s firing order. Sequencer Implementation To achieve this, we map the full four-stroke power cycle of 720◦crankshaft rotation onto a 2πradian interval. This allows for direct compatibility with our 2π-periodic continuous functions, such as the harmonic basis and phase-bent oscillators. 57 Each cylinder’s contribution is then defined by introducing a fixed phase offset according to the firing sequence and physical cylinder arrangement of the V8, as represented below. Left bank Right bank 1 5 2 6 3 7 4 8 In our configuration, the firing order is specified as: [1, 5, 4, 8, 6, 3, 7, 2]. This distributes the ignition events evenly across the cycle, but also leads to an alternating rhythm between the two cylinder banks. This alternation is a key perceptual cue of V-type engines, producing the characteristic uneven, “burbling” exhaust rhythm that distinguishes them from inline configurations. The individual piston phases are now computed by offsetting the global power-cycle phase by the corresponding firing angle and wrapping the result modulo 2π. This way inharmonicity, roughness, and modulation through are generated phase interference between cylinder outputs, rather than requiring explicit acoustic modeling of these effects. Cylinder Bank Summation After generating the individual cylinder pressure pulses, we sum the contributions within each cylinder bank to reflect the physical grouping of the V8 engine: Bank A (Left) = 4 X i=1 yi(t), Bank B (Right) = 8 X i=5 yi(t), (5.28) where yi(t)represents the pressure waveform of cylinder i. This separation allows us propagate further the pressure waves through independent signal paths. 58 5.3.3 Neural Parameter Control As in the HPN version, the input parameters to this synthesis architecture are derived from logits xiand transformed to meaningful time-varying synthesis parameters using dedicated model heads featuring activation functions and processing. The Pulse Wave Head extends the harmonic synthesis strategy by incorporating per-cylinder parameterization, phase offsets, and valve-shaping constraints, making it suitable for excitation modeling of individual combustion events. Cylinder Amplitudes. Each cylinder is assigned a time-varying amplitude Ci(t) = σ(PulseWaveHeadcyl_amp(xt))i,(5.29) which ensures positive contributions while allowing the network to control relative cylinder balance and dynamics across operating conditions. Phase Offsets and Jitter. Per-cylinder ignition phase offsets are predicted through ϕoffset i(t) = tanh(PulseWaveHeadcyl_phase(xt))i·∆ϕ,(5.30) where ∆ϕ=40 720 ·2πbounds deviations to ±40◦crank angle. To capture stochastic firing variability, a jitter term ϕjitter i(t)∼ N(0, σ2 ϕ)is added with σϕ=2 720 ·2π, corresponding to a ±1% crank fluctuation. The effective cylinder phase is then: ϕi(t) = ϕoffset i(t)+ϕjitter i(t).(5.31) Harmonic Distribution with Cylinder Decay. Raw harmonic amplitudes are predicted as Araw k(t) = clamp(PulseWaveHeadharm_amps(xt)k,−10,10),(5.32) 59 normalized with a softmax: Anorm k(t) = exp(Araw k(t)) Pjexp(Araw j(t)).(5.33) Per-cylinder exponential harmonic decays enforce natural damping of higher orders: λi(t) = σ(PulseWaveHeadharm_decay(xt)i)0.75,(5.34) Ai,k(t) = Anorm k(t)·exp(−λi(t)·k),(5.35) with re-normalization across k. An overall gain term G(t) = 2 ·[σ(PulseWaveHeadgain(xt))]ln(10) +ϵ(5.36) scales energy globally, ensuring stable synthesis. Phase Modulation. A modulation depth parameter controls curvature of "phase bending" as exponential slightly below unity: ν(t) = 1 −0.2·σ(PulseWaveHeadfm_depth(xt)),(5.37) resulting in variation between [0.8,1.0], centered at 0.9. Pulse Shaping via Valve Geometry. Each cylinder receives shaping coefficients αi, βifrom [αi, βi]=σ(PulseWaveHeadshape(xt))0.5 i,(5.38) where exponents bias the network toward longer decays (smoother pulses). After rescaling, αicorresponds to rounded opening angles (10–15°crank), and βito broader closing envelopes (50–100°crank): αi=deg2rad(5 ·αi+ 10),(5.39) βi=deg2rad(50 ·βi+ 50).(5.40) 60 Final Output. The head outputs three parameter groups per frame: Harmonic Amplitudes: Ai,k(t),(5.41) Frequency Modulation: ν(t),(5.42) Pulse Shapes: [αi, βi],(5.43) Phase Offsets: θi(t),(5.44) which together define the time-aligned, per-cylinder harmonic pulses for the PulseTrain Synthesizer. Up to this point, all synthesis parameters have been predicted by the network as time-varying outputs based on the embedding and decoding of control input sequences. In the following section, we present the PTR’s final processing stage and explain why its parameters are optimized directly, rather than being driven by timevarying control embeddings. 5.3.4 Differentiable Karplus-Strong Resonator Having generated pulse train signals for each cylinder bank through the pulse wave composition process, we now address the critical acoustic transformation that occurs between combustion events and the final audible engine sound. As established in the background review, the combustion noise itself is not what we perceive as engine sound—rather, it is the exhaust pulses after propagation through the complex exhaust system geometry. This motivates providing the network with differentiable processors capable of modeling the intricate acoustic phenomena occurring within exhaust systems: wave reflections, comb filtering effects, and complex interactions between oncoming pulses and pressure wave propagation. The exhaust system comprises a network of tubes with varying lengths, diameters, intersections, and conjunctions, typically engineered for optimal pressure flow and constructive pressure coupling to achieve streamlined exhaust flow. 61 To model this system, we implement two independent resonators processing the cylinder bank outputs separately, representing the distinct exhaust manifold paths with their characteristic lengths and geometries. These processed signals are subsequently combined within a final shared resonator, modeling the confluence into the common exhaust pipe system where further acoustic shaping occurs through the remaining exhaust components. Karplus-Strong Algorithm: A Recursive Feedback-Delay Line The Karplus-Strong algorithm provides an elegant framework for modeling resonant systems, such as strings, tubes and membranes, through a feedback delay line [59]. As shown in Figure 6, the basic structure consists of three key components: a delay line that stores previous output samples, a filter that processes the delayed signal before feedback, and a feedback mechanism that adds the scaled filtered delayed signal to the current input. Figure 6: Schematic of Karplus-Strong algorithm showing signal flow from input x[n]to output y[n]via delay line Z−Land second-order filter α·Z−L+β·Z−L−1 with feedback gain g. This can be expressed as the difference equation: y[n] = x[n] + g·h[n−L],(5.45) where Lis the delay length (determining fundamental pitch), gis the feedback gain, 62 and h[n−L]represents the filtered delayed signal: h[n−L]=α·y[n−L]+β·y[n−L−1].(5.46) This two-coefficient formulation (αand β) provides greater flexibility than the traditional Karplus-Strong implementation (where α=β= 0.5), allowing independent control over the filter characteristics within the feedback path. This enables more sophisticated timbre evolution across the delay, as different coefficient combinations can emphasize or attenuate specific spectral components. Extended versions of the Karplus-Strong algorithm incorporate additional filter stages and nonlinear transfer functions to more accurately model the physical behavior of stiff plucked strings [60]. For the purpose of engine sound synthesis, however, we adopt the original, more general form of the Karplus-Strong algorithm, in which a second-order filter with free (i.e., learnable or tunable) coefficients is applied in the feedback path, which is scaled by the feedback gain g: y[n] = x[n] + g·(α·y[n−L]+β·y[n−L−1]) (5.47) or equivalently, y[n] = x[n] + g· 1 X i=0 ai·y[n−L−i],(5.48) where a0=αand a1=β. Challenges with Recursivity As shown in the Karplus–Strong difference equation 5.48, the algorithm is inherently formulated as a recursive feedback system. This presents significant challenges when integrated into neural network architectures: Backpropagation Through Time (BPTT) Complexity The recursive dependency means that y[n]depends on all previous outputs y[n−1], y[n−2], . . . , y[0]. For gradient computation, this creates a dependency chain that must be unrolled completely: 63 •Computing ∂L ∂α requires gradients through the entire sequence •Each time step depends on all previous time steps •The computational graph becomes a long sequential chain This results in: •Computational complexity: –O(N·M)where Nis sequence length and Mis the number of parameters –Sequential processing that cannot be parallelized (each example in batch has unique recursions), making efficient implementation difficult •Gradient issues: Vanishing/exploding gradients over long sequences Stability Concerns Without proper constraints, recursive feedback filters can exhibit unstable behavior with outputs growing without bound. This is particularly problematic during neural network training, where parameters change rapidly and small numerical errors can accumulate over long sequences, leading to instable coefficients causing NaN values. A Related Solution: Differentiable All-Pole Filters The BPTT challenge is not unique to the Karplus-Strong algorithm; it arises in any recursive system. In digital signal processing, such recursive structures are most commonly encountered in digital filters. For the specific case of all-pole filters, Yu et al. [40] address the fundamental challenge of computing gradients through recursive IIR filters by reformulating the forward pass to eliminate recursion, thereby enabling stable and efficient backpropagation. 64 Chapter 6 Loss Functions The model is trained with a combination of loss functions to capture global spectral structure and harmonic details. 6.1 Multi-Resolution STFT Loss We employ a Multi-Resolution Short-Time Fourier Transform (STFT) loss composed of spectral convergence, linearand log-magnitude terms, supplemented by a spectral energy loss. The multi-resolution STFT was introduced to capture spectral structure across several time–frequency resolutions. Engel et al. [24] adopt this framework, though without both the log-magnitude and spectral convergence terms that were originally proposed by Arik et al. [64] and extended to multi-resolution by Yamamoto et al. [65]. Our implementation is based on the auraloss library [66], which provides multiresolution STFT losses with optional scale invariance. We extend it in two ways. First, we add an explicit spectral energy loss that supervises frame-wise energy, Ex,b,t =X f x2 mag,b,f,t, Ey,b,t =X f y2 mag,b,f,t,(6.1) E(xmag, ymag) = 1 BT X b,t d(Ex,b,t, Ey,b,t),(6.2) 71 where d(a, b)is the L1 distance of log-scaled energies, normalized across frames. This disentangles absolute loudness supervision from spectral shape, improving optimization stability by providing independent gradient signals for timbre and energy. Second, we introduce an equal-contribution normalization across resolutions, xmag ←xmag F, ymag ←ymag F,(6.3) which compensates for differing FFT sizes, ensuring that each resolution contributes comparably to the total loss. By scale-invariant rescaling [66], the target spectrum is normalized before computing STFT losses: αb=Pf,t xmag,b,f,t ymag,b,f,t Pf,t y2 mag,b,f,t , ymag,b ←αbymag,b.(6.4) The individual STFT terms are defined as: LSC =∥xmag −ymag∥F ∥ymag∥F ,(6.5) Llin =1 BFT X b,f,t |xmag,b,f,t −ymag,b,f,t|,(6.6) Llog =1 BFT X b,f,t |log(xmag,b,f,t +ϵ)−log(ymag,b,f,t +ϵ)|.(6.7) The total spectral loss is a weighted sum: L=wSC LSC +wlin Llin +wlog Llog +wenergy E(xmag, ymag),(6.8) where the weights wSC, wlin, wlog, wenergy control the relative contributions of each term and are all set to 1 in our experiments. The energy loss is computed before scale-invariant normalization to preserve absolute loudness supervision, while STFTbased terms are scale-invariant to focus on spectral shape. This formulation provides disentangled supervision for harmonic structure and loudness, while ensuring fair contribution from all resolutions, leading to more stable 72 and interpretable optimization. It also aligns conceptually with our model architecture, which predicts overall energy and harmonic distribution separately (see Section 5.2.2). 6.2 Harmonic Loss To supervise fine-grained harmonic content, we introduce a Harmonic Loss, inspired by Campbell diagrams from rotating machinery analysis. In a Campbell diagram, harmonic orders appear as diagonal lines when plotting resonant frequencies against rotational speed. Analogously, our loss emphasizes spectral energy along predicted harmonic tracks derived from the instantaneous RPM (or f0) of each frame, while deemphasizing inter-harmonic content. This allows the model to prioritize harmonic magnitudes over non-harmonic spectral components. The loss is instantiated with high spectral resolution (FFT size 65,536, window size 16,384, hop size 256 ata16 kHz sample rate) and zero-padding to minimize spectral leakage, ensuring that harmonic peaks are accurately captured. For vectorized computation, we construct a sinusoidal mask Mb,f,t whose maxima align with expected harmonic bins, and whose negative or inter-harmonic regions are clipped to reduce the contribution of non-harmonic energy. Each batch b, example, and frame treceives a mask based on the instantaneous RPM, capturing harmonics 0.5,1,1.5, . . . up to the desired number of harmonics. Let xmag, ymag ∈RB×F×Tdenote the masked magnitude spectrograms of the generated and target audio. The Harmonic Loss is computed as the frame-wise energy distance along the harmonic mask: Eharm b,t = F X f=1 xmag,b,f,t ·Mb,f,t2, Eharm b,t = F X f=1 ymag,b,f,t ·Mb,f,t2,(6.9) Lharm =1 B(T−1) B X b=1 T−1 X t=1 log(Eharm b,t+1 +ϵ)−log(Eharm b,t +ϵ),(6.10) 73 where ϵis a small constant for numerical stability. This formulation measures the temporal differences of logarithmic energy along harmonic bins, ensuring the model reproduces the dynamics of harmonic magnitudes. Figure 7: Log-magnitude spectra of prediction Xand target Yafter applying the Harmonic Mask M. The mask suppresses energy differences between harmonics and emphasizes harmonic regions, inspired by Campbell diagrams commonly used in NVH and engineering acoustics. 74 Chapter 7 Training 7.1 Model Architecture The model variants are both configured with 128 harmonic voices and 256 noise bands. The neural network architecture uses a hidden size of 256 dimensions with a single-layer GRU of 512 units, resulting in a total of 1,435,697 trainable parameters. Memory requirements include 5.74 MB for parameters, 14.33 MB for forward/backward passes, and an estimated total size of 20.61 MB. 7.2 Training and Validation Data We train our models on the first three subsets – A, B, C – of the Procedural Engine Sounds Dataset [51] described in Chapter 3. Each subset has nearly identical size, rpm and torque distributions, and coverage, enabling direct comparability while controlling for factors other than timbre. This design allows us to study generalization across different engine timbres. The subsets originate from three distinct real-world engines, with additional modifications to increase stochasticity and spectral complexity. Dataset Ais characterized by the harmonic spectrum of an inline-four engine, Bby a V8 configuration with low-frequency resonant exhaust characteristics, and Cby a V8 engine exhibiting prominent mid-range frequencies, intake noise, and metallic resonances. Based on 75 these traits, the sets can be ordered as A < B < C in terms of increasing spectral and temporal complexity. Each subset comprises approximately 2.5 hours (149 minutes) of audio data, which we split into 90% training and 10% validation data. 7.3 Data Processing For model training, audio data is processed in batches containing fixed-length monophonic chunks of 65,536 samples at 16 kHz sampling rate (approximately 4 seconds duration). These chunks are extracted at random positions throughout the dataset with 50% overlap to maximize data utilization. Audio signals are normalized to the range [−1,1] across the entire dataset, preserving the full dynamic range while maintaining relative energy differences that are essential for effective learning. Control data undergoes preprocessing to align with the model’s temporal resolution. Specifically, control signals are downsampled to match the model’s frame rate of 125 Hz (corresponding to a hop size of 128 samples) and subsequently standardized using zero mean and unit variance normalization. The standardization parameters (mean and standard deviation) are computed across the entire training dataset to ensure comparable feature magnitudes while preserving relative scaling relationships. In contrast, conditioning signals are provided at the original audio sample rate without scaling or normalization, as they capture absolute physical values that must be preserved. These preprocessing choices—including the 16 kHz sampling rate, 4-second chunk duration, and batch size of 8—align with established practices in DDSP-based timbre transfer and controllable synthesis systems. This configuration is well-suited to scenarios involving limited training data with narrow acoustic target timbres and minimal variety, leveraging strong inductive biases to circumvent the need for learning fundamental audio synthesis principles from raw data [24, 67, 32]. 76 7.4 Optimization Setup Training is conducted using the AdamW optimizer with an initial learning rate of 1×10−3, weight decay of 1×10−2, and beta parameters of (0.9, 0.999). Learning rate scheduling follows a one-cycle policy with a maximum learning rate of 2×10−4, where 15% of training steps constitute the warm-up phase. The learning rate increases by a factor of 10 from the initial value, then decreases by a factor of 100 in the final phase, following a cosine annealing strategy throughout the cycle. The complete training process spans 100 epochs, totaling approximately 45,000 training steps. 7.5 Parameter Initialization All linear layers employ Xavier initialization, which is appropriate for the tanh activations used in the synthesis parameter heads. GRU layers follow PyTorch’s default initialization scheme, where weights are drawn from a uniform distribution U(−√k, √k)with k=1 hidden_size and biases initialized to zero. Layer normalization parameters are initialized to unity for scale (γ= 1) and zero for shift (β= 0). 77 Chapter 8 Results In the subsequent sections, we compare the two model variants, HPN and PTR, by analyzing training dynamics on train and validation sets, evaluating cross-dataset generalization, and complementing objective metrics with subjective assessment. Throughout, A, B, C denote the respective training subsets introduced in Section 7.2. 8.1 Quantitative Analysis The following section examines convergence and performance, analyses loss components, and evaluates generalization capabilities across datasets. 8.1.1 Convergence and Performance Figure 8 illustrates the training dynamics for both model variants across each dataset. Both model variants demonstrate rapid convergence within the first 8,000 training steps, with total log-transformed validation loss (black dotted lines) decreasing sharply from approximately 0.55 to 0.16 on average across datasets. Convergence is effectively achieved by 15,000–25,000 training steps across all configurations, as indicated by the vertical dashed lines marking early stopping candidates (defined as positions with 8 following epochs of no significant improvement). These 78 candidates represent potential stopping points that could have achieved similar performance with significantly fewer training steps compared to the full 100-epoch training run. However, training was continued to capture the complete convergence dynamics, as validation loss showed continued gradual improvement without overfitting concerns, despite diminishing returns. Early stopping candidate loss values are detailed in Appendix B, and all subsequent analysis uses the final best models from the complete training run. Figure 8: Log-transformed Total Loss (training and validation) for both model variants (HPN, PTR; columns) across three datasets (A, B, C; rows). The x-axis denotes training steps and the y-axis the loss. Vertical dashed lines indicate early stop candidates (Esc), defined as points where no significant improvement occurred for 8 consecutive epochs. Training curves show per-step sampling while validation curves represent epoch-level measurements. 79 As shown in Table 3, PTR consistently outperforms HPN in total validation loss across all datasets, achieving a mean validation loss of 0.949 compared to HPN’s 1.006 (5.7% improvement). PTR shows superior validation performance across all three datasets, with improvements ranging from 3.8% to 7.6%. HPN PTR Dataset Harmonic STFT Total Harmonic STFT Total A 0.107 1.781 0.944 0.090 1.649 0.872 B 0.059 1.824 0.943 0.055 1.754 0.907 C 0.166 2.093 1.132 0.117 2.017 1.069 mean 0.111 1.899 1.006 0.088 1.807 0.949 Table 3: Best validation losses for HPN and PTR models, separated into harmonic, STFT, and total loss components. The final training losses in Table 4 reveal complementary strengths. HPN achieves marginally superior harmonic reconstruction during training (0.007 vs 0.008), but this advantage reverses on validation data (0.111 vs 0.088), suggesting HPN may overfit to harmonic details rather than learning generalizable synthesis patterns. PTR achieves superior STFT performance in both training (1.322 vs 1.377) and validation, indicating more robust spectral reconstruction. HPN PTR Dataset Harmonic STFT Total Harmonic STFT Total A0.007 1.245 0.643 0.009 1.220 0.646 B 0.008 1.401 0.728 0.007 1.354 0.702 C0.005 1.484 0.764 0.008 1.391 0.728 mean 0.007 1.377 0.712 0.008 1.322 0.692 Table 4: Best training losses for HPN and PTR models, separated into harmonic, STFT, and total loss components. The training-validation gap confirms this interpretation: PTR demonstrates superior consistency with a mean training-validation gap of 0.257 for total loss, while HPN exhibits a larger gap of 0.294. This indicates PTR’s learned representations and reconstruction strategies transfer more effectively to validation data. 80 with synthesis parameters alternating in an unstable manner. This behavior correlates with subset A’s characteristics: despite being spectrally least complex, it exhibits higher degrees of inharmonicities and shifts of large harmonic groups compared to other subsets. Furthermore, this engine timbre features typical inline-four characteristics, fundamentally different from the V8 firing sequence used as bias for the PTR variant. While the PTR model is theoretically capable of muting individual cylinders to utilize only four cylinders, this strategy appears difficult for the model to discover. Without this cylinder muting strategy, the PTR variant faces a fundamental limitation: firing eight cylinders instead of four inherently produces fundamental frequencies and harmonic content at twice the target values—a discrepancy that cannot be compensated through pulse shaping alone. Additionally, subset A’s pronounced harmonic deviations can be effectively addressed by the HPN variant’s Gaussian harmonic bending mechanism, while the PTR variant lacks equivalent direct means to target these singular harmonic deviations as effectively. The HPN variant does not exhibit this behavior, revealing superior robustness to transitional dynamics and greater flexibility in adapting to strong harmonic deviations. Idle State Challenges: Idle states present the most significant synthesis challenges, likely due to low frequency content and explicit rhythmic patterns that blur optimization targets without phase-alignment strategies. Additionally, idle sounds are underrepresented in the dataset compared to normal driving conditions (20003500 RPM), receiving proportionally less attention during training. Loss Function Limitations: Our loss functions do not incorporate phase-alignment strategies to synchronize output and target regarding engine cycle phase, naturally promoting an averaging strategy over time-aligned acoustic events. To circumvent this informative blindspot, both variants were architecturally constrained to explain the data in an inherently synchronized manner, as detailed in Chapter 5. While this approach successfully produces perceptually plausible rhythmic characteristics, the non-synchronized loss function may still provide inadequate informative feedback for precise envelope and modulation pattern updates. 87 Chapter 9 Conclusions and Future Work This chapter summarizes the main contributions and key findings, discusses their implications, and outlines directions for future work. 9.1 Summary of Contributions This thesis makes the following key contributions: •It integrates engine acoustics domain knowledge as inductive biases into neural audio architectures and differentiable synthesis components, proposing design patterns that may be transferable to other contexts of physically grounded audio synthesis. •It introduces the Procedural Engine Sounds Dataset [51], addressing the lack of suitable datasets for engine sound modeling and related inverse problems. •It provides the Procedural Engines Model (PRCE), including the complete training, inference, and analysis pipeline with model checkpoints and weights. 88 9.2 Key Findings The conducted experiments yield the following central insights: •Both PRCE model variants reproduce engine sounds with high perceptual plausibility, capturing timbre, dynamics, and responsiveness to control parameters beyond what objective loss metrics alone can describe. •Inductive biases leave audible signatures: while the PTR variant emphasizes rhythmic, mechanically grounded features, the HPN variant produces smoother and more flexible approximations across engine types. These differences confirm that model architecture directly influences solution strategies and shapes perceptual outcomes. •Both models generalize well by focusing on essential acoustic components while disregarding stochastic variations in target samples, indicating effective denoising and robustness for real-world applications. •Limitations include difficulties in reproducing idle states due to phase alignment challenges, underrepresentation of idle data, and the absence of phasesensitive loss functions. These shortcomings motivate the architectural and methodological directions outlined in Section 9.4. 9.3 Implications This work advances research and practice in several ways: •Academic Impact: The integration of domain knowledge into neural architectures illustrates the value of inductive biases for perceptually grounded sound synthesis. The openly released dataset and model provide benchmarks and resources that address a central bottleneck in the field, enabling reproducibility and further experimentation. 89 •Practical Applications: The controllable synthesis of realistic engine sounds has direct relevance for applications in automotive acoustics, gaming, and virtual prototyping. The responsiveness and accuracy of the PRCE model variants to control parameter variations makes them suitable for interactive use cases requiring real-time sound generation with high degree of perceptual realism. •Advancement of the State of the Art: By bridging traditional procedural sound generation with differentiable synthesis and machine learning, this thesis demonstrates a pathway toward scalable, explainable, and high-quality engine sound modeling. The released dataset, model, and pipeline establish a foundation for both generative and inverse tasks, including NVH analysis and simulation studies beyond the scope of engine sound reproduction. 9.4 Future Work This work opens several promising research directions for further advancements in neural engine sound synthesis. Learned Timbre Representations: The diverse engine configurations in our procedural dataset provide an opportunity to investigate whether the model naturally organizes learned representations into meaningful timbre spaces. This would reveal whether domain-specific inductive biases enable plausible and robust interpolation between engine types, allowing navigation through acoustic characteristics not explicitly present in the training data. Transfer Learning from Procedural Foundation: Our comprehensive procedural engine sounds dataset could serve as a pre-training foundation for timbre-specific fine-tuning with limited real-world data. This approach would leverage the systematic parameter coverage achieved through our procedural generation to enable high-quality synthesis with minimal additional training samples. 90 Inverse Parameter Prediction: Our detailed parameter annotations and demonstrated audio-to-parameter relationships enable training models on the inverse problem: predicting engine performance parameters from audio. This would enable automatic dataset enhancement with non-annotated real-world data, enabling real-world validation, architectural extensions, as well as non-generative, analytical tasks common to NVH and acoustic engineering, where time-aligned RPM and torque measurements are essential for identifying acoustic phenomena related to specific engine operation states. Real-World Validation: Incorporating real annotated vehicle recordings would validate whether our proposed synthesis approaches show robust performance on authentic engine acoustics, including scenarios with confounding environmental noises and limited data quantities that characterize real-world recording conditions. This validation step is crucial for establishing the practical applicability of our procedural generation approach and would inform subsequent research directions. Audio-Driven Architecture Extensions: Building on the inverse parameter prediction foundation, our network architecture could be extended to use audio as the primary input source, predicting the underlying performance parameters at an intermediate stage (similarly to f0and loudness in the original DDSP framework). This end-to-end audio-driven approach would enable learning from non-annotated audio collections and dramatically expand available training data beyond our current procedural generation approach. Broader Applications in Vehicle Noise Studies: Both our dataset and generative network architecture extend beyond the isolated scope of engine sounds, serving as integrable components in simulation setups that investigate other acoustically relevant sources within the same context. Potential applications include optimization of in-cabin speech detection systems, studies of urban noise pollution, and other research that benefits from realistic, holistic simulations with precise parametric control in the broader domain of vehicle acoustics. 91 Data and Code Availability The datasets, models, and code developed in this work are made publicly available to support reproducible research and facilitate future developments in engine sound synthesis: Procedural Engine Sounds Dataset: The complete dataset is available on Zenodo (https://doi.org/10.5281/zenodo.16883336) and Hugging Face Datasets (https://huggingface.co/datasets/rdoerfler/procedural-engine-sounds). PRCE Model and Code: The complete implementation, including training and inference pipelines, model checkpoints, and pre-trained weights, is available in the GitHub repository: https://github.com/rdoerfler/prce-model Audio Examples: Supplementary audio examples demonstrating model outputs are available at: https://rdoerfler.github.io/prce-examples/ 92 Appendix A Harmonic Deviation Analysis This appendix provides supplementary material motivating the systematic Gaussian bending strategy for inharmonicity, as described in Section 5.2.1. Harmonic deviations are reported as the ratio between identified harmonic components and their theoretical ideals, and have been obtained across the full dataset of real engine recordings, including all microphone positions, by systematic feature extraction and analysis. These deviations are plotted as functions of RPM and torque, enabling the detection of systematic inharmonicity phenomena, along with their magnitudes and patterns. Figure 11 shows that at specific rotational speeds, multiple engine order groups exhibit synchronized systematic deviations from their ideal harmonic ratios, exceeding ±4.0%. Furthermore, as highlighted in Figure 12, certain individual engine orders display independent positive deviations that remain relatively stable across wide rpm ranges, representing order-specific phenomena that persist throughout various operating conditions. These phenomena are consistently observable across different microphone positions, effectively ruling out microphone-specific measurement artifacts and suggesting that this behavior is inherent to the engine’s acoustic characteristics. 93 Figure 11: Deviations from ideal harmonics (defined as ratios between measured and ideal harmonics, centered around 0) plotted against RPM (X-axis). Y-axis shows harmonic indices, color intensity indicates degree and direction of deviation. Main plot shows analysis for channel 1, supplementary plots on the right show channel 2, 3, 4 respectively. Patterns of harmonic deviation observable across multiple microphone positions demonstrate that several groups of engine orders exhibit synchronized systematic drifts exceeding ±4.0% at specific rotational speeds. 94 Figure 12: Distinct dominant engine orders exhibit independent, stable frequency shifts exceeding 4.0% across wide RPM ranges. Layout and conventions as in Figure 11. 95 Appendix B Supplementary Training Results This appendix contains detailed metrics that support the quantitative analysis in Chapter 8. Dataset Model Early Stopping Best Loss Values Best Epoch Stop Epoch Harmonic STFT Total AHPN 67 75 0.133 1.793 0.963 PTR 59 67 0.131 1.681 0.906 BHPN 54 62 0.097 1.827 0.962 PTR 42 50 0.097 1.774 0.935 CHPN 26 34 0.223 2.306 1.265 PTR 43 51 0.201 2.072 1.136 Mean HPN - - 0.151 1.975 1.063 PTR - - 0.143 1.842 0.993 Table 5: Model performance summary showing final validation loss values at early stopping points, defined as epochs after which no significant improvement occurred for 8 consecutive epochs. HPN and PTR models are compared across datasets A, B, and C, with loss components including harmonic reconstruction, STFT spectral matching, and total combined loss. Mean values represent averages across all three datasets. Bold values indicate superior (lower) performance between HPN and PTR models for each dataset. 96 [52] Fletcher, N. H. The nonlinear physics of musical instruments. Reports on Progress in Physics 62, 723–764 (1999). [53] Murray, C. & Whitfield, D. S. Musical String Inharmonicity. ASTRA - The McNair Scholars’ Journal (2021). [54] Wolfe, J. Inharmonic resonances in wind instruments (2025). [55] Tervaniemi, M., Just, V., Koelsch, S., Widmann, A. & Schrger, E. Pitch discrimination accuracy in musicians vs nonmusicians: An event-related potential and behavioral study. Experimental Brain Research 161, 1–10 (2005). [56] Roads, C. Sound Composition with Pulsars. Journal of Audio Engineering Society 49, 134–147 (2001). [57] Kar, K., Roberts, S., Stone, R., Oldfield, M. & French, B. Instantaneous Exhaust Temperature Measurements Using Thermocouple Compensation Techniques. In SAE 2004 World Congress & Exhibition, 2004–01–1418 (2004). [58] Venkataraman, V., Hong, B. & Cronhjort, A. Analyzing Engine Exhaust Gas Temperature Pulsations and Gas-Dynamics Using Thin-Wire Thermocouples. Journal of Engineering for Gas Turbines and Power 146, 071002 (2024). [59] Smith, J. O. Physical Audio Signal Processing (W3K Publishing). [60] Jaffe, D. A. & Smith, J. O. Extensions of the Karplus-Strong Plucked-String Algorithm. Computer Music Journal 7, 56 (1983). 3680063. [61] Jang, E., Gu, S. & Poole, B. Categorical Reparameterization with GumbelSoftmax (2017). 1611.01144. [62] Smith, J. O. Introduction to Digital Filters with Audio Applications (W3K Publishing, 2007). [63] Yamaki, S., Abe, M. & Kawamata, M. Closed Form Solutions to L2-Sensitivity Minimization Subject to L2-Scaling Constraints for Second-Order State-Space Digital Filters with Real Poles. IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences E93-A, 476–487 (2010). 103 [64] Arik, S. O., Jun, H. & Diamos, G. Fast Spectrogram Inversion using Multihead Convolutional Neural Networks. IEEE Signal Processing Letters 26, 94–98 (2019). 1808.06719. [65] Yamamoto, R., Song, E. & Kim, J.-M. Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram (2020). 1910.11480. [66] Steinmetz, C. J. & Reiss, J. D. Auraloss: Audio-focused loss functions in PyTorch (2020). 2010.10291. [67] Hayes, B., Saitis, C. & Fazekas, G. Neural Waveshaping Synthesis (2021). 2107.05050. 104