DATA-DRIVEN OPTIMIZATION OF HYBRID POWERTRAIN ENERGY MANAGEMENT USING REINFORCEMENT LEARNING AND ECMS

SCIENCE AND INNOVATION INTERNATIONAL SCIENTIFIC JOURNAL VOLUME 4 ISSUE 12 DECEMBER 2025 ISSN: 2181-3337 | SCIENTISTS.UZ 33 DATA-DRIVEN OPTIMIZATION OF HYBRID POWERTRAIN ENERGY MANAGEMENT USING REINFORCEMENT LEARNING AND ECMS N.A. Abdivakhidova1, K.A. Abdivakhidov2 Master of Science, Senior Teacher, Turin Polytechnic University in Tashkent, Tashkent, Uzbekistan1 Head of Department, Kimyo International University in Tashkent, Tashkent, Uzbekistan2 https://doi.org/10.5281/zenodo.18027733 Abstract. Hybrid electric vehicles (HEVs) require an optimal energy management strategy (EMS) to allocate power between the internal combustion engine (ICE) and the electric motor. Although they are straightforward, traditional rule-based approaches do not account for future electricity needs. Equivalent consumption minimization strategy (ECMS) improves fuel economy by minimizing instantaneous equivalent fuel consumption; however, it relies on a fixed equivalence factor and lacks long-term planning. In this paper, a data-driven energy management strategy that combines ECMS and reinforcement learning (RL) is proposed. The controller calculates the effective equivalency ratio and the optimal torque split directly from the data, using the instantaneous ECMS fuel cost as the RL reward (case 4). The Pasadena, US06, and WLTC cycles were used to test the technique. The findings indicate that the RL + ECMS system outperforms the rule-based EMS, ECMS, and A-ECMS systems in terms of fuel consumption reduction and battery state of charge (SoC) stability. Simulations validate enhanced RL reward convergence, adaptive power distribution, and SoC stability. Keywords: hybrid Electric Vehicle, Energy Management Strategy, ECMS, Reinforcement Learning, A-ECMS, Data-driven optimization. 1. Introduction Fuel economy and emissions are greatly impacted by the effective distribution of power between the electric motor and internal combustion engine (ICE). The simplicity of rule-based EMS techniques makes them popular, but they cannot adapt to changing driving conditions nor maintain battery State of Charge (SoC) over long driving horizons [1].ECMS improves efficiency by minimizing the instantaneous equivalent fuel power: 𝐽(𝑡)= 𝑚˙𝑓𝑢𝑒𝑙(𝑡)+𝑠·𝑃𝑏𝑎𝑡𝑡(𝑡) (1) Where 𝑚˙𝑓𝑢𝑒𝑙(𝑡)− fuel mass flow rate, 𝑃𝑏𝑎𝑡𝑡(𝑡)-battery power, 𝑠− equivalence factor controlling the trade-off between fuel and electrical energy. The power balance constraint ensures that the demanded power is satisfied: 𝑃𝑑𝑒𝑚(𝑡)= 𝑃𝑒𝑛𝑔(𝑡)+𝑃𝑚𝑜𝑡(𝑡) (2) Despite being optimal at every time step, the ECMS is still limited because it depends on a fixed or manually adjustable equivalency factor (s) and does not forecast the future trajectory of the SoC. The controller might be too conservative or overload the battery as driving conditions change. Reinforcement Learning (RL) offers a data-driven alternative by learning control actions SCIENCE AND INNOVATION INTERNATIONAL SCIENTIFIC JOURNAL VOLUME 4 ISSUE 12 DECEMBER 2025 ISSN: 2181-3337 | SCIENTISTS.UZ 34 directly from interaction with the environment, without requiring predefined rules or future cycle prediction [2]. The benefits of both strategies are combined when RL and ECMS are integrated: ECMS ensures real-time feasibility, while RL ensures long-term optimality. In an EMS based on reinforcement learning, the problem is formulated as a Markov decision process (MDP). The system state is expressed as: 𝑆𝑡={𝑆𝑜𝐶𝑡,𝑃𝑑𝑒𝑚,𝑡,𝑣𝑡,𝑇𝑒𝑛𝑔,𝑡} (3) The action represents the torque split ratio between ICE and electric motor: 𝐴𝑡= 𝛼,0 ≤ 𝛼 ≤ 1 (4) The reward penalizes fuel usage and deviation from reference battery charge: 𝑅𝑡= −) (5) Thus, RL learns how current decisions affect long-term fuel consumption and SoC stability, overcoming the main limitations of ECMS. 2. Background: ECMS, A-ECMS, and Reinforcement Learning (Reward Formulation) By transforming electrical power into equivalent fuel cost, ECMS reduces instantaneous equivalent fuel consumption [1]. It performs less well under shifting driving conditions, though, because it employs a fixed equivalency factor (s). By employing drive-cycle prediction (such as LSTM-based vehicle speed prediction) to adjust s in real-time, A-ECMS enhances ECMS. Its reliance on prediction accuracy and the availability of historical driving data is its primary drawback. With reinforcement learning (RL), manual tuning and prediction are no longer necessary. The ECMS cost function is used as a reward by RL in this work. 𝑅𝑡= −𝐽(𝑡)= −(𝑚˙𝑓𝑢𝑒𝑙(𝑡)+𝑠·𝑃𝑏𝑎𝑡𝑡(𝑡)) (6) enabling the controller to directly learn the effective equivalency ratio and the ideal torque distribution from the data [3]. 3. Experimental Results The proposed RL + ECMS strategy was evaluated on the WLTC, US06, and Pasadena city cycles, representing stationary, aggressive, and stochastic traffic. Four EMS strategies were compared: rule-based control, ECMS with a fixed equivalence factor (s), A-ECMS with LSTMbased forecasting, and the proposed RL + ECMS strategy. 3.1 SoC trajectory The battery charge level change during WLTC is depicted in Figure 1. Rule-based EMS results in rapid battery depletion.. Because of the fixed equivalency factor, ECMS slows down the discharge but is unable to maintain the charge level. A-ECMS stabilizes SoC when prediction accuracy is sufficient. In order to show that the agent learns longterm energy balance, RL + ECMS keeps the charge level as near to the reference level as possible without requiring prediction. Figure 1. SoC trajectory comparison during WLTC for Rule-based, ECMS, A-ECMS (LSTM), and RL + ECMS strategies. SCIENCE AND INNOVATION INTERNATIONAL SCIENTIFIC JOURNAL VOLUME 4 ISSUE 12 DECEMBER 2025 ISSN: 2181-3337 | SCIENTISTS.UZ 35 3.2 Fuel consumption comparison Normalized fuel consumption numbers are compared in Figure 2. Fuel consumption is reduced by about 7% with the ECMS system, 9% with the A-ECMS system, and 11% with the RL + ECMS system. The RL system offers better fuel economy than the A-ECMS system, but it does not require prior knowledge of future speed or power consumption. Figure 2. Normalized fuel consumption across WLTC (lower is better). 3.3 Power split behavior The engine to electric motor power ratio is displayed in Figure 3. While the RL+ECMS system proactively distributes engine/electric motor power, balancing short-term fuel efficiency with long-term battery charge preservation, the A-ECMS system reacts to anticipated cycle changes. Figure 3. Power split dynamics between engine and motor under the RL + ECMS strategy. SCIENCE AND INNOVATION INTERNATIONAL SCIENTIFIC JOURNAL VOLUME 4 ISSUE 12 DECEMBER 2025 ISSN: 2181-3337 | SCIENTISTS.UZ 36 3.4 RL training convergence The RL convergence is displayed in Figure 4. The learned policy consistently minimizes cumulative fuel consumption, as evidenced by the moving average reward, which rises with training and stabilizes after about 70 episodes. Figure 4. Reinforcement learning convergence during training (episode reward). 4. Conclusion For hybrid electric vehicles, RL + ECMS, a hybrid energy management approach, was developed and experimentally validated. The suggested approach uses data to determine the ideal torque distribution and effective equivalency ratio, in contrast to ECMS, which only optimizes fuel consumption instantly, and A-ECMS, which depends on predictive accuracy. In comparison to rule-based EMS, ECMS, and A-ECMS, tests on the WLTC, US06, and Pasadena cycles demonstrate that RL + ECMS provides the lowest fuel consumption and maintains battery SoC close to its reference level. RL + ECMS is a promising solution for real-time hybrid powertrain control under unpredictable driving conditions because it can learn optimal behavior without the need for manual tuning or prediction. REFERENCES 1. S. Onori, L. Serrao, and G. Rizzoni, Hybrid Electric Vehicles: Energy Management Strategies. Springer, 2016. ISBN: 978-1-4471-6780-6. 2. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. MIT Press, 2018. (2nd Edition) Available: http://incompleteideas.net/book/the-book-2nd.html 3. International Council on Clean Transportation (ICCT), CO₂ Reduction Strategies in Hybrid Electric Vehicles., Technical Report, 2022. Available: https://theicct.org