Abstract:The thermal system of battery electric vehicles demands advanced control. Its thermal management needs to effectively control active components across varying operating conditions. While robust control function parametrization is required, current methodologies show significant drawbacks. They consume considerable time, human effort, and extensive real-world testing. Consequently, there is a need for innovative and intelligent solutions that are capable of autonomously parametrizing embedded controllers. Addressing this issue, our paper introduces a learning-based tuning approach. We propose a methodology that benefits from automated scenario generation for increased robustness across vehicle usage scenarios. Our deep reinforcement learning agent processes the tuning task context and incorporates an image-based interpretation of embedded parameter sets. We demonstrate its applicability to a valve controller parametrization task and verify it in real-world vehicle testing. The results highlight the competitive performance to baseline methods. This novel approach contributes to the shift towards virtual development of thermal management functions, with promising potential of large-scale parameter tuning in the automotive industry.

What problem does this paper attempt to address?

This paper attempts to solve the problem of feedback controller parameterization in the thermal management system (TS) of battery electric vehicles (BEV). Specifically, the current parameterization methods have the following problems: 1. **Time - consuming and labor - intensive**: The existing parameterization process requires a great deal of time and manpower and relies on extensive field tests. 2. **Difficult to adapt to different usage scenarios**: The existing methods are not flexible enough when dealing with different vehicle usage scenarios and cannot effectively cover all actual usage situations. 3. **Lack of automation**: The current methods have a low degree of automation and are difficult to cope with the complexity of the thermal management system and changing system configurations. To solve these problems, the paper proposes a learning - based parameter - tuning method based on deep reinforcement learning (DRL). This method uses automatic scenario generation techniques to improve robustness under different vehicle usage scenarios. In this way, the dependence on physical prototypes can be reduced, and efficient parameter optimization in the virtual development environment can be achieved. ### Main contributions 1. **Graphical ECU parameter representation**: A new graphical electronic control unit (ECU) parameter representation method is proposed to improve processing efficiency. 2. **Combination of scenario generation and DRL**: The thermal system scenario generation method is combined with the deep reinforcement learning agent for the parameterization of the feedback controller. 3. **Verification in practical applications**: The effectiveness of the proposed TM valve controller parameterization method is verified in actual battery electric vehicles. ### Formula summary - System dynamic equations: \[ \dot{T}_D = f(\alpha, \dot{V}, T_{U1}, T_{U2}, T_D) \] \[ \dot{T}_{U1} = f(\dot{Q}_{ED}, \dot{V}, T_{U1}) = f(P_{ED}, v_{veh}, \xi_{th}, \xi_{el}, k_{hyd}, u_{pmp}, T_{U1}) \] \[ \dot{T}_{U2} = f(\alpha, \dot{Q}_{HE}, \dot{V}, T_{U2}) = f(u_{vlv}, u_{fan}, u_{shu}, v_{veh}, \Delta T, k_{hyd}, u_{pmp}, T_{U2}) \] - Controller parameter optimization objective function: \[ \phi^* = \arg\min_\phi J \] - Reinforcement learning reward function: \[ r_J = -\frac{1}{N} \sum_{n = 0}^{N - 1}\left[b_1\left(\sqrt{|e_T(n)|}+e_T^2(n)\right)+b_2\cdot u_{vlv}^2(n)\right] \] where \(b_1 = 25\), \(b_2 = 0.1\). Through these methods and formulas, the paper aims to provide a more efficient and intelligent solution to deal with the complex parameter optimization problems in the thermal management system of battery electric vehicles.

Scenario-based Thermal Management Parametrization Through Deep Reinforcement Learning

Autonomous Vehicle Control Through the Dynamics and Controller Learning

An Efficient and Flexible Learning Framework for Dynamic Power and Thermal Co-Management

Supervised-Learning-Based Optimal Thermal Management in an Electric Vehicle

Reinforcement learning based EV energy management for integrated traction and cabin thermal management considering battery aging

Real-Time Battery Thermal Management for Electric Vehicles Based on Deep Reinforcement Learning

Virtual Development of Advanced Thermal Management Functions Using Model-in-the-Loop Applications

A deep reinforcement learning based EMS for PHEV considering temperature equilibrium of battery modules

Intelligent control of electric vehicle air conditioning system based on deep reinforcement learning

Reinforcement learning-based thermal comfort control for vehicle cabins

optimization of a Thermal Management System for Battery Electric Vehicles

Deep reinforcement learning-based energy management of hybrid battery systems in electric vehicles

Integrated Thermal and Energy Management of Connected Hybrid Electric Vehicles Using Deep Reinforcement Learning

Data-driven control of room temperature and bidirectional EV charging using deep reinforcement learning: Simulations and experiments

Thermal and Energy Management Based on Bimodal Airflow-Temperature Sensing and Reinforcement Learning

Battery Health-Aware and Deep Reinforcement Learning-Based Energy Management for Naturalistic Data-Driven Driving Scenarios

Cloud-based Health-Conscious Energy Management of Hybrid Battery Systems in Electric Vehicles with Deep Reinforcement Learning

A Fast Thermal Simulation and Dynamic Feedback Control Framework for Lithium-Ion Batteries

The integration of Model Predictive Control and deep Reinforcement Learning for efficient thermal control in thermoforming processes

Data-driven modeling and supervisory control system optimization for plug-in hybrid electric vehicles

Integrated Thermal Management of Electric Vehicles Based on Model Predictive Control With Approximated Value Function