An eco-driving algorithm for trains through distributing energy: A Q-Learning approach

Qingyang Zhu,Shuai Su,Tao Tang,Wentao Liu,Zixuan Zhang,Qinghao Tian
DOI: https://doi.org/10.1016/j.isatra.2021.04.036
IF: 7.3
2022-03-01
ISA Transactions
Abstract:The inverse problem of energy-efficient train control problem is formulated.A data-driven method based on Q-Learning approach is proposed.Two state definitions based on trip time and energy distribution are introduced.The energy-efficient train operation methodology is the focus of this paper, and a Q-Learning-based eco-driving approach is proposed. Firstly, the core idea of energy-distribution-based method (EDBM) that converts the eco-driving problem to the finite Markov decision process is presented. Secondly, Q-Learning approach is proposed to determine the optimal energy distribution policy. Specifically, two different state definitions, i.e., trip-time-relevant (TT) and energy-distribution-relevant (ED) state definitions, are introduced. Finally, the effectiveness of the proposed approach is verified in a deterministic and a stochastic environment. It is also illustrated that TT-state approach takes about 20 times more computation time compared with ED-state approach while the space complexity of TT-state approach is nearly constant. The hyperparameter sensitivity analysis demonstrates the robustness of the proposed approach.
automation & control systems,instruments & instrumentation,engineering, multidisciplinary
What problem does this paper attempt to address?