Deep Reinforcement Learning-based Energy Management for Hybrid Electric Vehicles (Synthesis Lectures on Advances in Automotive Technology)

The urgent need for vehicle electrification and improvement in fuel efficiency has gained increasing attention worldwide. Regarding this concern, the solution of hybrid vehicle systems has proven its value from academic research and industry applications, where energy management plays a key role in taking full advantage of hybrid electric vehicles (HEVs). There are many well-established energy management approaches, ranging from rules-based strategies to optimization-based methods, that can provide diverse options to achieve higher fuel economy performance. However, the research scope for energy management is still expanding with the development of intelligent transportation systems and the improvement in onboard sensing and computing resources. Owing to the boom in machine learning, especially deep learning and deep reinforcement learning (DRL), research on learning-based energy management strategies (EMSs) is gradually gaining more momentum. They have shown great promise in not onlybeing capable of dealing with big data, but also in generalizing previously learned rules to new scenarios without complex manually tunning. Focusing on learning-based energy management with DRL as the core, this book begins with an introduction to the background of DRL in HEV energy management. The strengths and limitations of typical DRL-based EMSs are identified according to the types of state space and action space in energy management. Accordingly, value-based, policy gradient-based, and hybrid action space-oriented energy management methods via DRL are discussed, respectively. Finally, a general online integration scheme for DRL-based EMS is described to bridge the gap between strategy learning in the simulator and strategy deployment on the vehicle controller.

Yuecheng Li is currently working at the Beijing Institute of Specialized Machinery. He obtained his Ph.D. from Beijing Institute of Technology in 2021 and studied in the Mechatronic Vehicle Systems Lab, University of Waterloo, as a visiting student from 2018–2019. His research interests include hybrid powertrains and energy management, intelligent control theories, and machine learning applied to vehicles. Hongwen He is currently a Professor with the National Engineering Laboratory for Electric Vehicles, School of Mechanical Engineering, Beijing Institute of Technology. He has authored or coauthored 126 EI-indexed papers, 82 SCI-indexed papers, and 17 ESI highly cited papers. He is the recipient of the second prize of the Chinese National Science and Technology Award, the first prize of natural science by the Ministry of Education, and the first prize of technological invention of China’s automobile industry.