Improve the Stability and Robustness of Power Management through Model-free Deep Reinforcement Learning
Lin Chen1, Xiao Li1 and Jiang Xu1,2
1Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology
2Microelectronics Thrust, The Hong Kong University of Science and Technology
jiang.xu@ust.hk
ABSTRACT
Achieving high performance with low energy consumption has become a primary design objective in multi-core systems. Recently, power management based on reinforcement learning has shown great potential in adapting to dynamic environments without much prior knowledge. However, conventional Q-learning (QL) algorithms adopted in most existing works encounter serious problems about scalability, instability, and overestimation. In this paper, we present a deep reinforcement learning-based approach to improve the stability and robustness of power management while reducing the energy-delay product (EDP) under user-specified performance requirements. The comprehensive status of the system is monitored periodically, making our controller sensitive to environmental change. To further improve the learning effectiveness, knowledge sharing among multiple devices is implemented in our approach. Experimental results on multiple realistic applications show that the proposed method can reduce the instability up to 68% compared with QL. Through knowledge sharing among multiple devices, our federated approach achieves around 4.8% EDP improvement over QL on average.
Keywords: Power Management, Deep Reinforcement Learning, Experience Replay, Federated Learning, Multicore System.
