A Coordinated Multi-Agent Reinforcement Learning Approach to Multi-Level Cache Co-partitioning
Rahul Jain1,a, Preeti Ranjan Panda1,b and Sreenivas Subramoney2
1Dept. of Computer Sc. and Engg., Indian Institute of Technology Delhi.
arahuljain@cse.iitd.ac.in
bpanda@cse.iitd.ac.in
2Microarchitecture Research Lab., Intel Technology India Pvt. Ltd., Bangalore.
sreenivas.subramoney@intel.com
ABSTRACT
The widening gap between the processor and memory performance has led to the inclusion of multiple levels of caches in the modern multi-core systems. Processors with simultaneous multithreading (SMT) support multiple hardware threads on the same physical core, which results in shared private caches. Any inefficiency in the cache hierarchy can negatively impact the system performance and motivates the need to perform a co-optimization of multiple cache levels by trading off individual application throughput for better system throughput and energydelay- product (EDP). We propose a novel coordinated multiagent reinforcement learning technique for performing Dynamic Cache Co-partitioning, called Machine Learned Caches (MLC). MLC has low implementation overhead and does not require any special hardware data profilers. We have validated our proposal with 15 8-core workloads created using Spec2006 benchmarks and found it to be an effective co-partitioning technique. MLC exhibited system throughput and EDP improvements of up to 14% (gmean:9.35%) and 19.2% (gmean: 13.5%) respectively. We believe this is the first attempt at addressing the problem of multi-level cache co-partitioning.
