RedMulE: A Compact FP16 Matrix-Multiplication Accelerator for Adaptive Deep Learning on RISC-V-Based Ultra-Low-Power SoCs
Yvan Tortorella1,a, Luca Bertaccini2,e, Davide Rossi1,b, Luca Benini1,2,c,f and Francesco Conti1,d
1University of Bologna, Bologna, Italy
ayvan.tortorella@unibo.it
bdavide.rossi@unibo.it
cluca.benini@unibo.it
df.conti@unibo.it
2ETH Zurich, Zurich, Switzerland
elbertaccini@iis.ethz.ch
flbenini@iis.ethz.ch
ABSTRACT
The fast proliferation of extreme-edge applications using Deep Learning (DL) based algorithms required dedicated hardware to satisfy extreme-edge applications’ latency, throughput, and precision requirements. While inference is achievable in practical cases, online finetuning and adaptation of general DL models are still highly challenging. One of the key stumbling stones is the need for parallel floating-point operations, which are considered unaffordable on sub-100mW extreme-edge SoCs. We tackle this problem with RedMulE (Reduced-precision matrix Multiplication Engine), a parametric low-power hardware accelerator for FP16 matrix multiplications - the main kernel of DL training and inference - conceived for tight integration within a cluster of tiny RISC-V cores based on the PULP (Parallel Ultra-Low-Power) architecture. In 22nm technology, a 32-FMA RedMulE instance occupies just 0.07mm2 (14% of an 8-core RISC-V cluster) and achieves up to 666MHz maximum operating frequency, for a throughput of 31.6MAC/cycle (98.8% utilization). We reach a cluster-level power consumption of 43.5mW and a full-cluster energy efficiency of 688 16-bit GFLOPS/W. Overall, RedMulE features up to 4.65× higher energy efficiency and 22× speedup over SW execution on 8 RISC-V cores.
