Multi-objective Precision Optimization of Deep Neural Networks for Edge Devices
Nhut-Minh Hoa, Ramesh Vaddib and Weng-Fai Wongc
School of Computing, National University of Singapore
aminhhn@comp.nus.edu.sg
bramesh@comp.nus.edu.sg
cwongwf@comp.nus.edu.sg
ABSTRACT
Precision tuning post-training is often needed for efficient implementation of deep neural networks especially when the inference implementation platform is resource constrained. While previous works have proposed many ad hoc strategies for this task, this paper describes a general method for allocating precision to trained deep neural networks data based on a property relating errors in a network. We demonstrate that the precision results of previous works for hardware accelerator or understanding cross layer precision requirement is subsumed by the proposed general method. It has achieved a 29% and 46% energy saving over the state-of-the-art search-based method for GoogleNet and VGG-19 respectively. Proposed precision allocation method can be used to optimize for different criteria based on hardware design constraints, allocating precision at the granularity of layers for very deep networks such as Resnet-152, which hitherto was not achievable.
