Deep Positron: A Deep Neural Network Using the Posit Number System
Zachariah Carmichael1, Hamed F. Langroudi1, Char Khazanov1, Jeffrey Lillie1, John L. Gustafson2 and Dhireesha Kudithipudi1
1Neuromorphic AI Lab, Rochester Institute of Technology, NY, USA
2National University of Singapore, Singapore
ABSTRACT
The recent surge of interest in Deep Neural Networks (DNNs) has led to increasingly complex networks that tax computational and memory resources. Many DNNs presently use 16-bit or 32-bit floating point operations. Significant performance and power gains can be obtained when DNN accelerators support low-precision numerical formats. Despite considerable research, there is still a knowledge gap on how low-precision operations can be realized for both DNN training and inference. In this work, we propose a DNN architecture, Deep Positron, with posit numerical format operating successfully at ≤8 bits for inference. We propose a precision-adaptable FPGA soft core for exact multiply-and-accumulate for uniform comparison across three numerical formats, fixed, floating-point and posit. Preliminary results demonstrate that 8-bit posit has better accuracy than 8-bit fixed or floating-point for three different low-dimensional datasets. Moreover, the accuracy is comparable to 32-bit floatingpoint on a Xilinx Virtex-7 FPGA device. The trade-offs between DNN performance and hardware resources, i.e. latency, power, and resource utilization, show that posit outperforms in accuracy and latency at 8-bit and below.
Keywords: Deep neural networks, Machine learning, DNN accelerators, Posits, Floating point, Tapered precision, Lowprecision.
