When Sorting Network Meets Parallel Bitstreams: A Fault-Tolerant Parallel Ternary Neural Network Accelerator based on Stochastic Computing
Yawen Zhang1, Sheng Lin2, Runsheng Wang1,a, Yanzhi Wang2,b, Yuan Wang1,c, Weikang Qian3,d and Ru Huang1
1Institute of Microelectronics, Peking University, Beijing, P.R. China
2Department of Electrical and Computer Engineering, Northeastern University, Boston, USA
3UM-SJTU Joint Institute, Shanghai Jiao Tong University, Shanghai, P.R. China
ar.wang@pku.edu.cn
byanz.wang@northeastern.edu
cwangyuan@pku.edu.cn
dqianwk@sjtu.edu.cn
ABSTRACT
Stochastic computing (SC) has been widely used in neural networks (NNs) due to its simple hardware cost and high fault tolerance. Conventionally, SC-based NN accelerators adopt a hybrid stochastic-binary format, using an accumulative parallel counter to convert bitstreams into a binary number. This method, however, sacrifices the fault tolerance and causes a high hardware cost. In order to fully exploit the superior fault tolerance of SC, taking a ternary neural network (TNN) as an example, we propose a parallel SC-based NN accelerator purely using bitstream computation. We apply a bitonic sorting network for simultaneously implementing the accumulation and activation function with parallel bitstreams. The proposed design not only has high fault tolerance, but also achieves at least 2:8⨯ energy efficiency improvement over the binary computing counterpart.
Keywords: Stochastic Computing, Ternary Neural Network, Bitonic Sort, Parallel Computing.
