Structural Design Optimization for Deep Convolutional Neural Networks using Stochastic Computing
Zhe Li1,a, Ao Ren1,b, Ji Li2, Qinru Qiu1,c, Bo Yuan3, Jeffrey Draper2,4 and Yanzhi Wang1,d
1Syracuse University, Syracuse, USA.
azli89@syr.edu
baren@syr.edu
cqiqiu@syr.edu
dywang393@syr.edu
2University of Southern California, Los Angeles, USA.
jli724@usc.edu
3City University of New York, New York, USA.
byuan@ccny.cuny.edu
4Information Sciences Institute, Marina Del Rey, USA.
draper@isi.edu
ABSTRACT
Deep Convolutional Neural Networks (DCNNs) have been demonstrated as effective models for understanding image content. The computation behind DCNNs highly relies on the capability of hardware resources due to the deep structure. DCNNs have been implemented on different large-scale computing platforms. However, there is a trend that DCNNs have been embedded into light-weight local systems, which requires low power/energy consumptions and small hardware footprints. Stochastic Computing (SC) radically simplifies the hardware implementation of arithmetic units and has the potential to satisfy the small low-power needs of DCNNs. Local connectivities and down-sampling operations have made DCNNs more complex to be implemented using SC. In this paper, eight feature extraction designs for DCNNs using SC in two groups are explored and optimized in detail from the perspective of calculation precision, where we permute two SC implementations for inner-product calculation, two down-sampling schemes, and two structures of DCNN neurons. We evaluate the network in aspects of network accuracy and hardware performance for each DCNN using one feature extraction design out of eight. Through exploration and optimization, the accuracies of SC-based DCNNs are guaranteed compared with software implementations on CPU/GPU/binarybased ASIC synthesis, while area, power, and energy are significantly reduced by up to 77638215;, 19038215;, and 3283538215.
