An Improved STBP for Training High-Accuracy and Low-Spike-Count Spiking Neural Networks
Pai-Yu Tan1, Cheng-Wen Wu1,2 and Juin-Ming Lu1,3
1Department of Electrical Engineering, National Tsing Hua University, Hsinchu, Taiwan
2Department of Electrical Engineering, National Cheng Kung University, Tainan, Taiwan
3Information and Communications Research Laboratories, Industrial Technology Research Institute, Hsinchu, Taiwan
ABSTRACT
Spiking Neural Networks (SNNs) that facilitate energy-efficient neuromorphic hardware are getting increasing attention. Directly training SNN with backpropagation has already shown competitive accuracy compared with Deep Neural Networks. Besides the accuracy, the number of spikes per inference has a direct impact on the processing time and energy once employed in the neuromorphic processors. However, previous direct-training algorithms do not put great emphasis on this metric. Therefore, this paper proposes four enhancing schemes for the existing direct-training algorithm, Spatio-Temporal Back- Propagation (STBP), to improve not only the accuracy but also the spike count per inference. We first modify the reset mechanism of the spiking neuron model to address the information loss issue, which enables the firing threshold to be a trainable variable. Then we propose two novel output spike decoding schemes to effectively utilize the spatio-temporal information. Finally, we reformulate the derivative approximation of the non-differentiable firing function to simplify the computation of STBP without accuracy loss. In this way, we can achieve higher accuracy and lower spike count per inference on image classification tasks. Moreover, the enhanced STBP is feasible for the on-line learning hardware implementation in the future.
Keywords: Artificial Intelligence (AI), Backpropagation, Direct-Training Algorithm, Image Classification, Machine Learning, Neuromorphic Computing, Spiking Neural Network (SNN).
