Ternarized TCN for µJ/Inference Gesture Recognition from DVS Event Frames
Georg Rutishauser1, Moritz Scherer1, Tim Fischer1 and Luca Benini1,2
1Departement Informationstechnologie und Elektrotechnik, ETH Zürich, Zürich, Switzerland
2Dipartimento di Ingegneria dell'Energia Elettrica e dell'Informazione, Università di Bologna, Bologna, Italy
ABSTRACT
Dynamic Vision Sensors (DVS) offer the opportunity to scale the energy consumption in image acquisition proportionally to the activity in the captured scene by only transmitting data when the captured image changes. Their potential for energyproportional sensing makes them highly attractive for severely energy-constrained sensing nodes at the edge. Most approaches to the processing of DVS data employ Spiking Neural Networks to classify the input from the sensor. In this paper, we propose an alternative, event frame-based approach to the classification of DVS video data. We assemble ternary video frames from the event stream and process them with a fully ternarized Temporal Convolutional Network which can be mapped to CUTIE, a highly energy-efficient Ternary Neural Network accelerator. The network mapped to the accelerator achieves a classification accuracy of 94:5%, matching the state of the art for embedded implementations. We implement the processing pipeline in a modern 22nm FDX technology and perform post-synthesis power simulation of the network running on the system, achieving an inference energy of 1:7 µJ, which is 647× lower than previously reported results based on Spiking Neural Networks.
Keywords: Machine Learning, Edge Computing, Classification Algorithms.
