An Efficient Bit-Flip Resilience Optimization Method for Deep Neural Networks
Christoph Schorn1,2,b, Andre Guntoro1,a and Gerd Ascheid2,c
1Robert Bosch GmbH, Corporate Research, Renningen, Germany
aandre.guntoro@de.bosch.com
2Institute for Communication Technologies and Embedded Systems, RWTH Aachen University, Germany
bchristoph.schorn@de.bosch.com
cgerd.ascheid@ice.rwth-aachen.de
ABSTRACT
Deep neural networks usually possess a high overall resilience against errors in their intermediate computations. However, it has been shown that error resilience is generally not homogeneous within a neural network and some neurons might be very sensitive to faults. Even a single bit-flip fault in one of these critical neuron outputs can result in a large degradation of the final network output accuracy, which cannot be tolerated in some safety-critical applications. While critical neuron computations can be protected using error correction techniques, a resilience optimization of the neural network itself is more desirable, since it can reduce the required effort for error correction and fault protection in hardware. In this paper, we develop a novel resilience optimization method for deep neural networks, which builds upon a previously proposed resilience estimation technique. The optimization involves only few steps and can be applied to pre-trained networks. In our experiments, we significantly reduce the worst-case failure rates after a bit-flip fault for deep neural networks trained on the MNIST, CIFAR-10 and ILSVRC classification benchmarks.
Keywords: Deep neural networks, Memory faults, Fault tolerance, Fault injection, Resilience optimization.
