ROQ: A Noise-Aware Quantization Scheme Towards Robust Optical Neural Networks with Low-bit Controls
Jiaqi Gu1,a, Zheng Zhao1,b, Chenghao Feng1,c, Hanqing Zhu2 ,Ray T. Chen1,d and David Z. Pan1,e
1ECE Department, The University of Texas at Austin
ajqgu@utexas.edu
bzhengzhao@utexas.edu
cfengchenghao1996@utexas.edu
dchenrt@austin.utexas.edu
edpan@ece.utexas.edu
2Department of Microelectronics, Shanghai Jiao Tong University, China
ABSTRACT
Optical neural networks (ONNs) demonstrate ordersof- magnitude higher speed in deep learning acceleration than their electronic counterparts. However, limited control precision and device variations induce accuracy degradation in practical ONN implementations. To tackle this issue, we propose a quantization scheme that adapts a full-precision ONN to low-resolution voltage controls. Moreover, we propose a protective regularization technique that dynamically penalizes quantized weights based on their estimated noise-robustness, leading to an improvement in noise robustness. Experimental results show that the proposed scheme effectively adapts ONNs to limited-precision controls and device variations. The resultant four-layer ONN demonstrates higher inference accuracy with lower variances than baseline methods under various control precisions and device noises.
