OnlineHD: Robust, Efficient, and Single-Pass Online Learning Using Hyperdimensional System
Alejandro Hernández-Cano1, Namiko Matsumoto2, Eric Ping2 and Mohsen Imani3
1Universidad Nacional Autónoma de México
2University of California San Diego
3University of California Irvine
m.imani@uci.edu
ABSTRACT
Hyper-Dimensional computing (HDC) is a braininspired learning approach for efficient and robust learning on today’s embedded devices. HDC supports single-pass learning, where it generates a classification model by one-time looking at each training data point. However, the single-pass model provides weak classification accuracy due to model saturation caused by naively accumulating high-dimensional data. Although the retraining model for hundreds of iterations addresses the model saturation and boosts the accuracy, it comes with significant training costs. In this paper, we propose OnlineHD, an adaptive HDC training framework for accurate, efficient, and robust learning. During single-pass training, OnlineHD identifies common patterns and eliminates model saturation. For each data point, OnlineHD updates the model depending on how similar it is to the existing model, instead of naive data accumulation. We expand the OnlineHD framework to support highly-accurate iterative training. We also exploit the holographic distribution of patterns in high-dimensional space to make OnlineHD ultra-robust against possible noise and hardware failure. Our evaluations on a wide range of classification problems show that OnlineHD adaptive training provides comparable classification accuracy to the retrained model while getting all efficiency benefits that a singlepass training provides. OnlineHD achieves, on average, 3.5 × and 6.9 × (3.7× and 5.8×) faster and more efficient training as compared to state-of-the-art machine learning (HDC algorithms), while providing similar classification accuracy and 8.5× higher robustness to a hardware error.
