AdaFlow: A Framework for Adaptive Dataflow CNN Acceleration on FPGAs
Guilherme Korol1,a, Michael Guilherme Jordan1,b, Mateus Beck Rutzig2 and Antonio Carlos Schneider Beck1,c
1Institute of Informatics - Universidade Federal do Rio Grande do Sul (UFRGS) - Porto Alegre - Brazil
agskorol@inf.ufrgs.br
bmgjordan@inf.ufrgs.br
ccaco@inf.ufrgs.br
2Electronics and Computing Department - Universidade Federal de Santa Maria (UFSM) - Santa Maria - Brazil
mateus@inf.ufsm.br
ABSTRACT
To meet latency and privacy requirements, resourcehungry deep learning applications have been migrating to the Edge, where IoT devices can offload the inference processing to local Edge servers. Since FPGAs have successfully accelerated an increasing number of deep learning applications (especially CNN-based ones), they emerge as an effective alternative for Edge platforms. However, Edge applications may present highly unpredictable workloads, requiring runtime adaptability in the inference processing. Although some works apply model switching on CPU and GPU platforms by exploiting different pruning rates at runtime, so the inference can adapt according to some qualityperformance trade-off, FPGA-based accelerators refrain from this approach since they are synthesized to specific CNN models. In this context, this work enables model switching on FPGAs by adding to the well-known FINN accelerator an extra level of adaptability (i.e., flexibility) and support to the dynamic use of pruning via fast model switch on flexible accelerators, at the cost of some extra logic, or via FPGA reconfigurations of fixed accelerators. From that, we developed AdaFlow: a framework that automatically builds, at design time, a library from these new available versions (flexible and fixed, pruned or not) that will be used, at runtime, to dynamically select a given version according to a user-configurable accuracy threshold and current workload conditions. We have evaluated AdaFlow under a smart Edge surveillance application with two CNN models and two datasets, showing that AdaFlow processes, on average, 1.3× more inferences and increases, on average, 1.4× the power efficiency over state-of-the-art statically deployed dataflow accelerators.
Keywords: Edge Computing, Adaptive Inference, CNN, FPGA.
