RAPNet: Resolution-Adaptive and Predictive Early Exit Network for Efficient Image Recognition

Youbing Hu,Yun Cheng,Zimu Zhou,Zhiqiang Cao,Anqi Lu,Jie Liu,Min Zhang,Zhijun Li

DOI: https://doi.org/10.1109/jiot.2024.3428554

2024-01-01

Abstract:Deploying compute-intensive deep neural networks (DNNs) on resource-constrained end devices has become a prominent trend, enabling localized intelligence. However, efficiently deploying these DNNs at scale poses challenges. To address this, extensive research has focused on the early exit architecture based on convolutional neural networks (CNNs), which dynamically adapt network depth to reduce inference computation. Nevertheless, the sequential execution of all internal classifiers (ICs) and subsequent termination based on an exit criterion is inefficient. Motivated by these insights, we introduce a resolution-adaptive prediction network (RAPNet) architecture. RAPNet comprises a lightweight prediction network that captures global image features and an inference network integrated with an early exit architecture. The prediction network accurately determines the optimal IC position conditioned on the input images for efficient image classification. Additionally, we incorporate resolution-adaptive inference and feature fusion mechanisms by computational reuse, to effectively mitigate image spatial redundancy and improve the accuracy of ICs. We conduct extensive experiments across various data sets and architectures to demonstrate that RAPNet achieves a significantly better accuracy versus computational tradeoff than other recently proposed early exit methods. For instance, when using MobileNet as the base network, RAPNet achieves significant accuracy improvements of 12% and 5.7% on the Tiny Imagenet and CIFAR-100 data sets, respectively, surpassing other early exit methods with similar computational constraints.