Abstract:Deploying deep models on embedded devices has been a challenging problem since the great success of deep learning based networks. Fixed-point networks, which represent their data with low bits fixed-point and thus give remarkable savings on memory usage, are generally preferred. Even though current fixed-point networks employ relative low bits (e.g. 8-bits), the memory saving is far from enough for the embedded devices. On the other hand, quantization deep networks, for example XNOR-Net and HWGQNet, quantize the data into 1 or 2 bits resulting in more significant memory savings but still contain lots of floatingpoint data. In this paper, we propose a fixed-point network for embedded vision tasks through converting the floatingpoint data in a quantization network into fixed-point. Furthermore, to overcome the data loss caused by the conversion, we propose to compose floating-point data operations across multiple layers (e.g. convolution, batch normalization and quantization layers) and convert them into fixedpoint. We name the fixed-point network obtained through such integrated conversion as Integrated Fixed-point Quantization Networks (IFQ-Net). We demonstrate that our IFQNet gives 2.16x and 18x more savings on model size and runtime feature map memory respectively with similar accuracy on ImageNet. Furthermore, based on YOLOv2, we design IFQ-Tinier-YOLO face detector which is a fixed-point network with 256x reduction in model size (246k Bytes) than Tiny-YOLO. We illustrate the promising performance of our face detector in terms of detection rate on Face Detection Data Set and Bencmark (FDDB) and qualitative results of detecting small faces of Wider Face dataset.

Integer Network for Cross Platform Graph Data Lossless Compression

1-Bit Compressive Diffusion in Networks Based on Bayesian Inference.

Fast Lossless Neural Compression with Integer-Only Discrete Flows

Unsupervised Network Quantization via Fixed-Point Factorization

Deep Network Quantization via Error Compensation

Efficient Integer-Arithmetic-Only Convolutional Neural Networks

Post-Training Quantization for Cross-Platform Learned Image Compression

Deep Neural Network Compression With Single and Multiple Level Quantization

Adaptive Layerwise Quantization for Deep Neural Network Compression

Accelerating Neural Network Inference by Overflow Aware Quantization

A Near Lossless Learned Image Coding Network Quantization Approach for Cross-Platform Inference.

Efficient Neural Image Decoding Via Fixed-Point Inference

A 4-Bit Integer-Only Neural Network Quantization Method Based on Shift Batch Normalization

Efficient Integer-Arithmetic-Only Convolutional Networks with Bounded ReLU

IFQ-Net: Integrated Fixed-point Quantization Networks for Embedded Vision

Quantization Networks

Self-Compressing Neural Networks

NeuZip: Memory-Efficient Training and Inference with Dynamic Compression of Neural Networks

A Universal Lossless Compression Method applicable to Sparse Graphs and Heavy-Tailed Sparse Graphs

Efficient Neural Compression with Inference-time Decoding

Deep Convolutional Neural Network Inference with Floating-point Weights and Fixed-point Activations