Abstract:We propose two efficient approximations to standard convolutional neural networks: Binary-Weight-Networks and XNOR-Networks. In Binary-Weight-Networks, the filters are approximated with binary values resulting in 32×\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\times $$\end{document} memory saving. In XNOR-Networks, both the filters and the input to convolutional layers are binary. XNOR-Networks approximate convolutions using primarily binary operations. This results in 58×\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\times $$\end{document} faster convolutional operations (in terms of number of the high precision operations) and 32×\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\times $$\end{document} memory savings. XNOR-Nets offer the possibility of running state-of-the-art networks on CPUs (rather than GPUs) in real-time. Our binary networks are simple, accurate, efficient, and work on challenging visual tasks. We evaluate our approach on the ImageNet classification task. The classification accuracy with a Binary-Weight-Network version of AlexNet is the same as the full-precision AlexNet. We compare our method with recent network binarization methods, BinaryConnect and BinaryNets, and outperform these methods by large margins on ImageNet, more than 16%\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$16\,\%$$\end{document} in top-1 accuracy. Our code is available at: http://allenai.org/plato/xnornet.

The Binary Output Units of Neural Network

Hyperbolic Binary Neural Network

Deep Neural Network Optimization Based on Binary Method for Handling Multi-Class Problems

A Multiclass Kernel Perceptron Algorithm

Learning Binary Codes and Binary Weights for Efficient Classification

Clustering of solutions in the symmetric binary perceptron

BCMLP: Binary-connected multilayer perceptrons

Morphological Perceptrons: Geometry and Training Algorithms

A Nonlinear Perceptron Algorithm Based on Kernel Functions

Training Multi-Layer Binary Neural Networks With Local Binary Error Signals

Binarizing by Classification: is Soft Function Really Necessary?

Binary Linear Compression for Multi-label Classification.

BiPer: Binary Neural Networks using a Periodic Function

A Geometric Theory for Binary Classification of Finite Datasets by DNNs with Relu Activations

Class Binarization to NeuroEvolution for Multiclass Classification

Discrepancy Algorithms for the Binary Perceptron

Perceptron Training Algorithms Designed Using Discrete-Time Control Liapunov Functions

An adiabatic method to train binarized artificial neural networks

Typical and atypical solutions in non-convex neural networks with discrete and continuous weights

XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks

TP-ADMM: An Efficient Two-Stage Framework for Training Binary Neural Networks