Abstract:We consider the problem of computing a sparse binary representation of an image. To be precise, given an image and an overcomplete, non-orthonormal basis, we aim to find a sparse binary vector indicating the minimal set of basis vectors that when added together best reconstruct the given input. We formulate this problem with an $L_2$ loss on the reconstruction error, and an $L_0$ (or, equivalently, an $L_1$) loss on the binary vector enforcing sparsity. This yields a so-called Quadratic Unconstrained Binary Optimization (QUBO) problem, whose solution is generally NP-hard to find. The contribution of this work is twofold. First, the method of unsupervised and unnormalized dictionary feature learning for a desired sparsity level to best match the data is presented. Second, the binary sparse coding problem is then solved on the Loihi 1 neuromorphic chip by the use of stochastic networks of neurons to traverse the non-convex energy landscape. The solutions are benchmarked against the classical heuristic simulated annealing. We demonstrate neuromorphic computing is suitable for sampling low energy solutions of binary sparse coding QUBO models, and although Loihi 1 is capable of sampling very sparse solutions of the QUBO models, there needs to be improvement in the implementation in order to be competitive with simulated annealing.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to use a neuromorphic processor based on spiking neural networks to solve the Quadratic Unconstrained Binary Optimization (QUBO) model of binary sparse coding. Specifically, given an image and an over - complete and non - orthogonal basis, the goal is to find a sparse binary vector that indicates the smallest set of basis vectors which, when added together, can best reconstruct the given input image. This problem can be expressed as the minimization problem of the following energy function: \[ E(x, a) = \min_a \left\{ \frac{1}{2} \| x - Da \|^2_2 + \lambda \| a \|_0 \right\} \] where: - $ x \in \mathbb{R}^m $ is the input signal (image). - $ D=\{ D_1,\ldots, D_n \} $ is an over - complete and non - orthogonal basis containing $ n > m $ vectors. - $ a \in B^n $ is a binary vector of length $ n $, $ B = \{0, 1\} $. - $ \| \cdot \|_2 $ represents the Euclidean norm. - $ \| \cdot \|_0 $ represents the number of non - zero elements in the vector. - $ \lambda > 0 $ is a parameter that controls the sparsity of the solution. Since this problem belongs to the 0 - 1 integer programming problem and its solutions usually have NP - hard complexity, it is very difficult to solve. The main contributions of the paper are: 1. Proposing an unsupervised and unnormalized dictionary feature learning method to match the optimal sparsity of the data. 2. Using the Loihi 1 neuromorphic chip to traverse the non - convex energy landscape through the neurons of the random network, thereby solving the QUBO problem of binary sparse coding. The paper also compares the performance of Loihi 1 and classical heuristic algorithms (such as simulated annealing) in solving the QUBO problem and shows the ability of neuromorphic computing to find sparse solutions with low energy consumption. Although Loihi 1 can find very sparse solutions, it still needs to be improved in some aspects to compete with simulated annealing.

Sampling binary sparse coding QUBO models using a spiking neuromorphic processor

Comparing Quantum Annealing and Spiking Neuromorphic Computing for Sampling Binary Sparse Coding QUBO Problems

Solving QUBO on the Loihi 2 Neuromorphic Processor

Binary Operations on Neuromorphic Hardware with Application to Linear Algebraic Operations and Stochastic Equations

Quadratic unconstrained binary optimization problem preprocessing: Theory and empirical analysis

Towards Arbitrary QUBO Optimization: Analysis of Classical and Quantum-Activated Feedforward Neural Networks

Ultra-low-power Image Classification on Neuromorphic Hardware

Quantum Sparse Coding and Decoding Based on Quantum Network

sBSNN: Stochastic-Bits Enabled Binary Spiking Neural Network with On-Chip Learning for Energy Efficient Neuromorphic Computing at the Edge

Implementing and Benchmarking the Locally Competitive Algorithm on the Loihi 2 Neuromorphic Processor

Classification and regression of spatio-temporal signals using NeuCube and its realization on SpiNNaker neuromorphic hardware

Stochastic Learning in Oxide Binary Synaptic Device for Neuromorphic Computing.

Quantum Image Denoising: A Framework via Boltzmann Machines, QUBO, and Quantum Annealing

Neuromorphic quadratic programming for efficient and scalable model predictive control

A Multilevel Approach For Solving Large-Scale QUBO Problems With Noisy Hybrid Quantum Approximate Optimization

Image recognition with an adiabatic quantum computer I. Mapping to quadratic unconstrained binary optimization

NBSSN: A Neuromorphic Binary Single-Spike Neural Network for Efficient Edge Intelligence.

Linearizing Binary Optimization Problems Using Variable Posets for Ising Machines

A binary integer programming-based method for qubit mapping in sparse architectures

Continuous black-box optimization with quantum annealing and random subspace coding

BitBrain and Sparse Binary Coincidence (SBC) memories: Fast, robust learning and inference for neuromorphic architectures