Abstract:In recent years, Deep Convolutional Neural Networks (DCNNs) have outreached the performance of classical algorithms for image restoration tasks. However, most of these methods are not suited for computational efficiency. In this work, we investigate Spiking Neural Networks (SNNs) for the specific and uncovered case of image denoising, with the goal of reaching the performance of conventional DCNN while reducing the computational cost. This task is challenging for two reasons. First, as denoising is a regression task, the network has to predict a continuous value (i.e., the noise amplitude) for each pixel of the image, with high precision. Moreover, state of the art results have been obtained with deep networks that are notably difficult to train in the spiking domain. To overcome these issues, we propose a formal analysis of the information conversion processing carried out by the Integrate and Fire (IF) spiking neurons and we formalize the trade-off between conversion error and activation sparsity in SNNs. We then propose, for the first time, an image denoising solution based on SNNs. The SNN networks are trained directly in the spike domain using surrogate gradient learning and backpropagation through time. Experimental results show that the proposed SNN provides a level of performance close to the state of the art with CNN based solutions. Specifically, our SNN achieves 30.18 dB of signal-to-noise ratio on the Set12 dataset, which is only 0.25 dB below the performance of the equivalent DCNN. Moreover we show that this performance can be achieved with low latency, i.e., using few timesteps, and with a significant level of sparsity. Finally, we analyze the energy consumption for different network latencies and network sizes. We show that the energy consumption of SNNs increases with longer latencies, making them more energy efficient compared to CNNs only for very small inference latencies. However, we also show that by increasing the network size, SNNs can provide competitive denoising performance while reducing the energy consumption by 20%.

SPN-CNN: Boosting Sensor-Based Source Camera Attribution With Deep Learning

Patch-Based Sensor Pattern Noise For Camera Source Identification

Source Camera Identification with Multi-Scale Feature Fusion Network

A Deep Learning Framework for Image Authentication: An Automatic Source Camera Identification Deep-Net

RSCNN: A CNN-Based Method to Enhance Low-Light Remote-Sensing Images

Generalizing Source Camera Identification Based on Integral Image Optimization and Constrained Neural Network

A Context-Adaptive SPN Predictor for Trustworthy Source Camera Identification

Deep CovDenseSNN: A Hierarchical Event-Driven Dynamic Framework with Spiking Neurons in Noisy Environment

CNN-based fast source device identification

Robust Source Camera Identification Against Adversarial Attacks

CSNN: an Augmented Spiking Based Framework with Perceptron-Inception

Source Camera Identification Based on an Adaptive Dual-Branch Fusion Residual Network

Efficient Source Camera Identification with Diversity-Enhanced Patch Selection and Deep Residual Prediction

Real-time Noise Source Estimation of a Camera System from an Image and Metadata

Semi-Blindly Enhancing Extremely Noisy Videos With Recurrent Spatio-Temporal Large-Span Network

DeNet: Scalable Real-Time Object Detection with Directed Sparse Sampling

SPIDEN: deep Spiking Neural Networks for efficient image denoising

Deep Burst Denoising

Denoising of Scintillation Camera Images Using a Deep Convolutional Neural Network: A Monte Carlo Simulation Approach

Deep joint demosaicking and denoising

Self-Supervised Mutual Learning for Dynamic Scene Reconstruction of Spiking Camera.