Abstract:Convolutional neural networks (CNNs) is becoming a critical role for deep learning-based computer vision applications. Through CNN we can extract meaningful information out of massive sensor data. Vision applications use this information to analyze the mainstream trends of the data and take immediate action based on these trends. However, CNN's energy consumption and bandwidth limitations make it difficult for CNN network systems to deploy in mobile systems with stringent energy limits. In this paper, we explore the simulation and the hardware method of optical convolution for low power image processing, which is inspired by the bio-image sensor Angle sensitive pixel (ASP). This optical computation method may be used to substitute the first convolution layer of the CNN network due to its energy-saving features and speed of light processing time. We adopt two-layer of customized transmission grating to perform this optical convolution computing. By generating the Talbot effect, the two-layer grating structure can perform optical convolution computation using Gabor wavelet filters, which will cause zero electrical power. We demonstrate both simulation and experiment results for optical convolution through our algorithm and prototype system, the convolution results can extract different meaningful information about the original image, which is very similar to edge filtering. This optical operation will hopefully be used to replace the first convolution layer of CNN since it can effectively reduce both the consumption of the energy power and the performing time.

Multi-layer Optical Convolutional Neural Network with Nonlinear Activation

Optical Convolution Based Computational Method for Low-Power Image Processing

Optical Convolutional Neural Network Based on an Amplitude Modulation Spatial Light Modulator and a 4-Level Phase Plate

Optical Nonlinear Activation Functions Based on MZI-Structure for Optical Neural Networks

Pre-sensor Computing with Compact Multilayer Optical Neural Network

Image sensing with multilayer nonlinear optical neural networks

Multi-layer optical Fourier neural network based on the convolution theorem

Programmable low-threshold optical nonlinear activation functions for photonic neural networks

Multilayer Optoelectronic Hybrid Convolutional Neural Network with an Optical 4F-System Recurrent Structure

Efficient Nonlinear Activation Function in Optical Neural Network

Optical-electronic Hybrid Fourier Convolutional Neural Network Based on Super-Pixel Complex-Valued Modulation

Compressed Meta-Optical Encoder for Image Classification

11 TOPS photonic convolutional accelerator for optical neural networks

All-Optical Nonlinear Activators Using Silicon-Based Waveguides and Microring Resonators for Photonic Neural Networks

11 TeraFLOPs per second photonic convolutional accelerator for deep learning optical neural networks

On-chip 4F-System Based on Concave Mirrors for Optical Neural Networks

High-accuracy optical convolution unit architecture for convolutional neural networks by cascaded acousto-optical modulator arrays

Lensless Opto-Electronic Neural Network with Quantum Dot Nonlinear Activation

Photonic Neural Activation Function Based on High-Speed Electro-Absorption Modulated Laser

Lensless opto-electronic neural network with quantum dots nonlinear activation

Optical convolutional neural network with atomic nonlinearity