Abstract:Ever-growing deep-learning technologies are making revolutionary changes for modern life.However,conventional computing architectures are designed to process sequential and digital programs but are burdened with performing massive parallel and adaptive deep-learning applications.Photonic integrated circuits provide an efficient approach to mitigate bandwidth limitations and the power-wall brought on by its electronic counterparts,showing great potential in ultrafast and energy-free high-performance computation.Here,we propose an optical computing ar-chitecture enabled by on-chip diffraction to implement convolutional acceleration,termed"optical convolution unit"(OCU).We demonstrate that any real-valued convolution kernels can be exploited by the OCU with a promi-nent computational throughput boosting via the concept of structral reparameterization.With the OCU as the fundamental unit,we build an optical convolutional neural network(oCNN)to implement two popular deep learn-ing tasks:classification and regression.For classification,Fashion Modified National Institute of Standards and Technology(Fashion-MNIST)and Canadian Institute for Advanced Research(CIFAR-4)data sets are tested with accuracies of 91.63％and 86.25％,respectively.For regression,we build an optical denoising convolutional neural network to handle Gaussian noise in gray-scale images with noise level σ=10,15,and 20,resulting in clean images with an average peak signal-to-noise ratio(PSNR)of 31.70,29.39,and 27.72 dB,respectively.The proposed OCU presents remarkable performance of low energy consumption and high information density due to its fully passive nature and compact footprint,providing a parallel while lightweight solution for future compute-in-memory architecture to handle high dimensional tensors in deep learning.

High-accuracy data readout in multi-dimensional optical data storage using convolutional neural networks

Error‐Free Long‐Lifespan Optical Storage Enhanced by Deep Learning

11 TeraFLOPs per second photonic convolutional accelerator for deep learning optical neural networks

Pushing the limits of optical information storage using deep learning

High-accuracy optical convolution unit architecture for convolutional neural networks by cascaded acousto-optical modulator arrays

Sophisticated Deep Learning with On-Chip Optical Diffractive Tensor Processing

Deep Learning with Passive Optical Nonlinear Mapping

11 TOPS photonic convolutional accelerator for optical neural networks

An optical neural network using less than 1 photon per multiplication

Multi-layer optical Fourier neural network based on the convolution theorem

Optical Convolutional Neural Network with WDM-Based Optical Patching and Microring Weighting Banks

Ultracompact Deep Neural Network for Ultrafast Optical Property Extraction in Spatial Frequency Domain Imaging (SFDI)

Decoding of Compressive Data Pages for Optical Data Storage Utilizing FFDNet

Memory-less scattering imaging with ultrafast convolutional optical neural networks

Femtosecond Laser Induced Amorphization of Quantum Dots and Application in Three‐dimensional Optical Data Storage

Multilayer Optoelectronic Hybrid Convolutional Neural Network with an Optical 4F-System Recurrent Structure

Optical neuromorphic processing at Tera-OP/s speeds based on Kerr soliton crystal microcombs

Deep learning with coherent VCSEL neural networks

All-Optical Computing Based on Convolutional Neural Networks

Optical Fourier convolutional neural network with high efficiency in image classification

High-energy-efficiency integrated photonic convolutional neural networks