Abstract:The explosive growth of computation and energy cost of artificial intelligence has spurred strong interests in new computing modalities as potential alternatives to conventional electronic processors. Photonic processors that execute operations using photons instead of electrons, have promised to enable optical neural networks with ultra-low latency and power consumption. However, existing optical neural networks, limited by the underlying network designs, have achieved image recognition accuracy far below that of state-of-the-art electronic neural networks. In this work, we close this gap by embedding massively parallelized optical computation into flat camera optics that perform neural network computation during the capture, before recording an image on the sensor. Specifically, we harness large kernels and propose a large-kernel spatially-varying convolutional neural network learned via low-dimensional reparameterization techniques. We experimentally instantiate the network with a flat meta-optical system that encompasses an array of nanophotonic structures designed to induce angle-dependent responses. Combined with an extremely lightweight electronic backend with approximately 2K parameters we demonstrate a reconfigurable nanophotonic neural network reaches 72.76\% blind test classification accuracy on CIFAR-10 dataset, and, as such, the first time, an optical neural network outperforms the first modern digital neural network -- AlexNet (72.64\%) with 57M parameters, bringing optical neural network into modern deep learning era.

High speed and reconfigurable optronic neural network with digital nonlinear activation

Convolutional Free-space Optical Neural Networks for Image Recognition

Optical Analog Computing Devices Designed by Deep Neural Network

Electrically Programmable Phase-Change Photonic Memory for Optical Neural Networks with Nanoseconds in Situ Training Capability

A Review of Optical Neural Networks

Programmable low-threshold optical nonlinear activation functions for photonic neural networks

On-chip 4F-System Based on Concave Mirrors for Optical Neural Networks

Spatially Varying Nanophotonic Neural Networks

All-Optical Nonlinear Activators Using Silicon-Based Waveguides and Microring Resonators for Photonic Neural Networks

Reconfigurable All-Optical Integrated Nonlinear Activator with Switchable Response Functions for Photonic Neural Networks

Photonic Neural Networks and Its Applications

A manufacture-friendly design framework for optical neural networks

End-to-End Optimization for a Compact Optical Neural Network Based on Nanostructured 2 x 2 Optical Processors

Low-power scalable multilayer optoelectronic neural networks enabled with incoherent light

Single-shot optical neural network

Space-efficient optical computing with an integrated chip diffractive neural network

An optical neural chip for implementing complex-valued neural network

Multi-layer optical Fourier neural network based on the convolution theorem

Multilayer Optoelectronic Hybrid Convolutional Neural Network with an Optical 4F-System Recurrent Structure

Lensless Opto-Electronic Neural Network with Quantum Dot Nonlinear Activation

Lensless opto-electronic neural network with quantum dots nonlinear activation