Abstract:The explosive growth of computation and energy cost of artificial intelligence has spurred strong interests in new computing modalities as potential alternatives to conventional electronic processors. Photonic processors that execute operations using photons instead of electrons, have promised to enable optical neural networks with ultra-low latency and power consumption. However, existing optical neural networks, limited by the underlying network designs, have achieved image recognition accuracy far below that of state-of-the-art electronic neural networks. In this work, we close this gap by embedding massively parallelized optical computation into flat camera optics that perform neural network computation during the capture, before recording an image on the sensor. Specifically, we harness large kernels and propose a large-kernel spatially-varying convolutional neural network learned via low-dimensional reparameterization techniques. We experimentally instantiate the network with a flat meta-optical system that encompasses an array of nanophotonic structures designed to induce angle-dependent responses. Combined with an extremely lightweight electronic backend with approximately 2K parameters we demonstrate a reconfigurable nanophotonic neural network reaches 72.76\% blind test classification accuracy on CIFAR-10 dataset, and, as such, the first time, an optical neural network outperforms the first modern digital neural network -- AlexNet (72.64\%) with 57M parameters, bringing optical neural network into modern deep learning era.

Real-time target recognition with all-optical neural networks for ghost imaging

Ghost Imaging with N-order Speckle Light

Foveated Ghost Imaging Based on Deep Learning

Ghost Imaging Based on Y-net: a Dynamic Coding and Decoding Approach.

Memory-less scattering imaging with ultrafast convolutional optical neural networks

Ghost imaging based on Y-net: a dynamic coding and conjugate-decoding approach

Real-time ghost image characterization for panoramic annular lenses

Spatially Varying Nanophotonic Neural Networks

Advances and Challenges of Optical Neural Networks

Deep-learned speckle pattern and its application to ghost imaging

Dynamic-excitation-based Steady-State Non-Line-of-sight Imaging Via Multi-Branch Convolutional Neural Network

Direct Object Recognition Without Line-of-Sight Using Optical Coherence

Optimizing Ghost Imaging via Analysis and Design of Speckle Patterns

Adaptive real-time single-pixel imaging

Wide-field scanning ghost imaging based on a local binary pattern and untrained neural network

Generative-Adversarial-Networks-based Ghost Recognition

Optimization of retina-like illumination patterns in ghost imaging

High-precision, real-time wavefront sensing via sparse diffractive deep neural networks

Imaging through a Scattering Medium under Different Intensities of Ambient Light Interference

Combined optimization ghost imaging based on random speckle field

Neuro-inspired optical sensor array for high-accuracy static image recognition and dynamic trace extraction