Abstract:Abstract Endowed with the superior computing speed and energy efficiency, optical neural networks (ONNs) have attracted ever-growing attention in recent years. Existing optical computing architectures are mainly single-channel due to the lack of advanced optical connection and interaction operators, solving simple tasks such as hand-written digit classification, saliency detection, etc. The limited computing capacity and scalability of single-channel ONNs restrict the optical implementation of advanced machine vision. Herein, we develop Monet: a multichannel optical neural network architecture for a universal multiple-input multiple-channel optical computing based on a novel projection-interference-prediction framework where the inter- and intra- channel connections are mapped to optical interference and diffraction. In our Monet, optical interference patterns are generated by projecting and interfering the multichannel inputs in a shared domain. These patterns encoding the correspondences together with feature embeddings are iteratively produced through the projection-interference process to predict the final output optically. For the first time, Monet validates that multichannel processing properties can be optically implemented with high-efficiency, enabling real-world intelligent multichannel-processing tasks solved via optical computing, including 3D/motion detections. Extensive experiments on different scenarios demonstrate the effectiveness of Monet in handling advanced machine vision tasks with comparative accuracy as the electronic counterparts yet achieving a ten-fold improvement in computing efficiency. For intelligent computing, the trends of dealing with real-world advanced tasks are irreversible. Breaking the capacity and scalability limitations of single-channel ONN and further exploring the multichannel processing potential of wave optics, we anticipate that the proposed technique will accelerate the development of more powerful optical AI as critical support for modern advanced machine vision.

Compute-first optical detection for noise-resilient visual perception

Optical Convolution Based Computational Method for Low-Power Image Processing

Countering Variations and Thermal Effects for Accurate Optical Neural Networks.

Solving Computer Vision Tasks with Diffractive Neural Networks

Resource-Saving and High-Robustness Image Sensing Based on Binary Optical Computing

Single-image-based Nonuniformity Correction of Uncooled Long-Wave Infrared Detectors: a Deep-Learning Approach.

Spatially Varying Nanophotonic Neural Networks

Noise-resilient single-pixel compressive sensing with single photon counting

A multichannel optical computing architecture for advanced machine vision

Scattering-induced entropy boost for highly-compressed optical sensing and encryption

Parallel Photoelectron Storage and Visual Preprocessing Based on Nanowire Defect Engineering for Image Degradation

A Highly Parallelized Signal and Image Recognition Method Based on Spatial Optics

A Self-Attention Armed Optronic Transformer in Imaging Through Scattering Media

Task-Oriented Real-time Visual Inference for IoVT Systems: A Co-design Framework of Neural Networks and Edge Deployment

Deep Learning with Passive Optical Nonlinear Mapping

Spectral Convolutional Neural Network Chip for In-sensor Edge Computing of Incoherent Natural Light

Retinomorphic Machine Vision in a Network Laser

An optical neural network using less than 1 photon per multiplication

Double-Attention YOLO: Vision Transformer Model Based on Image Processing Technology in Complex Environment of Transmission Line Connection Fittings and Rust Detection

Digital-analog hybrid matrix multiplication processor for optical neural networks

Progress in Bioinspired Photodetectors Design for Visual Information Processing