Abstract:As deep neural networks (DNNs) revolutionize machine learning, energy consumption and throughput are emerging as fundamental limitations of CMOS electronics. This has motivated a search for new hardware architectures optimized for artificial intelligence, such as electronic systolic arrays, memristor crossbar arrays, and optical accelerators. Optical systems can perform linear matrix operations at exceptionally high rate and efficiency, motivating recent demonstrations of low latency linear algebra and optical energy consumption below a photon per multiply-accumulate operation. However, demonstrating systems that co-integrate both linear and nonlinear processing units in a single chip remains a central challenge. Here we introduce such a system in a scalable photonic integrated circuit (PIC), enabled by several key advances: (i) high-bandwidth and low-power programmable nonlinear optical function units (NOFUs); (ii) coherent matrix multiplication units (CMXUs); and (iii) in situ training with optical acceleration. We experimentally demonstrate this fully-integrated coherent optical neural network (FICONN) architecture for a 3-layer DNN comprising 12 NOFUs and three CMXUs operating in the telecom C-band. Using in situ training on a vowel classification task, the FICONN achieves 92.7% accuracy on a test set, which is identical to the accuracy obtained on a digital computer with the same number of weights. This work lends experimental evidence to theoretical proposals for in situ training, unlocking orders of magnitude improvements in the throughput of training data. Moreover, the FICONN opens the path to inference at nanosecond latency and femtojoule per operation energy efficiency.

Nonlinear All-Optical Diffractive Deep Neural Network with 10.6 Μm Wavelength for Image Classification

Optical Analog Computing Devices Designed by Deep Neural Network

Programmable low-threshold optical nonlinear activation functions for photonic neural networks

Electrically Programmable Phase-Change Photonic Memory for Optical Neural Networks with Nanoseconds in Situ Training Capability

On-chip 4F-System Based on Concave Mirrors for Optical Neural Networks

Modeling Method of Miniaturized Nonlinear All-Optical Diffraction Deep Neural Network Based on 10.6 Μm Wavelength

1-bit Quantized On-chip Hybrid Diffraction Neural Network Enabled by Authentic All-optical Fully-connected Architecture

Space-efficient optical computing with an integrated chip diffractive neural network

Multimodal deep learning using on-chip diffractive optics with in situ training capability

Photonic machine learning with on-chip diffractive optics

A Simple Nonlinear Classifier Using a Multimode Optical Chip

An on-chip photonic deep neural network for image classification

A surface-normal photodetector as nonlinear activation function in diffractive optical neural networks

Simulating an Integrated Photonic Image Classifier for Diffractive Neural Networks

High speed and reconfigurable optronic neural network with digital nonlinear activation

Single chip photonic deep neural network with accelerated training

Integrated Diffractive Optical Neural Network with Space-Time Interleaving

Ti$_3$C$_2$T$_x$ MXene Enabled All-Optical Nonlinear Activation Function for On-Chip Photonic Deep Neural Networks

Ultra‐Compact and NonVolatile Nanophotonic Neural Networks

A Compact Butterfly-Style Silicon Photonic–Electronic Neural Chip for Hardware-Efficient Deep Learning

Optical Neural Network with Complementary Decomposition to Overcome the Phase Insensitive Constrains