Abstract:As deep neural networks (DNNs) revolutionize machine learning, energy consumption and throughput are emerging as fundamental limitations of CMOS electronics. This has motivated a search for new hardware architectures optimized for artificial intelligence, such as electronic systolic arrays, memristor crossbar arrays, and optical accelerators. Optical systems can perform linear matrix operations at exceptionally high rate and efficiency, motivating recent demonstrations of low latency linear algebra and optical energy consumption below a photon per multiply-accumulate operation. However, demonstrating systems that co-integrate both linear and nonlinear processing units in a single chip remains a central challenge. Here we introduce such a system in a scalable photonic integrated circuit (PIC), enabled by several key advances: (i) high-bandwidth and low-power programmable nonlinear optical function units (NOFUs); (ii) coherent matrix multiplication units (CMXUs); and (iii) in situ training with optical acceleration. We experimentally demonstrate this fully-integrated coherent optical neural network (FICONN) architecture for a 3-layer DNN comprising 12 NOFUs and three CMXUs operating in the telecom C-band. Using in situ training on a vowel classification task, the FICONN achieves 92.7% accuracy on a test set, which is identical to the accuracy obtained on a digital computer with the same number of weights. This work lends experimental evidence to theoretical proposals for in situ training, unlocking orders of magnitude improvements in the throughput of training data. Moreover, the FICONN opens the path to inference at nanosecond latency and femtojoule per operation energy efficiency.

Design and testing of silicon photonic 4F system for convolutional neural networks

On-chip 4F-System Based on Concave Mirrors for Optical Neural Networks

Optical Convolutional Neural Networks -- Combining Silicon Photonics and Fourier Optics for Computer Vision

Towards On-Chip Optical FFTs for Convolutional Neural Networks

11 Tera-OPs/s Photonic Convolutional Accelerator and Deep Optical Neural Network Based on an Integrated Kerr Soliton Crystal Microcomb

Optical-electronic Hybrid Fourier Convolutional Neural Network Based on Super-Pixel Complex-Valued Modulation

11 TeraFLOPs per second photonic convolutional accelerator for deep learning optical neural networks

A Winograd-based Integrated Photonics Accelerator for Convolutional Neural Networks

Compact optical convolution processing unit based on multimode interference

Integrated Photonic FFT for Optical Convolutions towards Efficient and High-Speed Neural Networks

Performance evaluation of an integrated photonic convolutional neural network based on delay buffering and wavelength division multiplexing

Physical Layer-Aware Digital-Analog Co-Design for Photonic Convolution Neural Network

11 TOPS photonic convolutional accelerator for optical neural networks

End-to-End Optimization for a Compact Optical Neural Network Based on Nanostructured 2 x 2 Optical Processors

Highly efficient convolution computing architecture based on silicon photonic Fano resonance devices

Single chip photonic deep neural network with accelerated training

Silicon Photonic Architecture for Training Deep Neural Networks with Direct Feedback Alignment

Inverse Design of High-Dimensional Nanostructured 2×2 Optical Processors Based On Deep Convolutional Neural Networks

Photonic Integrated Neuro-Synaptic Core for Convolutional Spiking Neural Network

All-Optical Computing Based on Convolutional Neural Networks

A Convolution Neural Network Implemented by Three 3 × 3 Photonic Integrated Reconfigurable Linear Processors