Abstract:The relentless growth of Artificial Intelligence (AI) workloads has fueled the drive towards non-Von Neuman architectures and custom computing hardware. Neuromorphic photonic engines aspire to synergize the low-power and high-bandwidth credentials of light-based deployments with novel architectures, towards surpassing the computing performance of their electronic counterparts. In this paper, we review recent progress in integrated photonic neuromorphic architectures and analyze the architectural and photonic hardware-based factors that limit their performance. Subsequently, we present our approach towards transforming silicon coherent neuromorphic layouts into high-speed and high-accuracy Deep Learning (DL) engines by combining robust architectures with hardware-aware DL training. Circuit robustness is ensured through a crossbar layout that circumvents insertion loss and fidelity constraints of state-of-the-art linear optical designs. Concurrently, we employ DL training models adapted to the underlying photonic hardware, incorporating noise- and bandwidth-limitations together with the supported activation function directly into Neural Network (NN) training. We validate experimentally the high-speed and high-accuracy advantages of hardware-aware DL models when combined with robust architectures through a SiPho prototype implementing a single column of a 4:4 photonic crossbar. This was utilized as the pen-ultimate hidden layer of a NN, revealing up to 5.93 accuracy improvement at 5GMACsecaxon when noise-aware training is enforced and allowing accuracies of 99.15 and 79.8 for the MNIST and CIFAR-10 classification tasks. Channel-aware training was then demonstrated by integrating the frequency response of the photonic hardware in NN training, with its experimental validation with the MNIST dataset revealing an accuracy increase of 12.93 at a record-high rate of 25GMACsecaxon.

Deep Learning with Coherent Nanophotonic Circuits

Deep learning with coherent VCSEL neural networks

Noise-resilient and high-speed deep learning with coherent silicon photonics

Deep Learning Enabled Nanophotonics

Deep Learning with Photonic Neural Cellular Automata

Single chip photonic deep neural network with accelerated training

Neuromorphic Silicon Photonics and Hardware-Aware Deep Learning for High-Speed Inference

Experimentally realized in situ backpropagation for deep learning in photonic neural networks

Convolutional Neural Networks with Coherent Nanophotonic Circuits

Low-power scalable multilayer optoelectronic neural networks enabled with incoherent light

Large-Scale Optical Neural Networks based on Photoelectric Multiplication

Spatially Varying Nanophotonic Neural Networks

Compatible Learning for Deep Photonic Neural Network

High-performance Optoelectronics for Integrated Photonic Neural Networks

Microring-based programmable coherent optical neural networks

Silicon Photonic Architecture for Training Deep Neural Networks with Direct Feedback Alignment

Deep Learning for Photonic Design and Analysis: Principles and Applications

Deep Learning for Design and Retrieval of Nano-photonic Structures

11 TeraFLOPs per second photonic convolutional accelerator for deep learning optical neural networks

InP Photonic Integrated Multi-Layer Neural Networks: Architecture and Performance Analysis