Abstract:The recent explosive compute growth, mainly fueled by the boost of artificial intelligence (AI) and deep neural networks (DNNs), is currently instigating the demand for a novel computing paradigm that can overcome the insurmountable barriers imposed by conventional electronic computing architectures. Photonic neural networks (PNNs) implemented on silicon integration platforms stand out as a promising candidate to endow neural network (NN) hardware, offering the potential for energy efficient and ultra-fast computations through the utilization of the unique primitives of photonics, i.e., energy efficiency, THz bandwidth, and low-latency. Thus far, several demonstrations have revealed the huge potential of PNNs in performing both linear and non-linear NN operations at unparalleled speed and energy consumption metrics. Transforming this potential into a tangible reality for deep learning (DL) applications requires, however, a deep understanding of the basic PNN principles, requirements, and challenges across all constituent architectural, technological, and training aspects. In this Tutorial, we, initially, review the principles of DNNs along with their fundamental building blocks, analyzing also the key mathematical operations needed for their computation in photonic hardware. Then, we investigate, through an intuitive mathematical analysis, the interdependence of bit precision and energy efficiency in analog photonic circuitry, discussing the opportunities and challenges of PNNs. Followingly, a performance overview of PNN architectures, weight technologies, and activation functions is presented, summarizing their impact in speed, scalability, and power consumption. Finally, we provide a holistic overview of the optics-informed NN training framework that incorporates the physical properties of photonic building blocks into the training process in order to improve the NN classification accuracy and effectively elevate neuromorphic photonic hardware into high-performance DL computational settings.

ADEPT: Automatic Differentiable DEsign of Photonic Tensor Cores

ADEPT-Z: Zero-Shot Automated Circuit Topology Search for Pareto-Optimal Photonic Tensor Cores

A manufacture-friendly design framework for optical neural networks

Easily Scalable Photonic Tensor Core Based on Tunable Units with Single Internal Phase Shifters

120 GOPS Photonic Tensor Core in Thin-film Lithium Niobate for Inference and in-situ Training

Photonic tensor cores for machine learning

M3ICRO: Machine Learning-Enabled Compact Photonic Tensor Core based on PRogrammable Multi-Operand Multimode Interference

Clements-Enhanced Complex-Valued Coherent Mesh with Balanced Detection Units for Photonic Neural Networks

Photonic tensor processing unit with single dataflow and programmable high-precision weighting control

TeMPO: Efficient Time-Multiplexed Dynamic Photonic Tensor Core for Edge AI with Compact Slow-Light Electro-Optic Modulator

Sophisticated Deep Learning with On-Chip Optical Diffractive Tensor Processing

DOCTOR: Dynamic On-Chip Temporal Variation Remediation Toward Self-Corrected Photonic Tensor Accelerators

Multimodal deep learning using on-chip diffractive optics with in situ training capability

POViT: Vision Transformer for Multi-Objective Design and Characterization of Photonic Crystal Nanocavities

PACE: Pacing Operator Learning to Accurate Optical Field Simulation for Complicated Photonic Devices

Integrated multi-operand optical neurons for scalable and hardware-efficient deep learning

Real-Time FJ/MAC PDE Solvers via Tensorized, Back-Propagation-Free Optical PINN Training

NeurOLight: A Physics-Agnostic Neural Operator Enabling Parametric Photonic Device Simulation

Photonic neural networks and optics-informed deep learning fundamentals

PIC2O-Sim: A Physics-Inspired Causality-Aware Dynamic Convolutional Neural Operator for Ultra-Fast Photonic Device FDTD Simulation