Abstract:As the ultra-high frequency nature of lightwave corresponds to great potential in wideband signal processing, nextgeneration electronic information systems of surveillance, radar and communications is promised with photonic signal processing systems. However, the sophisticated photonic systems suffer from various hardware defects, which severely limit the performance of signal processing. By introducing the emerging deep learning technology into the photonic system, the hardware defects can be recovered by the trained neural networks. Using different modified neural networks, we have demonstrated high-accuracy photonic analog-to-digital converters, Brillouin instantaneous frequency measurement, and high-fidelity photonic radar receivers. The demonstrated systems with simple configurations can outperform the conventional photonic processing system with complex configurations. Note that the adoption of neural networks may cause additional time delay to the signal flow. Photonic neural network accelerators (PNNs) become a promising solution to realize real-time signal processing. We propose and experimentally demonstrate several system architectures of photonic convolutional neural networks. The photonic dot product unit architecture implements the basic operation in convolution neural networks. And an optical patching scheme is demonstrated to enhances the power efficiency of the input ports in PNNs. Performance evaluations show that the proposed PNN architectures possess potential advantages of energy efficiency and computational power. We believe that, by combining the technical advantages of photonic signal processing and PNN acceleration, intelligent photonic signal processing systems with high-performance real-time wideband signal processing capabilities can be realized. Moreover, the large-scale photonic integration technology promises the fabrication of such hybrid systems in the future.

Photonic convolutional neural network with robustness against wavelength deviations

Color Router-based Long-wave Infrared Multispectral Imaging

Photonic convolution accelerator based on a hybrid integrated multi-wavelength laser array by photonic wire bonding for real-time image classification

Direct Optical Convolution Computing Based on Arrayed Waveguide Grating Router

Performance evaluation of an integrated photonic convolutional neural network based on delay buffering and wavelength division multiplexing

Photonic Convolution Neural Network Based on Interleaved Time-Wavelength Modulation

Compact optical convolution processing unit based on multimode interference

Experimentally realized in situ backpropagation for deep learning in photonic neural networks

11 Tera-OPs/s Photonic Convolutional Accelerator and Deep Optical Neural Network Based on an Integrated Kerr Soliton Crystal Microcomb

Inverse design of convolutional neural networks via nanophotonic kernels

Investigations on intelligent photonic signal processing technology

11 TOPS photonic convolutional accelerator for optical neural networks

Spatially Varying Nanophotonic Neural Networks

High-energy-efficiency integrated photonic convolutional neural networks

Diffraction‐Driven Parallel Convolution Processing with Integrated Photonics

Accelerating Convolutional Processing by Harnessing Channel Shifts in Arrayed Waveguide Gratings

11 TeraFLOPs per second photonic convolutional accelerator for deep learning optical neural networks

Multi-dimensional optical neural network

Design and testing of silicon photonic 4F system for convolutional neural networks

Compact, efficient, and scalable nanobeam core for photonic matrix-vector multiplication

Experimentally Realizing Convolution Processing in the Photonic Synthetic Frequency Dimension