Abstract:With the proliferation of ultrahigh-speed mobile networks and internet-connected devices, along with the rise of artificial intelligence (AI)<a href="/articles/s41586-020-03070-1#ref-CR1">1</a>, the world is generating exponentially increasing amounts of data that need to be processed in a fast and efficient way. Highly parallelized, fast and scalable hardware is therefore becoming progressively more important<a href="/articles/s41586-020-03070-1#ref-CR2">2</a>. Here we demonstrate a computationally specific integrated photonic hardware accelerator (tensor core) that is capable of operating at speeds of trillions of multiply-accumulate operations per second (1012 MAC operations per second or tera-MACs per second). The tensor core can be considered as the optical analogue of an application-specific integrated circuit (ASIC). It achieves parallelized photonic in-memory computing using phase-change-material memory arrays and photonic chip-based optical frequency combs (soliton microcombs<a href="/articles/s41586-020-03070-1#ref-CR3">3</a>). The computation is reduced to measuring the optical transmission of reconfigurable and non-resonant passive components and can operate at a bandwidth exceeding 14 gigahertz, limited only by the speed of the modulators and photodetectors. Given recent advances in hybrid integration of soliton microcombs at microwave line rates<a href="#ref-CR3">3</a>,<a href="#ref-CR4">4</a>,<a href="/articles/s41586-020-03070-1#ref-CR5">5</a>, ultralow-loss silicon nitride waveguides<a href="/articles/s41586-020-03070-1#ref-CR6">6</a>,<a href="/articles/s41586-020-03070-1#ref-CR7">7</a>, and high-speed on-chip detectors and modulators, our approach provides a path towards full complementary metal–oxide–semiconductor (CMOS) wafer-scale integration of the photonic tensor core. Although we focus on convolutional processing, more generally our results indicate the potential of integrated photonics for parallel, fast, and efficient computational hardware in data-heavy AI applications such as autonomous driving, live video processing, and next-generation cloud computing services.

Parallel Photonic Convolutional Processing On-Chip with Cross-Connect Architecture and Cyclic AWGs

Optical Convolution Based Computational Method for Low-Power Image Processing

Direct Optical Convolution Computing Based on Arrayed Waveguide Grating Router

Photonic Convolution Neural Network Based on Interleaved Time-Wavelength Modulation

On-chip 4F-System Based on Concave Mirrors for Optical Neural Networks

Accelerating Convolutional Processing by Harnessing Channel Shifts in Arrayed Waveguide Gratings

Diffraction‐Driven Parallel Convolution Processing with Integrated Photonics

A Winograd-based Integrated Photonics Accelerator for Convolutional Neural Networks

Parallel convolutional processing using an integrated photonic tensor core

Parallelizing Convolutional Neural Networks On Intel (R) Many Integrated Core Architecture

Non-volatile and Ultra-Fast Photonic Vector Accelerator with Optical Phase Change Materials and Integrated Microcomb

Compact optical convolution processing unit based on multimode interference

Performance evaluation of an integrated photonic convolutional neural network based on delay buffering and wavelength division multiplexing

An Energy-Efficient Convolutional Neural Network Processor Architecture Based on a Systolic Array

11 Tera-OPs/s Photonic Convolutional Accelerator and Deep Optical Neural Network Based on an Integrated Kerr Soliton Crystal Microcomb

A Parallel Photonic Chip for Nano-Second End-to-end Image Processing, Transmission, and Reconstruction

Optical Convolutional Neural Networks -- Combining Silicon Photonics and Fourier Optics for Computer Vision

Microcomb-based integrated photonic processing unit

High‐Resolution Optical Convolutional Neural Networks Using Phase‐Change Material‐Based Microring Hybrid Waveguides

Towards On-Chip Optical FFTs for Convolutional Neural Networks

Single chip photonic deep neural network with accelerated training