Abstract:The high demand for machine intelligence of doubling every three months is driving novel hardware solutions beyond charging of electrical wires given a resurrection to application specific integrated circuit (ASIC)-based accelerators. These innovations include photonic ASICs (P-ASIC) due to prospects of performing optical linear (and also nonlinear) operations, such as multiply-accumulate for vector matrix multiplications or convolutions, without iterative architectures. Such photonic linear algebra enables picosecond delay when photonic integrated circuits are utilized, via on-the-fly mathematics. However, the neurons full function includes providing a nonlinear activation function, knowns as thresholding, to enable decision making on inferred data. Many P-ASIC solutions performing this nonlinearity in the electronic domain, which brings challenges in terms of data throughput and delay, thus breaking the optical link and introducing increased system complexity via domain crossings. This work follows the notion of utilizing enhanced light-matter-interactions to provide efficient, compact, and engineerable electro-optic neuron nonlinearity. Here, we introduce and demonstrate a novel electro-optic device to engineer the shape of this optical nonlinearity to resemble a rectifying linear unit (ReLU) - the most-commonly used nonlinear activation function in neural networks. We combine the counter-directional transfer functions from heterostructures made out of two electro-optic materials to design a diode-like nonlinear response of the device. Integrating this nonlinearity into a photonic neural network, we show how the electrostatics of this thresholders gating junction improves machine learning inference accuracy and the energy efficiency of the neural network.

Programmable low-threshold optical nonlinear activation functions for photonic neural networks

Reconfigurable All-Optical Integrated Nonlinear Activator with Switchable Response Functions for Photonic Neural Networks

Optical Nonlinear Activation Functions Based on MZI-Structure for Optical Neural Networks

Digit Recognition Based on Programmable Nanophotonic Processor

All-Optical Nonlinear Activators Using Silicon-Based Waveguides and Microring Resonators for Photonic Neural Networks

Photonic Neural Activation Function Based on High-Speed Electro-Absorption Modulated Laser

Electrically Programmable Phase-Change Photonic Memory for Optical Neural Networks with Nanoseconds in Situ Training Capability

All-optical nonlinear activation function based on stimulated Brillouin scattering

Photonic Neural Networks and Its Applications

High-quality activation function for applications in neuromorphic photonic chips realized using low-quality nonlinear optical resonators

Ti$_3$C$_2$T$_x$ MXene Enabled All-Optical Nonlinear Activation Function for On-Chip Photonic Deep Neural Networks

Microdisk modulator-assisted optical nonlinear activation functions for photonic neural networks

Efficient Nonlinear Activation Function in Optical Neural Network

A surface-normal photodetector as nonlinear activation function in diffractive optical neural networks

All-optical neural network nonlinear activation function based on the optical bistability within a micro-ring resonator

All-Chalcogenide Programmable All-Optical Deep Neural Networks

Sub-milliwatt threshold power and tunable-bias all-optical nonlinear activation function using vanadium dioxide for wavelength-division multiplexing photonic neural networks

Optical ReLU-like Activation Function Based on a Semiconductor Laser with Optical Injection

Easily Scalable Photonic Tensor Core Based on Tunable Units with Single Internal Phase Shifters

An ITO Graphene hybrid integrated absorption modulator on Si-photonics for neuromorphic nonlinear activation

Ultra-broadband all-optical nonlinear activation function enabled by MoTe2/optical waveguide integrated devices