Abstract:The advent of big data and the Internet of Things has created urgent demands for in-sensor computing hardware with multimodal perception that can effectively resolve the inefficiency, high latency, and excessive energy consumption challenges faced by conventional sensory systems. Here, a simple-structured optoelectronic synaptic device of In2O3SnO2/Nb:SrTiO3 (ITO/NSTO) heterostructure is proposed, which vividly demonstrates in-sensor computing and multimodal perception capabilities. First, with the ingenious synaptic responses of the device under both optical and electrical stimuli, a multimodal in-sensor neuromorphic computing system capable of concurrently perceiving and processing visual and auditory information is constructed. Using this multimodal system to perform a human emotion recognition task, the misjudgment arising from single-modal cognition can thereby be effectively avoided. Second, utilizing the integrated sensing and processing functions, along with the dynamic memories of the device array, a neuromorphic vision system is implemented for real-time monitoring of moving vehicles, which displays high recognition efficiency and accuracy. This research not only provides an optoelectronic synaptic device that is low-cost and easy to mass-produce, but also paves the way for next-generation in-sensor computing that can efficiently process multimodal signals. The separated sensing, storing, and computing modules along with single-mode perception in the conventional sensory system cause time-delay, massive energy consumption, as well as incomprehensive cognition issues. Here a neuromorphic device capable of performing multimodal perception and in-sensor computation within a single device is proposed, enabling a novel multimodal sensory system that is efficient, low-energy, and has minimal time latency. image

Multisensory synapses based on Fe 3 O 4 /graphene transistors for neuromorphic computing

Carbon Nanotube-Based Flexible Ferroelectric Synaptic Transistors for Neuromorphic Computing

Multisensory Ferroelectric Semiconductor Synapse for Neuromorphic Computing

Multi-Stimuli-Responsive Synapse Based on Vertical Van Der Waals Heterostructures.

Optically Stimulated Synaptic Devices Based on the Hybrid Structure of Silicon Nanomembrane and Perovskite.

Multimodal In-Sensor Computing Implemented by Easily-Fabricated Oxide-Heterojunction Optoelectronic Synapses

CuInP2S6‐Based Electronic/Optoelectronic Synapse for Artificial Visual System Application

Flexible Metal Oxide/Graphene Oxide Hybrid Neuromorphic Transistors on Flexible Conducting Graphene Substrates

Multifunctional Two-Terminal Optoelectronic Synapse Based on Zinc Oxide/Poly(3-hexylthiophene) Heterojunction for Neuromorphic Computing

Low‐Power, Electrochemically Tunable Graphene Synapses for Neuromorphic Computing

A flexible PI/graphene heterojunction optoelectronic device modulated by TENG and UV light for neuromorphic vision system

A Multi-Input Light-Stimulated Synaptic Transistor for Complex Neuromorphic Computing

Dual-input Optoelectronic Synaptic Transistor Based on Amorphous ZnAlSnO for Multi-Target Neuromorphic Simulation

Bidirectional All‐Optical Synapses Based on a 2D Bi2O2Se/Graphene Hybrid Structure for Multifunctional Optoelectronics

Flexible electronic synapse enabled by ferroelectric field effect transistor for robust neuromorphic computing

All-Optically Controlled Artificial Synapses Based on Light-Induced Adsorption and Desorption for Neuromorphic Vision.

Multimodal 2D Ferroelectric Transistor with Integrated Perception-and-Computing-in-Memory Functions for Reservoir Computing

Optoelectronic Synapses Based on MoS2 Transistors for Accurate Image Recognition

A Photoelectric-Stimulated MoS2 Transistor for Neuromorphic Engineering

AlGaN/GaN MOS-HEMT enabled optoelectronic artificial synaptic devices for neuromorphic computing

Multifunctional Two-Terminal Optoelectronic Synapse Based on an Organic Semiconductor Film