Abstract:The advent of big data and the Internet of Things has created urgent demands for in-sensor computing hardware with multimodal perception that can effectively resolve the inefficiency, high latency, and excessive energy consumption challenges faced by conventional sensory systems. Here, a simple-structured optoelectronic synaptic device of In2O3SnO2/Nb:SrTiO3 (ITO/NSTO) heterostructure is proposed, which vividly demonstrates in-sensor computing and multimodal perception capabilities. First, with the ingenious synaptic responses of the device under both optical and electrical stimuli, a multimodal in-sensor neuromorphic computing system capable of concurrently perceiving and processing visual and auditory information is constructed. Using this multimodal system to perform a human emotion recognition task, the misjudgment arising from single-modal cognition can thereby be effectively avoided. Second, utilizing the integrated sensing and processing functions, along with the dynamic memories of the device array, a neuromorphic vision system is implemented for real-time monitoring of moving vehicles, which displays high recognition efficiency and accuracy. This research not only provides an optoelectronic synaptic device that is low-cost and easy to mass-produce, but also paves the way for next-generation in-sensor computing that can efficiently process multimodal signals. The separated sensing, storing, and computing modules along with single-mode perception in the conventional sensory system cause time-delay, massive energy consumption, as well as incomprehensive cognition issues. Here a neuromorphic device capable of performing multimodal perception and in-sensor computation within a single device is proposed, enabling a novel multimodal sensory system that is efficient, low-energy, and has minimal time latency. image

Object Motion Detection Enabled by Reconfigurable Neuromorphic Vision Sensor under Ferroelectric Modulation

A robust neuromorphic vision sensor with optical control of ferroelectric switching

Neuromorphic Vision Sensor Driven by Ferroelectric HfAlO

Low-Power Perovskite Neuromorphic Synapse with Enhanced Photon Efficiency for Directional Motion Perception

Multimodal In-Sensor Computing Implemented by Easily-Fabricated Oxide-Heterojunction Optoelectronic Synapses

In-plane Ferroelectric-Reconfigured Interface Towards Dual-Modal Intelligent Vision

High‐Contrast Bidirectional Optoelectronic Synapses based on 2D Molecular Crystal Heterojunctions for Motion Detection

A Reconfigurable All-Optical-Controlled Synaptic Device for Neuromorphic Computing Applications

High Photoresponsivity and Fast Response Speed Ferroelectric Photomemristor for Artificial Visual System Application

Integrated In‐Memory Sensor and Computing of Artificial Vision Based on Full‐vdW Optoelectronic Ferroelectric Field‐Effect Transistor

Optical-Electrical Coordinately Modulated Memristor Based on 2D Ferroelectric RP Perovskite for Artificial Vision Applications

Optoelectronic graded neurons for bioinspired in-sensor motion perception

A Multifunctional and Efficient Artificial Visual Perception Nervous System with Sb2Se3/CdS‐Core/Shell (SC) Nanorod Arrays Optoelectronic Memristor

Integration of Image Preprocessing and Recognition Functions in an Optoelectronic Coupling Organic Ferroelectric Retinomorphic Neuristor

Artificial Visual Systems Fabricated with Ferroelectric van der Waals Heterostructure for In-Memory Computing Applications

Interface Charge Engineering in Ferroelectric Neuristors for a Complete Machine Vision System

A Dual-Functional Integration of Photodetectors and Artificial Optoelectronic Synapses on a VO2/WO3 Heterojunction Device

Toward High Performance, Programmable Extreme-Edge Intelligence for Neuromorphic Vision Sensors utilizing Magnetic Domain Wall Motion-based MTJ

Non-volatile Rippled-Assisted Optoelectronic Array for All-Day Motion Detection and Recognition

Ultralow Power Wearable Organic Ferroelectric Device for Optoelectronic Neuromorphic Computing.

Object Motion Sensitivity: A Bio-inspired Solution to the Ego-motion Problem for Event-based Cameras