Abstract:The advent of big data and the Internet of Things has created urgent demands for in-sensor computing hardware with multimodal perception that can effectively resolve the inefficiency, high latency, and excessive energy consumption challenges faced by conventional sensory systems. Here, a simple-structured optoelectronic synaptic device of In2O3SnO2/Nb:SrTiO3 (ITO/NSTO) heterostructure is proposed, which vividly demonstrates in-sensor computing and multimodal perception capabilities. First, with the ingenious synaptic responses of the device under both optical and electrical stimuli, a multimodal in-sensor neuromorphic computing system capable of concurrently perceiving and processing visual and auditory information is constructed. Using this multimodal system to perform a human emotion recognition task, the misjudgment arising from single-modal cognition can thereby be effectively avoided. Second, utilizing the integrated sensing and processing functions, along with the dynamic memories of the device array, a neuromorphic vision system is implemented for real-time monitoring of moving vehicles, which displays high recognition efficiency and accuracy. This research not only provides an optoelectronic synaptic device that is low-cost and easy to mass-produce, but also paves the way for next-generation in-sensor computing that can efficiently process multimodal signals. The separated sensing, storing, and computing modules along with single-mode perception in the conventional sensory system cause time-delay, massive energy consumption, as well as incomprehensive cognition issues. Here a neuromorphic device capable of performing multimodal perception and in-sensor computation within a single device is proposed, enabling a novel multimodal sensory system that is efficient, low-energy, and has minimal time latency. image

A Dual-Mode Organic Memristor for Coordinated Visual Perceptive Computing

A Photoeletric Memristive Device for Artificial Visual Perception

A Brain-Inspired In-Memory Computing System for Neuronal Communication Via Memristive Circuits.

Multimodal In-Sensor Computing Implemented by Easily-Fabricated Oxide-Heterojunction Optoelectronic Synapses

A Memristive-Optoelectronic Sensor for Neuromorphic Visual Systems

Optical-Electrical Coordinately Modulated Memristor Based on 2D Ferroelectric RP Perovskite for Artificial Vision Applications

Reconfigurable Optoelectronic Memristor for In-Sensor Computing Applications

Hybrid Dual-Complementary Metal-Oxide-semiconductor/memristor Synapse-Based Neural Network with Its Applications in Image Super-Resolution

Coexistance of the Negative Photoconductance Effect and Analogue Switching Memory in the CuPc Organic Memristor for Neuromorphic Vision Computing

2D materials-based photo-memristors with tunable non-volatile responsivities for neuromorphic vision processing

Networking retinomorphic sensor with memristive crossbar for brain-inspired visual perception

Organic Memristor with Synaptic Plasticity for Neuromorphic Computing Applications.

Fully integrated multi-mode optoelectronic memristor array for diversified in-sensor computing

Optically Driven Intelligent Computing with ZnO Memristor

Wearable in-sensor reservoir computing using optoelectronic polymers with through-space charge-transport characteristics for multi-task learning

Neural Networks Based on In-Sensor Computing of Optoelectronic Memristor

Dual‐functional Perovskite‐based Photodetector And Memristor for Visual Memory

A Dual‐Functional Perovskite‐Based Photodetector and Memristor for Visual Memory

Graphene/MoS 2−x O X /graphene Photomemristor with Tunable Non-Volatile Responsivities for Neuromorphic Vision Processing

Integrated In-Memory Sensor and Computing of Artificial Vision System Based on Reversible Bonding Transition-Induced Nitrogen-Doped Carbon Quantum Dots (N-Cqds)

Low-Power Perovskite Neuromorphic Synapse with Enhanced Photon Efficiency for Directional Motion Perception