Abstract:The advent of big data and the Internet of Things has created urgent demands for in-sensor computing hardware with multimodal perception that can effectively resolve the inefficiency, high latency, and excessive energy consumption challenges faced by conventional sensory systems. Here, a simple-structured optoelectronic synaptic device of In2O3SnO2/Nb:SrTiO3 (ITO/NSTO) heterostructure is proposed, which vividly demonstrates in-sensor computing and multimodal perception capabilities. First, with the ingenious synaptic responses of the device under both optical and electrical stimuli, a multimodal in-sensor neuromorphic computing system capable of concurrently perceiving and processing visual and auditory information is constructed. Using this multimodal system to perform a human emotion recognition task, the misjudgment arising from single-modal cognition can thereby be effectively avoided. Second, utilizing the integrated sensing and processing functions, along with the dynamic memories of the device array, a neuromorphic vision system is implemented for real-time monitoring of moving vehicles, which displays high recognition efficiency and accuracy. This research not only provides an optoelectronic synaptic device that is low-cost and easy to mass-produce, but also paves the way for next-generation in-sensor computing that can efficiently process multimodal signals. The separated sensing, storing, and computing modules along with single-mode perception in the conventional sensory system cause time-delay, massive energy consumption, as well as incomprehensive cognition issues. Here a neuromorphic device capable of performing multimodal perception and in-sensor computation within a single device is proposed, enabling a novel multimodal sensory system that is efficient, low-energy, and has minimal time latency. image

A Brain-Inspired Hierarchical Interactive In-Memory Computing System and Its Application in Video Sentiment Analysis

A Brain-Inspired In-Memory Computing System for Neuronal Communication Via Memristive Circuits.

FeFET-Based In-Memory Hyperdimensional Encoding Design

Design and Implementation of a Flexible Neuromorphic Computing System for Affective Communication Via Memristive Circuits

Multimodal In-Sensor Computing Implemented by Easily-Fabricated Oxide-Heterojunction Optoelectronic Synapses

In-Memory Computing: Advances and Prospects

A Semantic and Syntactic Enhanced Neuromorphic Computing System and Its Application in Consumer Sentiment Analysis

MLG-NCS: Multimodal Local–Global Neuromorphic Computing System for Affective Video Content Analysis

Memristor-Based Hierarchical Attention Network for Multimodal Affective Computing in Mental Health Monitoring

In-Memory Computing Integrated Structure Circuit Based on Nonvolatile Flash Memory Unit

In-Memory Computing: The Next-Generation AI Computing Paradigm

In-memory Computing with Emerging Nonvolatile Memory Devices

An in-memory computing architecture based on a duplex 2D material structure for in-situ machine learning

A 3D MCAM Architecture Based on Flash Memory Enabling Binary Neural Network Computing for Edge AI

An In-Memory-Computing Structure with Quantum-Dot Transistor Toward Neural Network Applications: From Analog Circuits to Memory Arrays

A Heterogeneous In-Memory Computing Cluster For Flexible End-to-End Inference of Real-World Deep Neural Networks

A Hybrid Computing-In-Memory Architecture by Monolithic 3D Integration of BEOL CNT/IGZO-based CFET Logic and Analog RRAM

Hdc-Im: Hyperdimensional Computing In-Memory Architecture Based On Rram

Bioinspired Interactive Neuromorphic Devices

Ferroelectric materials for neuroinspired computing applications

Design and Implementation of a Charge-Sharing In-Memory-computing Macro with Sparse Feature for Quantized Neural Network