Abstract:The advent of big data and the Internet of Things has created urgent demands for in-sensor computing hardware with multimodal perception that can effectively resolve the inefficiency, high latency, and excessive energy consumption challenges faced by conventional sensory systems. Here, a simple-structured optoelectronic synaptic device of In2O3SnO2/Nb:SrTiO3 (ITO/NSTO) heterostructure is proposed, which vividly demonstrates in-sensor computing and multimodal perception capabilities. First, with the ingenious synaptic responses of the device under both optical and electrical stimuli, a multimodal in-sensor neuromorphic computing system capable of concurrently perceiving and processing visual and auditory information is constructed. Using this multimodal system to perform a human emotion recognition task, the misjudgment arising from single-modal cognition can thereby be effectively avoided. Second, utilizing the integrated sensing and processing functions, along with the dynamic memories of the device array, a neuromorphic vision system is implemented for real-time monitoring of moving vehicles, which displays high recognition efficiency and accuracy. This research not only provides an optoelectronic synaptic device that is low-cost and easy to mass-produce, but also paves the way for next-generation in-sensor computing that can efficiently process multimodal signals. The separated sensing, storing, and computing modules along with single-mode perception in the conventional sensory system cause time-delay, massive energy consumption, as well as incomprehensive cognition issues. Here a neuromorphic device capable of performing multimodal perception and in-sensor computation within a single device is proposed, enabling a novel multimodal sensory system that is efficient, low-energy, and has minimal time latency. image

Towards on-receptor computing: Electronic nociceptor embedded neuromorphic functionalities at nanoscale

On-receptor computing with classical associative learning in semiconductor oxide memristors

Multimodal In-Sensor Computing Implemented by Easily-Fabricated Oxide-Heterojunction Optoelectronic Synapses

A Brain-Inspired In-Memory Computing System for Neuronal Communication Via Memristive Circuits.

On-receptor computing utilizing vertical-structured cost-effective memristor

Emerging Iontronic Neural Devices for Neuromorphic Sensory Computing

Self healable neuromorphic memtransistor elements for decentralized sensory signal processing in robotics

An Artificial Spiking Afferent Nerve Based on Mott Memristors for Neurorobotics

An Artificial Spiking Nociceptor Integrating Pressure Sensors and Memristors

Neuromorphic Neuromodulation: Towards the next generation of on-device AI-revolution in electroceuticals

Neuro-inspired Thermoresponsive Nociceptor for Intelligent Sensory Systems

A Flexible Artificial Sensory Nerve Enabled by Nanoparticle‐Assembled Synaptic Devices for Neuromorphic Tactile Recognition (Adv. Sci. 24/2022)

A Biomimetic Nociceptor Based on a Vertical Multigate, Multichannel Neuromorphic Transistor

An evaluation of recent advancements in biological sensory organ-inspired neuromorphically tuned biomimetic devices

Freestanding Dual-Gate Oxide-Based Neuromorphic Transistors for Flexible Artificial Nociceptors

Cobalt-doped zinc oxide based memristors with nociceptor characteristics for bio-inspired technology

Ionic Diffusive Nanomemristors with Dendritic Competition and Cooperation Functions for Ultralow Voltage Neuromorphic Computing.

Artificial Leaky Integrate-and-Fire Sensory Neuron for In-Sensor Computing Neuromorphic Perception at the Edge

Stretchable Hierarchical Metal Wire Networks for Neuromorphic Emulation of Nociception and Anti-nociception

Sub 100nW volatile nano-metal-oxide memristor as synaptic-like encoder of neuronal spikes

Biopolymer based artificial synapse enables linear conductance tuning and low-power for neuromorphic computing