A Photoelectric Spiking Neuron for Visual Depth Perception

Chunsheng Chen,Yongli He,Huiwu Mao,Li Zhu,Xiangjing Wang,Ying Zhu,Yixin Zhu,Yi Shi,Changjin Wan,Qing Wan

DOI: https://doi.org/10.1002/adma.202201895

IF: 29.4

2022-04-11

Advanced Materials

Abstract:Advanced Materials, Accepted Article.

materials science, multidisciplinary,chemistry, physical,physics, applied, condensed matter,nanoscience & nanotechnology

What problem does this paper attempt to address?

A Spiking Visual Neuron for Depth Perceptual Systems

Chunsheng Chen,Yongli He,Huiwu Mao,L. Zhu,Xiangjing Wang,Ying Zhu,Yixin Zhu,Yi Shi,C. Wan,Qing Wan

DOI: https://doi.org/10.21203/rs.3.rs-990982/v1

2021-01-01

Abstract:The biological visual system encodes information into spikes and processes them parallelly by the neural network, which enables the perception with high throughput of visual information processing at an energy budget of a few watts. The parallelism and efficiency of bio-visual system motivates electronic implementation of this biological computing paradigm, which is challenged by the lack of bionic devices, such as spiking neurons that can mimic its biological counterpart. Here, we present a highly bio-realistic spiking visual neuron based on an Ag/TaOX/ITO memristor. Such spiking visual neuron collects visual information by a photodetector, encodes them into action potentials through the memristive spiking encoder, and interprets them for recognition tasks based on a network of neuromorphic transistors. The firing spikes generated by the memristive spiking encoders have a frequency range of 1-200 Hz and sub-micro watts power consumption, very close to the biological counterparts. Furthermore, a spiking visual system is demonstrated, replicating the distance-dependent response and eye fatigue of biological visual systems. The mimicked depth perception shows a recognition improvement by adapting to sights at different distance. Our design presents a fundamental building block for energy-efficient and biologically plausible artificial visual systems.
A Photoeletric Memristive Device for Artificial Visual Perception

Xuemeng Fan,Zijian Wang,Pengtao Li,Shengpeng Xing,Zhen Wang,Yishu Zhang

DOI: https://doi.org/10.1109/cstic61820.2024.10531854

2024-01-01

Abstract:Optoelectronic neuromorphic devices have driven the effective development of artificial vision systems, thanks to their ability to capture, store, and process visual information in the environment. We propose a single transistor based optoelectronic memristor that modulates the resistance of MOS 2 channels through photo-generated carriers, affects the voltage bias of the threshold switch (TS) drain, and achieves the basic neural functions of simulating leakage integration and excitation (LIF) and depolarization. Single transistor neurons drive the efficient development of neuromorphic systems.
A Photosensitive Spiking Neuron Using a Single Band-Modulation Device with Tunable Spiking Sensitivity

Yingxin Chen,Chenyu Liu,Tian,Hui Xie,Sorin Cristoloveanu,Peng Zhou,Yong Xu,Fanyu Liu,Jing Wan

DOI: https://doi.org/10.1109/ted.2024.3452073

IF: 3.1

2024-01-01

IEEE Transactions on Electron Devices

Abstract:Bio-inspired visual systems require a critical component to perceive optical stimuli and convert them into spiking signals. In this study, a compact photosensitive spiking neuron is demonstrated using a single band-modulation device called zero subthreshold swing and zero impact ionization FET (Z(2)-FET). This device is fully compatible with the standard CMOS process and features integrate-and-fire (IF) neural behavior. The firing threshold and spiking frequency of the Z(2)-FET neuron can be modulated by incident light with various intensities and wavelengths, similar to the photoresponse observed in biological neurons. TCAD simulations validate the working principle, originating from the deep depletion effect and photo-induced reduction of energy barriers. Inspired by biological neurons with adjustable spike encoding functionality, the Z(2)-FET neuron achieves controllable photoresponsive frequency when the input current, top-gate, and back-gate voltages are tuned. With its excellent scaling capability, low energy consumption, and tunable spiking sensitivity, the photosensitive Z(2)-FET neuron holds great potential in multifunctional neuromorphic applications, such as intelligent visual perception and neural computing.
A Bio-Inspired Spiking Vision Chip Based on SPAD Imaging and Direct Spike Computing for Versatile Edge Vision

Xu Yang,Chunhe Yao,Lei Kang,Qian Luo,Nan Qi,Runjiang Dou,Shuangming Yu,Peng Feng,Zhongming Wei,Jian Liu,Kaiyou Wang,Nanjian Wu,Liyuan Liu

DOI: https://doi.org/10.1109/jssc.2023.3340018

IF: 5.4

2024-01-01

IEEE Journal of Solid-State Circuits

Abstract:Edge vision devices face real 3-D scenes with lighting fluctuations. They require a vision chip capable of acquiring 2-D/3-D vision with a high dynamic range and performing real-time intelligent in situ processing, while maintaining low end-to-end latency. Aimed at this versatile edge vision, this article presents a bio-inspired full-spiking vision chip that consists of a single-photon-avalanche-diode (SPAD) image sensor and a spiking vision processor monolithically. To decrease latency while realizing versatile functions, we build a bio-inspired full spiking vision on chip by directly coupling spike-like SPAD imaging with a neuroinspired spike-computing paradigm. By modeling this spiking visual flow, we theoretically demonstrate direct coupling and derive a spike preprocessing method for dim-vision enhancement. The SPAD image sensor equipped with a configurable exposure gating scheme can realize three modes: 2-D, 3-D depth, and dim-vision imaging. The spiking vision processor contains a parallel processing element (PE) array and can be reconfigured as a preprocessor or a spiking neural network (SNN) processor. A prototype chip integrated with a $128\ttimes 128$ SPAD image sensor with a reconfigurable spiking vision processor (256 PEs) was implemented in a 0.18- $\mu$ m CMOS technology. The chip experimentally demonstrates three imaging modes, 99.30% classification accuracy on the MNIST dataset with 2-D vision, only 3.9% accuracy loss under 0.02 lux with dim-vision, 1.68 cm obstacle detection error with 3-D depth vision, and a 3.58 $\mu$ s latency for light adaptation. The chip realizes a 100 dB dynamic range, 100 000 spiking maps per second (SMps), and 81.92 GSOPS at an 80-MHz clock.
Hierarchical Spiking-Based Model for Efficient Image Classification with Enhanced Feature Extraction and Encoding.

Qi Xu,Yaxin Li,Jiangrong Shen,Pingping Zhang,Jian K. Liu,Huajin Tang,Gang Pan

DOI: https://doi.org/10.1109/tnnls.2022.3232106

IF: 14.255

2024-01-01

IEEE Transactions on Neural Networks and Learning Systems

Abstract:Thanks to their event-driven nature, spiking neural networks (SNNs) are surmised to be great computation-efficient models. The spiking neurons encode beneficial temporal facts and possess excessive anti-noise properties. However, the high-quality encoding of spatio-temporal complexity and also its training optimization of SNNs are restricted by means of the contemporary problem, this article proposes a novel hierarchical event-driven visual device to explore how information transmits and signifies in the retina the usage of biologically manageable mechanisms. This cognitive model is an augmented spiking-based framework consisting of the function learning capacity of convolutional neural networks (CNNs) with the cognition capability of SNNs. Furthermore, this visual device is modeled in a biological realism way with unsupervised learning rules and advanced spike firing rate encoding methods. We train and test them on some image datasets (Modified National Institute of Standards and Technology (MNIST), Canadian Institute for Advanced Research (CIFAR)10, and its noisy versions) to show that our mannequin can process greater vital data than present cognitive models. This article also proposes a novel quantization approach to make the proposed spiking-based model more efficient for neuromorphic hardware implementation. The outcomes show this joint CNN-SNN model can reap excessive focus accuracy and get more effective generalization ability.
Vertically integrated spiking cone photoreceptor arrays for color perception

Xiangjing Wang,Chunsheng Chen,Li Zhu,Kailu Shi,Baocheng Peng,Yixin Zhu,Huiwu Mao,Haotian Long,Shuo Ke,Chuanyu Fu,Ying Zhu,Changjin Wan,Qing Wan

DOI: https://doi.org/10.1038/s41467-023-39143-8

IF: 16.6

2023-06-10

Nature Communications

Abstract:Abstract The cone photoreceptors in our eyes selectively transduce the natural light into spiking representations, which endows the brain with high energy-efficiency color vision. However, the cone-like device with color-selectivity and spike-encoding capability remains challenging. Here, we propose a metal oxide-based vertically integrated spiking cone photoreceptor array, which can directly transduce persistent lights into spike trains at a certain rate according to the input wavelengths. Such spiking cone photoreceptors have an ultralow power consumption of less than 400 picowatts per spike in visible light, which is very close to biological cones. In this work, lights with three wavelengths were exploited as pseudo-three-primary colors to form ‘colorful’ images for recognition tasks, and the device with the ability to discriminate mixed colors shows better accuracy. Our results would enable hardware spiking neural networks with biologically plausible visual perception and provide great potential for the development of dynamic vision sensors.

multidisciplinary sciences
A Flexible Artificial Spiking Photoreceptor Enabled by a Single VO2 Mott Memristor for the Spike-Based Electronic Retina.

Chuan Yu Han,Shujing Zhao,Sheng Li Fang,Weihua Liu,Wing Man Tang,Peter To Lai,Can Li,Yuan Xiao Ma,Jia Qi Song,Xin Li,Xiao Li Wang,Wen Jun Ren,Rui Lin Wang,Xiao Dong Huang,Guo He Zhang,Li Geng

DOI: https://doi.org/10.1021/acsami.4c12874

IF: 9.5

2024-01-01

ACS Applied Materials & Interfaces

Abstract:The neuromorphic vision system that utilizes spikes as information carriers is crucial for the formation of spiking neural networks. Here, we present a bioinspired flexible artificial spiking photoreceptor (ASP), which is realized by using a single VO2 Mott memristor that can simultaneously sense and encode the stimulus light into spikes. The ASP has high spike-encoded photosensitivity and ultrawide photosensing range (405-808 nm) with good endurance (>7 x 10(7)) and high flexibility (bending radius similar to 5 mm). Then, we put forward an all-spike electronic retina architecture that comprises one layer of ASPs and one layer of artificial optical nerves (AONs) to process the spike information. Each AON consists of a single Mott memristor connected in series with a neuro-transistor that is a multiple-input floating-gate MOS transistor. Simulation results demonstrate that the all-spike electronic retina can successfully segment images with high Shannon entropy, thus laying the foundation for the development of a spike-based neuromorphic vision system.
Nearly Panoramic Neuromorphic Vision with Transparent Photosynapses

Xuemei Dong,Chen Chen,Keyuan Pan,Yinxiang Li,Zicheng Zhang,Zixi He,Bin Liu,Zhe Zhou,Yueyue Wu,Dengfeng Zhang,Hongchao Sun,Xinkai Qian,Min Xu,Wei Huang,Juqing Liu

DOI: https://doi.org/10.1002/advs.202303944

IF: 15.1

2023-08-29

Advanced Science

Abstract:A transparent planar photonic synapse (TPPS) is presented that offers dual‐side photosensitive capability for ≈360° neuromorphic visual perception and recognition of moving objects. The TPPS integrates an organic heterobilayer and carbon electrodes on a flat quartz substrate, exhibiting ultraviolet‐visible‐near‐infrared‐wavelength perception (365–970 nm) and various synaptic plasticity ascribed to persistent photoconductivity. Neuromorphic vision based on photonic synapses has the ability to mimic sensitivity, adaptivity, and sophistication of bio‐visual systems. Significant advances in artificial photosynapses are achieved recently. However, conventional photosyanptic devices normally employ opaque metal conductors and vertical device configuration, performing a limited hemispherical field of view. Here, a transparent planar photonic synapse (TPPS) is presented that offers dual‐side photosensitive capability for nearly panoramic neuromorphic vision. The TPPS consisting of all two dimensional (2D) carbon‐based derivatives exhibits ultra‐broadband photodetecting (365–970 nm) and ≈360° omnidirectional viewing angle. With its intrinsic persistent photoconductivity effect, the detector possesses bio‐synaptic behaviors such as short/long‐term memory, experience learning, light adaptation, and a 171% pair‐pulse‐facilitation index, enabling the synapse array to achieve image recognition enhancement (92%) and moving object detection.

materials science, multidisciplinary,nanoscience & nanotechnology,chemistry
High performance artificial visual perception and recognition with a plasmon-enhanced 2D material neural network

Tian Zhang,Xin Guo,Pan Wang,Xinyi Fan,Zichen Wang,Yan Tong,Decheng Wang,Limin Tong,Linjun Li

DOI: https://doi.org/10.1038/s41467-024-46867-8

2024-03-19

Abstract:The development of neuromorphic visual systems has recently gained momentum due to their potential in areas such as autonomous vehicles and robotics. However, current machine visual systems based on silicon technology usually contain photosensor arrays, format conversion, memory and processing modules. As a result, the redundant data shuttling between each unit, resulting in large latency and high-power consumption, seriously limits the performance of neuromorphic vision chips. Here, we demonstrate an artificial neural network (ANN) architecture based on an integrated 2D MoS2/Ag nanograting phototransistor array, which can simultaneously sense, pre-process and recognize optical images without latency. The pre-processing function of the device under photoelectric synergy ensures considerable improvement of efficiency and accuracy of subsequent image recognition. The comprehensive performance of the proof-of-concept device demonstrates great potential for machine vision applications in terms of large dynamic range (180 dB), high speed (500 ns) and low energy consumption per spike (2.4 × 10-17 J).
Three-terminal quantum dot light-emitting synapse with active adaptive photoelectric outputs for complex image processing/parallel computing

Cong Chen,Zhenjia Chen,Di Liu,Xianghong Zhang,Changsong Gao,Liuting Shan,Lujian Liu,Tianjian Chen,Tailiang Guo,Huipeng Chen

DOI: https://doi.org/10.1016/j.matt.2024.06.050

IF: 18.9

2024-01-01

Matter

Abstract:Machine vision enables machines to extract rich information from image or video data and make intelligent decisions. However, approaches using artificial synapse hardware systems significantly limit the real-time and accuracy in machine vision segmentation amid complex environments. Addressing this, we propose a novel three-terminal adaptive artificial-light-emitting synapse (AALS) capable of photoelectric double output along with adaptive behavior. The device uses silver nanowires (AgNWs) as polar conductive bridges to reduce reliance on transparent electrodes, while polyvinyl alcohol (PVA) dielectric layers adaptively modulate charge carrier concentrations in conductive channels. Additionally, we have designed an adaptive parallel neural network (APNN) and applied it to autonomous driving image processing. This innovation significantly reduces adaptation time and notably enhances mean pixel accuracy (MPA) for semantic segmentation under overexposure and low-light conditions by 142.2% and 304.4%, respectively. Therefore, this work introduces new strategies for advanced adaptive vision, promising significant potential in intelligent driving and neuromorphic computing.
A Flexible Artificial Spiking Photoreceptor Enabled by a Single VO 2 Mott Memristor for the Spike-Based Electronic Retina

Chuan Yu Han,Shujing Zhao,Sheng Li Fang,Weihua Liu,Wing Man Tang,Peter To Lai,Can Li,Yuan Xiao Ma,Jia Qi Song,Xin Li,Xiao Li Wang,Wen Jun Ren,Rui Lin Wang,Xiao Dong Huang,Guo He Zhang,Li Geng

DOI: https://doi.org/10.1021/acsami.4c12874

IF: 9.5

2024-10-10

ACS Applied Materials & Interfaces

Abstract:The neuromorphic vision system that utilizes spikes as information carriers is crucial for the formation of spiking neural networks. Here, we present a bioinspired flexible artificial spiking photoreceptor (ASP), which is realized by using a single VO(2) Mott memristor that can simultaneously sense and encode the stimulus light into spikes. The ASP has high spike-encoded photosensitivity and ultrawide photosensing range (405-808 nm) with good endurance (>7 × 107) and high flexibility (bending...

materials science, multidisciplinary,nanoscience & nanotechnology
High performance artificial visual system with plasmon-enhanced 2D material neural network

Tian Zhang,Xin Guo,Pan Wang,Linjun Li,Limin Tong

2023-07-25

Abstract:Artificial visual systems (AVS) have gained tremendous momentum because of its huge potential in areas such as autonomous vehicles and robotics as part of artificial intelligence (AI) in recent years. However, current machine visual systems composed of complex circuits based on complementary metal oxide semiconductor (CMOS) platform usually contains photosensor array, format conversion, memory and processing module. The large amount of redundant data shuttling between each unit, resulting in large latency and high power consumption, which greatly limits the performance of the AVS. Here, we demonstrate an AVS based on a new design concept, which consists of hardware devices connected in an artificial neural network (ANN) that can simultaneously sense, pre-process and recognize optical images without latency. The Ag nanograting and the two-dimensional (2D) heterostructure integrated plasmonic phototransistor array (PPTA) constitute the hardware ANN, and its synaptic weight is determined by the adjustable regularized photoresponsivity matrix. The eye-inspired pre-processing function of the device under photoelectric synergy ensures the considerable improvement of the efficiency and accuracy of subsequent image recognition. The comprehensive performance of the proof-of-concept device demonstrates great potential for machine vision applications in terms of large dynamic range (180 dB), high speed (500 ns) and ultralow energy consumption per spike (2.4e(-17) J).

Optics,Disordered Systems and Neural Networks,Applied Physics
Receptive Field-Based All-Optical Spiking Neural Network for Image Processing

Taiyi Chen,Yu Huang,Pei Zhou,Penghua Mu,Shuiying Xiang,V. N. Chizhevsky,Nianqiang Li

DOI: https://doi.org/10.1109/jqe.2023.3325227

IF: 2.5

2024-02-01

IEEE Journal of Quantum Electronics

Abstract:We report on a novel structure of a receptive field (RF)-based multi-layer all-optical neural network using a micropillar laser with a saturable absorber (SA) for image processing. From the perspective of biological vision, the realization of image processing based on the RF provides the biological rationality for the machine vision implemented by the spiking neural network (SNN). By exploiting the fast physical mechanisms of gain and absorption in the SA laser, the photonic spike-timing-dependent plasticity (STDP) curves are achieved to train the weights. Here, the source image pixels are mapped into the temporal information of spike trains injected into the neural network through the temporal coding method called time-to-first-spike encoding. Different source images are processed and tested by the proposed photonic SNN. Simulation results show that our proposed system can process not only simple binary images but also complex color images under the adjustment of STDP rules. When considering the robustness, we demonstrate the tolerance of the image segmentation to the time jitter. These results indicate that our proposed photonic SNN can achieve high-resolution processing of complex source images. Additionally, the time-multiplexing technique can be further adopted to simplify the RF structure, which is expected to reduce the complexity of the whole system, thus facilitating physical applications. Our work offers the prospect for a high-speed photonic spiking platform for image processing.

engineering, electrical & electronic,optics,physics, applied,quantum science & technology
A Spiking Artificial Vision Architecture Based on Fully Emulating the Human Vision

Yi Wu,Wenjie Deng,Kexin Li,Xiaoting Wang,Bo Liu,Jingzhen Li,Zhijie Chen,Yongzhe Zhang

DOI: https://doi.org/10.1002/adma.202312094

Abstract:Intelligent vision necessitates the deployment of detectors that are always-on and low-power, mirroring the continuous and uninterrupted responsiveness characteristic of human vision. Nonetheless, contemporary artificial vision systems attain this goal by the continuous processing of massive image frames and executing intricate algorithms, thereby expending substantial computational power and energy. In contrast, biological data processing, based on event-triggered spiking, has higher efficiency and lower energy consumption. Here, this work proposes an artificial vision architecture consisting of spiking photodetectors and artificial synapses, closely mirroring the intricacies of the human visual system. Distinct from previously reported techniques, the photodetector is self-powered and event-triggered, outputting light-modulated spiking signals directly, thereby fulfilling the imperative for always-on with low-power consumption. With the spiking signals processing through the integrated synapse units, recognition of graphics, gestures, and human action has been implemented, illustrating the potent image processing capabilities inherent within this architecture. The results prove the 90% accuracy rate in human action recognition within a mere five epochs utilizing a rudimentary artificial neural network. This novel architecture, grounded in spiking photodetectors, offers a viable alternative to the extant models of always-on low-power artificial vision system.
Single‐Transistor Optoelectronic Spiking Neuron with Optogenetics‐Inspired Spatiotemporal Dynamics

Hanxi Li,Jiayang Hu,Yishu Zhang,Anzhe Chen,Jiachao Zhou,Yuda Zhao,Yang Xu,Bin Yu

DOI: https://doi.org/10.1002/adfm.202314456

IF: 19

2024-01-01

Advanced Functional Materials

Abstract:The visual system is crucial for human perception and learning, and the retina is responsible for pre-processing visual information. Mimicking the neurobiological functions of the retina provides the basis for neuromorphic vision hardware. Here, an optogenetics-inspired prototype capable of modifying neuron-spiking behavior in a retina-like manner is demonstrated. In addition to basic information processing, neuronal signal transmission in the retina is bio-faithfully emulated using optical modulation, that is, neuronal firing is inhibited in the dark and activated upon illumination. The proposed single threshold-switch field-effect transistor optoelectronic neuron features an ultra-compact design by integrating a gate-modulated 2D MoS2 channel with a volatile threshold switch, and optoelectronic performance is achieved through a multi-physics resistance-matching mechanism. This study provides a promising pathway toward highly scalable and hierarchically designed spiking electronics, expanding the spectrum of neuromorphic hardware applications in the emerging field of biomimetic vision. A single-transistor optoelectronic neuron with optogenetics-inspired spatiotemporal dynamics, which not only replicates biological neuron behavior, but is also capable of modifying neuron spiking behavior through optical modulation in a retina-like manner. image
Artificial Visual Perception Nervous System Based on Low-Dimensional Material Photoelectric Memristors

Yifei Pei,Lei Yan,Zuheng Wu,Jikai Lu,Jianhui Zhao,Jingsheng Chen,Qi Liu,Xiaobing Yan

DOI: https://doi.org/10.1021/acsnano.1c04676

IF: 17.1

2021-11-23

ACS Nano

Abstract:The visual perception system is the most important system for human learning since it receives over 80% of the learning information from the outside world. With the exponential growth of artificial intelligence technology, there is a pressing need for high-energy and area-efficiency visual perception systems capable of processing efficiently the received natural information. Currently, memristors with their elaborate dynamics, excellent scalability, and information (e.g., visual, pressure, sound, etc.) perception ability exhibit tremendous potential for the application of visual perception. Here, we propose a fully memristor-based artificial visual perception nervous system (AVPNS) which consists of a quantum-dot-based photoelectric memristor and a nanosheet-based threshold-switching (TS) memristor. We use a photoelectric and a TS memristor to implement the synapse and leaky integrate-and-fire (LIF) neuron functions, respectively. With the proposed AVPNS we successfully demonstrate the biological image perception, integration and fire, as well as the biosensitization process. Furthermore, the self-regulation process of a speed meeting control system in driverless automobiles can be accurately and conceptually emulated by this system. Our work shows that the functions of the biological visual nervous system may be systematically emulated by a memristor-based hardware system, thus expanding the spectrum of memristor applications in artificial intelligence.
Spike Encoding with Optic Sensory Neurons Enable a Pulse Coupled Neural Network for Ultraviolet Image Segmentation

Quantan Wu,Bingjie Dang,Congyan Lu,Guangwei Xu,Guanhua Yang,Jiawei Wang,Xichen Chuai,Nianduan Lu,Di Geng,Hong Wang,Ling Li

DOI: https://doi.org/10.1021/acs.nanolett.0c02892

IF: 10.8

2020-10-16

Nano Letters

Abstract:Drawing inspiration from biology, neuromorphic systems are of great interest in direct interaction and efficient processing of analogue signals in the real world and could be promising for the development of smart sensors. Here, we demonstrate an artificial sensory neuron consisting of an InGaZnO4 (IGZO4)-based optical sensor and NbOx-based oscillation neuron in series, which can simultaneously sense the optical information even beyond the visible light region and encode them into electrical impulses. Such artificial vision sensory neurons can convey visual information in a parallel manner analogous to biological vision systems, and the output spikes can be effectively processed by a pulse coupled neural network, demonstrating the capability of image segmentation out of a complex background. This study could facilitate the construction of artificial visual systems and pave the way for the development of light-driven neurorobotics, bioinspired optoelectronics, and neuromorphic computing.The Supporting Information is available free of charge at <a class="ext-link" href="/doi/10.1021/acs.nanolett.0c02892?goto=supporting-info">https://pubs.acs.org/doi/10.1021/acs.nanolett.0c02892</a>.Material characterization of NbOx film, the electrical performance of the NbOx-based device, mechanism of the NbOx-based device, the electrical performance of the IGZO4-based device, and the formulation of the oscillation frequency (<a class="ext-link" href="/doi/suppl/10.1021/acs.nanolett.0c02892/suppl_file/nl0c02892_si_001.pdf">PDF</a>)This article has not yet been cited by other publications.

materials science, multidisciplinary,chemistry, physical,physics, applied, condensed matter,nanoscience & nanotechnology
StereoSpike: Depth Learning With a Spiking Neural Network

Javier Cuadrado-Anibarro,Benoit R. Cottereau,Ulysse Rancon,Timothee Masquelier

DOI: https://doi.org/10.1109/access.2022.3226484

IF: 3.9

2022-12-13

IEEE Access

Abstract:Depth estimation is an important computer vision task, useful in particular for navigation in autonomous vehicles, or for object manipulation in robotics. Here, we propose to solve it using StereoSpike, an end-to-end neuromorphic approach, combining two event-based cameras and a Spiking Neural Network (SNN) with a modified U-Net-like encoder-decoder architecture. More specifically, we used the Multi Vehicle Stereo Event Camera Dataset (MVSEC). It provides a depth ground-truth, which was used to train StereoSpike in a supervised manner, using surrogate gradient descent. We propose a novel readout paradigm to obtain a dense analog prediction–the depth of each pixel– from the spikes of the decoder. We demonstrate that this architecture generalizes very well, even better than its non-spiking counterparts, leading to near state-of-the-art test accuracy. To the best of our knowledge, it is the first time that such a large-scale regression problem is solved by a fully spiking neural network. Finally, we show that very low firing rates (< 5%) can be obtained via regularization, with a minimal cost in accuracy. This means that StereoSpike could be efficiently implemented on neuromorphic chips, opening the door for low power and real time embedded systems.

computer science, information systems,telecommunications,engineering, electrical & electronic
Degradable Photoelectric Synapses for VR Image Processing

Shuangjie Zhang,Ziyan He,Yu-Qing Zheng

DOI: https://doi.org/10.1016/j.device.2024.100397

2024-01-01

Device

Abstract:Photoelectric synapses, leveraging their fast response, low power consumption, and pre-processing capabilities, can help meet the demand for smaller and more efficient processors. In a recent study from Matter, Ding and co-workers present an organic and biodegradable neuromorphic vision sensor (ONeuVSs) to aid the effort to combat the mounting problem of electronic waste.
Spiking Information Processing in a Single Photonic Spiking Neuron Chip with Double Integrated Electronic Dendrites

Yahui Zhang,Shuiying Xiang,Xingxing Guo,Yanan Han,Yuechun Shi,Xiangfei Chen,Genquan Han,Yue Hao

DOI: https://doi.org/10.1364/prj.499767

IF: 7.6

2023-01-01

Photonics Research

Abstract:Dendrites, branches of neurons that transmit signals between synapses and soma, play a vital role in spiking information processing, such as nonlinear integration of excitatory and inhibitory stimuli. However, the investigation of nonlinear integration of dendrites in photonic neurons and the fabrication of photonic neurons including dendritic nonlinear integration in photonic spiking neural networks(SNNs) remain open problems.Here, we fabricate and integrate two dendrites and one soma in a single Fabry–Perot laser with an embedded saturable absorber(FP-SA) neuron to achieve nonlinear integration of excitatory and inhibitory stimuli. Note that the two intrinsic electrodes of the gain section and saturable absorber(SA) section in the FP-SA neuron are defined as two dendrites for two ports of stimuli reception, with one electronic dendrite receiving excitatory stimulus and the other receiving inhibitory stimulus. The stimuli received by two electronic dendrites are integrated non-linearly in a single FP-SA neuron, which generates spikes for photonic SNNs. The properties of frequency encoding and spatiotemporal encoding are investigated experimentally in a single FP-SA neuron with two electronic dendrites. For SNNs equipped with FP-SA neurons, the range of weights between presynaptic neurons and postsynaptic neurons is varied from negative to positive values by biasing the gain and SA sections of FP-SA neurons.Compared with SNN with all-positive weights realized by only biasing the gain section of photonic neurons, the recognition accuracy of Iris flower data is improved numerically in SNN consisting of FP-SA neurons. The results show great potential for multi-functional integrated photonic SNN chips.

A Photoelectric Spiking Neuron for Visual Depth Perception

A Spiking Visual Neuron for Depth Perceptual Systems

A Photoeletric Memristive Device for Artificial Visual Perception

A Photosensitive Spiking Neuron Using a Single Band-Modulation Device with Tunable Spiking Sensitivity

A Bio-Inspired Spiking Vision Chip Based on SPAD Imaging and Direct Spike Computing for Versatile Edge Vision

Hierarchical Spiking-Based Model for Efficient Image Classification with Enhanced Feature Extraction and Encoding.

Vertically integrated spiking cone photoreceptor arrays for color perception

A Flexible Artificial Spiking Photoreceptor Enabled by a Single VO2 Mott Memristor for the Spike-Based Electronic Retina.

Nearly Panoramic Neuromorphic Vision with Transparent Photosynapses

High performance artificial visual perception and recognition with a plasmon-enhanced 2D material neural network

Three-terminal quantum dot light-emitting synapse with active adaptive photoelectric outputs for complex image processing/parallel computing

A Flexible Artificial Spiking Photoreceptor Enabled by a Single VO 2 Mott Memristor for the Spike-Based Electronic Retina

High performance artificial visual system with plasmon-enhanced 2D material neural network

Receptive Field-Based All-Optical Spiking Neural Network for Image Processing

A Spiking Artificial Vision Architecture Based on Fully Emulating the Human Vision

Single‐Transistor Optoelectronic Spiking Neuron with Optogenetics‐Inspired Spatiotemporal Dynamics

Artificial Visual Perception Nervous System Based on Low-Dimensional Material Photoelectric Memristors

Spike Encoding with Optic Sensory Neurons Enable a Pulse Coupled Neural Network for Ultraviolet Image Segmentation

StereoSpike: Depth Learning With a Spiking Neural Network

Degradable Photoelectric Synapses for VR Image Processing

Spiking Information Processing in a Single Photonic Spiking Neuron Chip with Double Integrated Electronic Dendrites