Cognitive–behavioural therapy for schizophrenia

C. John,D. Turkington,D. Kingdon

DOI: https://doi.org/10.1192/bjp.165.5.695b

1994-11-01

Abstract:

What problem does this paper attempt to address?

A Reconfigurable Convolution-in-Pixel CMOS Image Sensor Architecture

Ruibing Song,Kejie Huang,Zongsheng Wang,Haibin Shen

DOI: https://doi.org/10.1109/tcsvt.2022.3179370

IF: 5.859

2022-01-01

IEEE Transactions on Circuits and Systems for Video Technology

Abstract:The separation of the data capture and analysis in modern vision systems has led to a massive amount of data transfer between the end devices and cloud computers, resulting in long latency, slow response, and high power consumption. Efficient hardware architectures are under focused development to enable Artificial Intelligence (AI) at the resource-limited sensing devices. One of the most promising solutions is to enable Processing-in-Pixel (PIP) scheme. However, the conventional schemes suffer from the low fill-factor issue. This paper proposes a PIP based Complementary Metal-Oxide-Semiconductor (CMOS) sensor architecture, which allows convolution operation before the column readout circuit to significantly reduce the overall power consumption while improving the resource utilization of the succeeding deep learning accelerator. The simulation results show that the proposed architecture could support the computing efficiency up to 3.37 TOPS/W at the 8-bit weight configuration, which is four times as high as the conventional schemes after normalization. The transistors required for each pixel are only 3.5T, significantly improving the fill-factor.
Live Video Analytics with FPGA-based Smart Cameras

Shang Wang,Chen Zhang,Yuanchao Shu,Yunxin Liu

DOI: https://doi.org/10.1145/3349614.3356027

2019-01-01

Abstract:Analyzing video feeds from large camera networks requires enormous compute and bandwidth. Edge computing has been proposed to ease the burden by bringing resources to the proximity of data. However, the number of cameras keeps growing and the associated computing resources on edge will again fall in short. To fundamentally solve the resource scarcity problem and make edge-based live video analytics scalable, we present an FPGA-based smart camera design that enables efficient in-situ streaming processing to meet the stringent low-power, energy-efficient, low-latency requirements of edge vision applications. By leveraging FPGA's intrinsic properties of architecture efficiency and exploiting its hardware support for parallelism, we demonstrate a 49x speedup over CPU and 6.4x more energy-efficiency than GPU, verified using a background subtraction algorithm.
Towards an Efficient CNN Inference Architecture Enabling In-Sensor Processing

Md Jubaer Hossain Pantho,Pankaj Bhowmik,Christophe Bobda

DOI: https://doi.org/10.3390/s21061955

2021-03-10

Abstract:The astounding development of optical sensing imaging technology, coupled with the impressive improvements in machine learning algorithms, has increased our ability to understand and extract information from scenic events. In most cases, Convolution neural networks (CNNs) are largely adopted to infer knowledge due to their surprising success in automation, surveillance, and many other application domains. However, the convolution operations' overwhelming computation demand has somewhat limited their use in remote sensing edge devices. In these platforms, real-time processing remains a challenging task due to the tight constraints on resources and power. Here, the transfer and processing of non-relevant image pixels act as a bottleneck on the entire system. It is possible to overcome this bottleneck by exploiting the high bandwidth available at the sensor interface by designing a CNN inference architecture near the sensor. This paper presents an attention-based pixel processing architecture to facilitate the CNN inference near the image sensor. We propose an efficient computation method to reduce the dynamic power by decreasing the overall computation of the convolution operations. The proposed method reduces redundancies by using a hierarchical optimization approach. The approach minimizes power consumption for convolution operations by exploiting the Spatio-temporal redundancies found in the incoming feature maps and performs computations only on selected regions based on their relevance score. The proposed design addresses problems related to the mapping of computations onto an array of processing elements (PEs) and introduces a suitable network structure for communication. The PEs are highly optimized to provide low latency and power for CNN applications. While designing the model, we exploit the concepts of biological vision systems to reduce computation and energy. We prototype the model in a Virtex UltraScale+ FPGA and implement it in Application Specific Integrated Circuit (ASIC) using the TSMC 90nm technology library. The results suggest that the proposed architecture significantly reduces dynamic power consumption and achieves high-speed up surpassing existing embedded processors' computational capabilities.
Bio-inspired smart vision sensor: toward a reconfigurable hardware modeling of the hierarchical processing in the brain

Pankaj Bhowmik,Md Jubaer Hossain Pantho,Christophe Bobda

DOI: https://doi.org/10.1007/s11554-020-00960-5

2020-03-19

Abstract:Biological vision systems inspire processing methods in computer vision applications. This paper employs the insights of vision systems in hardware and presents a pixel-parallel, reconfigurable, and layer-based hierarchical architecture for smart image sensors. The architecture aims to bring computation close to the sensor to achieve high acceleration for different machine vision applications while consuming low power. We logically divide the image into multiple regions and perform pixel-level and region-level processing after removing spatiotemporal redundancy. Those processors use bio-inspired algorithms to activate the regions with region of interest of a scene. The hierarchical processing breaks the traditional sequential image processing and introduces parallelism for machine vision applications. Also, we make the hardware design reconfigurable even after fabrication to make the hardware reusable for different applications. Simulation results show that the area overhead and power penalty for adding reconfigurable features stay in an acceptable range. We emphasize to maximize the operating speed and obtain 800 MHz. Besides, the design saves 84.01% and 96.91% dynamic power at the first and second stages of the hierarchy by removing redundant information. Furthermore, the sequential deployment of high-level reasoning only on the selected regions of the image becomes computationally inexpensive to execute a complex task in real time.

computer science, artificial intelligence,engineering, electrical & electronic,imaging science & photographic technology
A Manycore Vision Processor for Real-Time Smart Cameras

Bruno A da Silva,Arthur M Lima,Janier Arias-Garcia,Michael Huebner,Jones Yudi

DOI: https://doi.org/10.3390/s21217137

2021-10-27

Abstract:Real-time image processing and computer vision systems are now in the mainstream of technologies enabling applications for cyber-physical systems, Internet of Things, augmented reality, and Industry 4.0. These applications bring the need for Smart Cameras for local real-time processing of images and videos. However, the massive amount of data to be processed within short deadlines cannot be handled by most commercial cameras. In this work, we show the design and implementation of a manycore vision processor architecture to be used in Smart Cameras. With massive parallelism exploration and application-specific characteristics, our architecture is composed of distributed processing elements and memories connected through a Network-on-Chip. The architecture was implemented as an FPGA overlay, focusing on optimized hardware utilization. The parameterized architecture was characterized by its hardware occupation, maximum operating frequency, and processing frame rate. Different configurations ranging from one to eighty-one processing elements were implemented and compared to several works from the literature. Using a System-on-Chip composed of an FPGA integrated into a general-purpose processor, we showcase the flexibility and efficiency of the hardware/software architecture. The results show that the proposed architecture successfully allies programmability and performance, being a suitable alternative for future Smart Cameras.
A 4.57 Μw@120fps Vision System of Sensing with Computing for BNN-Based Perception Applications

Han Xu,Zheyu Liu,Ziwei Liz,Erxiang Ren,Maimaiti Nazhamati,Fei Qiao,Li Luo,Qi Wei,Xinjun Liu,Huazhong Yang

DOI: https://doi.org/10.1109/a-sscc53895.2021.9634759

2021-01-01

Abstract:In AIoT era, intelligent vision perception systems are widely deployed in edges. As shown in Fig. 1, due to limited energy budget, terminal devices usually adopt hierarchical processing architecture. A coarse object detection algorithm runs in always-on mode, and gets ready to trigger subsequent complex algorithms for precise recognition or segmentation. In conventional digital vision processing frameworks, light-induced photocurrents must be transformed to voltage ${\mathrm {(I_{ph}-to-V)}}$, converted to digital signals (A-to-D), transferred on-board to processors and exchanged between memory and processing elements. Smart vision chips provide promising solutions for cutting down these power overheads, such as placing analog processing circuits near the pixel array [2], customizing the analog-to-digital converter (ADC) which is capable of convolution [3] or adding processing circuits deeply into pixels to perform in-sensor current-domain MAC operations [4]. However, the photocurrent conversion ${\mathrm {(I_{ph}-to-V)}}$ circuits are still reserved in those works; besides, they could only complete 1st-layer convolution for low-level features extraction, and are unable to process subsequent layers for end-to-end perception tasks, which limits the processing capability with small CNN model. Additionally, systems that implement whole CNN algorithms are also proposed by integrating CIS with an analog processor in one chip [5] or stacking a CIS chip with a digital processor chip [6]. But power overheads on data transmission and memory access are still unsolved because these designs separate sensing and computing, and adopt conventional Von Neumann architecture with much memory access.
Hardware based spatio-temporal neural processing backend for imaging sensors: Towards a smart camera

Samiran Ganguly,Yunfei Gu,Mircea R. Stan,Avik W. Ghosh

DOI: https://doi.org/10.1117/12.2305137

2018-05-14

Abstract:In this work we show how we can build a technology platform for cognitive imaging sensors using recent advances in recurrent neural network architectures and training methods inspired from biology. We demonstrate learning and processing tasks specific to imaging sensors, including enhancement of sensitivity and signal-to-noise ratio (SNR) purely through neural filtering beyond the fundamental limits sensor materials, and inferencing and spatio-temporal pattern recognition capabilities of these networks with applications in object detection, motion tracking and prediction. We then show designs of unit hardware cells built using complementary metal-oxide semiconductor (CMOS) and emerging materials technologies for ultra-compact and energy-efficient embedded neural processors for smart cameras.
Energy-Efficient & Real-Time Computer Vision with Intelligent Skipping via Reconfigurable CMOS Image Sensors

Md Abdullah-Al Kaiser,Sreetama Sarkar,Peter A. Beerel,Akhilesh R. Jaiswal,Gourav Datta

2024-09-26

Abstract:Current video-based computer vision (CV) applications typically suffer from high energy consumption due to reading and processing all pixels in a frame, regardless of their significance. While previous works have attempted to reduce this energy by skipping input patches or pixels and using feedback from the end task to guide the skipping algorithm, the skipping is not performed during the sensor read phase. As a result, these methods can not optimize the front-end sensor energy. Moreover, they may not be suitable for real-time applications due to the long latency of modern CV networks that are deployed in the back-end. To address this challenge, this paper presents a custom-designed reconfigurable CMOS image sensor (CIS) system that improves energy efficiency by selectively skipping uneventful regions or rows within a frame during the sensor's readout phase, and the subsequent analog-to-digital conversion (ADC) phase. A novel masking algorithm intelligently directs the skipping process in real-time, optimizing both the front-end sensor and back-end neural networks for applications including autonomous driving and augmented/virtual reality (AR/VR). Our system can also operate in standard mode without skipping, depending on application needs. We evaluate our hardware-algorithm co-design framework on object detection based on BDD100K and ImageNetVID, and gaze estimation based on OpenEDS, achieving up to 53% reduction in front-end sensor energy while maintaining state-of-the-art (SOTA) accuracy.

Computer Vision and Pattern Recognition
Hardware-Algorithm Co-design Enabling Processing-in-Pixel-in-Memory (P2M) for Neuromorphic Vision Sensors

Md Abdullah-Al Kaiser,Akhilesh R. Jaiswal

2023-10-08

Abstract:The high volume of data transmission between the edge sensor and the cloud processor leads to energy and throughput bottlenecks for resource-constrained edge devices focused on computer vision. Hence, researchers are investigating different approaches (e.g., near-sensor processing, in-sensor processing, in-pixel processing) by executing computations closer to the sensor to reduce the transmission bandwidth. Specifically, in-pixel processing for neuromorphic vision sensors (e.g., dynamic vision sensors (DVS)) involves incorporating asynchronous multiply-accumulate (MAC) operations within the pixel array, resulting in improved energy efficiency. In a CMOS implementation, low overhead energy-efficient analog MAC accumulates charges on a passive capacitor; however, the capacitor's limited charge retention time affects the algorithmic integration time choices, impacting the algorithmic accuracy, bandwidth, energy, and training efficiency. Consequently, this results in a design trade-off on the hardware aspect-creating a need for a low-leakage compute unit while maintaining the area and energy benefits. In this work, we present a holistic analysis of the hardware-algorithm co-design trade-off based on the limited integration time posed by the hardware and techniques to improve the leakage performance of the in-pixel analog MAC operations.

Hardware Architecture,Image and Video Processing
Technology-Circuit-Algorithm Tri-Design for Processing-in-Pixel-in-Memory (P2M)

Md Abdullah-Al Kaiser,Gourav Datta,Sreetama Sarkar,Souvik Kundu,Zihan Yin,Manas Garg,Ajey P. Jacob,Peter A. Beerel,Akhilesh R. Jaiswal

DOI: https://doi.org/10.1145/3583781.3590235

2023-04-06

Abstract:The massive amounts of data generated by camera sensors motivate data processing inside pixel arrays, i.e., at the extreme-edge. Several critical developments have fueled recent interest in the processing-in-pixel-in-memory paradigm for a wide range of visual machine intelligence tasks, including (1) advances in 3D integration technology to enable complex processing inside each pixel in a 3D integrated manner while maintaining pixel density, (2) analog processing circuit techniques for massively parallel low-energy in-pixel computations, and (3) algorithmic techniques to mitigate non-idealities associated with analog processing through hardware-aware training schemes. This article presents a comprehensive technology-circuit-algorithm landscape that connects technology capabilities, circuit design strategies, and algorithmic optimizations to power, performance, area, bandwidth reduction, and application-level accuracy metrics. We present our results using a comprehensive co-design framework incorporating hardware and algorithmic optimizations for various complex real-life visual intelligence tasks mapped onto our P2M paradigm.

Image and Video Processing,Hardware Architecture
Voltage-Controlled Magnetic Tunnel Junction based ADC-less Global Shutter Processing-in-Pixel for Extreme-Edge Intelligence

Md Abdullah-Al Kaiser,Gourav Datta,Jordan Athas,Christian Duffee,Ajey P. Jacob,Pedram Khalili Amiri,Peter A. Beerel,Akhilesh R. Jaiswal

2024-10-14

Abstract:The vast amount of data generated by camera sensors has prompted the exploration of energy-efficient processing solutions for deploying computer vision tasks on edge devices. Among the various approaches studied, processing-in-pixel integrates massively parallel analog computational capabilities at the extreme-edge, i.e., within the pixel array and exhibits enhanced energy and bandwidth efficiency by generating the output activations of the first neural network layer rather than the raw sensory data. In this article, we propose an energy and bandwidth efficient ADC-less processing-in-pixel architecture. This architecture implements an optimized binary activation neural network trained using Hoyer regularizer for high accuracy on complex vision tasks. In addition, we also introduce a global shutter burst memory read scheme utilizing fast and disturb-free read operation leveraging innovative use of nanoscale voltage-controlled magnetic tunnel junctions (VC-MTJs). Moreover, we develop an algorithmic framework incorporating device and circuit constraints (characteristic device switching behavior and circuit non-linearity) based on state-of-the-art fabricated VC-MTJ characteristics and extensive circuit simulations using commercial GlobalFoundries 22nm FDX technology. Finally, we evaluate the proposed system's performance on two complex datasets - CIFAR10 and ImageNet, showing improvements in front-end and communication energy efficiency by 8.2x and 8.5x respectively and reduction in bandwidth by 6x compared to traditional computer vision systems, without any significant drop in the test accuracy.

Hardware Architecture,Image and Video Processing
Ultra-low Power In-Sensor Neuronal Computing with Oscillatory Retinal Neurons for Frequency-Multiplexed, Parallel Machine Vision

Ragib Ahsan,Hyun Uk Chae,Seyedeh Atiyeh Abbasi Jalal,Zezhi Wu,Jun Tao,Subrata Das,Hefei Liu,Jiang-Bin Wu,Stephen B. Cronin,Han Wang,Constantine Sideris,Rehan Kapadia

DOI: https://doi.org/10.1021/acsnano.4c09055

IF: 17.1

2024-01-01

ACS Nano

Abstract:In-sensor and near-sensor computing architectures enable multiply accumulate operations to be carried out directly at the point of sensing. In-sensor architectures offer dramatic power and speed improvements over traditional von Neumann architectures by eliminating multiple analog-to-digital conversions, data storage, and data movement operations. Current in-sensor processing approaches rely on tunable sensors or additional weighting elements to perform linear functions such as multiply accumulate operations as the sensor acquires data. This work implements in-sensor computing with an oscillatory retinal neuron device that converts incident optical signals into voltage oscillations. A computing scheme is introduced based on the frequency shift of coupled oscillators that enables parallel, frequency multiplexed, nonlinear operations on the inputs. An experimentally implemented 3 × 3 focal plane array of coupled neurons shows that functions approximating edge detection, thresholding, and segmentation occur in parallel. An example of inference on handwritten digits from the MNIST database is also experimentally demonstrated with a 3 × 3 array of coupled neurons feeding into a single hidden layer neural network, approximating a liquid-state machine. Finally, the equivalent energy consumption to carry out image processing operations, including peripherals such as the Fourier transform circuits, is projected to be <20 fJ/OP, possibly reaching as low as 15 aJ/OP.
Reconfiguring the Imaging Pipeline for Computer Vision

Mark Buckler,Suren Jayasuriya,Adrian Sampson

DOI: https://doi.org/10.1109/iccv.2017.111

2017-10-01

Abstract:Advancements in deep learning have ignited an explosion of research on efficient hardware for embedded computer vision. Hardware vision acceleration, however, does not address the cost of capturing and processing the image data that feeds these algorithms. We examine the role of the image signal processing (ISP) pipeline in computer vision to identify opportunities to reduce computation and save energy. The key insight is that imaging pipelines should be be configurable: to switch between a traditional photography mode and a low-power vision mode that produces lower-quality image data suitable only for computer vision. We use eight computer vision algorithms and a reversible pipeline simulation tool to study the imaging system's impact on vision performance. For both CNN-based and classical vision algorithms, we observe that only two ISP stages, demosaicing and gamma compression, are critical for task performance. We propose a new image sensor design that can compensate for these stages. The sensor design features an adjustable resolution and tunable analog-to-digital converters (ADCs). Our proposed imaging system's vision mode disables the ISP entirely and configures the sensor to produce subsampled, lower-precision image data. This vision mode can save ∼75% of the average energy of a baseline photography mode with only a small impact on vision task accuracy.
Sensor-level computer vision with pixel processor arrays for agile robots

Piotr Dudek,Thomas Richardson,Laurie Bose,Stephen Carey,Jianing Chen,Colin Greatwood,Yanan Liu,Walterio Mayol-Cuevas

DOI: https://doi.org/10.1126/scirobotics.abl7755

IF: 25

2022-07-10

Science Robotics

Abstract:Vision processing for control of agile autonomous robots requires low-latency computation, within a limited power and space budget. This is challenging for conventional computing hardware. Parallel processor arrays (PPAs) are a new class of vision sensor devices that exploit advances in semiconductor technology, embedding a processor within each pixel of the image sensor array. Sensed pixel data are processed on the focal plane, and only a small amount of relevant information is transmitted out of the vision sensor. This tight integration of sensing, processing, and memory within a massively parallel computing architecture leads to an interesting trade-off between high performance, low latency, low power, low cost, and versatility in a machine vision system. Here, we review the history of image sensing and processing hardware from the perspective of in-pixel computing and outline the key features of a state-of-the-art smart camera system based on a PPA device, through the description of the SCAMP-5 system. We describe several robotic applications for agile ground and aerial vehicles, demonstrating PPA sensing functionalities including high-speed odometry, target tracking, obstacle detection, and avoidance. In the conclusions, we provide some insight and perspective on the future development of PPA devices, including their application and benefits within agile, robust, adaptable, and lightweight robotics.

robotics
Hypoglycemia during intensified insulin therapy of young children.

C. de Beaufort

DOI: https://doi.org/10.1515/jpem.1998.11.s1.153

Abstract:
Processing Near Sensor Architecture in Mixed-Signal Domain with CMOS Image Sensor of Convolutional-Kernel-Readout Method

Zhe Chen,Xinjun Liu,Huazhong Yang,Huifeng Zhu,Erxiang Ren,Zheyu Liu,Kaige Jia,Li Luo,Xuan Zhang,Qi Wei,Fei Qiao

DOI: https://doi.org/10.1109/tcsi.2019.2937227

2020-01-01

Abstract:In the era of Artificial Intelligence (AI), bio-inspired perceptual computing system design brings favorable opportunities, while still facing considerable challenges in the meantime. Especially for tasks of image recognition in power-limited vision-based Internet of Things (IoT) devices, energy constraints due to the end of Dennard scaling limit the performance of Neural Network (NN) algorithms on popular digital platforms, which would not reach the energy efficiency requirement for embedded AI applications. In this paper, a processing near sensor architecture in mixed-signal domain with CMOS Image Sensor (CIS) of convolutional-kernel-readout method is proposed. Visual data is collected from a smart CIS, which can realize maximum $5 \times 5$ kernel-readout with minimum one slide step for convolutional operations. The outputs of CIS are directly processed by analog processing units locating near CIS without the constraint of digital clock and bottleneck of Analog-to-Digital Converter (ADC). By analyzing the effects of analog noise on classification accuracy, we further evaluate the fault-tolerance of the system to circuit noise and the device imperfection, such as mismatch and process variation. A mixed-signal visual perception chip is fabricated with a $32\times32$ image sensor and a Binarized Neural Network (BNN) processing array integrated with SMIC 180nm standard CMOS mixed-signal process. Measurement results show up to 545.4 GOPS/W energy efficiency with 1.8mW power consumption taking the advantages of ADC-free processing architecture. This work provides a promising alternative for low-power vision-based IoT intelligent applications.
An Encoded Photodetector Array for High-frame-rate Compressed Sensing and Computational Imaging

Wen Pan,Zhaolong Yang,Zhibiao Hao,Changzheng Sun,Bing Xiong,Jian Wang,Yanjun Han,Hongtao Li,Lin Gan,Yi Luo,Lai Wang

DOI: https://doi.org/10.1109/jsen.2024.3506937

IF: 4.3

2024-01-01

IEEE Sensors Journal

Abstract:Modern imaging technology, by exposure and subsequent readout from image sensors, has limited the existing frame rate in real-time scenes. In this article, we present an architecture based on computing in sensor array that can achieve higher frame rates than conventional cameras. This approach fuses the architecture with neural network algorithms for compressed sensing and computational imaging. It achieves a mean squared error of 10 −2 with fewer weights and sampling numbers (10%~40% of original data). It samples lower data to reconstruct images through fast recovery with an elapsed time of hundred microseconds. The feasibility has been verified through the encoded photodetector array chip consisting of 16×16 dual-gate silicon photodiodes. This type of chips represents a highly promising technology for constructing image sensors that delivers high-speed imaging capability of >10000 fps, which may also show potential superiority in terms of noise resistance and dynamic range.
An Analog-Memoryless Near Sensor Computing Architecture for Always-On Intelligent Perception Applications

Tianrui Ma,Kaige Jia,Xin Zhu,Fei QiaO,Qi Wei,Huichan Zhao,Xinjun Liu,Huazhong Yang

DOI: https://doi.org/10.1109/icta48799.2019.9012906

2019-01-01

Abstract:In always-on intelligent visual perception applications, Convolution Neural Network algorithm has been widely applied. However, the implementation of it's processing is limited by hardware resources and energy budgets, which come from data access, interface overhead and inflexible configuration. In this paper, we present a novel analog architecture that implements low-precision Binary Neural Network and fuses with CMOS image sensor for always-on applications. By enabling current-mode computation in analog domain, the overhead of analog-to-digital converter as well as analog buffer is eliminated. And in order to relax the overhead of CIS, conventional correlating-double-sampling is moved to Computation Module. Moreover, the architecture is also dynamic-configurable, which is optimized specifically for CIS data flow. Simulations with HSPICE show that the peak energy efficiency of proposed architecture can achieve 13.91 TOPS/W at the 8-bit percision of multiplication-and-accumulation analog computation, while accuracy loss is only about 5.66%.
A high-speed vision processor based on pixel-parallel PE array and its applications

Cong Shi,Nanjian Wu,Zhihua Wang

DOI: https://doi.org/10.1109/YCICT.2010.5713151

2010-01-01

Abstract:This paper proposes a novel high-speed vision processor based on pixel-parallel PE array. The processor consists of a pixel-parallel PE array, an embedded RISC core, an AHB bus, some SRAM blocks and other logical controllers. PE array performs the low- and mid-level vision processing, and RISC core carries out the high-level vision processing in succession. The vision processor can process vision information at a speed higher than 1000fps. Many application algorithms can be implemented by software programming on the vision processor. The processor prototype is implemented on a FPGA board with a 64×64 PE array and the clock frequency is 100MHz. It can realize moving detection in 128×128 image data at a rate of 4500fps, and 104fps in complicated face detection task for 160×120 video frame sequence. The results demonstrated that the vision processor outperforms the general serial CPUs for more than 60 times. © 2010 IEEE.
HyNNA: Improved Performance for Neuromorphic Vision Sensor based Surveillance using Hybrid Neural Network Architecture

Deepak Singla,Soham Chatterjee,Lavanya Ramapantulu,Andres Ussa,Bharath Ramesh,Arindam Basu

DOI: https://doi.org/10.48550/arXiv.2003.08603

2020-03-19

Abstract:Applications in the Internet of Video Things (IoVT) domain have very tight constraints with respect to power and area. While neuromorphic vision sensors (NVS) may offer advantages over traditional imagers in this domain, the existing NVS systems either do not meet the power constraints or have not demonstrated end-to-end system performance. To address this, we improve on a recently proposed hybrid event-frame approach by using morphological image processing algorithms for region proposal and address the low-power requirement for object detection and classification by exploring various convolutional neural network (CNN) architectures. Specifically, we compare the results obtained from our object detection framework against the state-of-the-art low-power NVS surveillance system and show an improved accuracy of 82.16% from 63.1%. Moreover, we show that using multiple bits does not improve accuracy, and thus, system designers can save power and area by using only single bit event polarity information. In addition, we explore the CNN architecture space for object classification and show useful insights to trade-off accuracy for lower power using lesser memory and arithmetic operations.

Image and Video Processing,Computer Vision and Pattern Recognition,Machine Learning

Cognitive–behavioural therapy for schizophrenia

A Reconfigurable Convolution-in-Pixel CMOS Image Sensor Architecture

Live Video Analytics with FPGA-based Smart Cameras

Towards an Efficient CNN Inference Architecture Enabling In-Sensor Processing

Bio-inspired smart vision sensor: toward a reconfigurable hardware modeling of the hierarchical processing in the brain

A Manycore Vision Processor for Real-Time Smart Cameras

A 4.57 Μw@120fps Vision System of Sensing with Computing for BNN-Based Perception Applications

Hardware based spatio-temporal neural processing backend for imaging sensors: Towards a smart camera

Energy-Efficient & Real-Time Computer Vision with Intelligent Skipping via Reconfigurable CMOS Image Sensors

Hardware-Algorithm Co-design Enabling Processing-in-Pixel-in-Memory (P2M) for Neuromorphic Vision Sensors

Technology-Circuit-Algorithm Tri-Design for Processing-in-Pixel-in-Memory (P2M)

Voltage-Controlled Magnetic Tunnel Junction based ADC-less Global Shutter Processing-in-Pixel for Extreme-Edge Intelligence

Ultra-low Power In-Sensor Neuronal Computing with Oscillatory Retinal Neurons for Frequency-Multiplexed, Parallel Machine Vision

Reconfiguring the Imaging Pipeline for Computer Vision

Sensor-level computer vision with pixel processor arrays for agile robots

Hypoglycemia during intensified insulin therapy of young children.

Processing Near Sensor Architecture in Mixed-Signal Domain with CMOS Image Sensor of Convolutional-Kernel-Readout Method

An Encoded Photodetector Array for High-frame-rate Compressed Sensing and Computational Imaging

An Analog-Memoryless Near Sensor Computing Architecture for Always-On Intelligent Perception Applications

A high-speed vision processor based on pixel-parallel PE array and its applications

HyNNA: Improved Performance for Neuromorphic Vision Sensor based Surveillance using Hybrid Neural Network Architecture