Abstract:The mnemonic descent method (MDM) algorithm is the first end-to-end recurrent convolutional system for high-accuracy face alignment. However, the heavy computational complexity and high memory access demands make it difficult to satisfy the requirements of real-time applications. To address this problem, an improved MDM (I-MDM) algorithm is proposed for efficient hardware implementation based on several hardware-oriented optimizations. First, a patch merging mechanism is introduced to dynamically cluster and eliminate redundant landmarks, which significantly reduces computational complexity with minimal accuracy loss. Second, a dedicated convolutional layer is inserted to halve the number of computations and memory access of the subsequent fully connected layer, yielding a 4.42% decrease in the failure rate. Third, a lightweight preprocessing method named dual regressors is proposed to reinitialize face images, which can greatly improve the overall accuracy. Moreover, compared with a similar method, the DR method can reduce computations and memory storage by nearly 99.9%. Overall and compared with the MDM algorithm, I-MDM not only reduces the number of computations by 23.5% but also decreases the failure rate by 17.9% on the 300 W test set. Based on the proposed I-MDM algorithm, an I-MDM-based hardwired accelerator is presented using the TSMC 65 nm CMOS process. First, compared with similar solutions, the gradient calculation operation is rearranged and loaded pixels are reused in the HoG feature extraction to eliminate all division operations and 25% off-chip memory access. Second, patch-independent central activations are used to enable patch-level pipelined operations, yielding a 2× acceleration in the overall process. This accelerator achieves 460 GOPS/W energy efficiency at 330 MHz, which is 38× higher than the most recent face alignment accelerator with the same process.

A 182 mW 94.3 f/s in Full HD Pattern-Matching Based Image Recognition Accelerator for an Embedded Vision System in 0.13-$\mu{\rm m}$ CMOS Technology

High-performance Reconfigurable DNN Accelerator on a Bandwidth-limited Embedded System

A Real-Time 17-Scale Object Detection Accelerator with Adaptive 2000-Stage Classification in 65nm CMOS.

A Binary-Feature-Based Object Recognition Accelerator with 22 M-Vector/s Throughput and 0.68 G-Vector/J Energy-Efficiency for Full-HD Resolution

A Multi-Mode Visual Recognition Hardware Accelerator for AR/MR Glasses

A high-speed vision processor based on pixel-parallel PE array and its applications

A 1000 Fps Vision Chip Based on a Dynamically Reconfigurable Hybrid Architecture Comprising a PE Array Processor and Self-Organizing Map Neural Network.

A 135-Frames/s 1080p 87.5-Mw Binary-Descriptor-Based Image Feature Extraction Accelerator

PISA: A Binary-Weight Processing-In-Sensor Accelerator for Edge Image Processing

Lightweight and Real-Time Infrared Image Processor Based on FPGA

A 460 GOPS/W Improved Mnemonic Descent Method-Based Hardwired Accelerator for Face Alignment.

Ultrahigh Framerate Vision Chip Featuring Central-Based Edge Detection Processed by All-Digital In-Imager Global-Parallel Processing Architecture

A high speed multi-level-parallel array processor for vision chips

ACNPU: A 4.75TOPS/W 1080P@30FPS Super Resolution Accelerator with Decoupled Asymmetric Convolution

A 127 fps in full hd accelerator based on optimized AKAZE with efficiency and effectiveness for image feature extraction

A 73.53TOPS/W 14.74TOPS Heterogeneous RRAM In-Memory and SRAM Near-Memory SoC for Hybrid Frame and Event-Based Target Tracking

A 181 GOPS AKAZE Accelerator Employing Discrete-Time Cellular Neural Networks for Real-Time Feature Extraction

Hardware Architecture for Fast General Object Detection using Aggregated Channel Features

A 92 fps and 2.56 mJ/frame Computing-in-Memory-based Human Pose Estimation Accelerator with Resource-Efficient Macro for Mobile Devices

7.3 A 1000fps Vision Chip Based on a Dynamically Reconfigurable Hybrid Architecture Comprising a PE Array and Self-Organizing Map Neural Network

A 22nm 3.5TOPS/W Flexible Micro-Robotic Vision SoC with 2MB Emram for Fully-on-Chip Intelligence.