Abstract:Deep Learning Accelerators are prone to faults which manifest in the form of errors in Neural Networks. Fault Tolerance in Neural Networks is crucial in real-time safety critical applications requiring computation for long durations. Neural Networks with high regularisation exhibit superior fault tolerance, however, at the cost of classification accuracy. In the view of difference in functionality, a Neural Network is modelled as two separate networks, i.e, the Feature Extractor with unsupervised learning objective and the Classifier with a supervised learning objective. Traditional approaches of training the entire network using a single supervised learning objective is insufficient to achieve the objectives of the individual components optimally. In this work, a novel multi-criteria objective function, combining unsupervised training of the Feature Extractor followed by supervised tuning with Classifier Network is proposed. The unsupervised training solves two games simultaneously in the presence of adversary neural networks with conflicting objectives to the Feature Extractor. The first game minimises the loss in reconstructing the input image for indistinguishability given the features from the Extractor, in the presence of a generative decoder. The second game solves a minimax constraint optimisation for distributional smoothening of feature space to match a prior distribution, in the presence of a Discriminator network. The resultant strongly regularised Feature Extractor is combined with the Classifier Network for supervised fine-tuning. The proposed Adversarial Fault Tolerant Neural Network Training is scalable to large networks and is independent of the architecture. The evaluation on benchmarking datasets: FashionMNIST and CIFAR10, indicates that the resultant networks have high accuracy with superior tolerance to stuck at "0" faults compared to widely used regularisers.

An Autonomous Error-Tolerant Architecture Featuring Self-reparation for Convolutional Neural Networks

Algorithm-Based Fault Tolerance for Convolutional Neural Networks

Designing Bio-Inspired Autonomous Error-Tolerant Massively Parallel Computing Architectures

Soft Error Tolerant Convolutional Neural Networks on FPGAs with Ensemble Learning

A Design of Autonomous Error-Tolerant Architectures for Massively Parallel Computing

Implementation of Highly Reliable Convolutional Neural Network with Low Overhead on Field-Programmable Gate Array

FTT-NAS: Discovering Fault-Tolerant Convolutional Neural Architecture

Soft Error Mitigation for Deep Convolution Neural Network on FPGA Accelerators.

Efficient Error-Tolerant Quantized Neural Network Accelerators

Improving Model Robustness Against Adversarial Examples with Redundant Fully Connected Layer.

Detect and Replace: Efficient Soft Error Protection of FPGA-Based CNN Accelerators

Reliable Classification with Ensemble Convolutional Neural Networks.

Functional Error Correction for Robust Neural Networks

Towards Enhancing Fault Tolerance in Neural Networks

FPGA Implementation of a Fault-Tolerant Fused and Branched CNN Accelerator With Reconfigurable Capabilities

Automated design of error-resilient and hardware-efficient deep neural networks

Fault-Tolerant Neural Networks from Biological Error Correction Codes

Artificial neural networks for online error detection

Systematic Reliability Evaluation of FPGA Implemented CNN Accelerators

Comparative study: AutoDPR-SEM for enhancing CNN reliability in SRAM-based FPGAs through autonomous reconfiguration

Magnetoresistive RAM for error resilient XNOR-Nets