Abstract:Deep Learning Accelerators are prone to faults which manifest in the form of errors in Neural Networks. Fault Tolerance in Neural Networks is crucial in real-time safety critical applications requiring computation for long durations. Neural Networks with high regularisation exhibit superior fault tolerance, however, at the cost of classification accuracy. In the view of difference in functionality, a Neural Network is modelled as two separate networks, i.e, the Feature Extractor with unsupervised learning objective and the Classifier with a supervised learning objective. Traditional approaches of training the entire network using a single supervised learning objective is insufficient to achieve the objectives of the individual components optimally. In this work, a novel multi-criteria objective function, combining unsupervised training of the Feature Extractor followed by supervised tuning with Classifier Network is proposed. The unsupervised training solves two games simultaneously in the presence of adversary neural networks with conflicting objectives to the Feature Extractor. The first game minimises the loss in reconstructing the input image for indistinguishability given the features from the Extractor, in the presence of a generative decoder. The second game solves a minimax constraint optimisation for distributional smoothening of feature space to match a prior distribution, in the presence of a Discriminator network. The resultant strongly regularised Feature Extractor is combined with the Classifier Network for supervised fine-tuning. The proposed Adversarial Fault Tolerant Neural Network Training is scalable to large networks and is independent of the architecture. The evaluation on benchmarking datasets: FashionMNIST and CIFAR10, indicates that the resultant networks have high accuracy with superior tolerance to stuck at "0" faults compared to widely used regularisers.

Fault models for artificial neural networks

Application of Neural Networks to Chemical Fault Diagnosis

Interpretability Based Neural Network Repair

Objective Function and Learning Algorithm for the General Node Fault Situation

Improving Fault Tolerance for Reliable DNN Using Boundary-Aware Activation

Using neural networks for fault diagnosis

Evolving Fault-Tolerant Neural Networks.

A FAULT DIAGNOSTIC MODEL BASED ON NOVEL NEURAL NETWORK CLASSIFIER

Fault-Tolerant Neural Networks from Biological Error Correction Codes

Improving Tolerance of Neural Networks Against Multi-Node Open Fault

Towards Enhancing Fault Tolerance in Neural Networks

Fault-Tolerant Analysis of Feedback Neural Networks with Threshold Neurons.

The superior fault tolerance of artificial neural network training with a fault/noise injection-based genetic algorithm

Characterizing a Neutron-Induced Fault Model for Deep Neural Networks

Exposing Reliability Degradation and Mitigation in Approximate DNNs under Permanent Faults

A Design Methodology for Fault-Tolerant Computing using Astrocyte Neural Networks

Fault Tolerance Based on Neural Networks for the Intelligent Distributed Framework

FTT-NAS: Discovering Fault-Tolerant Convolutional Neural Architecture

Faulty behavior of feedforward neural networks

Automatic Fault Detection for Deep Learning Programs Using Graph Transformations

A Neural Network Approach to Fault Diagnosis in Analog Circuits.