Abstract:Deep Learning Accelerators are prone to faults which manifest in the form of errors in Neural Networks. Fault Tolerance in Neural Networks is crucial in real-time safety critical applications requiring computation for long durations. Neural Networks with high regularisation exhibit superior fault tolerance, however, at the cost of classification accuracy. In the view of difference in functionality, a Neural Network is modelled as two separate networks, i.e, the Feature Extractor with unsupervised learning objective and the Classifier with a supervised learning objective. Traditional approaches of training the entire network using a single supervised learning objective is insufficient to achieve the objectives of the individual components optimally. In this work, a novel multi-criteria objective function, combining unsupervised training of the Feature Extractor followed by supervised tuning with Classifier Network is proposed. The unsupervised training solves two games simultaneously in the presence of adversary neural networks with conflicting objectives to the Feature Extractor. The first game minimises the loss in reconstructing the input image for indistinguishability given the features from the Extractor, in the presence of a generative decoder. The second game solves a minimax constraint optimisation for distributional smoothening of feature space to match a prior distribution, in the presence of a Discriminator network. The resultant strongly regularised Feature Extractor is combined with the Classifier Network for supervised fine-tuning. The proposed Adversarial Fault Tolerant Neural Network Training is scalable to large networks and is independent of the architecture. The evaluation on benchmarking datasets: FashionMNIST and CIFAR10, indicates that the resultant networks have high accuracy with superior tolerance to stuck at "0" faults compared to widely used regularisers.

Improving Tolerance of Neural Networks Against Multi-Node Open Fault

Interpretability Based Neural Network Repair

Evolving Fault-Tolerant Neural Networks.

Improving Fault Tolerance for Reliable DNN Using Boundary-Aware Activation

Towards Enhancing Fault Tolerance in Neural Networks

Fault-Tolerant Analysis of Feedback Neural Networks with Threshold Neurons.

AN ADAPTIVE FAULT-TOLERANT NEURAL NETWORK CLASSIFIER

Improving fault-tolerance of feedforward neural networks with genetic algorithms

FTT-NAS: Discovering Fault-Tolerant Convolutional Neural Architecture

A Multifault‐Tolerant Training Scheme for Nonideal Memristive Neural Networks

NV-DNN: Towards Fault-Tolerant DNN Systems with N-Version Programming

A Neural Network Approach to Fault Diagnosis in Analog Circuits.

The superior fault tolerance of artificial neural network training with a fault/noise injection-based genetic algorithm

A Modified Learning Algorithm for Improving the Fault Tolerance of BP Networks

Faulty behavior of feedforward neural networks

Enhancing Fault Resilience of QNNs by Selective Neuron Splitting

Objective Function and Learning Algorithm for the General Node Fault Situation

Using neural networks for fault diagnosis

Fault Tolerance Based on Neural Networks for the Intelligent Distributed Framework

Arithmetic-intensity-guided fault tolerance for neural network inference on GPUs

Exploring Winograd Convolution for Cost-effective Neural Network Fault Tolerance