Abstract:Deep Learning Accelerators are prone to faults which manifest in the form of errors in Neural Networks. Fault Tolerance in Neural Networks is crucial in real-time safety critical applications requiring computation for long durations. Neural Networks with high regularisation exhibit superior fault tolerance, however, at the cost of classification accuracy. In the view of difference in functionality, a Neural Network is modelled as two separate networks, i.e, the Feature Extractor with unsupervised learning objective and the Classifier with a supervised learning objective. Traditional approaches of training the entire network using a single supervised learning objective is insufficient to achieve the objectives of the individual components optimally. In this work, a novel multi-criteria objective function, combining unsupervised training of the Feature Extractor followed by supervised tuning with Classifier Network is proposed. The unsupervised training solves two games simultaneously in the presence of adversary neural networks with conflicting objectives to the Feature Extractor. The first game minimises the loss in reconstructing the input image for indistinguishability given the features from the Extractor, in the presence of a generative decoder. The second game solves a minimax constraint optimisation for distributional smoothening of feature space to match a prior distribution, in the presence of a Discriminator network. The resultant strongly regularised Feature Extractor is combined with the Classifier Network for supervised fine-tuning. The proposed Adversarial Fault Tolerant Neural Network Training is scalable to large networks and is independent of the architecture. The evaluation on benchmarking datasets: FashionMNIST and CIFAR10, indicates that the resultant networks have high accuracy with superior tolerance to stuck at "0" faults compared to widely used regularisers.

A Modified Learning Algorithm for Improving the Fault Tolerance of BP Networks

Improved BP algorithm for neural network and its application in welding process control

Unsteady Fault Diagnosis Based on Wavelet-Networks

Improving Fault Tolerance for Reliable DNN Using Boundary-Aware Activation

A Learning Algorithm for Solving the Over-Fitting Problem in BPANN

An Constraint-based Clustering Integrated BP Networks Approach for Analog Circuit Fault Diagnose

Evolving Neural Networks Using the Hybrid of Ant Colony Optimization and Bp Algorithms

Study of Bp Network for A Cylinder Shell'S Support Identification

Improved shuffled frog leaping algorithm-based BP neural network and its application in bearing early fault diagnosis

Brief Introduction of Back Propagation (BP) Neural Network Algorithm and Its Improvement

Towards Enhancing Fault Tolerance in Neural Networks

Objective Function and Learning Algorithm for the General Node Fault Situation

Effects of the Learning Algorithm on the Faulty Behavior of Feedback Neural Networks

AN ADAPTIVE FAULT-TOLERANT NEURAL NETWORK CLASSIFIER

A Fast Learning Algorithm of Neural Networks by Changing Error Functions

Evolutionary Weighted Broad Learning and Its Application to Fault Diagnosis in Self-Organizing Cellular Networks

Fault Location of Distribution Network Based on Back Propagation Neural Network Optimization Algorithm

Improving Tolerance of Neural Networks Against Multi-Node Open Fault

Evolving Fault-Tolerant Neural Networks.

The Improved Rapid Convergence Algorithm of the Connecting Rights in the BP Network

Improved BP Neural Network for Transformer Fault Diagnosis