Abstract:Deep Learning (DL) systems have proliferated in many applications, requiring specialized hardware accelerators and chips. In the nano-era, devices have become increasingly more susceptible to permanent and transient faults. Therefore, we need an efficient methodology for analyzing the resilience of advanced DL systems against such faults, and understand how the faults in neural accelerator chips manifest as errors at the DL application level, where faults can lead to undetectable and unrecoverable errors. Using fault injection, we can perform resilience investigations of the DL system by modifying neuron weights and outputs at the software-level, as if the hardware had been affected by a transient fault. Existing fault models reduce the search space, allowing faster analysis, but requiring a-priori knowledge on the model, and not allowing further analysis of the filtered-out search space. Therefore, we propose ISimDL, a novel methodology that employs neuron sensitivity to generate importance sampling-based fault-scenarios. Without any a-priori knowledge of the model-under-test, ISimDL provides an equivalent reduction of the search space as existing works, while allowing long simulations to cover all the possible faults, improving on existing model requirements. Our experiments show that the importance sampling provides up to 15x higher precision in selecting critical faults than the random uniform sampling, reaching such precision in less than 100 faults. Additionally, we showcase another practical use-case for importance sampling for reliable DNN design, namely Fault Aware Training (FAT). By using ISimDL to select the faults leading to errors, we can insert the faults during the DNN training process to harden the DNN against such faults. Using importance sampling in FAT reduces the overhead required for finding faults that lead to a predetermined drop in accuracy by more than 12x.

Machine Learning Methodologies to Predict the Results of Simulation-Based Fault Injection

Analog Circuit Simulation Approach with Multi-Fault Injector

Composing Graph Theory and Deep Neural Networks to Evaluate SEU Type Soft Error Effects

Use Neural Network to Improve Fault Injection Testing.

A Novel Method for Analog Fault Diagnosis Based on Neural Networks and Genetic Algorithms

A Novel Meta-Learning and Network Architecture Search Approach for Few-Shot High-Voltage Circuit Breaker Fault Diagnosis

Methodology of Fault Modeling,Injection and Judgment of Circuit Fault Simulation

An Automated Technique for Circuit-Level Simulation-Based Soft Fault Injection

Strategy for Identifying Analog Circuit Faults Using Improved Neural Network Algorithms

Simics-based System Level Fault Injection Platform

Fault Injection on Embedded Neural Networks: Impact of a Single Instruction Skip

Research on Fault Prediction of Nuclear Safety Class Crucial Circuit Based on SDAE- MKRVM

SpikingJET: Enhancing Fault Injection for Fully and Convolutional Spiking Neural Networks

A Neural Network Method For Multiple Fault Detection And Diagnosis Of Digital Circuits

A Deep Learning framework for simulation and defect prediction applied in microelectronics

DeepVigor+: Scalable and Accurate Semi-Analytical Fault Resilience Analysis for Deep Neural Network

Characterizing a Neutron-Induced Fault Model for Deep Neural Networks

Versatile and Efficient Techniques for Speeding-Up Circuit Level Simulated Fault-Injection Campaigns

Large-Scale Application of Fault Injection into PyTorch Models -- an Extension to PyTorchFI for Validation Efficiency

GNN4REL: Graph Neural Networks for Predicting Circuit Reliability Degradation

ISimDL: Importance Sampling-Driven Acceleration of Fault Injection Simulations for Evaluating the Robustness of Deep Learning