DeepXplore: Automated Whitebox Testing of Deep Learning Systems

Kexin Pei,Yinzhi Cao,Junfeng Yang,Suman Jana

DOI: https://doi.org/10.1145/3132747.3132785

2017-09-24

Abstract:Deep learning (DL) systems are increasingly deployed in safety- and security-critical domains including self-driving cars and malware detection, where the correctness and predictability of a system's behavior for corner case inputs are of great importance. Existing DL testing depends heavily on manually labeled data and therefore often fails to expose erroneous behaviors for rare inputs. We design, implement, and evaluate DeepXplore, the first whitebox framework for systematically testing real-world DL systems. First, we introduce neuron coverage for systematically measuring the parts of a DL system exercised by test inputs. Next, we leverage multiple DL systems with similar functionality as cross-referencing oracles to avoid manual checking. Finally, we demonstrate how finding inputs for DL systems that both trigger many differential behaviors and achieve high neuron coverage can be represented as a joint optimization problem and solved efficiently using gradient-based search techniques. DeepXplore efficiently finds thousands of incorrect corner case behaviors (e.g., self-driving cars crashing into guard rails and malware masquerading as benign software) in state-of-the-art DL models with thousands of neurons trained on five popular datasets including ImageNet and Udacity self-driving challenge data. For all tested DL models, on average, DeepXplore generated one test input demonstrating incorrect behavior within one second while running only on a commodity laptop. We further show that the test inputs generated by DeepXplore can also be used to retrain the corresponding DL model to improve the model's accuracy by up to 3%.

Machine Learning,Cryptography and Security,Software Engineering

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to automatically detect misbehaviors in deep - learning systems, especially in safety - and mission - critical tasks, such as self - driving cars and malware detection. Existing deep - learning testing methods rely on manually - labeled data, which often fails to expose misbehaviors under rare inputs. Therefore, the paper proposes a white - box framework named DeepXplore, aiming to systematically test actual deep - learning systems to discover misbehaviors that these systems may exhibit when handling edge - case inputs.

Specifically, DeepXplore solves the problem in the following ways:

1. **Introducing Neuron Coverage**: As a new test metric, it is used to measure the number of neurons activated by test inputs, thereby evaluating the extent to which test inputs cover the logic of the deep - learning system.

2. **Utilizing Multiple Deep - Learning Systems with Similar Functions as Cross - Referenced Oracles**: This avoids the need for manual inspection and identifies potential misbehaviors by comparing the outputs of different systems.

3. **Representing the Search for Inputs that Can Trigger Multiple Differential Behaviors and Have High Neuron Coverage as a Joint Optimization Problem**: And efficiently solve this problem through gradient - based search techniques.

Through these methods, DeepXplore can efficiently discover thousands of incorrect edge - case behaviors in large - scale actual deep - learning systems, and the generated test inputs can also be used to retrain the model and improve the model's accuracy.