Diffchaser: Detecting disagreements for deep neural networks

Xiaofei Xie, Lei Ma, Haijun Wang, Yuekang Li, Yang Liu, Xiaohong Li
2019-01-01
Abstract:The platform migration and customization have become an indispensable process of deep neural network (DNN) development lifecycle. A highprecision but complex DNN trained in the cloud on massive data and powerful GPUs often goes through an optimization phase (eg, quantization, compression) before deployment to a target device (eg, mobile device). A test set that effectively uncovers the disagreements of a DNN and its optimized variant provides certain feedback to debug and further enhance the optimization procedure. However, the minor inconsistency between a DNN and its optimized version is often hard to detect and easily bypasses the original test set. This paper proposes DiffChaser, an automated black-box testing framework to detect untargeted/targeted disagreements between version variants of a DNN. We demonstrate 1) its effectiveness by comparing with the state-of-the-art techniques, and 2) its usefulness in real-world DNN product deployment involved with quantization and optimization.
What problem does this paper attempt to address?