Abstract:A deep classifier is usually trained to (i) learn the numeric representation vector of samples and (ii) classify sample representations with learned classification boundaries. Time-travelling visualization, as an explainable AI technique, is designed to transform the model training dynamics into an animation of canvas with colorful dots and territories. Despite that the training dynamics of the highlevel concepts such as sample representations and classification boundaries are now observable, the model developers can still be overwhelmed by tens of thousands of moving dots across hundreds of training epochs (i.e., frames in the animation), which makes them miss important training events. In this work, we make the first attempt to develop the model time-travelling visualizers to the model time-travelling debuggers, for its practical use in model debugging tasks. Specifically, given an animation of model training dynamics of sample representation and classification landscape, we propose DeepDebugger solution to recommend the samples of user interest in a human-in-the-loop manner. On one hand, DeepDebugger monitors the training dynamics of samples and recommends suspicious samples based on their abnormality. On the other hand, our recommendation is interactive and fault-resilient for the model developers to explore the training process. By learning users' feedback, DeepDebugger refines its recommendation to fit their intention. Our extensive experiments on applying DeepDebugger on the known time-travelling visualizers show that DeepDebugger can (1) detect the majority of the abnormal movement of the training samples on canvas; (2) significantly boost the recommendation performance of samples of interest (5-10X more accurate than the baselines) with the runtime overhead of 0.015s per feedback; (3) be resilient under the 3%, 5%, 10% mistaken user feedback. Our user study of the tool shows that the interactive recommendation of DeepDebugger can help the participants accomplish the debugging tasks by saving 18.1% completion time and boosting the performance by 20.3%.

DeepVisualInsight: Time-Travelling Visualization for Spatio-Temporal Causality of Deep Classification Training

Aspect-based Sentiment Analysis with Opinion Tree Generation

Deep Clustering and Visualization for End-to-End High-Dimensional Data Analysis.

Structure-preserving visualization for single-cell RNA-Seq profiles using deep manifold transformation with batch-correction

DeepDebugger: An Interactive Time-Travelling Debugging Approach for Deep Classifiers

Time-Dependent VAE for Building Latent Representations from Visual Neural Activity with Complex Dynamics

An In-Situ Visual Analytics Framework for Deep Neural Networks

VisuaLizations As Intermediate Representations (VLAIR): an Approach for Applying Deep Learning-Based Computer Vision to Non-Image-based Data

DeepEye: Towards Automatic Data Visualization

NeuralVis: Visualizing and Interpreting Deep Learning Models

Interpreting Deep Classifier by Visual Distillation of Dark Knowledge.

On the Integration of Large-Scale Time Series Distance Matrices Into Deep Visual Analytic Tools

Illustrative Visualization of Time-Varying Features in Spatio-Temporal Data.

V-CNN: when Convolutional Neural Network Encounters Data Visualization.

Representation and analysis of time-series data via deep embedding and visual exploration

Deep Learning Driven Visual Path Prediction from a Single Image.

A Self-Interpretable Deep Learning Soft Sensor Based on Spatio-Temporal Causal Discovery Graph

Temporal-Spatial Causal Interpretations for Vision-Based Reinforcement Learning

Visual Point Cloud Forecasting enables Scalable Autonomous Driving

Mapping the Learning Curves of Deep Learning Networks

IntruDTS: Interactive Visual Analysis System for Intrusion Detection in Time Series