Abstract:A deep classifier is usually trained to (i) learn the numeric representation vector of samples and (ii) classify sample representations with learned classification boundaries. Time-travelling visualization, as an explainable AI technique, is designed to transform the model training dynamics into an animation of canvas with colorful dots and territories. Despite that the training dynamics of the highlevel concepts such as sample representations and classification boundaries are now observable, the model developers can still be overwhelmed by tens of thousands of moving dots across hundreds of training epochs (i.e., frames in the animation), which makes them miss important training events. In this work, we make the first attempt to develop the model time-travelling visualizers to the model time-travelling debuggers, for its practical use in model debugging tasks. Specifically, given an animation of model training dynamics of sample representation and classification landscape, we propose DeepDebugger solution to recommend the samples of user interest in a human-in-the-loop manner. On one hand, DeepDebugger monitors the training dynamics of samples and recommends suspicious samples based on their abnormality. On the other hand, our recommendation is interactive and fault-resilient for the model developers to explore the training process. By learning users' feedback, DeepDebugger refines its recommendation to fit their intention. Our extensive experiments on applying DeepDebugger on the known time-travelling visualizers show that DeepDebugger can (1) detect the majority of the abnormal movement of the training samples on canvas; (2) significantly boost the recommendation performance of samples of interest (5-10X more accurate than the baselines) with the runtime overhead of 0.015s per feedback; (3) be resilient under the 3%, 5%, 10% mistaken user feedback. Our user study of the tool shows that the interactive recommendation of DeepDebugger can help the participants accomplish the debugging tasks by saving 18.1% completion time and boosting the performance by 20.3%.

Aspect-based Sentiment Analysis with Opinion Tree Generation

DeepVisualInsight: Time-Travelling Visualization for Spatio-Temporal Causality of Deep Classification Training

Unifying Terrain Awareness Through Real-Time Semantic Segmentation

Structure-preserving visualization for single-cell RNA-Seq profiles using deep manifold transformation with batch-correction

Interpretable and Generalizable Spatiotemporal Predictive Learning with Disentangled Consistency

DeepDebugger: An Interactive Time-Travelling Debugging Approach for Deep Classifiers

Visualizing Large-Scale Spatial Time Series with GeoChron.

Representation and analysis of time-series data via deep embedding and visual exploration

ViTime: A Visual Intelligence-Based Foundation Model for Time Series Forecasting

Visual Analytics of Taxi Trajectory Data Via Topical Sub-trajectories.

Image-Based Storytelling Using Deep Learning.

Learn Decision Trees with Deep Visual Primitives

SemanticTraj: A New Approach to Interacting with Massive Taxi Trajectories.

TripVista: Triple Perspective Visual Trajectory Analytics and Its Application on Microscopic Traffic Data at a Road Intersection

Optimizing temporal topic segmentation for intelligent text visualization.

TV-Net: Temporal-Variable feature harmonizing Network for multivariate time series classification and interpretation

Learning Decomposed Spatial Relations for Multi-Variate Time-Series Modeling.

Transforming Multidimensional Time Series into Interpretable Event Sequences for Advanced Data Mining

VGbel: an Exploration of Ensemble Learning Incorporating Non-Euclidean Structural Representation for Time Series Classification

VisTR: Visualizations as Representations for Time-series Table Reasoning

DeepEye: Towards Automatic Data Visualization