Abstract:Conventional DNN training paradigms typically rely on one training set and one validation set, obtained by partitioning an annotated dataset used for training, namely gross training set, in a certain way. The training set is used for training the model while the validation set is used to estimate the generalization performance of the trained model as the training proceeds to avoid over-fitting. There exist two major issues in this paradigm. Firstly, the validation set may hardly guarantee an unbiased estimate of generalization performance due to potential mismatching with test data. Secondly, training a DNN corresponds to solve a complex optimization problem, which is prone to getting trapped into inferior local optima and thus leads to undesired training results. To address these issues, we propose a novel DNN training framework. It generates multiple pairs of training and validation sets from the gross training set via random splitting, trains a DNN model of a pre-specified structure on each pair while making the useful knowledge (e.g., promising network parameters) obtained from one model training process to be transferred to other model training processes via multi-task optimization, and outputs the best, among all trained models, which has the overall best performance across the validation sets from all pairs. The knowledge transfer mechanism featured in this new framework can not only enhance training effectiveness by helping the model training process to escape from local optima but also improve on generalization performance via implicit regularization imposed on one model training process from other model training processes. We implement the proposed framework, parallelize the implementation on a GPU cluster, and apply it to train several widely used DNN models. Experimental results demonstrate the superiority of the proposed framework over the conventional training paradigm.

Ultraschallgezielte Interventionen an peripheren Nerven: diagnostische und therapeutische Indikationen

Survey on Energy-Efficient Deep Neural Networks for Computer Vision

Deep Clustering and Visualization for End-to-End High-Dimensional Data Analysis.

Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers

NeuralVis: Visualizing and Interpreting Deep Learning Models

Understanding Neural Networks Through Deep Visualization

NeuralSI: Neural Design of Semantic Interaction for Interactive Deep Learning

Image-based Process Monitoring Using Deep Learning Framework

Metallic ureteric stents: early experience.

DeepVisualInsight: Time-Travelling Visualization for Spatio-Temporal Causality of Deep Classification Training

DVTest: Deep Neural Network Visualization Testing Framework

Towards Efficient Visual Simplification of Computational Graphs in Deep Neural Networks

Towards Better Analysis of Deep Convolutional Neural Networks

iNNspector: Visual, Interactive Deep Model Debugging

Visual Analytics for Explainable Deep Learning

Distributed Neural Representation for Reactive in situ Visualization

Visual Analysis of Prediction Uncertainty in Neural Networks for Deep Image Synthesis

A Novel DNN Training Framework via Data Sampling and Multi-Task Optimization

A Novel Neural Network Training Framework with Data Assimilation

Toward Efficient and Adaptive Design of Video Detection System with Deep Neural Networks

Analyzing the Training Processes of Deep Generative Models.