Abstract:Curriculum learning and self-paced learning are the training strategies that gradually feed the samples from easy to more complex. They have captivated increasing attention due to their excellent performance in robotic vision. Most recent works focus on designing curricula based on difficulty levels in input samples or smoothing the feature maps. However, smoothing labels to control the learning utility in a curriculum manner is still unexplored. In this work, we design a paced curriculum by label smoothing (P-CBLS) using paced learning with uniform label smoothing (ULS) for classification tasks and fuse uniform and spatially varying label smoothing (SVLS) for semantic segmentation tasks in a curriculum manner. In ULS and SVLS, a bigger smoothing factor value enforces a heavy smoothing penalty in the true label and limits learning less information. Therefore, we design the curriculum by label smoothing (CBLS). We set a bigger smoothing value at the beginning of training and gradually decreased it to zero to control the model learning utility from lower to higher. We also designed a confidence-aware pacing function and combined it with our CBLS to investigate the benefits of various curricula. The proposed techniques are validated on four robotic surgery datasets of multi-class, multi-label classification, captioning, and segmentation tasks. We also investigate the robustness of our method by corrupting validation data into different severity levels. Our extensive analysis shows that the proposed method improves prediction accuracy and robustness. The code is publicly available at https://github.com/XuMengyaAmy/P-CBLS. Note to Practitioners—The motivation of this article is to improve the performance and robustness of deep neural networks in safety-critical applications such as robotic surgery by controlling the learning ability of the model in a curriculum learning manner and allowing the model to imitate the cognitive process of humans and animals. The designed approaches do not add parameters that require additional computational resources.

Self-Paced Boost Learning for Classification.

SPLBoost: An Improved Robust Boosting Algorithm Based on Self-Paced Learning

Multi-class AdaBoost ELM

Multiclass AdaBoost ELM and Its Application in LBP Based Face Recognition

Self-Paced Balance Learning for Clinical Skin Disease Recognition

What Objective Does Self-paced Learning Indeed Optimize?

A Robust Semi-Supervised Broad Learning System Guided by Ensemble-Based Self-Training

Self-paced Ensemble for Highly Imbalanced Massive Data Classification

Contrastive Learning with Boosted Memorization.

A Scalable Stagewise Approach to Large-Margin Multiclass Loss-Based Boosting

Bridging the Gap: Learning Pace Synchronization for Open-World Semi-Supervised Learning

Balanced Self-Paced Learning for AUC Maximization

BalancedBoost: A hybrid approach for real-time network traffic classification

Confidence-Aware Paced-Curriculum Learning by Label Smoothing for Surgical Scene Understanding

Efficient Cluster-Based Boosting for Semisupervised Classification.

A Direct Approach to Multi-class Boosting and Extensions

Self-PU: Self Boosted and Calibrated Positive-Unlabeled Training

From Pretext to Purpose: Batch-Adaptive Self-Supervised Learning

Self-adaptive cost weights-based support vector machine cost-sensitive ensemble for imbalanced data classification

Boosting Bootstrap Fld Subspaces In Classification

S 2 Match: Self-paced sampling for data-limited semi-supervised learning