Abstract:Lifelong machine learning can learn a sequence of consecutive robotic perception tasks via transferring previous experiences. However, 1) most existing lifelong learning based perception methods only take advantage of visual information for robotic tasks, while neglecting another important tactile sensing modality to capture discriminative material properties; 2) Meanwhile, they cannot explore the intrinsic relationships across different modalities and the common characterization among different tasks of each modality, due to the distinct divergence between heterogeneous feature distributions. To address above challenges, we propose a new Lifelong Visual-Tactile Learning (LVTL) model for continuous robotic visual-tactile perception tasks, which fully explores the latent correlations in both intra-modality and cross-modality aspects. Specifically, a modality-specific knowledge library is developed for each modality to explore common intra-modality representations across different tasks, while narrowing intra-modality mapping divergence between semantic and feature spaces via an auto-encoder mechanism. Moreover, a sparse constraint based modality-invariant space is constructed to capture underlying cross-modality correlations and identify the contributions of each modality for new coming visual-tactile tasks. We further propose a modality consistence regularizer to efficiently align the heterogeneous visual and tactile samples, which ensures the semantic consistence between different modality-specific knowledge libraries. After deriving an efficient model optimization strategy, we conduct extensive experiments on several representative datasets to validate the superiority of our LVTL model. Comparison experiments show that our proposed model significantly outperforms existing state-of-the-art methods with about 1.29%<math>∼</math>15.36% improvement under different lifelong visual-tactile perception scenarios.

Proper Reuse of Features Extractor for Real-time Continual Learning

RECL: Responsive Resource-Efficient Continuous Learning for Video Analytics

Recurrent Volume-based 3D Feature Fusion for Real-time Multi-view Object Pose Estimation

Preserving Earlier Knowledge in Continual Learning with the Help of All Previous Feature Extractors

Adaptive online continual multi-view learning

Online Active Continual Learning for Robotic Lifelong Object Recognition

Effective Continual Learning for Text Classification with Lightweight Snapshots.

Lifelong robotic visual-tactile perception learning

Online Continual Learning Via Candidates Voting

Look-Ahead Selective Plasticity for Continual Learning of Visual Tasks

Adaptive Progressive Continual Learning.

Recent Advances of Continual Learning in Computer Vision: An Overview

Continual Panoptic Perception: Towards Multi-modal Incremental Interpretation of Remote Sensing Images

Long-Term Visual Object Tracking Via Continual Learning

Continual Learning with Strong Experience Replay

Continual Predictive Learning from Videos

Adaptive Rentention & Correction for Continual Learning

Exemplar-free Continual Learning of Vision Transformers via Gated Class-Attention and Cascaded Feature Drift Compensation

Continual Recognition with Adaptive Memory Update.

Continual Learning of Image Classes with Language Guidance from a Vision-Language Model

Dynamic Memory-Based Continual Learning with Generating and Screening