Abstract:Lifelong machine learning can learn a sequence of consecutive robotic perception tasks via transferring previous experiences. However, 1) most existing lifelong learning based perception methods only take advantage of visual information for robotic tasks, while neglecting another important tactile sensing modality to capture discriminative material properties; 2) Meanwhile, they cannot explore the intrinsic relationships across different modalities and the common characterization among different tasks of each modality, due to the distinct divergence between heterogeneous feature distributions. To address above challenges, we propose a new Lifelong Visual-Tactile Learning (LVTL) model for continuous robotic visual-tactile perception tasks, which fully explores the latent correlations in both intra-modality and cross-modality aspects. Specifically, a modality-specific knowledge library is developed for each modality to explore common intra-modality representations across different tasks, while narrowing intra-modality mapping divergence between semantic and feature spaces via an auto-encoder mechanism. Moreover, a sparse constraint based modality-invariant space is constructed to capture underlying cross-modality correlations and identify the contributions of each modality for new coming visual-tactile tasks. We further propose a modality consistence regularizer to efficiently align the heterogeneous visual and tactile samples, which ensures the semantic consistence between different modality-specific knowledge libraries. After deriving an efficient model optimization strategy, we conduct extensive experiments on several representative datasets to validate the superiority of our LVTL model. Comparison experiments show that our proposed model significantly outperforms existing state-of-the-art methods with about 1.29%<math>∼</math>15.36% improvement under different lifelong visual-tactile perception scenarios.

A Cross-Domain Lifelong Learning Model for Visual Understanding

MVDC: A Multi-View Domain Confusion Strategy for Cross Domain Change Detection

Life-long Cross-media Correlation Learning

Lifelong robotic visual-tactile perception learning

Learning a Deep Structural Subspace Across Hyperspectral Scenes with Cross-Domain VAE

Advancing Cross-domain Discriminability in Continual Learning of Vison-Language Models

Cross-Domain Collaborative Learning in Social Multimedia

Manifold Regularized Cross-Modal Embedding for Zero-Shot Learning

Cross-Domain Collaborative Learning Via Discriminative Nonparametric Bayesian Model

Advancing Cross-domain Discriminability in Continual Learning of Vision-Language Models

Lifelong-MonoDepth: Lifelong Learning for Multi-Domain Monocular Metric Depth Estimation

Depthwise Convolution is All You Need for Learning Multiple Visual Domains

Cross-lingual Lifelong Learning

Enhancing few-shot lifelong learning through fusion of cross-domain knowledge

Cross-Modality Bridging and Knowledge Transferring for Image Understanding

Hierarchical Visual-Textual Knowledge Distillation for Life-Long Correlation Learning

VLLaVO: Mitigating Visual Gap through LLMs

DualCross: Cross-Modality Cross-Domain Adaptation for Monocular BEV Perception

Cross-Domain Web Image Annotation

Cross-Domain Visual Matching via Generalized Similarity Measure and Feature Learning

Cross-Domain Feature Learning in Multimedia