Abstract:Lifelong machine learning can learn a sequence of consecutive robotic perception tasks via transferring previous experiences. However, 1) most existing lifelong learning based perception methods only take advantage of visual information for robotic tasks, while neglecting another important tactile sensing modality to capture discriminative material properties; 2) Meanwhile, they cannot explore the intrinsic relationships across different modalities and the common characterization among different tasks of each modality, due to the distinct divergence between heterogeneous feature distributions. To address above challenges, we propose a new Lifelong Visual-Tactile Learning (LVTL) model for continuous robotic visual-tactile perception tasks, which fully explores the latent correlations in both intra-modality and cross-modality aspects. Specifically, a modality-specific knowledge library is developed for each modality to explore common intra-modality representations across different tasks, while narrowing intra-modality mapping divergence between semantic and feature spaces via an auto-encoder mechanism. Moreover, a sparse constraint based modality-invariant space is constructed to capture underlying cross-modality correlations and identify the contributions of each modality for new coming visual-tactile tasks. We further propose a modality consistence regularizer to efficiently align the heterogeneous visual and tactile samples, which ensures the semantic consistence between different modality-specific knowledge libraries. After deriving an efficient model optimization strategy, we conduct extensive experiments on several representative datasets to validate the superiority of our LVTL model. Comparison experiments show that our proposed model significantly outperforms existing state-of-the-art methods with about 1.29%<math>∼</math>15.36% improvement under different lifelong visual-tactile perception scenarios.

Evaluation of Continual Learning Methods for Object Hardness Recognition

Continual Learning in Human Activity Recognition: an Empirical Analysis of Regularization

Towards Lifelong Object Recognition: A Dataset and Benchmark

Is Continual Learning Ready for Real-world Challenges?

Lifelong Visual-Tactile Cross-Modal Learning for Robotic Material Perception

Continual Learning through Human-Robot Interaction -- Human Perceptions of a Continual Learning Robot in Repeated Interactions

Lifelong robotic visual-tactile perception learning

Probabilistic Spiking Neural Network for Robotic Tactile Continual Learning

OpenLORIS-Object: A Dataset and Benchmark towards Lifelong Object Recognition

CLFR-M: Continual Learning Framework for Robots Via Human Feedback and Dynamic Memory

Online Active Continual Learning for Robotic Lifelong Object Recognition

Towards Robust Evaluations of Continual Learning

Improving Robotic Tactile Localization Super-resolution via Spatiotemporal Continuity Learning and Overlapping Air Chambers

The Role of Tactile Sensing in Learning and Deploying Grasp Refinement Algorithms

A Review of Continual Learning for Robotics

The Empirical Impact of Forgetting and Transfer in Continual Visual Odometry

OpenLORIS-Object: A Robotic Vision Dataset and Benchmark for Lifelong Deep Learning

Recent progress on tactile object recognition

Don't Stop Learning: Towards Continual Learning for the CLIP Model

Benchmarking Continual Learning from Cognitive Perspectives

Continual Predictive Learning from Videos