Learn to Memorize and to Forget: A Continual Learning Perspective of Dynamic SLAM

Baicheng Li,Zike Yan,Dong Wu,Hanqing Jiang,Hongbin Zha

2024-07-18

Abstract:Simultaneous localization and mapping (SLAM) with implicit neural representations has received extensive attention due to the expressive representation power and the innovative paradigm of continual learning. However, deploying such a system within a dynamic environment has not been well-studied. Such challenges are intractable even for conventional algorithms since observations from different views with dynamic objects involved break the geometric and photometric consistency, whereas the consistency lays the foundation for joint optimizing the camera pose and the map parameters. In this paper, we best exploit the characteristics of continual learning and propose a novel SLAM framework for dynamic environments. While past efforts have been made to avoid catastrophic forgetting by exploiting an experience replay strategy, we view forgetting as a desirable characteristic. By adaptively controlling the replayed buffer, the ambiguity caused by moving objects can be easily alleviated through forgetting. We restrain the replay of the dynamic objects by introducing a continually-learned classifier for dynamic object identification. The iterative optimization of the neural map and the classifier notably improves the robustness of the SLAM system under a dynamic environment. Experiments on challenging datasets verify the effectiveness of the proposed framework.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper primarily aims to address the problem of achieving stable and efficient Simultaneous Localization and Mapping (SLAM) in dynamic environments. Specifically, it focuses on how to effectively perform SLAM in environments containing moving objects, as the geometric and photometric consistency in such environments is disrupted, posing a challenge to traditional SLAM systems.

To solve this problem, the authors propose a novel SLAM framework based on continual learning. The core of this framework lies in its ability to distinguish which information should be memorized and which should be forgotten. By introducing a continual learning classifier to identify dynamic objects in the scene and control the exclusion of these objects' information from subsequent map updates, the framework effectively handles dynamic environments. Additionally, the method leverages implicit neural representations to construct continuous representations of the scene, which helps improve the system's robustness and adaptability.

The specific contributions include:
1. Proposing the first dense neural SLAM framework designed to tackle challenging dynamic environments, capable of reliable motion segmentation, stable camera tracking, and convenient map updates in changing environments.
2. Designing a continual learning approach to update a classifier that records the motion states of objects in the environment. This classifier can achieve both forward and backward knowledge transfer and integrate prior knowledge to enhance its performance.
3. Demonstrating that the forgetting mechanism of continual learning can be utilized to update neural scene representations based on environmental changes.

In summary, this research provides an innovative solution for achieving stable and efficient SLAM systems in complex and dynamic environments.