Abstract:Endoscopy is a widely used technique for the early detection of diseases or robotic-assisted minimally invasive surgery (RMIS). Numerous deep learning (DL)-based research works have been developed for automated diagnosis or processing of endoscopic view. However, existing DL models may suffer from catastrophic forgetting. When new target classes are introduced over time or cross institutions, the performance of old classes may suffer severe degradation. More seriously, data privacy and storage issues may lead to the unavailability of old data when updating the model. Therefore, it is necessary to develop a continual learning (CL) methodology to solve the problem of catastrophic forgetting in endoscopic image segmentation. To tackle this, we propose a Endoscopy Continual Semantic Segmentation (EndoCSS) framework that does not involve the storage and privacy issues of exemplar data. The framework includes a mini-batch pseudo-replay (MB-PR) mechanism and a self-adaptive noisy cross-entropy (SAN-CE) loss. The MB-PR strategy circumvents privacy and storage issues by generating pseudo-replay images through a generative model. Meanwhile, the MB-PR strategy can also correct the model deviation to the replay data and current training data, which is aroused by the significant difference in the amount of current and replay images. Therefore, the model can perform effective representation learning on both new and old tasks. SAN-CE loss can help model fitting by adjusting the model's output logits, and also improve the robustness of training. Extensive continual semantic segmentation (CSS) experiments on public datasets demonstrate that our method can robustly and effectively address the catastrophic forgetting brought by class increment in endoscopy scenes. The results show that our framework holds excellent potential for real-world deployment in a streaming learning manner.

Segmentation of Multiple Sclerosis Lesions across Hospitals: Learn Continually or Train from Scratch?

SegHeD+: Segmentation of Heterogeneous Data for Multiple Sclerosis Lesions with Anatomical Constraints and Lesion-aware Augmentation

Improving Multiple Sclerosis Lesion Segmentation Across Clinical Sites: A Federated Learning Approach with Noise-Resilient Training

SegHeD: Segmentation of Heterogeneous Data for Multiple Sclerosis Lesions with Anatomical Constraints

Toward Generalizable Multiple Sclerosis Lesion Segmentation Models

Longitudinal Segmentation of MS Lesions via Temporal Difference Weighting

CoactSeg: Learning from Heterogeneous Data for New Multiple Sclerosis Lesion Segmentation

Continual Learning of Medical Image Classification Based on Feature Replay

What is Wrong with Continual Learning in Medical Image Segmentation?

Rethinking Exemplars for Continual Semantic Segmentation in Endoscopy Scenes: Entropy-based Mini-Batch Pseudo-Replay

Automated Segmentation of Multiple Sclerosis Lesions using Deep Learning

Towards Synchronous Memorizability and Generalizability with Site-Modulated Diffusion Replay for Cross-Site Continual Segmentation

Distribution-Aware Replay for Continual MRI Segmentation

Contrast-Enhancing Lesion Segmentation in Multiple Sclerosis: A Deep Learning Approach Validated in a Multicentric Cohort

Deep Attention and Graphical Neural Network for Multiple Sclerosis Lesion Segmentation From MR Imaging Sequences

Multiple Sclerosis Lesion Segmentation -- A Survey of Supervised CNN-Based Methods

Multiple Sclerosis Lesion Segmentation from Brain MRI via Fully Convolutional Neural Networks

Unsupervised Domain Adaptation With Optimal Transport in Multi-Site Segmentation of Multiple Sclerosis Lesions From MRI Data

Multiple Sclerosis Lesion Activity Segmentation with Attention-Guided Two-Path CNNs

Incremental Learning for Heterogeneous Structure Segmentation in Brain Tumor MRI