Abstract:Background and Objective: Deep learning models often suffer from performance degradations when deployed in real clinical environments due to appearance shifts between training and testing images. Most extant methods use training-time adaptation , which almost require target domain samples in the training phase. However, these solutions are limited by the training process and cannot guarantee the accurate prediction of test samples with unforeseen appearance shifts. Further, it is impractical to collect target samples in advance. In this paper, we provide a general method of making existing segmentation models robust to samples with unknown appearance shifts when deployed in daily clinical practice. Methods: Our proposed test-time bi-directional adaptation framework combines two complementary strategies. First, our image-to-model (I2M) adaptation strategy adapts appearance-agnostic test images to the learned segmentation model using a novel plug-and-play statistical alignment style transfer module during testing. Second, our model-to-image (M2I) adaptation strategy adapts the learned segmentation model to test images with unknown appearance shifts. This strategy applies an augmented self-supervised learning module to fine-tune the learned model with proxy labels that it generates. This innovative procedure can be adaptively constrained using our novel proxy consistency criterion. This complementary I2M and M2I framework demonstrably achieves robust segmentation against unknown appearance shifts using existing deep-learning models. Results: Extensive experiments on 10 datasets containing fetal ultrasound, chest X-ray, and retinal fundus images demonstrate that our proposed method achieves promising robustness and efficiency in segmenting images with unknown appearance shifts. Conclusions: To address the appearance shift problem in clinically acquired medical images, we provide robust segmentation by using two complementary strategies. Our solution is general and amenable for deployment in clinical settings.

Unseen Object Instance Segmentation with Fully Test-time RGB-D Embeddings Adaptation.

Learning to Simulate Complex Scenes for Street Scene Segmentation

Recurrent Volume-based 3D Feature Fusion for Real-time Multi-view Object Pose Estimation

Learning Instance-Specific Adaptation for Cross-Domain Segmentation

Zero-Shot 3d Pose Estimation of Unseen Object by Two-Step Rgb-D Fusion

Test-time Bi-directional Adaptation between Image and Model for Robust Segmentation

Multichannel Semantic Segmentation with Unsupervised Domain Adaptation

Detection and Segmentation of Unlearned Objects in Unknown Environment

Fully Test-Time Adaptation for Image Segmentation

Data-efficient 3D instance segmentation by transferring knowledge from synthetic scans

Learning Semantic Segmentation from Synthetic Data: A Geometrically Guided Input-Output Adaptation Approach

Segmentation Networks Domain Adaptation A real image ( a ) Without adaptation ( b ) With adaptation Real images without annotations Synthetic images with annotations ... ... ? ?

A Two-Pipeline Instance Segmentation Network via Boundary Enhancement for Scene Understanding

Increasing the Robustness of Deep Learning Models for Object Segmentation: A Framework for Blending Automatically Annotated Real and Synthetic Data.

ACE: Adapting to Changing Environments for Semantic Segmentation

Unsupervised Urban Scene Segmentation Via Domain Adaptation

Double-domain Adaptation Semantics for Retrieval-based Long-term Visual Localization

Benchmarking and Analysis of Unsupervised Object Segmentation from Real-World Single Images

Test-Time Adaptation with Shape Moments for Image Segmentation

A Multilevel-Guided Curriculum Domain Adaptation Approach to Semantic Segmentation for High-Resolution Remote Sensing Images.

AsymFormer: Asymmetrical Cross-Modal Representation Learning for Mobile Platform Real-Time RGB-D Semantic Segmentation