Test-time adaptation for geospatial point cloud semantic segmentation with distinct domain shifts

Puzuo Wang,Wei Yao,Jie Shao,Zhiyi He

2024-07-08

Abstract:Domain adaptation (DA) techniques help deep learning models generalize across data shifts for point cloud semantic segmentation (PCSS). Test-time adaptation (TTA) allows direct adaptation of a pre-trained model to unlabeled data during inference stage without access to source data or additional training, avoiding privacy issues and large computational resources. We address TTA for geospatial PCSS by introducing three domain shift paradigms: photogrammetric to airborne LiDAR, airborne to mobile LiDAR, and synthetic to mobile laser scanning. We propose a TTA method that progressively updates batch normalization (BN) statistics with each testing batch. Additionally, a self-supervised learning module optimizes learnable BN affine parameters. Information maximization and reliability-constrained pseudo-labeling improve prediction confidence and supply supervisory signals. Experimental results show our method improves classification accuracy by up to 20\% mIoU, outperforming other methods. For photogrammetric (SensatUrban) to airborne (Hessigheim 3D) adaptation at the inference stage, our method achieves 59.46\% mIoU and 85.97\% OA without retraining or fine-turning.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

### Problems the Paper Aims to Solve

The paper primarily focuses on addressing the issue of data shift between different domains in the task of geospatial point cloud semantic segmentation (PCSS) by utilizing test-time adaptation (TTA) techniques. Specifically, the goals of the paper include:

1. **Proposing a TTA Method**: By adjusting the batch normalization (BN) layer statistics in the pre-trained model, the method aims to adapt the model to the data distribution of the target domain, thereby improving classification accuracy.
2. **Constructing Benchmark Tests**: To validate the effectiveness of the proposed TTA method, the paper constructs three practical domain adaptation benchmark scenarios:
- Photogrammetric point cloud to airborne laser scanning (ALS)
- Airborne laser scanning to mobile laser scanning (MLS)
- Synthetic data to mobile laser scanning
3. **Optimizing BN Layer Parameters**: In addition to updating the BN layer statistics, a self-supervised module is introduced to optimize the learnable affine parameters of the BN layer, further enhancing model performance.

Through these methods, the paper aims to address the common issue of cross-domain data shift in geospatial point cloud semantic segmentation and significantly improve the classification accuracy of the model in the target domain. Experimental results show that, compared to directly applying the pre-trained model, the proposed method improves the mean Intersection over Union (mIoU) by up to 20% and is highly efficient, requiring no model retraining.