Abstract:Research on loss surface geometry, such as Sharpness-Aware Minimization (SAM), shows that flatter minima improve generalization. Recent studies further reveal that flatter minima can also reduce the domain generalization (DG) gap. However, existing flatness-based DG techniques predominantly operate within a full-precision training process, which is impractical for deployment on resource-constrained edge devices that typically rely on lower bit-width representations (e.g., 4 bits, 3 bits). Consequently, low-precision quantization-aware training is critical for optimizing these techniques in real-world applications. In this paper, we observe a significant degradation in performance when applying state-of-the-art DG-SAM methods to quantized models, suggesting that current approaches fail to preserve generalizability during the low-precision training process. To address this limitation, we propose a novel Gradient-Adaptive Quantization-Aware Training (GAQAT) framework for DG. Our approach begins by identifying the scale-gradient conflict problem in low-precision quantization, where the task loss and smoothness loss induce conflicting gradients for the scaling factors of quantizers, with certain layers exhibiting opposing gradient directions. This conflict renders the optimization of quantized weights highly unstable. To mitigate this, we further introduce a mechanism to quantify gradient inconsistencies and selectively freeze the gradients of scaling factors, thereby stabilizing the training process and enhancing out-of-domain generalization. Extensive experiments validate the effectiveness of the proposed GAQAT framework. On PACS, our 3-bit and 4-bit models outperform direct DG-QAT integration by up to 4.5%. On DomainNet, the 4-bit model achieves near-lossless performance compared to full precision, with improvements of 1.39% (4-bit) and 1.06% (3-bit) over the SOTA QAT baseline.

Seeking Consistent Flat Minima for Better Domain Generalization via Refining Loss Landscapes

Flatness-Aware Minimization for Domain Generalization

Knowledge Distillation-based Domain-invariant Representation Learning for Domain Generalization

Exploring Flat Minima for Domain Generalization With Large Learning Rates

Sharpness-Aware Gradient Matching for Domain Generalization

Make Continual Learning Stronger via C-Flat

Flatten Long-Range Loss Landscapes for Cross-Domain Few-Shot Learning

Domain-Inspired Sharpness-Aware Minimization Under Domain Shifts

Sharpness-Aware Model-Agnostic Long-Tailed Domain Generalization

Bilateral Sharpness-Aware Minimization for Flatter Minima

Meta Curvature-Aware Minimization for Domain Generalization

Efficient Loss Landscape Reshaping for Convolutional Neural Networks

Effective Domain Knowledge Transfer with Soft Fine-tuning

GAQAT: gradient-adaptive quantization-aware training for domain generalization

Unknown Domain Inconsistency Minimization for Domain Generalization

Sharpness-Aware Training for Free

When Do Flat Minima Optimizers Work?

Friendly Sharpness-Aware Minimization

LFME: A Simple Framework for Learning from Multiple Experts in Domain Generalization

Domain-Aware Fine-Tuning: Enhancing Neural Network Adaptability