Gradient Aligned Domain Generalization with a Mutual Teaching Teacher-student Network for Intelligent Fault Diagnosis

Yulin Ma,Jun Yang,Lei Li
DOI: https://doi.org/10.1016/j.ress.2023.109516
2023-07-27
Abstract:Domain generalization for intelligent fault diagnosis handles multiple source domains and generalizes learned knowledge to unseen target domains. Recent efforts actively advocate for learning domain invariances that retain only domain-invariant features and reduce domain-specific ones. However, most of their invariance criteria might not be effective, compared with the simple empirical risk minimization. Besides, pursuing a similar training procedure has shown the potential to promote domain invariances, but it involves gradient alignments that are quite hard to manipulate. To address these issues, a Gradient Aligned Domain Generalization method (GADG) with a mutual teaching teacher-student network is proposed. Specifically, domain-invariant features are first thoroughly identified using gradients derived from multiple task-specific classifiers, which also intend to offer structure benefits by taking disentangled features for respective tasks. Then, to capture highly expressive domain invariances in the gradient space, a granular gradient alignment is initiated to emphasize high-order domain consistencies, which collaborate with low-order invariant distributional statistics to guarantee safe gradient alignments. Finally, to satisfy consistent gradient alignments, a mutual teaching teacher-student network is proposed, in which modules are assigned with respective tasks and recursively trained. Comprehensive experiments on various generalization tasks validate the efficacy of GADG.
engineering, industrial,operations research & management science
What problem does this paper attempt to address?