Source-Function Weighted-Transfer Learning for Nonparametric Regression with Seemingly Similar Sources

Lu Lin,Weiyu Li

2023-01-01

Abstract: The homogeneity, or more generally, the similarity between source domains and a target domain seems to be essential to a positive transfer learning. In practice, however, the similarity condition is difficult to check and is often violated. In this paper, instead of the popularly used similarity condition, a seeming similarity is introduced, which is defined by a non-orthogonality together with a smoothness. Such a condition is naturally satisfied under common situations and even implies the dissimilarity in some sense. Based on the seeming similarity together with an $L_2$-adjustment, a source-function weighted-transfer learning estimation (sw-TLE) is constructed. By source-function weighting, an adaptive transfer learning is achieved in the sense that it is applied to similar and dissimilar scenarios with a relatively high estimation efficiency. Particularly, under the case with homogenous source and target models, the sw-TLE even can be competitive with the full data estimator. The hidden relationship between the source-function weighting estimator and the James-Stein estimator is established as well, which reveals the structural reasonability of our methodology. Moreover, the strategy does apply to nonparametric and semiparametric models. The comprehensive simulation studies and real data analysis can illustrate that the new strategy is significantly better than the competitors.