Efficient and Controllable Remote Sensing Fake Sample Generation Based on Diffusion Model
Zhiqiang Yuan,Chongyang Hao,Ruixue Zhou,Jialiang Chen,Miao Yu,Wenkai Zhang,Hongqi Wang,Xian Sun
DOI: https://doi.org/10.1109/tgrs.2023.3268331
IF: 8.2
2023-01-01
IEEE Transactions on Geoscience and Remote Sensing
Abstract:In this paper, we propose an efficient remote sensing fake sample generation (RSFSG) framework based on the diffusion model, so as to generate controllable samples consistent with real scenes. Firstly, in order to alleviate the huge time consumption caused by the large parameters of the diffusion model, we come up with a multi-frequency dynamic knowledge distillation based on the consistent power spectrum of predicted Gaussian noise. The proposed multi-frequency knowledge transfer balances the ununiform output of the lightweight model by learning different frequency outputs during diffusion process at different stages. Secondly, to address the problem of slow training of diffusion models, we propose a progressive training strategy (PTS), inspired by the fitting mechanism of deep networks from low to high frequencies. PTS enables fast fitting of diffusion models by enabling the model to learn from low frequency information such as color at low resolution, and gradually move to high resolution images full of details such as texture. The established above two method achieve well generation performance under lightweight parameters, within almost half of the training time consumption. Extensive evaluations demonstrate that the proposed model significantly outperforms the state-of-the-art methods on RS controllable fake sample generation. With in our knowledge, we are the first work to introduce the diffusion model into the RSFSG task and obtain well performance, the code and the corresponding pre-trained files have released at Link.
imaging science & photographic technology,remote sensing,engineering, electrical & electronic,geochemistry & geophysics