Sat2Vid - Street-view Panoramic Video Synthesis from a Single Satellite Image.

Zuoyue Li,Zhenqiang Li,Zhaopeng Cui,Rongjun Qin,Marc Pollefeys,Martin R. Oswald

DOI: https://doi.org/10.1109/iccv48922.2021.01221

2021-01-01

Abstract:We present a novel method for synthesizing both temporally and geometrically consistent street-view panoramic video from a single satellite image and camera trajectory. Existing cross-view synthesis approaches focus on images, while video synthesis in such a case has not yet received enough attention. For geometrical and temporal consistency, our approach explicitly creates a 3D point cloud representation of the scene and maintains dense 3D-2D correspondences across frames that reflect the geometric scene configuration inferred from the satellite view. As for synthesis in the 3D space, we implement a cascaded network architecture with two hourglass modules to generate point-wise coarse and fine features from semantics and per-class latent vectors, followed by projection to frames and an up-sampling module to obtain the final realistic video. By leveraging computed correspondences, the produced streetview video frames adhere to the 3D geometric scene structure and maintain temporal consistency. Qualitative and quantitative experiments demonstrate superior results compared to other state-of-the-art synthesis approaches that either lack temporal consistency or realistic appearance. To the best of our knowledge, our work is the first one to synthesize cross-view images to videos..