Abstract:This paper presents a novel approach for cross-view synthesis aimed at generating plausible ground-level images from corresponding satellite imagery or vice versa. We refer to these tasks as satellite-to-ground (Sat2Grd) and ground-to-satellite (Grd2Sat) synthesis, respectively. Unlike previous works that typically focus on one-to-one generation, producing a single output image from a single input image, our approach acknowledges the inherent one-to-many nature of the problem. This recognition stems from the challenges posed by differences in illumination, weather conditions, and occlusions between the two views. To effectively model this uncertainty, we leverage recent advancements in diffusion models. Specifically, we exploit random Gaussian noise to represent the diverse possibilities learnt from the target view data. We introduce a Geometry-guided Cross-view Condition (GCC) strategy to establish explicit geometric correspondences between satellite and street-view features. This enables us to resolve the geometry ambiguity introduced by camera pose between image pairs, boosting the performance of cross-view image synthesis. Through extensive quantitative and qualitative analyses on three benchmark cross-view datasets, we demonstrate the superiority of our proposed geometry-guided cross-view condition over baseline methods, including recent state-of-the-art approaches in cross-view image synthesis. Our method generates images of higher quality, fidelity, and diversity than other state-of-the-art approaches.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to solve the key challenges in **cross - view image synthesis**, especially generating ground - level images from satellite images (Sat2Grd) or satellite images from ground - level images (Grd2Sat). Specifically, the paper focuses on the following issues: 1. **One - to - many mapping problem**: - Cross - view image synthesis is essentially a **non - deterministic task**, that is, one input image can correspond to multiple reasonable output images. This is mainly due to the differences in lighting, weather conditions and occlusions between different viewpoints. Traditional methods usually generate only a single output image, ignoring this diversity. 2. **Geometric alignment problem**: - There are significant perspective changes between satellite images and ground - level images, resulting in very little field - of - view overlap, severe occlusions and huge differences in image content and visual features. How to maintain geometric consistency during the generation process is an important challenge. 3. **Uncertainty and ambiguity**: - Differences in camera pose introduce geometric uncertainty, especially more obvious in the ground - to - satellite synthesis task. Traditional GAN - based methods are difficult to handle this ambiguity. ### Solutions proposed in the paper To solve the above problems, the paper proposes a new method - **Geometry - guided Cross - view Diffusion**. The main innovations of this method include: - **Geometry - guided Cross - view Condition (GCC)**: - Map the extracted image features to the target perspective through explicit 3D geometric projection, thereby establishing the geometric correspondence between satellite and street - view features. GCC can effectively alleviate the geometric ambiguity caused by camera pose differences. - **Using diffusion models**: - Diffusion models can represent multiple possibilities in the target - view data by introducing random Gaussian noise, so as to better model the uncertainty in cross - view synthesis tasks. - **Multi - level feature aggregation**: - Extract and aggregate features at different levels to ensure the semantic and geometric accuracy of the generated image. ### Experimental verification The paper verifies the effectiveness of the proposed method through extensive experiments on multiple benchmark datasets (such as KITTI, CVUSA, CVACT). The results show that this method is not only superior to existing methods in quantitative indicators (such as SSIM, PSNR, LPIPS, FID), but also performs well in the quality, fidelity and diversity of the generated images. In summary, by introducing the geometry - guided cross - view condition and diffusion models, this paper successfully solves the geometric alignment and one - to - many mapping problems in cross - view image synthesis, providing new research ideas and technical means for this field.

Geometry-guided Cross-view Diffusion for One-to-many Cross-view Image Synthesis

CrossViewDiff: A Cross-View Diffusion Model for Satellite-to-Street View Synthesis

Cross-View Meets Diffusion: Aerial Image Synthesis with Geometry and Text Guidance

Geometry-Guided Street-View Panorama Synthesis from Satellite Imagery

Geospecific View Generation -- Geometry-Context Aware High-resolution Ground View Inference from Satellite Views

Geometry-Aware Satellite-to-Ground Image Synthesis for Urban Areas.

Coming Down to Earth: Satellite-to-Street View Synthesis for Geo-Localization

UAV-Satellite View Synthesis for Cross-view Geo-Localization

GeoGS3D: Single-view 3D Reconstruction via Geometric-aware Diffusion Model and Gaussian Splatting

Retrieval-guided Cross-view Image Synthesis

Consistent-1-to-3: Consistent Image to 3D View Synthesis via Geometry-aware Diffusion Models

Cross-View Image Synthesis Using Conditional GANs

Sat2Vid - Street-view Panoramic Video Synthesis from a Single Satellite Image.

SGD: Street View Synthesis with Gaussian Splatting and Diffusion Prior

Ground–Satellite Coupling for Cross-View Geolocation Combined With Multiscale Fusion of Spatial Features

Towards Cross-View Consistency in Semantic Segmentation While Varying View Direction.

Mixed-View Panorama Synthesis using Geospatially Guided Diffusion

Wonder3D: Single Image to 3D Using Cross-Domain Diffusion

MagicDrive: Street View Generation with Diverse 3D Geometry Control

A Construct-Optimize Approach to Sparse View Synthesis without Camera Pose

Drone-assisted Road Gaussian Splatting with Cross-view Uncertainty