Reconstructing Satellites in 3D from Amateur Telescope Images

Zhiming Chang,Boyang Liu,Yifei Xia,Youming Guo,Boxin Shi,He Sun
2024-04-29
Abstract:This paper proposes a framework for the 3D reconstruction of satellites in low-Earth orbit, utilizing videos captured by small amateur telescopes. The video data obtained from these telescopes differ significantly from data for standard 3D reconstruction tasks, characterized by intense motion blur, atmospheric turbulence, pervasive background light pollution, extended focal length and constrained observational perspectives. To address these challenges, our approach begins with a comprehensive pre-processing workflow that encompasses deep learning-based image restoration, feature point extraction and camera pose initialization. We proceed with the application of an improved 3D Gaussian splatting algorithm for reconstructing the 3D model. Our technique supports simultaneous 3D Gaussian training and pose estimation, enabling the robust generation of intricate 3D point clouds from sparse, noisy data. The procedure is further bolstered by a post-editing phase designed to eliminate noise points inconsistent with our prior knowledge of a satellite's geometric constraints. We validate our approach using both synthetic datasets and actual observations of China's Space Station, showcasing its significant advantages over existing methods in reconstructing 3D space objects from ground-based observations.
Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
The paper aims to address the problem of 3D reconstruction of low Earth orbit satellites based on images taken by ground-based telescopes. Specifically, the research team proposed an innovative method that can recover accurate 3D models from blurry and noisy amateur telescope videos that are severely affected by atmospheric turbulence and light pollution. The paper mainly addresses the following key issues: 1. **Impact of Atmospheric Turbulence and Light Pollution**: Images observed by ground-based telescopes are usually affected by atmospheric turbulence, leading to image distortion; additionally, light pollution caused by artificial light sources reduces the visibility of celestial targets. 2. **Complexity Due to Limited Viewing Angles**: The fixed observation position of the telescope means that only limited angle information can be captured as the satellite moves rapidly, further increasing the difficulty of 3D reconstruction. 3. **High Noise Levels and Blurry Images**: The rapid movement of the satellite requires short exposure times, resulting in low signal-to-noise ratios and blurry images. To address these challenges, the authors proposed a new method consisting of three main steps: 1. **Preprocessing**: Improving the quality of the original images through techniques such as lucky imaging, wavelet decomposition sharpening, and Video Restoration Transformer. 2. **Joint Optimization of Pose Estimation and 3D Reconstruction**: Initializing camera poses using the Structure from Motion (SFM) algorithm and iteratively optimizing with an improved 3D Gaussian point cloud reconstruction technique. 3. **Post-Editing**: Applying geometric filtering to the reconstructed point cloud to remove outliers and improve the accuracy of the final model. Experimental results show that this method not only performs well on synthetic datasets but also achieves excellent results on actual observations of the Chinese space station, particularly in terms of training speed, significantly outperforming traditional NeRF methods.