Abstract:This paper aims to present the design and validation of a deep learning-based pipeline for estimating the pose of an uncooperative target spacecraft, from a single grayscale monocular image. The possibility of enabling autonomous vision-based relative navigation in close proximity to a non-cooperative space object has recently gained interest in the space industry. In particular, such a technology would be especially appealing for Active Debris Removal (ADR) missions as well as in an on-orbit servicing scenario. The use of a simple camera, compared to more complex sensors such as a LiDAR, has numerous advantages in terms of lower mass, volume, and power requirements. This would translate into a substantially cheaper chaser space-craft, at the expense of increased complexity of the image processing algorithms. The Relative Pose Estimation Pipeline (RPEP) proposed in this work leverages state-of-the-art Convolutional Neural Network (CNN) architectures to detect the features of the target spacecraft from a single monocular image. Speciﬁcally, the overall pipeline is composed of three main subsystems. The input image is ﬁrst of all processed using an object detection CNN that localizes the portion of the image enclosing our target, i.e. the bounding box. This is followed by a second CNN that regresses the location of semantic keypoints of the spacecraft. Eventually, a geometric optimization algorithm exploits the detected keypoint locations to solve for the ﬁnal relative pose, based on the knowledge of camera intrinsics and of a wireframe model of the target satellite. The Spacecraft PosE Estimation Dataset (SPEED), a collection of 15300 images of the Tango spacecraft released by the Space rendezvous LABoratory (SLAB), has been used for training the deep neural networks employed in our pipeline, as well as for evaluating performance and estimation uncertainty. The proposed RPEP pipeline guarantees on SPEED a centimeter-level position accuracy and degree-level attitude accuracy, along with considerable robustness to changes in illumination and background conditions. In addition, our architecture showed to generalize well on real images, despite having exclusively exploited synthetic data to train the CNNs.

Neural Network-Based Navigation Filter for Monocular Pose and Motion Tracking of Noncooperative Spacecraft

Pose Estimation and Neural Implicit Reconstruction Towards Non-Cooperative Spacecraft Without Offline Prior Information

Adaptive Neural Network-based Unscented Kalman Filter for Robust Pose Tracking of Noncooperative Spacecraft

FlexKalmanNet: A Modular AI-Enhanced Kalman Filter Framework Applied to Spacecraft Motion Estimation

Multi-Sensors Data Tracking Fusion Based on a Neural Network Filter

DEEP LEARNING-BASED MONOCULAR RELATIVE POSE ESTIMATION OF UNCOOPERATIVE SPACECRAFT

Inertial-Aided Metric States and Surface Normal Estimation Using a Monocular Camera.

Continuous-discrete extended Kalman filtering based on the neural ordinary differential equations method

Spacecraft autonomous navigation using line-of-sight directions of non-cooperative targets by improved Q-learning based extended Kalman filter

A Combined Visual-Inertial Navigation System of MSCKF and EKF-SLAM

The Modified Cubature Kalman Filter for Relative Navigation of Space Target Tracking

Tight Fusion of a Monocular Camera, MEMS-IMU, and Single-Frequency Multi-GNSS RTK for Precise Navigation in GNSS-Challenged Environments

Localization Through Particle Filter Powered Neural Network Estimated Monocular Camera Poses

Machine Learning for LiDAR-Based Navigation System

Intelligent Tracking Method for Aerial Maneuvering Target Based on Unscented Kalman Filter

An Application Of Unscented Kalman Filter For Pose And Motion Estimation Based On Monocular Vision

ChiNet: Deep Recurrent Convolutional Learning for Multimodal Spacecraft Pose Estimation

Trainable Quaternion Extended Kalman Filter with Multi-Head Attention for Dead Reckoning in Autonomous Ground Vehicles

A Distributionally Robust Fusion Framework for Autonomous Multisensor Spacecraft Navigation during Entry Phase of Mars Entry, Descent, and Landing

Pose Estimation for Non-Cooperative Spacecraft Rendezvous Using Convolutional Neural Networks

Online Supervised Training of Spaceborne Vision during Proximity Operations using Adaptive Kalman Filtering