Abstract:Deep learning plays a critical role in vision-based satellite pose estimation. However, the scarcity of real data from the space environment means that deep models need to be trained using synthetic data, which raises the Sim2Real domain gap problem. A major cause of the Sim2Real gap are novel lighting conditions encountered during test time. Event sensors have been shown to provide some robustness against lighting variations in vision-based pose estimation. However, challenging lighting conditions due to strong directional light can still cause undesirable effects in the output of commercial off-the-shelf event sensors, such as noisy/spurious events and inhomogeneous event densities on the object. Such effects are non-trivial to simulate in software, thus leading to Sim2Real gap in the event domain. To close the Sim2Real gap in event-based satellite pose estimation, the paper proposes a test-time self-supervision scheme with a certifier module. Self-supervision is enabled by an optimisation routine that aligns a dense point cloud of the predicted satellite pose with the event data to attempt to rectify the inaccurately estimated pose. The certifier attempts to verify the corrected pose, and only certified test-time inputs are backpropagated via implicit differentiation to refine the predicted landmarks, thus improving the pose estimates and closing the Sim2Real gap. Results show that the our method outperforms established test-time adaptation schemes.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is **the domain gap (Sim2Real gap) between the simulated environment and the real environment in event - based satellite attitude estimation**, especially when facing extreme illumination conditions, this gap is particularly obvious. Specifically: 1. **Reasons for the Sim2Real Gap**: - Due to the lack of real data from the space environment, deep - learning models are usually trained with synthetic data, which leads to the domain gap between simulated data and real data. - Extreme illumination conditions (such as strong directional light) will generate noise / spurious events and non - uniform event densities in the output of commercial off - the - shelf event sensors, and these effects are difficult to accurately simulate in software, further exacerbating the Sim2Real gap. 2. **Limitations of Existing Methods**: - Although existing domain adaptation methods can partially alleviate this problem, they still have deficiencies when dealing with event data under extreme illumination conditions. - Although using event sensors can improve the robustness to illumination changes to a certain extent, extreme illumination conditions will still cause adverse effects. 3. **The Method Proposed in the Paper**: - The paper proposes a test - time self - supervision scheme, combined with a certifier module, to reduce the Sim2Real gap. - Self - supervision attempts to correct inaccurate attitude estimates by aligning the predicted satellite attitude dense point cloud with the event data through an optimization procedure. - The validation module attempts to validate the corrected attitude and only allows the validated test inputs to pass through implicit differential back - propagation to update the predicted landmarks, thereby improving the attitude estimate. 4. **Experimental Results**: - The experimental results show that this method is significantly superior to existing test - time adaptation schemes when dealing with event data under extreme illumination conditions, effectively narrowing the Sim2Real gap. In summary, this paper aims to solve the Sim2Real gap problem in event - based satellite attitude estimation caused by extreme illumination conditions by introducing a novel test - time self - supervision framework, thereby improving the performance of the model in the real environment.

Test-Time Certifiable Self-Supervision to Bridge the Sim2Real Gap in Event-Based Satellite Pose Estimation