AI-Powered Augmented Reality for Satellite Assembly, Integration and Test

Alvaro Patricio,Joao Valente,Atabak Dehban,Ines Cadilha,Daniel Reis,Rodrigo Ventura

2024-09-27

Abstract:The integration of Artificial Intelligence (AI) and Augmented Reality (AR) is set to transform satellite Assembly, Integration, and Testing (AIT) processes by enhancing precision, minimizing human error, and improving operational efficiency in cleanroom environments. This paper presents a technical description of the European Space Agency's (ESA) project "AI for AR in Satellite AIT," which combines real-time computer vision and AR systems to assist technicians during satellite assembly. Leveraging Microsoft HoloLens 2 as the AR interface, the system delivers context-aware instructions and real-time feedback, tackling the complexities of object recognition and 6D pose estimation in AIT workflows. All AI models demonstrated over 70% accuracy, with the detection model exceeding 95% accuracy, indicating a high level of performance and reliability. A key contribution of this work lies in the effective use of synthetic data for training AI models in AR applications, addressing the significant challenges of obtaining real-world datasets in highly dynamic satellite environments, as well as the creation of the Segmented Anything Model for Automatic Labelling (SAMAL), which facilitates the automatic annotation of real data, achieving speeds up to 20 times faster than manual human annotation. The findings demonstrate the efficacy of AI-driven AR systems in automating critical satellite assembly tasks, setting a foundation for future innovations in the space industry.

Computer Vision and Pattern Recognition,Artificial Intelligence

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to improve operational precision, reduce human errors and enhance operational efficiency in the satellite assembly, integration and test (AIT) process by combining artificial intelligence (AI) and augmented reality (AR) technologies. Specifically, the paper mainly focuses on the following aspects:

1. **Improving precision and reducing human errors**: In satellite manufacturing, any minor error may lead to huge economic losses. Therefore, the paper proposes a system based on AI and AR, aiming to help technicians complete complex assembly tasks more accurately by providing real - time guidance and feedback, thus reducing human errors.

2. **Improving operational efficiency**: The traditional satellite assembly process usually depends on paper manuals or electronic documents, which is not only time - consuming but also error - prone. The system proposed in the paper projects operation guidelines directly into the actual working environment through AR technology, enabling technicians to complete tasks more quickly and efficiently.

3. **Solving the data acquisition problem**: Satellite components often change, and it is very difficult to obtain real - world training data. For this reason, the paper proposes a method of using synthetic data to train AI models and develops an automatic annotation tool SAMAL, which greatly improves the efficiency of data preparation.

4. **Achieving automation and intelligence**: By integrating advanced computer vision algorithms, such as object detection, 6D pose estimation and optical character recognition (OCR), the paper presents a highly automated system that can provide accurate guidance and support in the satellite assembly process.

In conclusion, the main goal of this paper is to bring revolutionary improvements to the satellite assembly, integration and test process through the combination of AI and AR technologies, and improve the precision, efficiency and reliability of the entire process.