Abstract:On-site human-robot collaboration is one form of lunar exploration, where astronauts work side-by-side with rovers to accomplish tasks. Mixed reality (MR) is being tried in astronaut-rover teams. While research is primarily focused on MR technology to facilitate natural human-robot interaction, it often neglects the importance of incorporating transparency into decision-making processes. Our proposal is to utilize shared MR and optimized perception-localization techniques for human-robot collaboration in lunar exploration. The establishment of consensus between astronauts and rovers during decision-making processes can be facilitated through shared spatial context and interactive holographic content. This technology combines the strengths of astronauts and rovers for decision-making during rover navigation missions. It avoids blindly relying on rovers or only using manual manipulation by astronauts. In order to improve terrain perception and facilitate visualization for astronaut-rover teams during lunar navigation tasks, we develop a risk-aware lunar terrain parsing method that utilizes multiscale eigenvalue-based features and an optimized Random Forest classifier. Our method outperforms others with an impressive accuracy of 94.2%. Our co-location MR system incorporates a marker & instance-based spatial anchor method, customized specifically for the unique topography of the lunar terrain and optimized for resource conservation. Our experiments on lunar navigation with three terrain conditions and four configurations confirm that that shared MR could improve task performance and reduce workload. The proposed shared MR paradigm and related technologies can provide a reference in future lunar exploration missions.

CISRU: a robotics software suite to enable complex rover-rover and astronaut-rover interaction

Enabling In-Situ Resources Utilisation by leveraging collaborative robotics and astronaut-robot interaction

Immersive Rover Control and Obstacle Detection based on Extended Reality and Artificial Intelligence

Design of a user-friendly control system for planetary rovers with CPS feature

RUR53: an Unmanned Ground Vehicle for Navigation, Recognition and Manipulation

iCub-HRI: A Software Framework for Complex Human-Robot Interaction Scenarios on the iCub Humanoid Robot

Development of a Human–Robot Interface for Cobot Trajectory Planning Using Mixed Reality

Enhancing Robotic Perception through Synchronized Simulation and Physical Common-Sense Reasoning

A Novel Methodology for Autonomous Planetary Exploration Using Multi-Robot Teams

Modularity for lunar exploration: European Moon Rover System Pre-Phase A Design and Field Test Campaign Results

The European Moon Rover System: a modular multipurpose rover for future complex lunar missions

CASPER: Cognitive Architecture for Social Perception and Engagement in Robots

Multi-Robot Interfaces and Operator Situational Awareness: Study of the Impact of Immersion and Prediction

The Internet of Cooperative Agents Architecture (X-IoCA) for Robots, Hybrid Sensor Networks, and MEC Centers in Complex Environments: A Search and Rescue Case Study

Vision Guided Modeling and Simulation for CAS Rover.

NASA Space Robotics Challenge 2 Qualification Round: An Approach to Autonomous Lunar Rover Operations

Preserving HRI Capabilities: Physical, Remote and Simulated Modalities in the SciRoc 2021 Competition

On-site human-robot collaboration for lunar exploration based on shared mixed reality

RISE: an open-source architecture for interdisciplinary and reproducible human-robot interaction research

Towards Autonomous Planetary Exploration

Spatial Computing and Intuitive Interaction: Bringing Mixed Reality and Robotics Together