Abstract:This paper presents a study on path planning for 6-DOF free-floating space robotic manipulators using Deep Deterministic Policy Gradient-based Reinforcement Learning. The focus is the development of a novel reward function tailored to address critical requirements for efficient and effective manipulation in space. These requirements include accurate pose alignment between the end-effector and the target, collision avoidance with both the target and other links of the manipulator, smoothing of joint velocities, adaptability to strong dynamic coupling between the manipulator and its base spacecraft due to high manipulator-spacecraft mass ratio, and resilience to noise in the state observations. Uniquely, the proposed reward function employs quaternions for orientation control to reduce pose misalignments and dynamic singularities, as opposed to traditional Euler angles. Our findings demonstrate that the Reinforcement Learning algorithm, when guided by this new reward function that integrates these enhancements and constraints, not only achieves the desired path planning objectives more efficiently but also exhibits faster convergence. Furthermore, the Reinforcement Learning successfully manages significant dynamic coupling effects caused by a high mass ratio between the robotic manipulator and the base spacecraft. Even under the challenge of noisy state observations, the trained agent successfully completes the path planning task, proving the Reinforcement Learning's applicability to real-space mission designs where the noise in observation is inevitable. The study highlights the critical role of reward function design in the Reinforcement Learning training process and its consequential impact on the solution quality, providing a solid foundation for future advancements in free-floating space robotic missions.

Six degree-of-freedom body-fixed hovering over unmapped asteroids via LIDAR altimetry and reinforcement meta-learning

Seeker based Adaptive Guidance via Reinforcement Meta-Learning Applied to Asteroid Close Proximity Operations

Reinforcement Meta-Learning for Interception of Maneuvering Exoatmospheric Targets with Parasitic Attitude Loop

Integrated and Adaptive Guidance and Control for Endoatmospheric Missiles via Reinforcement Learning

Collision-Free Trajectory Planning for a 6-DoF Free-Floating Space Robot via Hierarchical Decoupling Optimization

Reinforcement Learning and Sim-to-Real Transfer of Reorientation and Landing Control for Quadruped Robots on Asteroids

Orbit–attitude dynamics and control of spacecraft hovering over a captured asteroid in the earth–moon system

Coordinated Control Based on Reinforcement Learning for Dual-Arm Continuum Manipulators in Space Capture Missions

Autonomy in the Real-World: Autonomous Trajectory Planning for Asteroid Reconnaissance via Stochastic Optimization

Immersion-and Invariance-Based Adaptive Control of Asteroid-Orbiting and - Hovering Spacecraft

Path Planning of 6-DOF Free-Floating Space Robotic Manipulators Using Reinforcement Learning

Body-fixed orbit-attitude hovering at equilibria near an asteroid using non-canonical Hamiltonian structure

Autonomous Six-Degree-of-Freedom Spacecraft Docking Maneuvers via Reinforcement Learning

A Multi-Target Trajectory Planning of a 6-Dof Free-Floating Space Robot Via Reinforcement Learning

Adaptive Guidance and Integrated Navigation with Reinforcement Meta-Learning

Trajectory Design and Guidance for Far-range Proximity Operations of Active Debris Removal Missions with Angles-only Navigation and Safety Considerations

Autonomous Spacecraft Collision Avoidance with Multiple Space Debris Based on Reinforcement Learning

Autonomous and Robust Orbit-keeping for Small Body Missions

Imitation Learning for Autonomous Trajectory Learning of Robot Arms in Space

Mobility Strategy of Multi-Limbed Climbing Robots for Asteroid Exploration

Redundant Space Manipulator Autonomous Guidance for In-Orbit Servicing via Deep Reinforcement Learning