Abstract:This paper presents a study on path planning for 6-DOF free-floating space robotic manipulators using Deep Deterministic Policy Gradient-based Reinforcement Learning. The focus is the development of a novel reward function tailored to address critical requirements for efficient and effective manipulation in space. These requirements include accurate pose alignment between the end-effector and the target, collision avoidance with both the target and other links of the manipulator, smoothing of joint velocities, adaptability to strong dynamic coupling between the manipulator and its base spacecraft due to high manipulator-spacecraft mass ratio, and resilience to noise in the state observations. Uniquely, the proposed reward function employs quaternions for orientation control to reduce pose misalignments and dynamic singularities, as opposed to traditional Euler angles. Our findings demonstrate that the Reinforcement Learning algorithm, when guided by this new reward function that integrates these enhancements and constraints, not only achieves the desired path planning objectives more efficiently but also exhibits faster convergence. Furthermore, the Reinforcement Learning successfully manages significant dynamic coupling effects caused by a high mass ratio between the robotic manipulator and the base spacecraft. Even under the challenge of noisy state observations, the trained agent successfully completes the path planning task, proving the Reinforcement Learning's applicability to real-space mission designs where the noise in observation is inevitable. The study highlights the critical role of reward function design in the Reinforcement Learning training process and its consequential impact on the solution quality, providing a solid foundation for future advancements in free-floating space robotic missions.

Training in Task Space to Speed Up and Guide Reinforcement Learning

Learning with Training Wheels: Speeding up Training with a Simple Controller for Deep Reinforcement Learning

Learning Task Space Actions for Bipedal Locomotion

Dexterous Legged Locomotion in Confined 3D Spaces with Reinforcement Learning

How does the structure embedded in learning policy affect learning quadruped locomotion?

Learning Bipedal Walking for Humanoids with Current Feedback

Hybrid Zero Dynamics Inspired Feedback Control Policy Design for 3D Bipedal Locomotion using Reinforcement Learning

Path Planning of 6-DOF Free-Floating Space Robotic Manipulators Using Reinforcement Learning

A Hierarchical Reinforcement Learning Approach to Control Legged Mobile Manipulators

Enabling Efficient, Reliable Real-World Reinforcement Learning with Approximate Physics-Based Models

RL + Model-based Control: Using On-demand Optimal Control to Learn Versatile Legged Locomotion

Guiding real-world reinforcement learning for in-contact manipulation tasks with Shared Control Templates

Robust Walking and Sim-to-Real Optimization for Quadruped Robots via Reinforcement Learning

Subspace-wise Hybrid RL for Articulated Object Manipulation

One-shot sim-to-real transfer policy for robotic assembly via reinforcement learning with visual demonstration

Training Efficient Controllers via Analytic Policy Gradient

Learning Modular Robot Visual-motor Locomotion Policies

Modelling Generalized Forces with Reinforcement Learning for Sim-to-Real Transfer

Run Time Assured Reinforcement Learning for Six Degree-of-Freedom Spacecraft Inspection

Just Round: Quantized Observation Spaces Enable Memory Efficient Learning of Dynamic Locomotion

Evaluation of Constrained Reinforcement Learning Algorithms for Legged Locomotion