Abstract:This paper presents a study on path planning for 6-DOF free-floating space robotic manipulators using Deep Deterministic Policy Gradient-based Reinforcement Learning. The focus is the development of a novel reward function tailored to address critical requirements for efficient and effective manipulation in space. These requirements include accurate pose alignment between the end-effector and the target, collision avoidance with both the target and other links of the manipulator, smoothing of joint velocities, adaptability to strong dynamic coupling between the manipulator and its base spacecraft due to high manipulator-spacecraft mass ratio, and resilience to noise in the state observations. Uniquely, the proposed reward function employs quaternions for orientation control to reduce pose misalignments and dynamic singularities, as opposed to traditional Euler angles. Our findings demonstrate that the Reinforcement Learning algorithm, when guided by this new reward function that integrates these enhancements and constraints, not only achieves the desired path planning objectives more efficiently but also exhibits faster convergence. Furthermore, the Reinforcement Learning successfully manages significant dynamic coupling effects caused by a high mass ratio between the robotic manipulator and the base spacecraft. Even under the challenge of noisy state observations, the trained agent successfully completes the path planning task, proving the Reinforcement Learning's applicability to real-space mission designs where the noise in observation is inevitable. The study highlights the critical role of reward function design in the Reinforcement Learning training process and its consequential impact on the solution quality, providing a solid foundation for future advancements in free-floating space robotic missions.

Path Planning Method for Manipulators Based on Improved Twin Delayed Deep Deterministic Policy Gradient and RRT*

An Efficiently Convergent Deep Reinforcement Learning-Based Trajectory Planning Method for Manipulators in Dynamic Environments

Motion Planning of Robot Manipulators for a Smoother Path Using a Twin Delayed Deep Deterministic Policy Gradient with Hindsight Experience Replay

An Improved Rapidly-exploring Random Tree Star Algorithm for Manipulator Path Planning

Multi-point Traversal Path Planning of Manipulator Based on Improved RRT Algorithm

Path planning and collision avoidance based on the RRT*FN framework for a robotic manipulator in various scenarios

Manipulator Motion Planning Based on Improved RRT Algorithm

Path planning of a manipulator based on an improved P_RRT* algorithm

Path Planning for Robotic Manipulator in Complex Multi-Obstacle Environment Based on Improved_RRT

Rapid motion planning of manipulator in three-dimensional space under multiple scenes

A Path Planning Method with a Bidirectional Potential Field Probabilistic Step Size RRT for a Dual Manipulator

Improved Motion Planning Algorithms Based on Rapidly-exploring Random Tree: A Review.

A Method on Dynamic Path Planning for Robotic Manipulator Autonomous Obstacle Avoidance Based on an Improved RRT Algorithm

RRT-Based Path Planning for Follow-the-Leader Motion of Hyper-Redundant Manipulators.

Rrt-Gd: An Efficient Rapidly-Exploring Random Tree Approach With Goal Directionality For Redundant Manipulator Path Planning

Path Planning of 6-DOF Free-Floating Space Robotic Manipulators Using Reinforcement Learning

A General Framework of Motion Planning for Redundant Robot Manipulator Based on Deep Reinforcement Learning

Research on Decoupling Motion Path Planning of Two Manipulators of Live Working Robot Based on Hybrid RRT Algorithm

6R Serial Manipulator Space Path Planning Based on RRT

Deep Reinforcement Learning Integrated RRT Algorithm for Path Planning

Safe Interval Randomized Path Planing For Manipulators