Abstract:This paper presents a study on path planning for 6-DOF free-floating space robotic manipulators using Deep Deterministic Policy Gradient-based Reinforcement Learning. The focus is the development of a novel reward function tailored to address critical requirements for efficient and effective manipulation in space. These requirements include accurate pose alignment between the end-effector and the target, collision avoidance with both the target and other links of the manipulator, smoothing of joint velocities, adaptability to strong dynamic coupling between the manipulator and its base spacecraft due to high manipulator-spacecraft mass ratio, and resilience to noise in the state observations. Uniquely, the proposed reward function employs quaternions for orientation control to reduce pose misalignments and dynamic singularities, as opposed to traditional Euler angles. Our findings demonstrate that the Reinforcement Learning algorithm, when guided by this new reward function that integrates these enhancements and constraints, not only achieves the desired path planning objectives more efficiently but also exhibits faster convergence. Furthermore, the Reinforcement Learning successfully manages significant dynamic coupling effects caused by a high mass ratio between the robotic manipulator and the base spacecraft. Even under the challenge of noisy state observations, the trained agent successfully completes the path planning task, proving the Reinforcement Learning's applicability to real-space mission designs where the noise in observation is inevitable. The study highlights the critical role of reward function design in the Reinforcement Learning training process and its consequential impact on the solution quality, providing a solid foundation for future advancements in free-floating space robotic missions.

A Deep Reinforcement Learning Solution for the Low Level Motion Control of a Robot Manipulator System

Model-free Maneuvering Control of Fixed-Wing UAVs Based on Deep Reinforcement Learning

Open-Loop Motion Control of a Hydraulic Soft Robotic Arm Using Deep Reinforcement Learning

Integrating DeepRL with Robust Low-Level Control in Robotic Manipulators for Non-Repetitive Reaching Tasks

Dynamic Obstacle Avoidance Algorithm for Robot Arm Based on Deep Reinforcement Learning

Deep Reinforcement Learning for Motion Control Algorithms in Robotics

Safety Guaranteed Manipulation Based on Reinforcement Learning Planner and Model Predictive Control Actor

A Deep Reinforcement-Learning Approach for Inverse Kinematics Solution of a High Degree of Freedom Robotic Manipulator

Deep Reinforcement Learning Based Trajectory Planning Under Uncertain Constraints

Deep Reinforcement Learning for Robotic Manipulation-The state of the art

Vision-based reinforcement learning control of soft robot manipulators

Deep Reinforcement Learning for the Control of Robotic Manipulation: A Focussed Mini-Review

Trajectory tracking control based on deep reinforcement learning and ensemble random network distillation for robotic manipulator

A Hierarchical Reinforcement Learning Approach to Control Legged Mobile Manipulators

Provably Safe Deep Reinforcement Learning for Robotic Manipulation in Human Environments

Trajectory Planning for Teleoperated Space Manipulators Using Deep Reinforcement Learning

A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation

Reinforcement Learning Controllers for Soft Robots using Learned Environments

Path Planning of 6-DOF Free-Floating Space Robotic Manipulators Using Reinforcement Learning

Motion Planning and Obstacle Avoidance for Robot Manipulators Using Model Predictive Control-based Reinforcement Learning

Solving Challenging Control Problems Using Two-Staged Deep Reinforcement Learning