Abstract:This paper presents a study on path planning for 6-DOF free-floating space robotic manipulators using Deep Deterministic Policy Gradient-based Reinforcement Learning. The focus is the development of a novel reward function tailored to address critical requirements for efficient and effective manipulation in space. These requirements include accurate pose alignment between the end-effector and the target, collision avoidance with both the target and other links of the manipulator, smoothing of joint velocities, adaptability to strong dynamic coupling between the manipulator and its base spacecraft due to high manipulator-spacecraft mass ratio, and resilience to noise in the state observations. Uniquely, the proposed reward function employs quaternions for orientation control to reduce pose misalignments and dynamic singularities, as opposed to traditional Euler angles. Our findings demonstrate that the Reinforcement Learning algorithm, when guided by this new reward function that integrates these enhancements and constraints, not only achieves the desired path planning objectives more efficiently but also exhibits faster convergence. Furthermore, the Reinforcement Learning successfully manages significant dynamic coupling effects caused by a high mass ratio between the robotic manipulator and the base spacecraft. Even under the challenge of noisy state observations, the trained agent successfully completes the path planning task, proving the Reinforcement Learning's applicability to real-space mission designs where the noise in observation is inevitable. The study highlights the critical role of reward function design in the Reinforcement Learning training process and its consequential impact on the solution quality, providing a solid foundation for future advancements in free-floating space robotic missions.

Control of Space Flexible Manipulator Using Soft Actor-Critic and Random Network Distillation

Model-free Maneuvering Control of Fixed-Wing UAVs Based on Deep Reinforcement Learning

Robotic Manipulator in Dynamic Environment with SAC Combing Attention Mechanism and LSTM

Trajectory tracking control based on deep reinforcement learning and ensemble random network distillation for robotic manipulator

Path Planning of 6-DOF Free-Floating Space Robotic Manipulators Using Reinforcement Learning

Coordinated Control Based on Reinforcement Learning for Dual-Arm Continuum Manipulators in Space Capture Missions

A Deep Reinforcement Learning Solution for the Low Level Motion Control of a Robot Manipulator System

Optimized SAC Deep Reinforcement Learning Control for Electro-hydraulic Servo Systems

Path Planning for Multi-Arm Manipulators Using Deep Reinforcement Learning: Soft Actor–Critic with Hindsight Experience Replay

N-space (NS)-Framework: a generalized solution for soft continuum robots via reinforcement learning

Constrained motion planning of free-float dual-arm space manipulator via deep reinforcement learning

Learning to Control a Free-floating Space Robot Using Deep Reinforcement Learning

An Efficiently Convergent Deep Reinforcement Learning-Based Trajectory Planning Method for Manipulators in Dynamic Environments

Control Of Flexible Manipulator Based On Reinforcement Learning

A Cable‐Actuated Soft Manipulator for Dexterous Grasping Based on Deep Reinforcement Learning

Trajectory Planning for Teleoperated Space Manipulators Using Deep Reinforcement Learning

Adversarial Skill Learning for Robust Manipulation

Reinforcement Learning Control of a Single-Link Flexible Robotic Manipulator

Deep MARL Based Resilient Motion Planning for Decentralized Space Manipulator

Uncertainty Aware Model Predictive Control for Free-Floating Space Manipulator Based on Probabilistic Ensembles Neural Network

A deep reinforcement learning based method for real-time path planning and dynamic obstacle avoidance