Abstract:This paper presents a study on path planning for 6-DOF free-floating space robotic manipulators using Deep Deterministic Policy Gradient-based Reinforcement Learning. The focus is the development of a novel reward function tailored to address critical requirements for efficient and effective manipulation in space. These requirements include accurate pose alignment between the end-effector and the target, collision avoidance with both the target and other links of the manipulator, smoothing of joint velocities, adaptability to strong dynamic coupling between the manipulator and its base spacecraft due to high manipulator-spacecraft mass ratio, and resilience to noise in the state observations. Uniquely, the proposed reward function employs quaternions for orientation control to reduce pose misalignments and dynamic singularities, as opposed to traditional Euler angles. Our findings demonstrate that the Reinforcement Learning algorithm, when guided by this new reward function that integrates these enhancements and constraints, not only achieves the desired path planning objectives more efficiently but also exhibits faster convergence. Furthermore, the Reinforcement Learning successfully manages significant dynamic coupling effects caused by a high mass ratio between the robotic manipulator and the base spacecraft. Even under the challenge of noisy state observations, the trained agent successfully completes the path planning task, proving the Reinforcement Learning's applicability to real-space mission designs where the noise in observation is inevitable. The study highlights the critical role of reward function design in the Reinforcement Learning training process and its consequential impact on the solution quality, providing a solid foundation for future advancements in free-floating space robotic missions.

A Trajectory Planning Method for Capture Operation of Space Robotic Arm Based on Deep Reinforcement Learning

Discrete trajectory planning for robotic arms

Autonomous trajectory planner for space telerobots capturing space debris under the teleprogramming framework

A Multi-Target Trajectory Planning of a 6-Dof Free-Floating Space Robot Via Reinforcement Learning

Trajectory Planning for Teleoperated Space Manipulators Using Deep Reinforcement Learning

A Trajectory Planning Method For Rapid Capturing An Unknown Space Tumbling Target

An Integrated Tracking Control Approach Based on Reinforcement Learning for a Continuum Robot in Space Capture Missions

Reinforcement learning in dual-arm trajectory planning for a free-floating space robot

Coordinated Control Based on Reinforcement Learning for Dual-Arm Continuum Manipulators in Space Capture Missions

An End-to-End Trajectory Planning Strategy for Free-floating Space Robots

Multi-Objective Optimal Trajectory Planning for Robotic Arms Using Deep Reinforcement Learning

Area-oriented Coordinated Trajectory Planning of Dual-Arm Space Robot for Capturing a Tumbling Target

Trajectory planning of a dual-arm space robot for target capturing with minimizing base disturbance

Path Planning of 6-DOF Free-Floating Space Robotic Manipulators Using Reinforcement Learning

Modeling and Planning of a Space Robot for Capturing Tumbling Target by Approaching the Dynamic Closest Point

Trajectory Planning and Coordination Control of a Space Robot for Detumbling a Flexible Tumbling Target in Post-Capture Phase

Capture and Detumbling Control for Active Debris Removal by a Dual-Arm Space Robot

A Learning System for Motion Planning of Free-Float Dual-Arm Space Manipulator towards Non-Cooperative Object

Directly Skill-Transferred DMP-based Trajectory Planning for Robot Soft-Capture of Space Tumbling Targets

Improving Soft-Capture Phase Success in Space Debris Removal Missions: Leveraging Deep Reinforcement Learning and Tactile Feedback

Trajectory Planning of Space Robot System for Target Berthing and Reorientation after Capturing