Abstract:This paper presents a study on path planning for 6-DOF free-floating space robotic manipulators using Deep Deterministic Policy Gradient-based Reinforcement Learning. The focus is the development of a novel reward function tailored to address critical requirements for efficient and effective manipulation in space. These requirements include accurate pose alignment between the end-effector and the target, collision avoidance with both the target and other links of the manipulator, smoothing of joint velocities, adaptability to strong dynamic coupling between the manipulator and its base spacecraft due to high manipulator-spacecraft mass ratio, and resilience to noise in the state observations. Uniquely, the proposed reward function employs quaternions for orientation control to reduce pose misalignments and dynamic singularities, as opposed to traditional Euler angles. Our findings demonstrate that the Reinforcement Learning algorithm, when guided by this new reward function that integrates these enhancements and constraints, not only achieves the desired path planning objectives more efficiently but also exhibits faster convergence. Furthermore, the Reinforcement Learning successfully manages significant dynamic coupling effects caused by a high mass ratio between the robotic manipulator and the base spacecraft. Even under the challenge of noisy state observations, the trained agent successfully completes the path planning task, proving the Reinforcement Learning's applicability to real-space mission designs where the noise in observation is inevitable. The study highlights the critical role of reward function design in the Reinforcement Learning training process and its consequential impact on the solution quality, providing a solid foundation for future advancements in free-floating space robotic missions.

Deep Reinforcement Learning for Multi-contact Motion Planning of Hexapod Robots

Hierarchical Free Gait Motion Planning for Hexapod Robots Using Deep Reinforcement Learning

Deep Reinforcement Learning Approach with Multiple Experience Pools for UAV's Autonomous Motion Planning in Complex Unknown Environments

A RDA-Based Deep Reinforcement Learning Approach for Autonomous Motion Planning of UAV in Dynamic Unknown Environments

A Motion Planning and Control Method of Quadruped Robot Based on Deep Reinforcement Learning

Motion planning and contact force distribution for heavy‐duty hexapod robots walking on unknown rugged terrains

Coordinated Motion Planning of Dual-arm Space Robot with Deep Reinforcement Learning

A Multitasking-Oriented Robot Arm Motion Planning Scheme Based on Deep Reinforcement Learning and Twin Synchro-Control

Deep Reinforcement Learning Based Trajectory Planning Under Uncertain Constraints

A General Framework of Motion Planning for Redundant Robot Manipulator Based on Deep Reinforcement Learning

Collaborative motion planning for multi-manipulator systems through Reinforcement Learning and Dynamic Movement Primitives

Multi-Robot Motion Planning: A Learning-Based Artificial Potential Field Solution

Criticality-Guided Deep Reinforcement Learning for Motion Planning

Path Planning of 6-DOF Free-Floating Space Robotic Manipulators Using Reinforcement Learning

CPG-based Motion Planning of Hybrid Underwater Hexapod Robot for Wall Climbing and Transition

CPG-Based Hierarchical Locomotion Control for Modular Quadrupedal Robots Using Deep Reinforcement Learning.

A Hierarchical Framework for Quadruped Robots Gait Planning Based on DDPG

Imitation-Enhanced Reinforcement Learning with Privileged Smooth Transition for Hexapod Locomotion

Contact Sequence Planning for Hexapod Robots in Sparse Foothold Environment Based on Monte-Carlo Tree

Motion Coordination Of Multiple Robots Based On Deep Reinforcement Learning