Abstract:Efficient and stable exploration remains a key challenge for deep reinforcement learning (DRL) operating in high-dimensional action and state spaces. Recently, a more promising approach by combining the exploration in the action space with the exploration in the parameters space has been proposed to get the best of both methods. In this article, we propose a new iterative and close-loop framework by combining the evolutionary algorithm (EA), which does explorations in a gradient-free manner directly in the parameters space with an actor-critic, and the deep deterministic policy gradient (DDPG) reinforcement learning algorithm, which does explorations in a gradient-based manner in the action space to make these two methods cooperate in a more balanced and efficient way. In our framework, the policies represented by the EA population (the parametric perturbation part) can evolve in a guided manner by utilizing the gradient information provided by the DDPG and the policy gradient part (DDPG) is used only as a fine-tuning tool for the best individual in the EA population to improve the sample efficiency. In particular, we propose a criterion to determine the training steps required for the DDPG to ensure that useful gradient information can be generated from the EA generated samples and the DDPG and EA part can work together in a more balanced way during each generation. Furthermore, within the DDPG part, our algorithm can flexibly switch between fine-tuning the same previous RL-Actor and fine-tuning a new one generated by the EA according to different situations to further improve the efficiency. Experiments on a range of challenging continuous control benchmarks demonstrate that our algorithm outperforms related works and offers a satisfactory trade-off between stability and sample efficiency.

Toward solving the critical care manpower crisis.

Optimization of Robotic Arm Grasping through Fractional-Order Deep Deterministic Policy Gradient Algorithm

Regularly Updated Deterministic Policy Gradient Algorithm

Network Architecture for Optimizing Deep Deterministic Policy Gradient Algorithms

Path Following for Autonomous Ground Vehicle Using DDPG Algorithm: A Reinforcement Learning Approach

Path Planning in Complex Environments Using Attention-Based Deep Deterministic Policy Gradient

PP-PG: Combining Parameter Perturbation with Policy Gradient Methods for Effective and Efficient Explorations in Deep Reinforcement Learning

Power Allocation for Full-Duplex Communication Systems Based on Deep Deterministic Policy Gradient

Edge Delayed Deep Deterministic Policy Gradient: efficient continuous control for edge scenarios

Research on Collision-free Control and Simulation of Single-Agent Based on An Improved DDPG Algorithm

Improved Exploration through Latent Trajectory Optimization in Deep Deterministic Policy Gradient

A path planning algorithm fusion of obstacle avoidance and memory functions

Prioritized experience replay in path planning via multi-dimensional transition priority fusion

UAV Path Planning Based on Multicritic-Delayed Deep Deterministic Policy Gradient

A decision-making of autonomous driving method based on DDPG with pretraining

Deterministic Value-Policy Gradients

Self-Adaptive Double Bootstrapped DDPG

Efficient Path Planning for Mobile Robot Based on Deep Deterministic Policy Gradient

Diffusion Policy Policy Optimization

ETGL-DDPG: A Deep Deterministic Policy Gradient Algorithm for Sparse Reward Continuous Control

Improved Communication Efficiency in Federated Natural Policy Gradient via ADMM-based Gradient Updates