Abstract:The approach of directly combining clustering method and reinforcement learning (RL) will lead to encounter the issue where states may have different state transition processes under the same action, resulting in poor policy performance. To address this challenge with multi-dimensional continuous observation data, an improved reinforcement learning method based on unsupervised learning is proposed with a novel framework. Instead of dimensionality reduction methods, unsupervised clustering is employed to indirectly capture the underlying structure of the data. First, the proposed framework incorporates multi-dimensional information, including the current observation data, the next observation data and reward information, during the clustering process, leading to a more accurate and comprehensive low-dimensional discrete representation of the observation data while retaining preserving transition of Markov decision process. Second, by compressing the observation data into a well-defined state space, the resulting cluster labels serve as the low-dimensional discrete label-states for reinforcement learning to generate more effective and robust policies. Comparative analysis with state-of-the-art RL methods demonstrates that the improved RL methods base on framework achieves higher rewards, indicating its superior performance. Furthermore, the framework exhibits computational efficiency, as evidenced by its reasonable time complexity. This structural innovation allows for better exploration and exploitation of the transition, leading to improved policy performance in engineering applications.

Reinforcement Learning Method of Continuous State Adaptively Discretized Based on K-means Clustering

Successive Convex Approximation Based Off-Policy Optimization for Constrained Reinforcement Learning

Hierarchical Clustering-Based State Grouping Reinforcement Learning for Switching Decision of Autonomous Vehicles

An Improved Reinforcement Learning Method Based on Unsupervised Learning

Reinforcement Learning Method For Continuous State Space Based On Dynamic Neural Network

Clustered Reinforcement Learning

A Q-learning Method for Continuous Space Based on Self-organizing Fuzzy RBF Network

Fuzzy CMAC with automatic state partition for reinforcementlearning

Continuous-action reinforcement learning with fast policy search and adaptive basis function selection

An efficient reinforcement learning algorithm for continuous actions

Effective Experiences Collection and State Aggregation in Reinforcement Learning

Tree Based Discretization for Continuous State Space Reinforcement Learning

Deep Reinforcement Clustering

Incremental Reinforcement Learning in Continuous Spaces Via Policy Relaxation and Importance Weighting.

Enhancing Online Continual Learning with Plug-and-Play State Space Model and Class-Conditional Mixture of Discretization

Deep Multi-Agent Reinforcement Learning with Discrete-Continuous Hybrid Action Spaces

Kernel-Based Continuous-Action Actor-Critic Learning

Clustering state membership-based Q-learning for dynamic scheduling

Continual Reinforcement Learning with Diversity Exploration and Adversarial Self-Correction

Regularization-Based Efficient Continual Learning in Deep State-Space Models

Hierarchical Subspaces of Policies for Continual Offline Reinforcement Learning