Abstract:Q-learning is widely recognized as an effective approach for synthesizing controllers to achieve specific goals. However, handling challenges posed by continuous state-action spaces remains an ongoing research focus. This paper presents a systematic analysis that highlights a major drawback in space discretization methods. To address this challenge, the paper proposes a symbolic model that represents behavioral relations, such as alternating simulation from abstraction to the controlled system. This relation allows for seamless application of the synthesized controller based on abstraction to the original system. Introducing a novel Q-learning technique for symbolic models, the algorithm yields two Q-tables encoding optimal policies. Theoretical analysis demonstrates that these Q-tables serve as both upper and lower bounds on the Q-values of the original system with continuous spaces. Additionally, the paper explores the correlation between the parameters of the space abstraction and the loss in Q-values. The resulting algorithm facilitates achieving optimality within an arbitrary accuracy, providing control over the trade-off between accuracy and computational complexity. The obtained results provide valuable insights for selecting appropriate learning parameters and refining the controller. The engineering relevance of the proposed Q-learning based symbolic model is illustrated through two case studies.

A Novel Q-Learning Approach with Continuous States and Actions

Adaptive Double Fuzzy Systems Based Q-Learning for Pursuit-Evasion Game

Dynamic fuzzy Q-learning and control of mobile robots

Dynamic Fuzzy Q-Learning and Its Real-Time Application in Embedded System

Real-time Dynamic Fuzzy Q-learning and Control of Mobile Robots

Automatic generation of fuzzy inference systems by dynamic fuzzy Q-learning

A Q-learning Method for Continuous Space Based on Self-organizing Fuzzy RBF Network

A Fuzzy Multi-Step Q Learning Algorithm Based On Q(Lambda)- Learning And Its Application

Dynamic Self-Generated Fuzzy Systems for Reinforcement Learning.

A kind of weighted Q-learning for continuous state and action spaces

A PROPOSAL OF WEIGHTED Q-LEARNING FOR CONTINUOUS STATE AND ACTION SPACES

Online Tuning of Fuzzy Inference Systems Using Dynamic Fuzzy Q-Learning

An efficient reinforcement learning algorithm for continuous actions

A Novel Approach For Generation Of Fuzzy Neural Networks

Q learning based on self-organizing fuzzy radial basis function network

Self-Learning in Obstacle Avoidance of a Mobile Robot via Dynamic Self-Generated Fuzzy Q-Learning

Dynamic neural network control through fuzzy Q-learning algorithms

Multi-Agent Reward-Iteration Fuzzy Q-Learning

How to discretize continuous state-action spaces in Q-learning: A symbolic control approach

Automatic Generation Of Fuzzy Inference Systems Using Incremental-Topological-Preserving-Map-Based Fuzzy Q-Learning

Parametrized deep q-networks learning: Reinforcement learning with discrete-continuous hybrid action space