Dynamic Fuzzy Q-Learning and Its Real-Time Application in Embedded System
LU Yong-Kui,XU Min,LI Yong-Xin,DU Hua-Sheng,WU Yue-Hua,Jie Yang
DOI: https://doi.org/10.3969/j.issn.1003-6059.2006.04.002
2006-01-01
Pattern Recognition and Artificial Intelligence
Abstract:A new dynamic fuzzy Q-learning (DFQL) method is presented in this paper which is capable of tuning fuzzy inference systems (FIS) online. In DFQL system, the generation of continuous actions depends upon a discrete number of actions of every fuzzy rule and the vector of firing strengths of fuzzy rule. In order to explore the set of possible actions and acquire experiences through the reinforcement signals, the actions are selected using an exploration-exploitation strategy based on the expended greedy algorithm. A function Q that gives the action quality with eligibility trace and meta learning rule is used here to speed up learning, e-completeness of fuzzy rules criterion and temporal-difference (TD) error criterion are considered for rule generation. The DFQL approach has been applied to a real-time control caterpillar robot for the wall following task. Experimental results and comparative studies with the fuzzy Q-learning and continuous-action Q-learning in the wall-following task of mobile robots demonstrate that the proposed DFQL method is superior.