A new XCS algorithm based on the metropolis criterion

Guoqiang Li,Hua Zou,Fangchun Yang

2011-01-01

Journal of Computational Information Systems

Abstract:LCS (Learning classifier system) was first proposed by Holland, which is an adaptive learning mechanism combined both machine learning and genetic computing. XCS is the most successful and popular LCS implementation to date, not only because of its simplistic and generality but also for its contribution to the bridging between LCS and reinforcement learning. The action selection procedure of XCS uses a combination of both exploration and exploitation, which is also called the ε-greedy selection in reinforcement learning literature. The execution probability of exploration scheme is Pexp, and otherwise the exploitation scheme will be executed. But with the increasing of XCS's accuracy, the action exploration probability should be decreased correspondingly. For this reason the ε-greedy selection scheme based XCS converges very slowly. Therefore we proposed to use the metropolis criterion to adjust the execution probability of exploration and exploitation scheme dynamically to seek the balance between the converging time and avoid the local optimization. The experiments conducted showed that the simulated annealing based XCS could have a much shorter converging time on the premise of prediction accuracy. © 2011 Binary Information Press December, 2011.