Abstract:Learning automata (LA), a powerful tool for reinforcement learning in the field of machine learning, could explore its optimal state by continuously interacting with an external environment. Generally, the traditional LA algorithms, especially estimator LA algorithms, can be ultimately abstracted out as P- or Q-models, which are simply located in the stationary environments. A more comprehensive consideration would be S-model operating in the non-stationary environment. For this specific LA, presently the most popular achievement belongs to stochastic estimator LA (SELA). However, synchronously handing four parameters involved in SELA is an intractable job, as these parameters may vary dramatically in values under different environments, making it essential to develop a strategy for parameter tuning. In this paper, we first propose a scheme to determine the parameter searching scope and subsequently present a series of parameter searching methods, including a four-dimensional method and a two-dimensional method, making SELA applicable for any environment with switching non-stationary characteristics. Furthermore, to decrease the tuning cost, a reduced parameter SELA supported by the new two-dimensional parameter searching method emerges. And to break the traditional limit that the environmental reward probability must be symmetrically distributed, the S-model is constructed from a new perspective, thus forming a novel reduced parameter S-model of SELA (rpS-SELA). A detailed mathematical proof theoretically reveals the absolute expediency of rpS-SELA. In addition, it is demonstrated by experimental simulations that rpS-SELA outperforms others with a reduced tuning cost, a minor time consumption, a higher accuracy rate, and above all, a stronger tracking ability to the environmental switches.

Weak Estimator-Based Stochastic Searching on the Line in Dynamic Dual Environments.

Random Walk-Based Solution to Triple Level Stochastic Point Location Problem

Solving Stationary and Stochastic Point Location Problem with Optimal Computing Budget Allocation

Learning to Seek: Multi-Agent Online Source Seeking Against Non-Stochastic Disturbances

Estimation of Distribution Algorithm with Scatter Search for Dynamic Optimisation Problems

Thompson Sampling Guided Stochastic Searching on the Line for Deceptive Environments with Applications to Root-Finding Problems

Distributed Extremum Seeking with Stochastic Perturbations

A General Strategy for Solving the Stochastic Point Location Problem by Utilizing the Correlation of Three Adjacent Nodes

Tracking Property of UMDA in Dynamic Environment by Landscape Framework

Probabilistic Line Searches for Stochastic Optimization

A Stochastically Evolving Non-local Search and Solutions to Inverse Problems with Sparse Data

Cooperative Active Learning-Based Dual Control for Exploration and Exploitation in Autonomous Search

Balancing Search and Estimation in Random Search Based Stochastic Simulation Optimization

Dynamic Tracking with Fuzzy Rules for Evolutionary Dynamic Constrained Optimization

Multistep Dual Control for Exploration and Exploitation in Autonomous Search With Convergence Guarantee

Stochastic Location Optimization in A Dynamic Environment

A Mahalanobis Distance-based Approach for Dynamic Multi-objective Optimization with Stochastic Changes

A Novel Reduced Parameter S-Model of Estimator Learning Automata in the Switching Non-Stationary Environment

A Stability Principle for Learning under Non-Stationarity

Stochastic Local Search Using the Search Space Smoothing Meta-Heuristic: A Case Study.

REAL-TIME HUMAN TRACKING BASED ON SWITCHING LINEAR DYNAMIC SYSTEM COMBINED WITH ADAPTIVE MEANSHIFT TRACKER