Abstract:In the distributed network service systems such as streaming-media systems and resource-sharing systems with multiple service nodes, admission control (AC) technology is an essential way to enhance performance. Model-based optimization approaches are good ways to be applied to analyze and solve the optimal AC policy. However, due to “the curse of dimensionality”, computing such policy for practical systems is a rather difficult task. In this paper, we consider a general model of the distributed network service systems, and address the problem of designing an optimal AC policy. An analytical model is presented for the system with fixed parameters based on semi-Markov decision process (SMDP). We design an event-driven AC policy, and the stationary randomized policy is taken as the policy structure. To solve the SMDP, both the state aggregation approach and the reinforcement-learning (RL) method with online policy optimization algorithm are applied. Then, we extend the problem by considering the system with time-varying parameters, where the arrival rates of requests at each service node may change over time. In view of this situation, an AC policy switching mechanism is presented. This mechanism allows the system to decide whether to adjust its AC policy according to the policy switching rule. And in order to maximize the gain of system, that is, to obtain the optimal AC policy switching rule, another RL-based algorithm is applied. To assess the effectiveness of SMDP-based AC policy and policy switching mechanism for the system, numerical experiments are presented. We compare the performance of optimal policies obtained by the solutions of proposed methods with other classical AC policies. The simulation results illustrate that higher performance and computational efficiency could be achieved by using the SMDP model and RL-based algorithms proposed in this paper.

A General Model for Non-Markovian Stochastic Decision Discrete-Event Systems

An MDP-based Handoff Decision Algorithm for Multi-Domain Heterogeneous Wireless Access Networks

A general framework for analyzing the optimal call admission control in DS-CDMA cellular network

Stochastic Hybrid Netcad Systems for Modeling Call Admission and Routing Control in Networks

Novel Stochastic Model For Call Admission Control In Broadband Wireless Multimedia Networks

Intelligent Gateway Selection and User Scheduling in Non-Stationary Air-Ground Networks

Solving the non-preemptive two queue polling model with generally distributed service and switch-over durations and Poisson arrivals as a Semi-Markov Decision Process

A reinforcement-learning approach for admission control in distributed network service systems

Markov Decision Processes with State-Dependent Discount Factors and Unbounded Rewards/costs.

A Generalized Markov Chain Model for IEEE 802.11 Distributed Coordination Function.

Stochastic Control of Network Systems I: NETCAD State Space Structure & Dynamics

Towards Analysis Of Semi-Markov Decision Processes

Mixed Markov Decision Processes in a Semi-Markov Environment with Discounted Criterion

Switching-POMDP Based Admission Control Policies for Service Systems with Distributed Architecture.

Network Security Defense Decision-Making Method Based on Stochastic Game and Deep Reinforcement Learning

Modeling Decision Process in Multi-Agent Systems: A Graphical Markov Game Based Approach

A New Stochastic Admission Control Scheme for Wireless Networks

Wi-Fi Optimization with Deep Diffusion Deterministic Policy

Research On Modeling The Nonhomogenous Markov Decision Systems With Dynamic Bayesian Network

An adaptive call admission policy for broadband wireless multimedia networks using stochastic control

Semi-Infinitely Constrained Markov Decision Processes and Efficient Reinforcement Learning