Abstract:As is well known, average-cost optimality inequalities imply the existence of stationary optimal policies for Markov decision processes with average costs per unit time, and these inequalities hold under broad natural conditions. This paper provides sufficient conditions for the validity of the average-cost optimality equation for an infinite state problem with weakly continuous transition probabilities and with possibly unbounded one-step costs and noncompact action sets. These conditions also imply the convergence of sequences of discounted relative value functions to average-cost relative value functions and the continuity of average-cost relative value functions. As shown in this paper, the classic periodic-review setup-cost inventory control problem with backorders and convex holding/backlog costs satisfies these conditions. Therefore, the optimality inequality holds in the form of an equality with a continuous average-cost relative value function for this problem. In addition, the K-convexity of discounted relative value functions and their convergence to average-cost relative value functions, when the discount factor increases to 1, imply the K-convexity of average-cost relative value functions. This implies that average-cost optimal (s, S) policies for the inventory control problem can be derived from the average-cost optimality equation.

Average Cost Minimization Problems Subject to State Constraints

Lmi Approach To Optimal Guaranteed Cost Control For A Class Of Linear Uncertain Discrete Systems

Necessary Optimality Conditions For Average Cost Minimization Problems

Optimal Guaranteed Cost Control for a Class of Linear Uncertain Time-Delay Systems

Optimal Guaranteed Cost Control of Dynamic Interval Systems: an LMI Approach

Optimal Control Problems with Incomplete and Different Integral Time Domains in the Objective and Constraints

Average cost optimal control under weak ergodicity hypotheses: Relative value iterations

Solution to the risk-sensitive average cost optimality equation in a class of Markov decision processes with finite state space

OPTIMALITY STRATEGY OF AVERAGE COST BASED PERFORMANCE POTENTIALS FOR MARKOV CONTROL PROCESS

Discrete‐Time Optimal Control of State‐Constrained Nonlinear Systems Using Approximate Dynamic Programming

On the optimality equation for average cost Markov decision processes and its validity for inventory control

Average-Cost MDPs with Infinite State and Action Sets: New Sufficient Conditions for Optimality Inequalities and Equations

On an Approximation of Average Cost per Unit Time Impulse Control of Markov Processes

Average Cost Optimality of Partially Observed MDPs: Contraction of Nonlinear Filters and Existence of Optimal Solutions and Approximations

Optimality conditions in control problems with random state constraints in probabilistic or almost-sure form

Characterization of the optimal average cost in Markov decision chains driven by a risk-seeking controller

Average Cost Optimality of Partially Observed MDPS: Contraction of Non-linear Filters, Optimal Solutions and Approximations

The necessary and sufficient conditions for stochastic differential systems with multi-time states cost functional

Lax Formulae for Efficiently Solving Two Classes of State-Constrained Optimal Control Problems

Optimal Control for a Class of Infinite Dimensional Systems Involving an $L^\infty$-term in the Cost Functional

Discretized Approximations for POMDP with Average Cost