Abstract:For optimal control of diffusions under several criteria, due to computational or analytical reasons, many studies have a apriori assumed control policies to be Lipschitz or smooth, often with no rigorous analysis on whether this restriction entails loss. While optimality of Markov/stationary Markov policies for expected finite horizon/infinite horizon (discounted/ergodic) cost and cost-up-to-exit time optimal control problems can be established under certain technical conditions, an optimal solution is typically only measurable in the state (and time, if the horizon is finite) with no apriori additional structural properties. In this paper, building on our recent work [S. Pradhan and S. Yüksel, Continuity of cost in Borkar control topology and implications on discrete space and time approximations for controlled diffusions under several criteria, Electronic Journal of Probability (2024)] establishing the regularity of optimal cost on the space of control policies under the Borkar control topology for a general class of diffusions, we establish near optimality of smooth/Lipschitz continuous policies for optimal control under expected finite horizon, infinite horizon discounted/average, and up-to-exit time cost criteria. Under mild assumptions, we first show that smooth/Lipschitz continuous policies are dense in the space of Markov/stationary Markov policies under the Borkar topology. Then utilizing the continuity of optimal costs as a function of policies on the space of Markov/stationary policies under the Borkar topology, we establish that optimal policies can be approximated by smooth/Lipschitz continuous policies with arbitrary precision. While our results are extensions of our recent work, the practical significance of an explicit statement and accessible presentation dedicated to Lipschitz and smooth policies, given their prominence in the literature, motivates our current paper.

Local Poisson Equations Associated with Discrete-Time Markov Control Processes

Solution to the risk-sensitive average cost optimality equation in a class of Markov decision processes with finite state space

On Average Risk-sensitive Markov Control Processes

Risk-Sensitive Optimal Control in Communicating Average Markov Decision Chains

Characterization of the optimal average cost in Markov decision chains driven by a risk-seeking controller

Average Continuous Control of Piecewise Deterministic Markov Processes

Risk-Sensitive Average Markov Decision Processes in General Spaces

Optimal Stationary Policies for a Class of Countable Markov Control Processes

Discrete-Time Hybrid Control Processes with Unbounded Costs

Average cost optimal control under weak ergodicity hypotheses: Relative value iterations

Risk-sensitive Markov control processes

Controlled Semi-Markov Chains with Risk-Sensitive Average Cost Criterion

Stability analysis of optimal control problems with time-dependent costs

Optimal control of nonlocal continuity equations: numerical solution

Optimal Stationary Policies for Semi-Markov Control Processes with Discounted-Cost Criteria

Necessary Optimality Conditions For Average Cost Minimization Problems

Asynchronous Control for Discrete-Time Markovian Jump Systems with Multiplicative Noise

Near Optimality of Lipschitz and Smooth Policies in Controlled Diffusions

Optimal control formulation of transition path problems for Markov Jump Processes

Mean Field Markov Decision Processes

Constrained Markov Decision Processes with Non-constant Discount Factor