Abstract:For optimal control of diffusions under several criteria, due to computational or analytical reasons, many studies have a apriori assumed control policies to be Lipschitz or smooth, often with no rigorous analysis on whether this restriction entails loss. While optimality of Markov/stationary Markov policies for expected finite horizon/infinite horizon (discounted/ergodic) cost and cost-up-to-exit time optimal control problems can be established under certain technical conditions, an optimal solution is typically only measurable in the state (and time, if the horizon is finite) with no apriori additional structural properties. In this paper, building on our recent work [S. Pradhan and S. Yüksel, Continuity of cost in Borkar control topology and implications on discrete space and time approximations for controlled diffusions under several criteria, Electronic Journal of Probability (2024)] establishing the regularity of optimal cost on the space of control policies under the Borkar control topology for a general class of diffusions, we establish near optimality of smooth/Lipschitz continuous policies for optimal control under expected finite horizon, infinite horizon discounted/average, and up-to-exit time cost criteria. Under mild assumptions, we first show that smooth/Lipschitz continuous policies are dense in the space of Markov/stationary Markov policies under the Borkar topology. Then utilizing the continuity of optimal costs as a function of policies on the space of Markov/stationary policies under the Borkar topology, we establish that optimal policies can be approximated by smooth/Lipschitz continuous policies with arbitrary precision. While our results are extensions of our recent work, the practical significance of an explicit statement and accessible presentation dedicated to Lipschitz and smooth policies, given their prominence in the literature, motivates our current paper.

On Borkar and Young Relaxed Control Topologies and Continuous Dependence of Invariant Measures on Control Policy

Near Optimality of Lipschitz and Smooth Policies in Controlled Diffusions

Another Look at Partially Observed Optimal Stochastic Control: Existence, Ergodicity, and Approximations without Belief-Reduction

Existence of Markov equilibrium control in discrete time

Controlled Diffusions under Full, Partial and Decentralized Information: Existence of Optimal Policies and Discrete-Time Approximations

Continuous-Time Controlled Markov Chains

On Strategic Measures and Optimality Properties in Discrete-Time Stochastic Control with Universally Measurable Policies

Drift and Monotonicity Conditions for Continuous-Time Controlled Markov Chains with an Average Criterion.

Strong and Weak Equilibria for Time-Inconsistent Stochastic Control in Continuous Time

Maximizing the probability of visiting a set infinitely often for a Markov decision process with Borel state and action spaces

Stochastic Near-Optimal Controls for Path-Dependent Systems

Continuous time Stochastic optimal control under discrete time partial observations

Partially Observed Optimal Stochastic Control: Regularity, Optimality, Approximations, and Learning

REPRESENTATION FORMULAS FOR LIMIT VALUES OF LONG RUN STOCHASTIC OPTIMAL CONTROLS

Isomorphism Properties of Optimality and Equilibrium Solutions under Equivalent Information Structure Transformations I: Stochastic Dynamic Teams

Controllability of Measure Driven Evolution Systems with Nonlocal Conditions

Robust affine control of linear stochastic systems

Robust Policy Optimization in Continuous-time Mixed $\mathcal{H}_2/\mathcal{H}_\infty$ Stochastic Control

The minimal hitting probability of continuous-time controlled Markov systems with countable states

Robustness of Optimal Controlled Diffusions with Near-Brownian Noise via Rough Paths Theory

Chaos of Transformations Induced Onto the Space of Probability Measures