Abstract:We present a probabilistic formulation of risk aware optimal control problems for stochastic differential equations. Risk awareness is in our framework captured by objective functions in which the risk neutral expectation is replaced by a risk function, a nonlinear functional of random variables that accounts for the controller's risk preferences. We state and prove a risk aware minimum principle that gives necessary and sufficient conditions for optimality of generalized control processes taking values on probability measures defined on a given action space. We show that going from the risk neutral to the risk aware case, the minimum principle is modified by the introduction of one additional real-valued stochastic process that acts as a risk adjustment factor. This adjustment process is explicitly given as the expectation, conditional on the filtration at the given time, of an appropriately defined functional derivative of the risk function evaluated at the random total cost. The control model we employ differs from standard relaxed controls, and we elaborate on the differences, and benefits and drawbacks, of the control types; we further give conditions under which the generalized control can be realized using a strict control process. We present an application of the results for a portfolio allocation problem and show that the risk awareness of the objective function gives rise to a risk premium term that is characterized by the risk adjustment process described above. This suggests uses of our results in e.g. pricing of risk modeled by generic risk functions in financial applications.

Duality Between Large Deviation Control and Risk-Sensitive Control for Markov Decision Processes.

On Maximizing Probabilities for Over-Performing a Target for Markov Decision Processes

Dual Adaptive Control

Risk-Sensitivity Vanishing Limit for Controlled Markov Processes

Risk Aware Minimum Principle for Optimal Control of Stochastic Differential Equations

Risk-sensitive Markov control processes

Information Relaxation and Dual Formulation of Controlled Markov Diffusions

On Average Risk-sensitive Markov Control Processes

Risk‐sensitive maximum principle for stochastic optimal control of mean‐field type Markov regime‐switching jump‐diffusion systems

Convex duality for stochastic singular control problems

A Discount Vanishing Approximation for Markov Decision Processes with Risk Sensitivity

Optimal Control of Uncertain Stochastic Systems with Markovian Switching and Its Applications to Portfolio Decisions

Markov Decision Processes under Risk Sensitivity: A Discount Vanishing Approach

Strong Duality in Risk-Constrained Nonconvex Functional Programming

Integrating Risk-Averse and Constrained Reinforcement Learning for Robust Decision-Making in High-StakesScenarios

Risk-sensitive Continuous-Time Markov Decision Processes with Unbounded Rates and Borel Spaces

Finite Horizon Partially Observable Semi-Markov Decision Processes under Risk Probability Criteria

Primal-Dual Regression Approach for Markov Decision Processes with General State and Action Spaces

On an Ergodic Two-Sided Singular Control Problem

Duality in Regret Measures and Risk Measures

Near-Optimal Mean-Variance Controls under Two-time-scale Formulations and Applications