Risk-sensitive control of reflected diffusion processes on orthrant

Sunil Kumar Gauttam,K. Suresh Kumar,Chandan Pal

DOI: https://doi.org/10.48550/arXiv.1701.01213

2017-01-05

Optimization and Control

Abstract:In this article, we prove the existence of optimal risk-sensitive control with state constraints. We use near monotone assumption on the running cost to prove the existence of optimal risk-sensitive control.

What problem does this paper attempt to address?

On optimal control of reflected diffusions

Adam Jonsson

2023-11-07

Abstract:We study a simple singular control problem for a Brownian motion with constant drift and variance reflected at the origin. Exerting control pushes the process towards the origin and generates a concave increasing state-dependent yield which is discounted at a fixed rate. The most interesting feature of the problem is that its solution can be more complicated than anticipated. Indeed, for some parameter values, the optimal policy involves two reflecting barriers and one repelling boundary, the action region being the union of two disjoint intervals. We also show that the apparent anomaly can be understood as involving a switch between two strategies with different risk profiles: The risk-neutral decision maker initially gambles on the more risky strategy, but lowers risk if this strategy underperforms.

Probability,Optimization and Control
Discrete-Time Approximation of Risk-Averse Control Problems for Diffusion Processes

Andrzej Ruszczynski,Jianing Yao

DOI: https://doi.org/10.48550/arXiv.1508.05316

2016-08-20

Abstract:We consider optimal control problems for diffusion processes, where the objective functional is defined by a time-consistent dynamic risk measure. We focus on coherent risk measures defined by $g$-evaluations. For such problems, we construct a family of time and space perturbed systems with piecewise-constant control functions. We obtain a regularized optimal value function by a special mollification procedure. This allows us to establish a bound on the difference between the optimal value functions of the original problem and of the problem with piecewise-constant controls.

Optimization and Control
Data-driven rules for multidimensional reflection problems

Sören Christensen,Asbjørn Holk Thomsen,Lukas Trottner

2023-11-12

Abstract:Over the recent past data-driven algorithms for solving stochastic optimal control problems in face of model uncertainty have become an increasingly active area of research. However, for singular controls and underlying diffusion dynamics the analysis has so far been restricted to the scalar case. In this paper we fill this gap by studying a multivariate singular control problem for reversible diffusions with controls of reflection type. Our contributions are threefold. We first explicitly determine the long-run average costs as a domain-dependent functional, showing that the control problem can be equivalently characterized as a shape optimization problem. For given diffusion dynamics, assuming the optimal domain to be strongly star-shaped, we then propose a gradient descent algorithm based on polytope approximations to numerically determine a cost-minimizing domain. Finally, we investigate data-driven solutions when the diffusion dynamics are unknown to the controller. Using techniques from nonparametric statistics for stochastic processes, we construct an optimal domain estimator, whose static regret is bounded by the minimax optimal estimation rate of the unreflected process' invariant density. In the most challenging situation, when the dynamics must be learned simultaneously to controlling the process, we develop an episodic learning algorithm to overcome the emerging exploration-exploitation dilemma and show that given the static regret as a baseline, the loss in its sublinear regret per time unit is of natural order compared to the one-dimensional case.

Optimization and Control,Probability,Statistics Theory,Machine Learning
Robustness of Stochastic Optimal Control to Approximate Diffusion Models under Several Cost Evaluation Criteria

Somnath Pradhan,Serdar Yuksel

2023-09-16

Abstract:In control theory, typically a nominal model is assumed based on which an optimal control is designed and then applied to an actual (true) system. This gives rise to the problem of performance loss due to the mismatch between the true model and the assumed model. A robustness problem in this context is to show that the error due to the mismatch between a true model and an assumed model decreases to zero as the assumed model approaches the true model. We study this problem when the state dynamics of the system are governed by controlled diffusion processes. In particular, we will discuss continuity and robustness properties of finite horizon and infinite-horizon $\alpha$-discounted/ergodic optimal control problems for a general class of non-degenerate controlled diffusion processes, as well as for optimal control up to an exit time. Under a general set of assumptions and a convergence criterion on the models, we first establish that the optimal value of the approximate model converges to the optimal value of the true model. We then establish that the error due to mismatch that occurs by application of a control policy, designed for an incorrectly estimated model, to a true model decreases to zero as the incorrect model approaches the true model. We will see that, compared to related results in the discrete-time setup, the continuous-time theory will let us utilize the strong regularity properties of solutions to optimality (HJB) equations, via the theory of uniformly elliptic PDEs, to arrive at strong continuity and robustness properties.

Optimization and Control
On the hierarchical risk-averse control problems for diffusion processes

Getachew K. Befekadu,Alexander Veremyev,Eduardo L. Pasiliao

DOI: https://doi.org/10.48550/arXiv.1603.03359

2016-03-10

Optimization and Control

Abstract:In this paper, we consider a risk-averse control problem for diffusion processes, in which there is a partition of the admissible control strategy into two decision-making groups (namely, the {\it leader} and {\it follower}) with different cost functionals and risk-averse satisfactions. Our approach, based on a hierarchical optimization framework, requires that a certain level of risk-averse satisfaction be achieved for the {\it leader} as a priority over that of the {\it follower's} risk-averseness. In particular, we formulate such a risk-averse control problem involving a family of time-consistent dynamic convex risk measures induced by conditional $g$-expectations (i.e., filtration-consistent nonlinear expectations associated with the generators of certain backward stochastic differential equations). Moreover, under suitable conditions, we establish the existence of optimal risk-averse solutions, in the sense of viscosity solutions, for the corresponding risk-averse dynamic programming equations. Finally, we briefly comment on the implication of our results.
Risk Aware Minimum Principle for Optimal Control of Stochastic Differential Equations

Jukka Isohtl,William B. Haskell,Jukka Isohatala

DOI: https://doi.org/10.1109/tac.2021.3123229

IF: 6.549

2021-01-01

IEEE Transactions on Automatic Control

Abstract:We present a probabilistic formulation of risk aware optimal control problems for stochastic differential equations. Risk awareness is in our framework captured by objective functions in which the risk neutral expectation is replaced by a risk function, a nonlinear functional of random variables that accounts for the controller's risk preferences. We state and prove a risk aware minimum principle that gives necessary and sufficient conditions for optimality of generalized control processes taking values on probability measures defined on a given action space. We show that going from the risk neutral to the risk aware case, the minimum principle is modified by the introduction of one additional real-valued stochastic process that acts as a risk adjustment factor. This adjustment process is explicitly given as the expectation, conditional on the filtration at the given time, of an appropriately defined functional derivative of the risk function evaluated at the random total cost. The control model we employ differs from standard relaxed controls, and we elaborate on the differences, and benefits and drawbacks, of the control types; we further give conditions under which the generalized control can be realized using a strict control process. We present an application of the results for a portfolio allocation problem and show that the risk awareness of the objective function gives rise to a risk premium term that is characterized by the risk adjustment process described above. This suggests uses of our results in e.g. pricing of risk modeled by generic risk functions in financial applications.

automation & control systems,engineering, electrical & electronic
Risk-sensitive control of continuous time Markov chains

Mrinal K. Ghosh,Subhamay Saha

DOI: https://doi.org/10.48550/arXiv.1409.4032

2014-09-14

Abstract:We study risk-sensitive control of continuous time Markov chains taking values in discrete state space. We study both finite and infinite horizon problems. In the finite horizon problem we characterise the value function via HJB equation and obtain an optimal Markov control. We do the same for infinite horizon discounted cost case. In the infinite horizon average cost case we establish the existence of an optimal stationary control under certain Lyapunov condition. We also develop a policy iteration algorithm for finding an optimal control.

Optimization and Control
Risk-averse risk-constrained optimal control

Pantelis Sopasakis,Mathijs Schuurmans,Panagiotis Patrinos

DOI: https://doi.org/10.48550/arXiv.1903.06749

2019-03-16

Abstract:Multistage risk-averse optimal control problems with nested conditional risk mappings are gaining popularity in various application domains. Risk-averse formulations interpolate between the classical expectation-based stochastic and minimax optimal control. This way, risk-averse problems aim at hedging against extreme low-probability events without being overly conservative. At the same time, risk-based constraints may be employed either as surrogates for chance (probabilistic) constraints or as a robustification of expectation-based constraints. Such multistage problems, however, have been identified as particularly hard to solve. We propose a decomposition method for such nested problems that allows us to solve them via efficient numerical optimization methods. Alongside, we propose a new form of risk constraints which accounts for the propagation of uncertainty in time.

Optimization and Control
On a Class of Singular Stochastic Control Problems for Reflected Diffusions

Giorgio Ferrari

DOI: https://doi.org/10.48550/arXiv.1711.03741

2017-11-10

Abstract:Reflected diffusions naturally arise in many problems from applications ranging from economics and mathematical biology to queueing theory. In this paper we consider a class of infinite time-horizon singular stochastic control problems for a general one-dimensional diffusion that is reflected at zero. We assume that exerting control leads to a state-dependent instantaneous reward, whereas reflecting the diffusion at zero gives rise to a proportional cost with constant marginal value. The aim is to maximize the total expected reward, minus the total expected cost of reflection. We show that depending on the properties of the state-dependent instantaneous reward we can have qualitatively different kinds of optimal strategies. The techniques employed are those of stochastic control and of the theory of linear diffusions.

Optimization and Control
On Average Risk-sensitive Markov Control Processes

Yun Shen,Klaus Obermayer,Wilhelm Stannat

DOI: https://doi.org/10.48550/arXiv.1403.3321

2015-07-22

Abstract:We introduce the Lyapunov approach to optimal control problems of average risk-sensitive Markov control processes with general risk maps. Motivated by applications in particular to behavioral economics, we consider possibly non-convex risk maps, modeling behavior with mixed risk preference. We introduce classical objective functions to the risk-sensitive setting and we are in particular interested in optimizing the average risk in the infinite-time horizon for Markov Control Processes on general, possibly non-compact, state spaces allowing also unbounded cost. Existence and uniqueness of an optimal control is obtained with a fixed point theorem applied to the nonlinear map modeling the risk-sensitive expected total cost. The necessary contraction is obtained in a suitable chosen seminorm under a new set of conditions: 1) Lyapunov-type conditions on both risk maps and cost functions that control the growth of iterations, and 2) Doeblin-type conditions, known for Markov chains, generalized to nonlinear mappings. In the particular case of the entropic risk map, the above conditions can be replaced by the existence of a Lyapunov function, a local Doeblin-type condition for the underlying Markov chain, and a growth condition on the cost functions.

Optimization and Control
Stochastic Control Problems with State Reflections Arising from Relaxed Benchmark Tracking

Lijun Bo,Yijie Huang,Xiang Yu

DOI: https://doi.org/10.1287/moor.2023.0265

IF: 2.215

2024-10-02

Mathematics of Operations Research

Abstract:This paper studies stochastic control problems motivated by optimal consumption with wealth benchmark tracking. The benchmark process is modeled by a combination of a geometric Brownian motion and a running maximum process, indicating its increasing trend in the long run. We consider a relaxed tracking formulation such that the wealth compensated by the injected capital always dominates the benchmark process. The stochastic control problem is to maximize the expected utility of consumption deducted by the cost of the capital injection under the dynamic floor constraint. By introducing two auxiliary state processes with reflections, an equivalent auxiliary control problem is formulated and studied, which leads to the Hamilton-Jacobi-Bellman equation with two Neumann boundary conditions. We establish the existence of a unique classical solution to the dual partial differential equation using some novel probabilistic representations involving the local time of some dual processes together with a tailor-made decomposition-homogenization technique. The proof of the verification theorem on the optimal feedback control can be carried out by some stochastic flow analysis and technical estimations of the optimal control. Funding: L. Bo and Y. Huang are supported by the National Natural Science Foundation of China [Grant 12471451], the Natural Science Basic Research Program of Shaanxi [Grant 2023-JC-JQ-05], the Shaanxi Fundamental Science Research Project for Mathematics and Physics [Grant 23JSZ010], and Fundamental Research Funds for the Central Universities [Grant 20199235177]. X. Yu is supported by the Hong Kong RGC General Research Fund (GRF) [Grant 15304122] and the Research Centre for Quantitative Finance at the Hong Kong Polytechnic University [Grant P0042708].

mathematics, applied,operations research & management science
Risk-sensitive Markov control processes

Yun Shen,Wilhelm Stannat,Klaus Obermayer

DOI: https://doi.org/10.1137/120899005

2014-01-24

Abstract:We introduce a general framework for measuring risk in the context of Markov control processes with risk maps on general Borel spaces that generalize known concepts of risk measures in mathematical finance, operations research and behavioral economics. Within the framework, applying weighted norm spaces to incorporate also unbounded costs, we study two types of infinite-horizon risk-sensitive criteria, discounted total risk and average risk, and solve the associated optimization problems by dynamic programming. For the discounted case, we propose a new discount scheme, which is different from the conventional form but consistent with the existing literature, while for the average risk criterion, we state Lyapunov-like stability conditions that generalize known conditions for Markov chains to ensure the existence of solutions to the optimality equation.

Optimization and Control,Computational Engineering, Finance, and Science,Dynamical Systems,Machine Learning
A risk-sensitive global maximum principle for controlled fully coupled FBSDEs with applications

Jingtao Lin,Jingtao Shi

DOI: https://doi.org/10.3934/mcrf.2024006

2024-03-06

Mathematical Control and Related Fields

Abstract:This paper is concerned with a kind of risk-sensitive optimal control problem for fully coupled forward-backward stochastic systems. The control variable enters the diffusion term of the state equation and the control domain is not necessarily convex. A new global maximum principle is obtained without assuming that the value function is smooth. The maximum condition, the first- and second-order adjoint equations heavily depend on the risk-sensitive parameter. An optimal control problem with a fully coupled linear forward-backward stochastic system and an exponential-quadratic cost functional is discussed. The optimal feedback control and optimal cost are obtained by using Girsanov's theorem and completion-of-squares approach via risk-sensitive Riccati equations. A local solvability result of coupled risk-sensitive Riccati equations is given by Picard-Lindel f's Theorem.

mathematics, applied
Stochastic control problems with state-reflections arising from relaxed benchmark tracking

Lijun Bo,Yijie Huang,Xiang Yu

2024-04-25

Abstract:This paper studies stochastic control problems motivated by optimal consumption with wealth benchmark tracking. The benchmark process is modeled by a combination of a geometric Brownian motion and a running maximum process, indicating its increasing trend in the long run. We consider a relaxed tracking formulation such that the wealth compensated by the injected capital always dominates the benchmark process. The stochastic control problem is to maximize the expected utility of consumption deducted by the cost of the capital injection under the dynamic floor constraint. By introducing two auxiliary state processes with reflections, an equivalent auxiliary control problem is formulated and studied, which leads to the HJB equation with two Neumann boundary conditions. We establish the existence of a unique classical solution to the dual PDE using some novel probabilistic representations involving the local time of some dual processes together with a tailor-made decomposition-homogenization technique. The proof of the verification theorem on the optimal feedback control can be carried out by some stochastic flow analysis and technical estimations of the optimal control.

Optimization and Control,Mathematical Finance
Risk-sensitive Necessary and Sufficient Optimality Conditions and Financial Applications: Fully Coupled Forward-Backward Stochastic Differential Equations with Jump diffusion

Rania Khallout,Adel Chala

DOI: https://doi.org/10.48550/arXiv.1903.02072

2019-03-05

Optimization and Control

Abstract:Throughout this paper, we focused our aim on the problem of optimal control under a risk-sensitive performance functional, where the system is given by a fully coupled forward-backward stochastic differential equation with jump. The risk neutral control system has been used as preliminary step, where the admissible controls are convex, and the optimal solution exists. The necessary as well as sufficient optimality conditions for risk-sensitive performance are proved. At the end of this work, we illustrate our main result by giving an example of mean-variance for risk sensitive control problem applied in cash flow market.
Stochastic Near-Optimal Controls for Path-Dependent Systems

Dorival Leão,Alberto Ohashi,Francys Souza

DOI: https://doi.org/10.48550/arXiv.1707.04976

2018-01-18

Abstract:In this article, we present a general methodology for control problems driven by the Brownian motion filtration including non-Markovian and non-semimartingale state processes controlled by mutually singular measures. The main result of this paper is the development of a concrete pathwise method for characterizing and computing near-optimal controls for abstract controlled Wiener functionals. The theory does not require ad hoc functional differentiability assumptions on the value process and elipticity conditions on the diffusion components. The analysis is pathwise over suitable finite dimensional spaces and it is based on the weak differential structure introduced by Leão, Ohashi and Simas jointly with measurable selection arguments. The theory is applied to stochastic control problems based on path-dependent SDEs where both drift and possibly degenerated diffusion components are controlled. Optimal control of drifts for path-dependent SDEs driven by fractional Brownian motion is also discussed. We finally provide an application in the context of financial mathematics. Namely, we construct near-optimal controls in a non-Markovian portfolio optimization problem.

Probability,Optimization and Control
Stochastic Minimum Principle for Partially Observed Systems Subject to Continuous and Jump Diffusion Processes and Driven by Relaxed Controls

Nasir U. Ahmed,Charalambos D. Charalambous

DOI: https://doi.org/10.48550/arXiv.1302.3455

2013-02-15

Abstract:In this paper we consider non convex control problems of stochastic differential equations driven by relaxed controls. We present existence of optimal controls and then develop necessary conditions of optimality. We cover both continuous diffusion and Jump processes.

Optimization and Control
Robustness of Optimal Controlled Diffusions with Near-Brownian Noise via Rough Paths Theory

Somnath Pradhan,Zachary Selk,Serdar Yüksel

2023-12-06

Abstract:In this article we show a robustness theorem for controlled stochastic differential equations driven by approximations of Brownian motion. Often, Brownian motion is used as an idealized model of a diffusion where approximations such as Wong-Zakai, Karhnen-Loève or fractional Brownian motion are often seen as more physical. However, there has been extensive literature on solving control problems driven by Brownian motion and little on control problems driven by more realistic models that are only approximately Brownian. The question of robustness naturally arises from such approximations. We show robustness using rough paths theory, which allows for a pathwise theory of stochastic differential equations. To this end, in particular, we show that within the class of Lipschitz continuous control policies, an optimal solution for the Brownian idealized model is near optimal for a true system driven by a non-Brownian (but near-Brownian) noise.

Optimization and Control,Probability

Risk-sensitive control of reflected diffusion processes on orthrant

On optimal control of reflected diffusions

Discrete-Time Approximation of Risk-Averse Control Problems for Diffusion Processes

Data-driven rules for multidimensional reflection problems

Robustness of Stochastic Optimal Control to Approximate Diffusion Models under Several Cost Evaluation Criteria

On the hierarchical risk-averse control problems for diffusion processes

Risk Aware Minimum Principle for Optimal Control of Stochastic Differential Equations

Risk-sensitive control of continuous time Markov chains

Risk-averse risk-constrained optimal control

On a Class of Singular Stochastic Control Problems for Reflected Diffusions

On Average Risk-sensitive Markov Control Processes

Stochastic Control Problems with State Reflections Arising from Relaxed Benchmark Tracking

Risk-sensitive Markov control processes

A risk-sensitive global maximum principle for controlled fully coupled FBSDEs with applications

Stochastic control problems with state-reflections arising from relaxed benchmark tracking

Risk-sensitive Necessary and Sufficient Optimality Conditions and Financial Applications: Fully Coupled Forward-Backward Stochastic Differential Equations with Jump diffusion

Stochastic Near-Optimal Controls for Path-Dependent Systems

Stochastic Minimum Principle for Partially Observed Systems Subject to Continuous and Jump Diffusion Processes and Driven by Relaxed Controls

Robustness of Optimal Controlled Diffusions with Near-Brownian Noise via Rough Paths Theory