Abstract:This article proposes a novel approach to construct data-driven online solutions to optimization problems (P) subject to a class of distributionally uncertain dynamical systems. The introduced framework allows for the simultaneous learning of distributional system uncertainty via a parameterized, control-dependent ambiguity set using a finite historical dataset, and its use to make online decisions with probabilistic regret function bounds. Leveraging the merits of machine learning, the main technical approach relies on the theory of distributional robust optimization (DRO), to hedge against uncertainty and provide less conservative results than standard robust optimization approaches. Starting from recent results that describe ambiguity sets via parameterized, and control-dependent empirical distributions as well as ambiguity radii, we first present a tractable reformulation of the corresponding optimization problem while maintaining the probabilistic guarantees. We then specialize these problems to the cases of 1) optimal one-stage control of distributionally uncertain nonlinear systems, and 2) resource allocation under distributional uncertainty. A novelty of this work is that it extends DRO to online optimization problems subject to a distributionally uncertain dynamical system constraint, handled via a control-dependent ambiguity set that leads to online-tractable optimization with probabilistic guarantees on regret bounds. Further, we introduce an online version of the Nesterov's accelerated-gradient algorithm, and analyze its performance to solve this class of problems via the dissipativity theory.

Distributionally Robust Policy Learning under Concept Drifts

Distributionally Robust Batch Contextual Bandits

Distributionally Robust Policy Evaluation under General Covariate Shift in Contextual Bandits

Distributionally Robust Policy Learning with Wasserstein Distance

Wasserstein Distributionally Robust Policy Evaluation and Learning for Contextual Bandits

Distributionally Robust Learning With Stable Adversarial Training

The Curious Price of Distributional Robustness in Reinforcement Learning with a Generative Model

Distributionally Robust Optimization with Bias and Variance Reduction

Upper and Lower Bounds for Distributionally Robust Off-Dynamics Reinforcement Learning

A Robust Learning Approach for Regression Models Based on Distributionally Robust Optimization.

Robustified Multivariate Regression and Classification Using Distributionally Robust Optimization under the Wasserstein Metric

A Robust Learning Algorithm for Regression Models Using Distributionally Robust Optimization under the Wasserstein Metric

Double pessimism is provably efficient for distributionally robust offline reinforcement learning: Generic algorithm and robust partial coverage

Minimax Regret Optimization for Robust Machine Learning under Distribution Shift

Distributionally Robust Optimization with Markovian Data

Online Optimization and Ambiguity-Based Learning of Distributionally Uncertain Dynamic Systems

Distributionally Robust Infinite-horizon Control: from a pool of samples to the design of dependable controllers

Distributionally Robust Offline Reinforcement Learning with Linear Function Approximation

Stable Adversarial Learning under Distributional Shifts

Robust Distribution Learning with Local and Global Adversarial Corruptions

On the Foundation of Distributionally Robust Reinforcement Learning