Abstract:We consider the periodic review dynamic pricing and inventory control problem with fixed ordering cost. Demand is random and price dependent, and unsatisfied demand is backlogged. With complete demand information, the celebrated (s,S,p) policy is proved to be optimal, where s and S are the reorder point and order-up-to level for ordering strategy, and p, a function of on-hand inventory level, characterizes the pricing strategy. In this paper, we consider incomplete demand information and develop online learning algorithms whose average profit approaches that of the optimal (s,S,p) with a tight O(T^(1/2)) regret rate. A number of salient features differentiate our work from the existing online learning researches in the OM literature. First, computing the optimal (s,S,p) policy requires solving a dynamic programming (DP) over multiple periods involving unknown quantities, which is different from the majority of learning problems in operations management that only require solving single-period optimization questions. It is hence challenging to establish stability results through DP recursions, which we accomplish by proving uniform convergence of the profit-to-go function. The necessity of analyzing action-dependent state transition over multiple periods resembles the reinforcement learning question, considerably more difficult than existing bandit learning algorithms. Second, the pricing function p is of infinite dimension, and approaching it is much more challenging than approaching a finite number of parameters as seen in existing researches. The demand-price relationship is estimated based on upper confidence bound, but the confidence interval cannot be explicitly calculated due to the complexity of the DP recursion. Finally, due to the multi-period nature of (s,S,p) policies the actual distribution of the randomness in demand plays an important role in determining the optimal pricing strategy p, which is unknown to the learner a priori. In this paper, the demand randomness is approximated by an empirical distribution constructed using dependent samples, and a novel Wasserstein metric based argument is employed to prove convergence of the empirical distribution.

A Gradient-Based Method to Calculate (s, S) Policies

A Unified Convergence Theory of a Numerical Method, and Applications to the Replenishment Policies.

A Condition-Based Replacement and Spare Provisioning Policy for Deteriorating Systems with Uncertain Deterioration to Failure.

Performance Evaluation of Modified Echelon <i>(r, Q)</I> Policies for Stochastic Serial Inventory Systems

Optimality of (s, S, P) Policy in a General Inventory-Pricing Model with Uniform Demands.

Performance Bounds and Asymptotic Optimality of Modified (r, Q) Policies for Stochastic Distribution Inventory Systems

Modified Echelon (r, Q) Policies with Guaranteed Performance Bounds for Stochastic Serial Inventory Systems

Stochastic Dynamic Programming Heuristic for the (R, s, S) Policy Parameters Computation

Stochastic Cubic-Regularized Policy Gradient Method

Computing Optimal (R, s, S) Policy Parameters by a Hybrid of Branch-and-Bound and Stochastic Dynamic Programming

A Periodic Review Inventory System with S-Shaped Utility Function

Score-Aware Policy-Gradient Methods and Performance Guarantees using Local Lyapunov Conditions: Applications to Product-Form Stochastic Networks and Queueing Systems

Infinite-Horizon Policy-Gradient Estimation with Variable Discount Factor for Markov Decision Process

Off-Policy Policy Gradient with State Distribution Correction

A Minibatch Stochastic Gradient Descent-Based Learning Metapolicy for Inventory Systems with Myopic Optimal Policy

Supplement to 'A Simple Heuristic Policy for Stochastic Distribution Inventory Systems with Fixed Shipment Costs'

A New Approach to (s, S) Inventory Systems

A Payoff-Based Policy Gradient Method in Stochastic Games with Long-Run Average Payoffs

Dynamic inventory replenishment strategy for aerospace manufacturing supply chain: combining reinforcement learning and multi-agent simulation

Dynamic Pricing and Inventory Control with Fixed Ordering Cost and Incomplete Demand Information

A Minibatch-SGD-Based Learning Meta-Policy for Inventory Systems with Myopic Optimal Policy