Abstract:This book is about dynamic programming and its applications in economics, finance, and adjacent fields. It brings together recent innovations in the theory of dynamic programming and provides applications and code that can help readers approach the research frontier. The book is aimed at graduate students and researchers, although most chapters are accessible to undergraduate students with solid quantitative backgrounds.

What problem does this paper attempt to address?

The paper primarily explores dynamic programming and its applications in economics, finance, and related fields. Specifically, it covers the classic results of dynamic programming and introduces recent theoretical innovations, including recursive preferences, robust control, continuous-time models, and time-varying discount rates. These extensions often do not satisfy the contraction mapping restrictions that traditional methods rely on. To handle these applications, the book adopts the abstract framework proposed by Bertsekas. The paper first introduces the basic concepts and background material for solving fixed-point problems and computing lifecycle value through Chapters 1 to 3. Chapters 4 and 5 discuss optimal stopping and Markov decision processes, respectively. Chapter 6 extends the Markov decision framework to situations where the discount rate varies over time. Chapter 7 explores recursive preferences. The main dynamic programming theoretical results in Chapters 4 to 6 are special cases of the more general results in Chapters 8 to 9. Chapter 10 briefly discusses continuous-time models. Additionally, the paper emphasizes the importance of computer code, providing code written in the Julia language, and all code can be found on GitHub. The choice of Julia is due to its open-source nature and its ability to make computer code as close as possible to the relevant mathematical formulas. Although Python code is also provided in the book, the authors believe that Julia code is superior in terms of clarity. In summary, the problem the paper attempts to solve is how to effectively apply dynamic programming in a finite state space to solve optimization problems in economics and finance, and it provides a systematic approach to analyze these issues.

Dynamic Programming: Finite States

Optimizing Operations Management and Business Analytics Strategies under Uncertainty: Dynamic Programming

Using dynamic programming with adaptive grid scheme for optimal control problems in economics

Gradient-Bounded Dynamic Programming with Submodular and Concave Extensible Value Functions

A Course in Dynamic Optimization

The Principle of Optimality in Dynamic Programming: A Pedagogical Note

Some Applications of the Theory of Dynamic Programming—A Review

Dynamic Programming for Structured Continuous Markov Decision Problems

Dynamic programming applications in water resources

Safe Dynamic Programming

Approximate dynamic programming for continuous state and control problems

Approximate Dynamic Programming with Feasibility Guarantees

Dynamic Programs on Partially Ordered Sets

Stochastic Dynamic Programming with Non-linear Discounting

Differential dynamic programming with stagewise equality and inequality constraints using interior point method

Efficient algorithms of pathwise dynamic programming for decision optimization in mining operations

Dynamic Programming Approaches for Resource Allocation in Project Scheduling: Maximizing Efficiency under Time and Budget Constraints

Speed-Up in Dynamic-Programming

Improved Dynamic Programming Methods for Optimal Control of Lumped-Parameter Stochastic Systems

A kernel-based approximate dynamic programming approach: Theory and application

Adaptive Dynamic Programming for Control: A Survey and Recent Advances