Abstract:Calibrating agent-based models (ABMs) in economics and finance typically involves a derivative-free search in a very large parameter space. In this work, we benchmark a number of search methods in the calibration of a well-known macroeconomic ABM on real data, and further assess the performance of "mixed strategies" made by combining different methods. We find that methods based on random-forest surrogates are particularly efficient, and that combining search methods generally increases performance since the biases of any single method are mitigated. Moving from these observations, we propose a reinforcement learning (RL) scheme to automatically select and combine search methods on-the-fly during a calibration run. The RL agent keeps exploiting a specific method only as long as this keeps performing well, but explores new strategies when the specific method reaches a performance plateau. The resulting RL search scheme outperforms any other method or method combination tested, and does not rely on any prior information or trial and error procedure.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to solve the calibration problem of agent - based models (ABM) in the economic and financial fields. Specifically: 1. **Search challenges in high - dimensional parameter spaces**: - ABM calibration usually involves searching in very large, derivative - free parameter spaces. Since the computational cost of ABM simulations is high, how to efficiently search for the optimal parameter combination becomes a key issue. 2. **Limitations of existing methods**: - Existing search methods have their own advantages and disadvantages, and a single method may converge to a sub - optimal solution due to its own bias. For example, the machine - learning - based surrogate method using random forests is effective, but may encounter bottlenecks when dealing with highly non - convex and discretely varying loss functions; the Best Batch Sampler (BB) of the genetic algorithm class has poor performance when used in isolation and is prone to getting trapped in local minima. 3. **The need to combine multiple search strategies**: - The paper finds that combining different search methods can significantly improve overall performance because the biases of different methods can cancel each other out. However, manually selecting and combining these methods requires a great deal of trial and error and is inefficient. 4. **The need for automatic selection and combination of search methods**: - To solve the above problems, the paper proposes a scheme based on reinforcement learning (RL) that can automatically select and combine different search methods during the calibration process. The RL agent will dynamically adjust the strategy according to the current performance, thus avoiding the manual trial - and - error process and being able to adaptively optimize the search method without prior information. ### Specific objectives of the reinforcement learning scheme - **Automatically select the best search method**: Through the RL agent, automatically select the search method that is most likely to find a better parameter combination based on the current loss value and historical performance. - **Dynamically combine multiple methods**: When a method reaches a performance plateau, the RL agent will explore new strategies instead of fixedly using a single method. - **Surpass single or simple combination methods**: Experimental results show that the RL scheme outperforms any single method or simple combination method under all tested parameter settings. ### Summary The core problem of the paper is to realize the automatic selection and combination of search methods in the economic ABM calibration process by introducing reinforcement learning techniques, so as to improve the calibration efficiency and accuracy. This not only solves the limitations of existing methods but also provides a more flexible and efficient calibration framework.

Reinforcement Learning for Combining Search Methods in the Calibration of Economic ABMs

Learning and Calibrating Heterogeneous Bounded Rational Market Behaviour with Multi-Agent Reinforcement Learning

Simulating the Economic Impact of Rationality through Reinforcement Learning and Agent-Based Modelling

Reinforcement Learning in Economics and Finance

Calibration of Derivative Pricing Models: a Multi-Agent Reinforcement Learning Perspective

Adaptive learning for financial markets mixing model-based and model-free RL for volatility targeting

Combining Tree-Search, Generative Models, and Nash Bargaining Concepts in Game-Theoretic Reinforcement Learning

Deep reinforcement learning on a multi-asset environment for trading

Bridging the gap between Markowitz planning and deep reinforcement learning

A mouse monoclonal antibody reactive preferentially with human IgM lambda.

MetaTrader: An Reinforcement Learning Approach Integrating Diverse Policies for Portfolio Optimization

Deep Reinforcement Trading with Predictable Returns

Bridging the Micro and Macro: Calibration of Agent-Based Model Using Mean-Field Dynamics

Towards Multi-Agent Reinforcement Learning driven Over-The-Counter Market Simulations

Optimizing Trading Strategies in Quantitative Markets using Multi-Agent Reinforcement Learning

Analyzing Micro-Founded General Equilibrium Models with Many Agents using Deep Reinforcement Learning

A Review of Reinforcement Learning in Financial Applications

Many learning agents interacting with an agent-based market model

Model-based Deep Reinforcement Learning for Dynamic Portfolio Optimization

Evaluation of Deep Reinforcement Learning Algorithms for Portfolio Optimisation

Reinforcement Learning for Economic Policy: A New Frontier?