Abstract:Financial portfolio management (PM) is one of the most applicable problems in reinforcement learning (RL) owing to its sequential decision-making nature. However, existing RL-based approaches rarely focus on scalability or reusability to adapt to the ever-changing markets. These approaches are rigid and unscalable to accommodate the varying number of assets of portfolios and increasing need for heterogeneous data. Also, RL agents in the existing systems are ad-hoc trained and hardly reusable for different portfolios. To confront the above problems, a modular design is desired for the systems to be compatible with reusable asset-dedicated agents. In this paper, we propose a multi-agent RL-based system for PM (MSPM). MSPM involves two types of asynchronously-updated modules: Evolving Agent Module (EAM) and Strategic Agent Module (SAM). An EAM is an information-generating module with a DQN agent, and it receives heterogeneous data and generates signal-comprised information for a particular asset. An SAM is a decision-making module with a PPO agent for portfolio optimization, and it connects to EAMs to reallocate the assets in a portfolio. Trained EAMs can be connected to any SAM at will. With its modularized architecture, the multi-step condensation of volatile market information, and the reusable design of EAM, MSPM simultaneously addresses the two challenges in RL-based PM: scalability and reusability. Experiments on 8-year U.S. stock market data prove the effectiveness of MSPM in profit accumulation by its outperformance over five baselines in terms of accumulated rate of return (ARR), daily rate of return, and Sortino ratio. MSPM improves ARR by at least 186.5% compared to CRP, a widely-used PM strategy. To validate the indispensability of EAM, we back-test and compare MSPMs on four portfolios. EAM-enabled MSPMs improve ARR by at least 1341.8% compared to EAM-disabled MSPMs.

PyCaMa: Python for cash management

A multi-objective approach to the cash management problem

A stochastic goal programming model to derive stable cash management policies

Managing Inventory for a Multidivisional Firm with Cash Pooling.

On the formal foundations of cash management systems

cmaes : A Simple yet Practical Python Library for CMA-ES

Fitting random cash management models to data

Cash flow management by risk-neutral and risk-averse stochastic approaches

A multidimensional review of the cash management problem

Solving Constrained CASH Problems with ADMM

MSPM: A Modularized and Scalable Multi-Agent Reinforcement Learning-based System for Financial Portfolio Management

The Generalized Cash Balance Problem: Optimization-Based One Step Ahead Optimal Control.

Optimizing Cash Management for Large Scale Bank Operations

CaΣoS: A nonlinear sum-of-squares optimization suite

pysamoo: Surrogate-Assisted Multi-Objective Optimization in Python

FCMpy: a python module for constructing and analyzing fuzzy cognitive maps

Multi-Task Combinatorial Bandits for Budget Allocation

Hybrid MCDM method on pythagorean fuzzy set and its application

Cashflow Management Practices and Financial Performance of Small and Medium Business Enterprises in Kenya

Investment Portfolio with Convex Optimization and Risk Adjustment Using Multi-Factor Model and Multi-Armed Bandit Algorithm

Improving cash logistics in bank branches by coupling machine learning and robust optimization