Abstract:While research of reinforcement learning applied to financial markets predominantly concentrates on finding optimal behaviours, it is worth to realize that the reinforcement learning returns $G_t$ and state value functions themselves are of interest and play a pivotal role in the evaluation of assets. Instead of focussing on the more complex task of finding optimal decision rules, this paper studies and applies the power of distributional state value functions in the context of financial market valuation and machine learning based trading algorithms. Accurate and trustworthy estimates of the distributions of $G_t$ provide a competitive edge leading to better informed decisions and more optimal behaviour. Herein, ideas from predictive knowledge and deep reinforcement learning are combined to introduce a novel family of models called CDG-Model, resulting in a highly flexible framework and intuitive approach with minimal assumptions regarding underlying distributions. The models allow seamless integration of typical financial modelling pitfalls like transaction costs, slippage and other possible costs or benefits into the model calculation. They can be applied to any kind of trading strategy or asset class. The frameworks introduced provide concrete business value through their potential in market valuation of single assets and portfolios, in the comparison of strategies as well as in the improvement of market timing. They can positively impact the performance and enhance the learning process of existing or new trading algorithms. They are of interest from a scientific point-of-view and open up multiple areas of future research. Initial implementations and tests were performed on real market data. While the results are promising, applying a robust statistical framework to evaluate the models in general remains a challenge and further investigations are needed.

A reinforcement learning extension to the Almgren-Chriss model for optimal trade execution

Many learning agents interacting with an agent-based market model

A simple learning agent interacting with an agent-based market model

Optimal Execution with Reinforcement Learning

Deep reinforcement learning on a multi-asset environment for trading

Auto Uning of Price Prediction Models for High-Frequency Trading Via Reinforcement Learning

Using Reinforcement Learning in the Algorithmic Trading Problem

Reinforcement Learning for Optimal Execution when Liquidity is Time-Varying

Exploiting Distributional Value Functions for Financial Market Valuation, Enhanced Feature Creation and Improvement of Trading Algorithms

Reinforcement Learning Framework for Quantitative Trading

An Adaptive Dual-level Reinforcement Learning Approach for Optimal Trade Execution

A reinforcement learning approach to improve the performance of the Avellaneda-Stoikov market-making algorithm

Reinforcement Learning for Corporate Bond Trading: A Sell Side Perspective

Deep differentiable reinforcement learning and optimal trading

Practical Application of Deep Reinforcement Learning to Optimal Trade Execution

Learning and Calibrating Heterogeneous Bounded Rational Market Behaviour with Multi-Agent Reinforcement Learning

Reinforcement Learning for Continuous-Time Optimal Execution: Actor-Critic Algorithm and Error Analysis

Multi-agent reinforcement learning in a realistic limit order book market simulation

Extending Deep Reinforcement Learning Frameworks in Cryptocurrency Market Making

Multi-Agent Deep Reinforcement Learning for Liquidation Strategy Analysis