Abstract:Recently using machine learning (ML) based techniques to optimize modern database management systems has attracted intensive interest from both industry and academia. With an objective to tune a specific component of a DBMS (e.g., index selection, knobs tuning), the ML-based tuning agents have shown to be able to find better configurations than experienced database administrators. However, one critical yet challenging question remains unexplored -- how to make those ML-based tuning agents work collaboratively. Existing methods do not consider the dependencies among the multiple agents, and the model used by each agent only studies the effect of changing the configurations in a single component. To tune different components for DBMS, a coordinating mechanism is needed to make the multiple agents cognizant of each other. Also, we need to decide how to allocate the limited tuning budget among the agents to maximize the performance. Such a decision is difficult to make since the distribution of the reward for each agent is unknown and non-stationary. In this paper, we study the above question and present a unified coordinating framework to efficiently utilize existing ML-based agents. First, we propose a message propagation protocol that specifies the collaboration behaviors for agents and encapsulates the global tuning messages in each agent's model. Second, we combine Thompson Sampling, a well-studied reinforcement learning algorithm with a memory buffer so that our framework can allocate budget judiciously in a non-stationary environment. Our framework defines the interfaces adapted to a broad class of ML-based tuning agents, yet simple enough for integration with existing implementations and future extensions. We show that it can effectively utilize different ML-based agents and find better configurations with 1.4~14.1X speedups on the workload execution time compared with baselines.

A Sampling-based Learning Framework for Big Databases

Policy Optimization with Stochastic Mirror Descent.

A Rank-Based Sampling Framework for Offline Reinforcement Learning

Improving Deep Reinforcement Learning with Mirror Loss

DBMind

Learning to Optimize Join Queries With Deep Reinforcement Learning

Discovered Policy Optimisation

Experience-Enhanced Learning: One Size Still does not Fit All in Automatic Database

A Unified and Efficient Coordinating Framework for Autonomous DBMS Tuning

KnobTree: Intelligent Database Parameter Configuration via Explainable Reinforcement Learning

Machine Learning for Databases

Sparse Q-learning with Mirror Descent

Policy Search using Dynamic Mirror Descent MPC for Model Free Off Policy RL

Opengauss

Multi-agent Databases via Independent Learning

Automated Database Indexing using Model-free Reinforcement Learning

Boosting Offline Reinforcement Learning via Data Rebalancing

Enhanced DBR mirror design via D3QN: A reinforcement learning approach

DeepCAT+: A Low-Cost and Transferrable Online Configuration Auto-Tuning Approach for Big Data Frameworks

Adaptive Policy Learning for Offline-to-Online Reinforcement Learning

AI Meets Database: AI4DB and DB4AI