Abstract:Restless multi-armed bandits (RMAB) have demonstrated success in optimizing resource allocation for large beneficiary populations in public health settings. Unfortunately, RMAB models lack flexibility to adapt to evolving public health policy priorities. Concurrently, Large Language Models (LLMs) have emerged as adept automated planners across domains of robotic control and navigation. In this paper, we propose a Decision Language Model (DLM) for RMABs, enabling dynamic fine-tuning of RMAB policies in public health settings using human-language commands. We propose using LLMs as automated planners to (1) interpret human policy preference prompts, (2) propose reward functions as code for a multi-agent RMAB environment, and (3) iterate on the generated reward functions using feedback from grounded RMAB simulations. We illustrate the application of DLM in collaboration with ARMMAN, an India-based non-profit promoting preventative care for pregnant mothers, that currently relies on RMAB policies to optimally allocate health worker calls to low-resource populations. We conduct a technology demonstration in simulation using the Gemini Pro model, showing DLM can dynamically shape policy outcomes using only human prompts as input.

What problem does this paper attempt to address?

This paper attempts to solve the problem that it is difficult to dynamically adjust resource allocation strategies in public health settings. Specifically, although the existing Restless Multi - Armed Bandits (RMAB) models perform well in optimizing resource allocation for large - scale beneficiary populations, they lack flexibility and cannot adapt to the ever - changing public health policy priorities. In addition, these models mainly focus on fixed targets and require a great deal of manual design to achieve new health goals. To solve these problems, the author proposes a Decision - Language Model (DLM), which can use human language commands to dynamically fine - tune RMAB strategies. The main contributions of DLM are as follows: 1. **Utilizing large - language models (LLMs) to interpret human policy preferences**: Through natural language processing techniques, DLM can understand and interpret human policy preferences, thereby generating corresponding reward functions. 2. **Generating reward function code**: DLM can directly generate reward function code in a multi - agent RMAB environment to achieve specific RMAB goals. 3. **Self - reflection mechanism based on simulation feedback**: DLM simulates the results of RMAB strategies and iteratively improves the generated reward functions based on these results without real - world feedback. The paper also demonstrates the application potential of DLM in actual public health scenarios through cooperation with ARMMAN, a non - profit organization in India. ARMMAN provides preventive health information to pregnant women through an automated telephone service, but due to limited resources, it needs to effectively allocate the telephone visits of health workers. DLM can improve the overall project participation and effectiveness by dynamically adjusting strategies to give priority to supporting specific vulnerable groups. In conclusion, this paper aims to provide a flexible and efficient resource allocation method by combining large - language models and RMAB techniques to meet the dynamic needs in the public health field.

A Decision-Language Model (DLM) for Dynamic Restless Multi-Armed Bandit Tasks in Public Health

A Decision-Language Model (DLM) for Dynamic Restless Multi-Armed Bandit Tasks in Public Health

Scalable Decision-Focused Learning in Restless Multi-Armed Bandits with Application to Maternal and Child Health

Balancing Act: Prioritization Strategies for LLM-Designed Restless Bandit Rewards

Equitable Restless Multi-Armed Bandits: A General Framework Inspired By Digital Health

Efficient Resource Allocation with Fairness Constraints in Restless Multi-Armed Bandits

Policy-focused Agent-based Modeling using RL Behavioral Models

A Bayesian Approach to Online Learning for Contextual Restless Bandits with Applications to Public Health

Efficient Public Health Intervention Planning Using Decomposition-Based Decision-Focused Learning

The Bandit Whisperer: Communication Learning for Restless Bandits

Limited Resource Allocation in a Non-Markovian World: The Case of Maternal and Child Healthcare

IRL for Restless Multi-Armed Bandits with Applications in Maternal and Child Health

A Demonstration of Adaptive Collaboration of Large Language Models for Medical Decision-Making

Language Models are Alignable Decision-Makers: Dataset and Application to the Medical Triage Domain

Decision-Making Behavior Evaluation Framework for LLMs under Uncertain Context

MDAgents: An Adaptive Collaboration of LLMs for Medical Decision-Making

Online Restless Multi-Armed Bandits with Long-Term Fairness Constraints

Efficiently Deploying LLMs with Controlled Risk

Large Language Model As Autonomous Decision Maker

Enabling Intelligent Interactions between an Agent and an LLM: A Reinforcement Learning Approach

Optimal Decision Making Through Scenario Simulations Using Large Language Models