The Power of Resets in Online Reinforcement Learning

Zakaria Mhammedi,Dylan J. Foster,Alexander Rakhlin
2024-04-26
Abstract:Simulators are a pervasive tool in reinforcement learning, but most existing algorithms cannot efficiently exploit simulator access -- particularly in high-dimensional domains that require general function approximation. We explore the power of simulators through online reinforcement learning with {local simulator access} (or, local planning), an RL protocol where the agent is allowed to reset to previously observed states and follow their dynamics during training. We use local simulator access to unlock new statistical guarantees that were previously out of reach:
Machine Learning,Artificial Intelligence
What problem does this paper attempt to address?