Abstract:Deep Reinforcement Learning (DRL) algorithms have been increasingly employed during the last decade to solve various decision-making problems such as autonomous driving and robotics. However, these algorithms have faced great challenges when deployed in safety-critical environments since they often exhibit erroneous behaviors that can lead to potentially critical errors. One way to assess the safety of DRL agents is to test them to detect possible faults leading to critical failures during their execution. This raises the question of how we can efficiently test DRL policies to ensure their correctness and adherence to safety requirements. Most existing works on testing DRL agents use adversarial attacks that perturb states or actions of the agent. However, such attacks often lead to unrealistic states of the environment. Their main goal is to test the robustness of DRL agents rather than testing the compliance of agents' policies with respect to requirements. Due to the huge state space of DRL environments, the high cost of test execution, and the black-box nature of DRL algorithms, the exhaustive testing of DRL agents is impossible. In this paper, we propose a Search-based Testing Approach of Reinforcement Learning Agents (STARLA) to test the policy of a DRL agent by effectively searching for failing executions of the agent within a limited testing budget. We use machine learning models and a dedicated genetic algorithm to narrow the search towards faulty episodes. We apply STARLA on Deep-Q-Learning agents which are widely used as benchmarks and show that it significantly outperforms Random Testing by detecting more faults related to the agent's policy. We also investigate how to extract rules that characterize faulty episodes of the DRL agent using our search results. Such rules can be used to understand the conditions under which the agent fails and thus assess its deployment risks.

RLStop: A Reinforcement Learning Stopping Method for TAR

Stopping Methods for Technology Assisted Reviews based on Point Processes

Combining Counting Processes and Classification Improves a Stopping Rule for Technology Assisted Review

When to Stop Reviewing in Technology-Assisted Reviews

Using Chao's Estimator as a Stopping Criterion for Technology-Assisted Review

Engineering Quality and Reliability in Technology-Assisted Review

Countering Reward Over-optimization in LLM with Demonstration-Guided Reinforcement Learning

A Method for Stopping Active Learning Based on Stabilizing Predictions and the Need for User-Adjustable Stopping

A Reproducibility Study of Goldilocks: Just-Right Tuning of BERT for TAR

A Search-Based Testing Approach for Deep Reinforcement Learning Agents

Deep Reinforcement Learning for Optimal Stopping with Application in Financial Engineering

ROER: Regularized Optimal Experience Replay

Efficient Sparse-Reward Goal-Conditioned Reinforcement Learning with a High Replay Ratio and Regularization

AdaStop: adaptive statistical testing for sound comparisons of Deep RL agents

Edit Distance based RL for RNNT decoding

RLRF4Rec: Reinforcement Learning from Recsys Feedback for Enhanced Recommendation Reranking

Stabilizing RLHF through Advantage Model and Selective Rehearsal

Focused ReAct: Improving ReAct through Reiterate and Early Stop

Computationally Efficient Reinforcement Learning: Targeted Exploration leveraging Simple Rules

Can't Remember Details in Long Documents? You Need Some R&R

Improving Large Language Models via Fine-grained Reinforcement Learning with Minimum Editing Constraint