Abstract:Data augmentation (DA) is a crucial technique for enhancing the sample efficiency of visual reinforcement learning (RL) algorithms. Notably, employing simple observation transformations alone can yield outstanding performance without extra auxiliary representation tasks or pre-trained encoders. However, it remains unclear which attributes of DA account for its effectiveness in achieving sample-efficient visual RL. To investigate this issue and further explore the potential of DA, this work conducts comprehensive experiments to assess the impact of DA's attributes on its efficacy and provides the following insights and improvements: (1) For individual DA operations, we reveal that both ample spatial diversity and slight hardness are indispensable. Building on this finding, we introduce Random PadResize (Rand PR), a new DA operation that offers abundant spatial diversity with minimal hardness. (2) For multi-type DA fusion schemes, the increased DA hardness and unstable data distribution result in the current fusion schemes being unable to achieve higher sample efficiency than their corresponding individual operations. Taking the non-stationary nature of RL into account, we propose a RL-tailored multi-type DA fusion scheme called Cycling Augmentation (CycAug), which performs periodic cycles of different DA operations to increase type diversity while maintaining data distribution consistency. Extensive evaluations on the DeepMind Control suite and CARLA driving simulator demonstrate that our methods achieve superior sample efficiency compared with the prior state-of-the-art methods.

Unified State Representation Learning under Data Augmentation

Domain Adaptation In Reinforcement Learning Via Latent Unified State Representation

Domain Adaptive State Representation Alignment for Reinforcement Learning

State Representation Learning for Effective Deep Reinforcement Learning.

Automatic Data Augmentation for Generalization in Deep Reinforcement Learning

A stable data-augmented reinforcement learning method with ensemble exploration and exploitation

Generalization of Reinforcement Learning with Policy-Aware Adversarial Data Augmentation

State Representation Learning with Adjacent State Consistency Loss for Deep Reinforcement Learning.

Automatic Data Augmentation for Generalization in Reinforcement Learning

A Recipe for Unbounded Data Augmentation in Visual Reinforcement Learning

Reinforcement Learning with Euclidean Data Augmentation for State-Based Continuous Control

A Comprehensive Survey of Data Augmentation in Visual Reinforcement Learning

Don’t Touch What Matters: Task-Aware Lipschitz Data Augmentation for Visual Reinforcement Learning

Towards More Sample Efficiency in Reinforcement Learning with Data Augmentation.

Learning Better with Less: Effective Augmentation for Sample-Efficient Visual Reinforcement Learning

Don't Touch What Matters: Task-Aware Lipschitz Data Augmentation for Visual Reinforcement Learning

Contrastive State Augmentations for Reinforcement Learning-Based Recommender Systems

Revisiting Data Augmentation in Deep Reinforcement Learning

Mask-based Latent Reconstruction for Reinforcement Learning

Simple Noisy Environment Augmentation for Reinforcement Learning