Abstract:Despite progress in visual perception tasks such as image classification and detection, computers still struggle to understand the interdependency of objects in the scene as a whole, e.g., relations between objects or their attributes. Existing methods often ignore global context cues capturing the interactions among different object instances, and can only recognize a handful of types by exhaustively training individual detectors for all possible relationships. To capture such global interdependency, we propose a deep Variation-structured Reinforcement Learning (VRL) framework to sequentially discover object relationships and attributes in the whole image. First, a directed semantic action graph is built using language priors to provide a rich and compact representation of semantic correlations between object categories, predicates, and attributes. Next, we use a variation-structured traversal over the action graph to construct a small, adaptive action set for each step based on the current state and historical actions. In particular, an ambiguity-aware object mining scheme is used to resolve semantic ambiguity among object categories that the object detector fails to distinguish. We then make sequential predictions using a deep RL framework, incorporating global context cues and semantic embeddings of previously extracted phrases in the state vector. Our experiments on the Visual Relationship Detection (VRD) dataset and the large-scale Visual Genome dataset validate the superiority of VRL, which can achieve significantly better detection results on datasets involving thousands of relationship and attribute types. We also demonstrate that VRL is able to predict unseen types embedded in our action graph by learning correlations on shared graph nodes.

Visual Sensor Network Reconfiguration with Deep Reinforcement Learning

DEAR: Disentangled Environment and Agent Representations for Reinforcement Learning without Reconstruction

A Partially Supervised Reinforcement Learning Framework for Visual Active Search

Reinforcement Learning Meets Visual Odometry

Visual Diagnostics for Deep Reinforcement Learning Policy Development

Deep Variation-structured Reinforcement Learning for Visual Relationship and Attribute Detection

Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control

Visual Reinforcement Learning with Self-Supervised 3D Representations

RePo: Resilient Model-Based Reinforcement Learning by Regularizing Posterior Predictability

Visual Kinematics Representation Via Self-Supervised Object Tracking for Deep Reinforcement Learning

ReLIC: A Recipe for 64k Steps of In-Context Reinforcement Learning for Embodied AI

Image-Based Deep Reinforcement Learning with Intrinsically Motivated Stimuli: On the Execution of Complex Robotic Tasks

Deep Reinforcement Learning: A Brief Survey

Deductive Reinforcement Learning for Visual Autonomous Urban Driving Navigation.

Masked World Models for Visual Control

Vision-based navigation and obstacle avoidance via deep reinforcement learning

A Brief Survey of Deep Reinforcement Learning

ViSaRL: Visual Reinforcement Learning Guided by Human Saliency

RDDRL: a recurrent deduction deep reinforcement learning model for multimodal vision-robot navigation

Deep SIMBAD: Active Landmark-based Self-localization Using Ranking -based Scene Descriptor