Abstract:We present PACE, a novel method for modifying motion-captured virtual agents to interact with and move throughout dense, cluttered 3D scenes. Our approach changes a given motion sequence of a virtual agent as needed to adjust to the obstacles and objects in the environment. We first take the individual frames of the motion sequence most important for modeling interactions with the scene and pair them with the relevant scene geometry, obstacles, and semantics such that interactions in the agents motion match the affordances of the scene (e.g., standing on a floor or sitting in a chair). We then optimize the motion of the human by directly altering the high-DOF pose at each frame in the motion to better account for the unique geometric constraints of the scene. Our formulation uses novel loss functions that maintain a realistic flow and natural-looking motion. We compare our method with prior motion generating techniques and highlight the benefits of our method with a perceptual study and physical plausibility metrics. Human raters preferred our method over the prior approaches. Specifically, they preferred our method 57.1% of the time versus the state-of-the-art method using existing motions, and 81.0% of the time versus a state-of-the-art motion synthesis method. Additionally, our method performs significantly higher on established physical plausibility and interaction metrics. Specifically, we outperform competing methods by over 1.2% in terms of the non-collision metric and by over 18% in terms of the contact metric. We have integrated our interactive system with Microsoft HoloLens and demonstrate its benefits in real-world indoor scenes. Our project website is available at <a class="link-external link-https" href="https://gamma.umd.edu/pace/" rel="external noopener nofollow">this https URL</a>.

Virtual Agent Positioning Driven by Personal Characteristics

Placement Retargeting of Virtual Avatars to Dissimilar Indoor Environments

Interactive Animation of Virtual Characters: Application to Virtual Kung-Fu Fighting

VirtualOffshore: an Interactive 3D Offshore Environment for Embodied Agents

Responsive Action Generation By Physically-Based Motion Retrieval And Adaptation

Inserting Virtual Pedestrians into Pedestrian Groups Video with Behavior Consistency

Expressing social attitudes in virtual agents for social training games

TravelAgent: An AI Assistant for Personalized Travel Planning

PACE: Data-Driven Virtual Agent Interaction in Dense and Cluttered Environments

Exploring User Experience and Performance of a Tedious Task Through Human–agent Relationship

Online Inserting Virtual Characters into Dynamic Video Scenes

VAEnvGen: A Real-Time Virtual Agent Environment Generation System Based on Large Language Models

A virtual reality platform for dynamic human-scene interaction

Virtual space ontologies for scripting agents

FVA: Modeling Perceived Friendliness of Virtual Agents Using Movement Characteristics

AgentAvatar: Disentangling Planning, Driving and Rendering for Photorealistic Avatar Agents

Multi-Agent Deep Reinforcement Learning for Dynamic Avatar Migration in AIoT-enabled Vehicular Metaverses with Trajectory Prediction

Imagined Potential Games: A Framework for Simulating, Learning and Evaluating Interactive Behaviors

Learning Human Search Behavior from Egocentric Visual Inputs

Behavioral Analysis of Vision-and-Language Navigation Agents

Autonomic Visiting in Digital 3D Scenes