Abstract:Data-efficiency is crucial for autonomous robots to adapt to new tasks and environments. In this work we focus on robotics problems with a budget of only 10-20 trials. This is a very challenging setting even for data-efficient approaches like Bayesian optimization (BO), especially when optimizing higher-dimensional controllers. Simulated trajectories can be used to construct informed kernels for BO. However, previous work employed supervised ways of extracting low-dimensional features for these. We propose a model and architecture for a sequential variational autoencoder that embeds the space of simulated trajectories into a lower-dimensional space of latent paths in an unsupervised way. We further compress the search space for BO by reducing exploration in parts of the state space that are undesirable, without requiring explicit constraints on controller parameters. We validate our approach with hardware experiments on a Daisy hexapod robot and an ABB Yumi manipulator. We also present simulation experiments with further comparisons to several baselines on Daisy and two manipulators. Our experiments indicate the proposed trajectory-based kernel with dynamic compression can offer ultra data-efficient optimization.

What problem does this paper attempt to address?

This paper attempts to solve the data - efficiency problem encountered when using Bayesian Optimization (BO) for controller parameter tuning in robotics, especially in the case of having only 10 - 20 trial budgets. Specifically, the paper focuses on how to achieve super - efficient data utilization in high - dimensional control spaces so that robots can quickly adapt to new tasks or environments. To address this challenge, the authors propose a new method, that is, improving Bayesian Optimization through dynamic compression in the variational latent space. The core of this method lies in: 1. **Application of Variational Autoencoder (VAE)**: The authors develop a Sequential Variational Autoencoder (SVAE) to embed simulated trajectories into a low - dimensional latent path space. This process is unsupervised and does not require manual feature extraction. 2. **Dynamic Compression**: Dynamically compress the search space by reducing exploration in the undesired parts of the state space. This step does not require explicit constraints on controller parameters. Specifically, for those controllers that lead to undesired behaviors, reduce the sampling times of these controllers by scaling the distances between them, so that BO can more quickly rule out these sub - optimal controllers and increase the exploration of ideal controllers. 3. **Constructing Trajectory - Based Kernel Functions**: Utilize the latent paths learned by SVAE to construct a latent - path - based kernel function for optimizing controller parameters in the BO process. This method takes into account not only the similarity between controller parameters but also their distribution characteristics in the latent path space. Through these techniques, the paper aims to achieve a framework that can complete efficient optimization within a very limited number of trials, especially suitable for autonomous robot systems that need to quickly adapt to new tasks or environments. Experimental results show that this method can significantly improve data efficiency in both actual hardware (such as the Daisy hexapod robot and ABB Yumi manipulator) and simulation environments, especially outstanding in high - dimensional control spaces.

Bayesian Optimization in Variational Latent Spaces with Dynamic Compression

High-dimensional Bayesian optimization using low-dimensional feature spaces

Safe Sim-to-Real Robot Exploration with Constrained Bayesian Optimization

CARE: Confidence-rich Autonomous Robot Exploration Using Bayesian Kernel Inference and Optimization

Joint Composite Latent Space Bayesian Optimization

Advancing Bayesian Optimization via Learning Correlated Latent Space

Latent Space Bayesian Optimization with Latent Data Augmentation for Enhanced Exploration

Local Latent Space Bayesian Optimization over Structured Inputs

High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learning

Cautious Bayesian Optimization for Efficient and Scalable Policy Search

Learning the Latent Space of Robot Dynamics for Cutting Interaction Inference

Geometry-aware Bayesian Optimization in Robotics using Riemannian Matérn Kernels

Inversion-based Latent Bayesian Optimization

Mold into a Graph: Efficient Bayesian Optimization over Mixed-Spaces

Latent Dynamics Model Learning as Multi-objective Optimization with Augmented Tchebyshev Function

Approximation-Aware Bayesian Optimization

Bayesian Optimization for Sample-Efficient Policy Improvement in Robotic Manipulation

Violation-aware contextual Bayesian optimization for controller performance optimization with unmodeled constraints

Deep Kernel and Image Quality Estimators for Optimizing Robotic Ultrasound Controller using Bayesian Optimization

Scalable Bayesian Optimization via Focalized Sparse Gaussian Processes

Using a Variational Autoencoder to Learn Valid Search Spaces of Safely Monitored Autonomous Robots for Last-Mile Delivery