Abstract:We explore learning-based approaches for feedback control of a dexterous five-finger hand performing non-prehensile manipulation. First, we learn local controllers that are able to perform the task starting at a predefined initial state. These controllers are constructed using trajectory optimization with respect to locally-linear time-varying models learned directly from sensor data. In some cases, we initialize the optimizer with human demonstrations collected via teleoperation in a virtual environment. We demonstrate that such controllers can perform the task robustly, both in simulation and on the physical platform, for a limited range of initial conditions around the trained starting state. We then consider two interpolation methods for generalizing to a wider range of initial conditions: deep learning, and nearest neighbors. We find that nearest neighbors achieve higher performance. Nevertheless, the neural network has its advantages: it uses only tactile and proprioceptive feedback but no visual feedback about the object (i.e. it performs the task blind) and learns a time-invariant policy. In contrast, the nearest neighbors method switches between time-varying local controllers based on the proximity of initial object states sensed via motion capture. While both generalization methods leave room for improvement, our work shows that (i) local trajectory-based controllers for complex non-prehensile manipulation tasks can be constructed from surprisingly small amounts of training data, and (ii) collections of such controllers can be interpolated to form more global controllers. Results are summarized in the supplementary video: https://youtu.be/E0wmO6deqjo

Design and Control Co-Optimization for Automated Design Iteration of Dexterous Anthropomorphic Soft Robotic Hands

Designing Anthropomorphic Soft Hands through Interaction

Co-optimization of robotic design and skill inspired by human hand evolution

Robotic in-hand manipulation with relaxed optimization

Dexterous Manipulation Control of a Bionic Prosthesis in Cooperation with Human Upper Limbs

Simultaneous Optimization of Discrete and Continuous Parameters Defining a Robot Morphology and Controller

Evolving Robotic Hand Morphology Through Grasping and Learning

Co-Designing Tools and Control Policies for Robust Manipulation

Optimization Design for Dexterous-Hand of Mobile Robot

Object-Centric Dexterous Manipulation from Human Motion Data

Learning to Design and Use Tools for Robotic Manipulation

Kinematic Optimization of a Robotic Arm for Automation Tasks with Human Demonstration

Contact-Implicit Model Predictive Control for Dexterous In-hand Manipulation: A Long-Horizon and Robust Approach

Simulation Data Driven Design Optimization for Reconfigurable Soft Gripper System

Dexterous Soft Hands Linearize Feedback-Control for In-Hand Manipulation

Co-Optimization of Robot Design and Control: Enhancing Performance and Understanding Design Complexity

DEFT: Dexterous Fine-Tuning for Real-World Hand Policies

The Role of Learning and Kinematic Features in Dexterous Manipulation: A Comparative Study with Two Robotic Hands

Design Optimizer for Planar Soft-Growing Robot Manipulators

Learning Dexterous Manipulation Policies from Experience and Imitation

Design and Gesture Optimization of a Soft-Rigid Robotic Hand for Adaptive Grasping