Abstract:Clothes grasping and unfolding is a core step in robotic-assisted dressing. Most existing works leverage depth images of clothes to train a deep learning-based model to recognize suitable grasping points. These methods often utilize physics engines to synthesize depth images to reduce the cost of real labeled data collection. However, the natural domain gap between synthetic and real images often leads to poor performance of these methods on real data. Furthermore, these approaches often struggle in scenarios where grasping points are occluded by the clothing item itself. To address the above challenges, we propose a novel Bi-directional Fractal Cross Fusion Network (BiFCNet) for semantic segmentation, enabling recognition of graspable regions in order to provide more possibilities for grasping. Instead of using depth images only, we also utilize RGB images with rich color features as input to our network in which the Fractal Cross Fusion (FCF) module fuses RGB and depth data by considering global complex features based on fractal geometry. To reduce the cost of real data collection, we further propose a data augmentation method based on an adversarial strategy, in which the color and geometric transformations simultaneously process RGB and depth data while maintaining the label correspondence. Finally, we present a pipeline for clothes grasping and unfolding from the perspective of semantic segmentation, through the addition of a strategy for grasp point selection from segmentation regions based on clothing flatness measures, while taking into account the grasping direction. We evaluate our BiFCNet on the public dataset NYUDv2 and obtained comparable performance to current state-of-the-art models. We also deploy our model on a Baxter robot, running extensive grasping and unfolding experiments as part of our ablation studies, achieving an 84% success rate.

A Continuous Robot Vision Approach for Predicting Shapes and Visually Perceived Weights of Garments

Continuous Perception Garment Classification Based on Optical Flow Variation.

GarNet: A Continuous Robot Vision Approach for Predicting Shapes and Visually Perceived Weights of Garments

Deforming Garment Classification with Shallow Temporal Extraction and Tree-Based Fusion

Automatic Measurement of Garment Sizes Using Image Recognition

Visual-Tactile Learning of Garment Unfolding for Robot-Assisted Dressing

UniGarmentManip: A Unified Framework for Category-Level Garment Manipulation Via Dense Visual Correspondence

Robot Vision Architecture for Autonomous Clothes Manipulation

Garment Recognition and Reconstruction Using Object Simultaneous Localization and Mapping

Learning to Grasp Clothing Structural Regions for Garment Manipulation Tasks

GarmentTracking: Category-Level Garment Pose Tracking

Predicting ready-made garment dressing fit for individuals based on highly reliable examples.

GarmentLab: A Unified Simulation and Benchmark for Garment Manipulation

SKT: Integrating State-Aware Keypoint Trajectories with Vision-Language Models for Robotic Garment Manipulation

GarmNet: Improving Global with Local Perception for Robotic Laundry Folding

RoboHanger: Learning Generalizable Robotic Hanger Insertion for Diverse Garments

Learning Dense Visual Object Descriptors to Fold Two-Dimensional Deformable Fabrics

Learning Dense Visual Correspondences in Simulation to Smooth and Fold Real Fabrics

Active Clothing Material Perception Using Tactile Sensing and Deep Learning

Clothes Grasping and Unfolding Based on RGB-D Semantic Segmentation

Single-shot clothing category recognition in free-configurations with application to autonomous clothes sorting