Abstract:Simulation-to-real is the task of training and developing machine learning models and deploying them in real settings with minimal additional training. This approach is becoming increasingly popular in fields such as robotics. However, there is often a gap between the simulated environment and the real world, and machine learning models trained in simulation may not perform as well in the real world. We propose a framework that utilizes a message-passing pipeline to minimize the information gap between simulation and reality. The message-passing pipeline is comprised of three modules: scene understanding, robot planning, and performance validation. First, the scene understanding module aims to match the scene layout between the real environment set-up and its digital twin. Then, the robot planning module solves a robotic task through trial and error in the simulation. Finally, the performance validation module varies the planning results by constantly checking the status difference of the robot and object status between the real set-up and the simulation. In the experiment, we perform a case study that requires a robot to make a cup of coffee. Results show that the robot is able to complete the task under our framework successfully. The robot follows the steps programmed into its system and utilizes its actuators to interact with the coffee machine and other tools required for the task. The results of this case study demonstrate the potential benefits of our method that drive robots for tasks that require precision and efficiency. Further research in this area could lead to the development of even more versatile and adaptable robots, opening up new possibilities for automation in various industries.

Parallel Pipelining Simulation and Control System of Prototype Space Robot

A Comprehensive Simulation Testbench For Aerial Robot In Dynamic Scenario Using Gazebo-Ros

FastSim: A Modular and Plug-and-Play Simulator for Aerial Robots

HumRoboSim: an Autonomous Humanoid Robot Simulation System

A Ground Teleoperation Experimental System of Space Robot Using Hybrid Approach

Sim2Plan: Robot Motion Planning via Message Passing between Simulation and Reality

Design and Implementation of Simulation System for the Competitive Robot System

A Ground-Based Validation System of Teleoperation for a Space Robot

The Ground-based Validation Technology of Teleoperation for Space Robot

Design And Implementation Of The Universal Servo Control Algorithm Verification System Based On High-Speed Communication Fieldbus

Open Architecture Multi-Axis Motion Control System for Industrial Robot Based on Can Bus

Design and Experimental Study of Space Manipulator Simulator System

General scheme of teleoperation for space robot

Development of Ground Experiment System for Space Robot Performing Fine Manipulation

A Ground Experiment System of Free-floating Robot for Capturing Space Target

The Dsp Implementation Of Rt Control Of 6-Dof Parallel Robot Based On Rcp

Study on Three Dimensional Predictive Simulation System of Teleoperation for Free Flying Space Robot

Experiment and study of a flight siumlator prototype system

A Ground Experiment System of a Free-Floating Robot for Fine Manipulation

Study On Dynamic Modeling And Control Simulation Of Space Robot

Unified Simulation and Test Platform for Control Systems of Unmanned Vehicles