Abstract:This paper focuses on addressing the visual tracking problem using learning-based methods for object tracking tasks. This problem contains a major difficulty, i.e., how to acquire a satisfactory generalization ability of the developed system? In this paper, firstly, the object state tracking system, including a camera-in-hand, a 3D camera and a Rethink Baxter robot, is introduced. The problem formulation is also presented. Secondly, we propose a Kalman-based estimation strategy to acquire the object’s state. In addition, a learning-based tracking controller is developed using the Gaussian mixture models (GMM) method to steer the robot end-effector to track the mobile object. Thirdly, to guarantee system stability (i.e., the position and velocity errors between the object and end-effector will always converge to zeros), the controller parameter constraints are derived, which is a theoretical contribution of this paper. The controller parameter adjustment is avoided by the proposed training process. Thus, the proposed method becomes easy to implement, which is a practical contribution. Finally, the effectiveness of the proposed method is demonstrated by simulation and experimental examples, and the proposed method has satisfactory generalization ability. Note to Practitioners—This paper studies object tracking problems for different practical applications, such as industrial cutting, grasping and dynamic monitoring. Different trajectory tracking methods have been widely applied in the industrial area. However, users always complain that when the object or trajectory is changed, the tracking controller more or less needs to be re-adjusted. This re-adjust process always requires professional knowledge and programming experience, and thus a factory must employ some professional engineers. In addition, since the objects may be diverse in shape, color and size, to acquire the accurate object position and velocity, an appropriate solution is necessary. Motivated by the above introductions, this paper aims to develop a learning-based controller to track different complex trajectories without frequent and specific parameter adjustment processes. Firstly, a visual measurement system is developed to quickly find and estimate the position and velocity of an object. Secondly, with the object’s information, the learning from demonstration method (GMM method) is applied for the control policy design. Thirdly, the detailed system stability analysis is presented, and the corresponding controller parameter constraints are derived and considered in the proposed control policy. Subsequently, with the demonstration data, the packaged learning algorithm will automatically compute the controller parameters, and users can change the controller performance only by providing the desired demonstrations. In summary, this paper proposes a systematic object tracking solution, and it may bring a new idea to develop a practical object tracking system, using both the learning-based methods to improve the ability of generalization for tracking different objects.

A Robot Learning from Demonstration Platform Based on Optical Motion Capture

Human Demonstration Trajectory Refinement for Redundant Manipulators.

COMBINATION OF AFFINE DEFORMATION AND DYNAMIC MOVEMENT PRIMITIVE IN LEARNING HUMAN MOTION FOR REDUNDANT MANIPULATOR

Development of a Human-Robot Hybrid Intelligent System Based on Brain Teleoperation and Deep Learning SLAM

Trajectory Generation with Multi-Stage Cost Functions Learned from Demonstrations

Learning From Human Demonstrations for Wheel Mobile Manipulator: An Unscented Model Predictive Control Approach

A Human–Robot Collaboration Method Using a Pose Estimation Network for Robot Learning of Assembly Manipulation Trajectories From Demonstration Videos

DeMoBot: Deformable Mobile Manipulation with Vision-based Sub-goal Retrieval

Demonstration Learning and Generalization of Robotic Motor Skills Based on Wearable Motion Tracking Sensors

Learning Articulated Motions From Visual Demonstration

Biologically Inspired Motion Modeling and Neural Control for Robot Learning From Demonstrations

SLoMo: A General System for Legged Robot Motion Imitation from Casual Videos

TeleMoMa: A Modular and Versatile Teleoperation System for Mobile Manipulation

A Combined Learning and Optimization Framework to Transfer Human Whole-body Loco-manipulation Skills to Mobile Manipulators

Learning Physical Collaborative Robot Behaviors From Human Demonstrations

Learning Robotic Manipulation from Demonstrations by Combining Deep Generative Model and Dynamic Control System

Intelligent robotic arm for human pose recognition based on teleoperation system

Skill Learning in Robot-Assisted Micro-Manipulation Through Human Demonstrations with Attention Guidance

RoMo: A Robust Solver for Full-body Unlabeled Optical Motion Capture

A Learning-Based Object Tracking Strategy Using Visual Sensors and Intelligent Robot Arm

DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulation