Abstract:This paper focuses on addressing the visual tracking problem using learning-based methods for object tracking tasks. This problem contains a major difficulty, i.e., how to acquire a satisfactory generalization ability of the developed system? In this paper, firstly, the object state tracking system, including a camera-in-hand, a 3D camera and a Rethink Baxter robot, is introduced. The problem formulation is also presented. Secondly, we propose a Kalman-based estimation strategy to acquire the object’s state. In addition, a learning-based tracking controller is developed using the Gaussian mixture models (GMM) method to steer the robot end-effector to track the mobile object. Thirdly, to guarantee system stability (i.e., the position and velocity errors between the object and end-effector will always converge to zeros), the controller parameter constraints are derived, which is a theoretical contribution of this paper. The controller parameter adjustment is avoided by the proposed training process. Thus, the proposed method becomes easy to implement, which is a practical contribution. Finally, the effectiveness of the proposed method is demonstrated by simulation and experimental examples, and the proposed method has satisfactory generalization ability. Note to Practitioners—This paper studies object tracking problems for different practical applications, such as industrial cutting, grasping and dynamic monitoring. Different trajectory tracking methods have been widely applied in the industrial area. However, users always complain that when the object or trajectory is changed, the tracking controller more or less needs to be re-adjusted. This re-adjust process always requires professional knowledge and programming experience, and thus a factory must employ some professional engineers. In addition, since the objects may be diverse in shape, color and size, to acquire the accurate object position and velocity, an appropriate solution is necessary. Motivated by the above introductions, this paper aims to develop a learning-based controller to track different complex trajectories without frequent and specific parameter adjustment processes. Firstly, a visual measurement system is developed to quickly find and estimate the position and velocity of an object. Secondly, with the object’s information, the learning from demonstration method (GMM method) is applied for the control policy design. Thirdly, the detailed system stability analysis is presented, and the corresponding controller parameter constraints are derived and considered in the proposed control policy. Subsequently, with the demonstration data, the packaged learning algorithm will automatically compute the controller parameters, and users can change the controller performance only by providing the desired demonstrations. In summary, this paper proposes a systematic object tracking solution, and it may bring a new idea to develop a practical object tracking system, using both the learning-based methods to improve the ability of generalization for tracking different objects.

Tracking Control for Motion Constrained Robotic System Via Dynamic Event-Sampled Intelligent Learning Method.

Optimal Tracking Control for Motion Constrained Robot Systems Via Event-Sampled Critic Learning

Tracking Control for Robot Manipulator Based on Deterministic Learning and Event-Triggered Mechanism

Fuzzy Adaptive Tracking of Constrained Nonlinear Systems with Event-Sampling Reinforcement Learning

Dynamic Learning from Adaptive Neural Control of Uncertain Robots with Guaranteed Full-State Tracking Precision

Event-Triggered Learning Robust Tracking Control of Robotic Systems with Unknown Uncertainties

Event-triggered Learning-Based Robust Tracking Control for Robotic Manipulators with Uncertain Dynamics and Non-Zero Equilibrium

Cooperative Tracking Control for Nonlinear MASs under Event-Triggered Communication.

Predefined-time event-triggered adaptive neural practical tracking control for flexible-joint robot

Learning-Based Tracking Control of Unknown Robot Systems with Online Parameter Estimation

Time-Varying Transformation-Based Adaptive Tracking Control of Uncertain Robotic Systems With Event-Triggered Mechanism

Event-triggered practical tracking control for an uncertain free-flying flexible-joint space robot with dead-zone input

Neural‐based adaptive event‐triggered tracking control for flexible‐joint robots with random noises

A Learning-Based Object Tracking Strategy Using Visual Sensors and Intelligent Robot Arm

Neural Control of Robotic Manipulator with Guaranteed Tracking Precision and Velocity Constraints

Adaptive neural network iterative learning control of long-stroke hybrid robots with initial errors and full state constraints

Neural-Learning Trajectory Tracking Control of Flexible-Joint Robot Manipulators with Unknown Dynamics

Event-triggered neural adaptive tracking control for output constrained nonlinear multi-agent systems with unmodeled dynamics and unknown signs

Event-triggered Adaptive Tracking Control for Stochastic Nonlinear Systems With State Constraints

Dynamic Learning from Adaptive Neural Control for Flexible Joint Robot with Tracking Error Constraints Using High-Gain Observer

Model-free tracking control of complex dynamical trajectories with machine learning