Abstract:Visual servo control rules that refer to the control methods of robot motion planning using image data acquired from the camera mounted on the robot have been widely applied to the motion control of robotic arms or mobile robots. The methods are usually classified as image-based visual servo, position-based visual servo, and hybrid visual servo (HVS) control rules. Mobile manipulation enhances the working range and flexibility of robotic arms. However, there is little work on applying visual servo control rules to the motion of the whole mobile manipulation robot. We propose an HVS motion control method for a mobile manipulation robot which combines a six-degree-of-freedom (6-DOF) robotic arm with a nonholonomic mobile base. Based on the kinematic differential equations of the mobile manipulation robot, the global Jacobian matrix of the whole robot is derived, and the HVS control equation is derived using the whole Jacobian matrix combined with position and visual image information. The distance between the gripper and target is calculated through the observation of the marker by a camera mounted on the gripper. The differences between the positions of the markers’ feature points and the expected positions of them in the image coordinate system are also calculated. These differences are substituted into the control equation to obtain the speed control law of each degree of freedom of the mobile manipulation robot. To avoid the position error caused by observation, we also introduce the Kalman filter to correct the positions and orientations of the end of the manipulator. Finally, the proposed algorithm is validated on a mobile manipulation platform consisting of a Bulldog chassis, a UR5 robotic arm, and a ZED camera.

Robotic Inserting a Moving Object using Visual-based Control with Time-Delay Compensator

Hybrid Visual Servoing for Rivet-in-hole Insertion Based on Super-twisting Sliding Mode Control

Learning-based Optimization Algorithms Combining Force Control Strategies for Peg-in-Hole Assembly.

Sensor-Based Control Using an Image Point and Distance Features for Rivet-in-Hole Insertion.

Iteratively Successive Projection: A Novel Continuous Approach for the Task-Based Control of Redundant Robots.

A Noncontact Control Strategy for Circular Peg-in-Hole Assembly Guided by the 6-DOF Robot Based on Hybrid Vision

Visual Tracking Control with Time-delay Compensation

Visual Tracking Control of Cable-Driven Hyper-Redundant Snake-Like Manipulator

Vision-based robotic peg-in-hole research: integrating object recognition, positioning, and reinforcement learning

Laser tracker-based control for peg-in-hole assembly robot

Position/Force Visual-Sensing-Based Robotic Sheet-Like Peg-in-Hole Assembly

A Hybrid Visual Servo Control Method for Simultaneously Controlling a Nonholonomic Mobile and a Manipulator.

Visual Tracking And Grasping Of Moving Objects And Its Application To An Industrial Robot

Eye-in-Hand Robotic Arm Gripping System Based on Machine Learning and State Delay Optimization

Robotic Multiple Peg-in-Hole Assembly with Deviation Estimation and Fast Insertion Control

Transferable Trial-Minimizing Progressive Peg-in-hole Model

Autonomous 6-DOF Manipulator Operation for Moving Target by a Capture and Placement Control System

Multiple Peg-in-Hole Assembly of Tightly Coupled Multi-manipulator Using Learning-based Visual Servo

Robot Manipulator Curve Tracking in Unknown Plane Based on Stereo Vision

Time Delay Compensation of a Robotic Arm based on Multiple Sensors for Indirect Teaching

Quickly Inserting Pegs into Uncertain Holes using Multi-view Images and Deep Network Trained on Synthetic Data