Abstract:The automatic control of excavator operation trajectories is a pivotal technology for autonomous excavators, with its essential prerequisite being the real-time measurement of manipulator poses. Given the complexity of the operating environment, traditional sensor-based measurement methods face limitations, whereas visual measurement emerges as a promising technique. Achieving accurate visual measurement of excavator manipulator poses involves a crucial aspect: mapping the relationship between image information and poses. Firstly, to address the significant errors in pose prediction encountered with machine learning techniques like Artificial Neural Networks (ANN), this work introduces a mathematical model specifically for mapping this relationship, referred to as the pose mapping mathematical model (PMMM), which includes the calibration of model parameters. Secondly, to address the sensitivity of initial values in the calibration process, we propose the residual-guided initialization algorithm. This algorithm aims to ensure that initial values closely approximate the ground truth values, thus preventing matrix singularity at the source and avoiding parameter estimation divergence. Thirdly, to address challenges such as those caused by unstable lighting conditions and discrepancies between the dataset and the mathematical model, we introduce the RANSAC (random sample consensus)-Driven LM (Levenberg–Marquardt) parameter optimization algorithm to enhance the accuracy of parameter estimation. Experiments with static and dynamic online measurement demonstrate that our method reduces pose measurement errors compared to existing methods. This research lays a solid foundation for the development of excavator visual measurement techniques and automated manipulator control based on visual measurements, also serving as a valuable reference for research on mechanical arms.

A New Measurement Method of Real-time Pose Estimation for an Automatic Hydraulic Excavator

Vision-based Excavator Pose Estimation for Automatic Control

An ANN-Based Precision Compensation Method for Industrial Manipulators Via Optimization of Point Selection

Modeling and Control of General Hydraulic Excavator for Human-in-the-loop Automation

Calibration of Visual Measurement System for Excavator Manipulator Pose

A vision-based end pose estimation method for excavator manipulator

Full-body pose estimation for excavators based on data fusion of multiple onboard sensors

An Accurate And Stable Pose Estimation Method Based On Geometry For Port Hoisting Machinery

Visual Sensor Target Tracking and Localization Method for Automatic Excavators

Research on Trajectory Planning and Autodig of Hydraulic Excavator

Study of IMU Installation Position for Posture Estimation of Excavators

Real-Time Bucket Pose Estimation Based on Deep Neural Network and Registration Using Onboard 3D Sensor

Dynamic Hazardous Proximity Zone Design for Excavator Based on 3D Mechanical Arm Pose Estimation Via Computer Vision

Three-Dimensional Working Pose Estimation in Industrial Scenarios with Monocular Camera

Method for the Relative Pose Reconstruction of Hydraulic Supports Driven by Digital Twins

A Novel Method for Measuring Pose of Hydraulic Supports Relative to Inspection Robot Using LiDAR

Vision-Based Excavator Detection and Tracking Using Hybrid Kinematic Shapes and Key Nodes

Working Stage Identification Of Excavators Based On Control Signals Of Operating Handles

Pose Estimation Technique of Scattered Pistons Based on CAD Model and Global Feature

An Improved Method of Real-Time Camera Pose Estimation Based on Descriptor Tracking.

A Posture Measurement Approach for an Articulated Manipulator by RGB-D Cameras