Abstract:An advanced method is presented for improving motion estimation in autonomous excavation operations within unstructured environments, addressing the significant challenges posed by dynamic objects and non-textured surfaces commonly encountered in earthwork scenarios. The proposed approach integrates Oriented Fast and Rotated BRIEF (ORB) feature points with semantic masks to enhance the accuracy and reliability of visual motion estimation. To obtain pixel level real-time semantic masks, a specialized semantic segmentation dataset was constructed, and a real-time segmentation method based on the DeeplabV3+ framework and MobileNetV2 backbone was implemented. The experimental platform, a modified excavator equipped with a stereo camera, was tested in an open, unstructured environment designed to simulate real-world earthmoving tasks. Extensive ablation and comparative experiments demonstrate that the proposed method substantially outperforms traditional visual Simultaneous Localization and Mapping (SLAM) algorithms particularly in scenarios involving static arm data, where the elimination of irrelevant dynamic feature points resulted in a significant improvement in both Absolute Trajectory Error (ATE) and Relative Pose Error (RPE). Even in cases where the arm is in motion, the method maintains superior accuracy through precise segmentation of dynamic and static feature points. Additionally, the optimized CPU and GPU computation times for mask generation further contribute to a frame processing time that meets the demands of autonomous excavation tasks, underscoring its practical applicability. In summary, our solution effectively addresses motion estimation challenges in complex and dynamic environments, particularly in continuous trench excavation with dynamic objects and non-textured surfaces. This advancement minimizes human intervention and promotes the automation of excavation construction for construction machinery companies, ultimately improving project efficiency and safety.

Vision-based Excavator Pose Estimation for Automatic Control

A New Measurement Method of Real-time Pose Estimation for an Automatic Hydraulic Excavator

Modeling and Control of General Hydraulic Excavator for Human-in-the-loop Automation

Calibration of Visual Measurement System for Excavator Manipulator Pose

Research on Trajectory Planning and Autodig of Hydraulic Excavator

Visual Sensor Target Tracking and Localization Method for Automatic Excavators

A vision-based end pose estimation method for excavator manipulator

Full-body pose estimation for excavators based on data fusion of multiple onboard sensors

Construction path tracking and pose estimation of unmanned bulldozer

Position-Based Visual Servoing Control for Multi-Joint Hydraulic Manipulator

Construction site safety monitoring and excavator activity analysis system

Overview of Servo Control Technologies for Excavators

Study of IMU Installation Position for Posture Estimation of Excavators

Vision-Based Categorical Object Pose Estimation and Manipulation.

A Comprehensive Study of 3-D Vision-Based Robot Manipulation

Intelligent Technologies for Construction Machinery Using Data-Driven Methods

Computer Vision Applications In Construction And Asset Management Phases: A Literature Review

Vision-Based Control of an Industrial Vehicle in Unstructured Environments

Vision-Based Detection and Real-Time Adaptive Control Schemes for Autonomous Ultrasound Scanning Robots

Automatic vision-based calculation of excavator earthmoving productivity using zero-shot learning activity recognition

Enhanced Motion Estimation for Autonomous Excavation: Accelerated Semantic Segmentation and ORB Features for Unstructured Environments