Abstract:In order to improve the target visual recognition and localization accuracy of robotic arms in complex scenes with similar targets, hybrid recognition and localization methods based on an industrial camera and depth camera are proposed. First, according to the speed and accuracy requirements of target recognition and localization, YOLOv5s is introduced as the basic algorithm model for target hybrid recognition and localization. Then, in order to improve the accuracy of target recognition and coarse localization based on an industrial camera (eye-to-hand), the AFPN feature fusion module, simple and parameter-free attention module (SimAM), and soft non-maximum suppression (Soft NMS) are introduced. In order to improve the accuracy of target recognition and fine localization based on a depth camera (eye-in-hand), the SENetV2 backbone network structure, dynamic head module, deformable attention mechanism, and chain-of-thought prompted adaptive enhancer network are introduced. After that, on the basis of constructing a dual camera platform for target hybrid recognition and localization, the hand–eye calibration, collection and production of image datasets required for model training are completed. Finally, for the docking of the oil filling port, the hybrid recognition and localization experimental tests are completed in sequence. The test results show that in target recognition and coarse localization based on the industrial camera, the recognition accuracy of the designed model reaches 99%, and the average localization errors in the horizontal and vertical directions are 2.22 mm and 3.66 mm, respectively. In target recognition and fine localization based on the depth camera, the recognition accuracy of the designed model reaches 98%, and the average errors in depth, horizontal, and vertical directions are 0.12 mm, 0.28 mm, and 0.16 mm, respectively. These not only verify the effectiveness of the target hybrid recognition and localization methods based on dual cameras, but also demonstrate that they meet the high-precision recognition and localization requirements in complex scenes.

A Cockpit Panel Detection Visual System with Panel Localization and Button State Recognition.

Teleoperate System of Underwater Cleaning Robot Based on HUD.

Hand Posture Recognition and Tracking Based on Bag-of-Words for Human Robot Interaction

Real-time Dynamic Gesture Recognition and Hand Servo Tracking Using PTZ Camera

Collaborative Recognition of Feasible Region with Aerial and Ground Robots Through DPCN

A Human-Robot Collaboration System for Object Handover

A Robot-in-the-loop Face Detection and Recognition System with Coordinaive Control Platform

Multi-class Assembly Parts Recognition Using Composite Feature and Random Forest for Robot Programming by Demonstration.

A Two-Stage Monocular Vision Detection Method for 6D Pose Estimation in Multi-Heterogeneous Robot Systems

YOLOv8-PoseBoost: Advancements in Multimodal Robot Pose Keypoint Detection

Target Recognition of Multi Source Machine Vision Pan Tilt Integrated Inspection Robot for Power Inspection

A Mobile Robotic Arm Grasping System with Autonomous Navigation and Object Detection

Enhanced Hand–Eye Coordination Control for Six-Axis Robots Using YOLOv5 with Attention Module

An approach based on deep learning methods to detect the condition of solar panels in solar power plants

Research on Target Hybrid Recognition and Localization Methods Based on an Industrial Camera and a Depth Camera in Complex Scenes

HRC of intelligent assembly system based on multimodal gesture control

Intelligent Mobile Robot Controller Design for Hotel Room Service with Deep Learning Arm-Based Elevator Manipulator

Accurate Robot Arm Attitude Estimation Based on Multi-View Images and Super-Resolution Keypoint Detection Networks

Continuous Hand Gestures Detection and Recognition in Emergency Human-Robot Interaction Based on the Inertial Measurement Unit

A Searching Space Constrained Partial to Full Registration Approach With Applications in Airport Trolley Deployment Robot

Design and Implementation of Automatic Assisted Aiming System For Robomaster EP Based on YOLOv5