Abstract:This research combines the NAO robot platform to develop three single scene modes (visual line-following navigation, object location and grabbing, and moving object tracking and obstacle avoidance), and finally through the frame based expert system to form a knowledge base of each mode, and uses the inference engine to achieve multi-mode fusion. In the development of visual line-following navigation, the proposed fast path extraction processing method can not only accurately extract the path information in the case of noisy interference, but also improve the running speed to ensure the real-time performance of the robot. A slope compensation PID controller is proposed to control the parameters of the robot during walking, which ensures low error and high stability when the robot follows the navigation line. In the development of object positioning and grabbing, a monocular localization algorithm is proposed using the characteristics of QR code, then the kinematics of the robot arm is modeled. Finally, the end effector of the robot is moved to the specified position according to the three-dimensional coordinates of the QR code to grasp the target object. In the development of moving object tracking and obstacle avoidance, the NaoMark marker is used to simplify the characteristics of the moving target, and the parameters of the robot walking are controlled according to the central position of the marker in the image. In order to avoid the collision of the robot with the obstacle during the movement, the obstacle avoidance is realized by combining the ultrasonic sensor measurement and the artificial potential field algorithm. Finally, the implementation methods of each part of the above three modes are formed into a knowledge base in the form of a framework based expert system, and the knowledge is combined by the inference engine to realize the fusion of multiple modes to complete tasks that require multiple modes. This research has presented an original solution, as a model, which can enable robots to perform complex service tasks consisting of multiple connected actions in a dynamic environment. This proposed methodology can let the intelligent operation of the robot serve in different scenes and fast response based on its actual requirements. This studying will enhance the prototype robot vision function and development of additional value for consolidation robot mobility based on multi-mode fusion to increase service robots in the home environment of intelligent control capability. In future, it will be extended to the hospital medical care and public safety, and other professional purpose.

Multi‐target detection and grasping control for humanoid robot NAO

Multi - target objects and complex color recognition model based on humanoid robot

Robot grasping system based on deep learning target detection

Kinematics Modeling and Trajectory Planning for NAO Robot Object Grasping Based on Image Analysis

Target Localization and Grasping of NAO Robot Based on YOLOv8 Network and Monocular Ranging

Target-Oriented Object Grasping via Multimodal Human Guidance

A Novel Robotic Grasping Method for Moving Objects Based on Multi-Agent Deep Reinforcement Learning

Vision-Based Robotic Object Grasping—A Deep Reinforcement Learning Approach

A Robotic Semantic Grasping Method for Pick-and-place Tasks

Grasping Control for a Neuro-Cognitive Robot.

Grasping detection of dual manipulators based on Markov decision process with neural network

Detecting And Tracking Objects In Hri: Yolo Networks For The Nao "I See You" Function

Deep learning-based control framework for dynamic contact processes in humanoid grasping

Visual-Guided Robotic Object Grasping Using Dual Neural Network Controllers

Deep learning-based method for vision-guided robotic grasping of unknown objects

Humanoid robot action using multi-mode fusion based on frame expert system

Enhanced Hand–Eye Coordination Control for Six-Axis Robots Using YOLOv5 with Attention Module

Vision-Based Robotic Arm Control Algorithm Using Deep Reinforcement Learning for Autonomous Objects Grasping

Grasping Diverse Objects with Simulated Humanoids

Robotic Object Recognition And Grasping With A Natural Background

A Multi-task Convolutional Neural Network for Autonomous Robotic Grasping in Object Stacking Scenes