Abstract:This research combines the NAO robot platform to develop three single scene modes (visual line-following navigation, object location and grabbing, and moving object tracking and obstacle avoidance), and finally through the frame based expert system to form a knowledge base of each mode, and uses the inference engine to achieve multi-mode fusion. In the development of visual line-following navigation, the proposed fast path extraction processing method can not only accurately extract the path information in the case of noisy interference, but also improve the running speed to ensure the real-time performance of the robot. A slope compensation PID controller is proposed to control the parameters of the robot during walking, which ensures low error and high stability when the robot follows the navigation line. In the development of object positioning and grabbing, a monocular localization algorithm is proposed using the characteristics of QR code, then the kinematics of the robot arm is modeled. Finally, the end effector of the robot is moved to the specified position according to the three-dimensional coordinates of the QR code to grasp the target object. In the development of moving object tracking and obstacle avoidance, the NaoMark marker is used to simplify the characteristics of the moving target, and the parameters of the robot walking are controlled according to the central position of the marker in the image. In order to avoid the collision of the robot with the obstacle during the movement, the obstacle avoidance is realized by combining the ultrasonic sensor measurement and the artificial potential field algorithm. Finally, the implementation methods of each part of the above three modes are formed into a knowledge base in the form of a framework based expert system, and the knowledge is combined by the inference engine to realize the fusion of multiple modes to complete tasks that require multiple modes. This research has presented an original solution, as a model, which can enable robots to perform complex service tasks consisting of multiple connected actions in a dynamic environment. This proposed methodology can let the intelligent operation of the robot serve in different scenes and fast response based on its actual requirements. This studying will enhance the prototype robot vision function and development of additional value for consolidation robot mobility based on multi-mode fusion to increase service robots in the home environment of intelligent control capability. In future, it will be extended to the hospital medical care and public safety, and other professional purpose.

Multi - target objects and complex color recognition model based on humanoid robot

Multi-objects Real Time Recognition Based on Color Information

Fast Recognition and Precise Localization of Humanoid Soccer Robot Vision System

Multi‐target detection and grasping control for humanoid robot NAO

Multi-modal 3D Human Tracking for Robots in Complex Environment with Siamese Point-Video Transformer

Humanoid robot action using multi-mode fusion based on frame expert system

Intelligent communication of two humanoid robots based on computer vison

Target recognition method based on NAO robot

Multi-sensor information fusion: Combination of fuzzy systems and evidence theory approaches in color recognition for the NAO humanoid robot

Detecting And Tracking Objects In Hri: Yolo Networks For The Nao "I See You" Function

Humanoid robots play chess using visual control

Improved Objects Matching with a Single RGB Camera

Multi-Class Detection and Segmentation of Objects in Depth

Intelligent Perception System of Robot Visual Servo for Complex Industrial Environment

Robot recognizing humans intention and interacting with humans based on a multi-task model combining ST-GCN-LSTM model and YOLO model

A Two-Stage Monocular Vision Detection Method for 6D Pose Estimation in Multi-Heterogeneous Robot Systems

Research on Target Hybrid Recognition and Localization Methods Based on an Industrial Camera and a Depth Camera in Complex Scenes

Multi-View Active Sensing for Human-Robot Interaction via Hierarchically Connected Tree

Object Detection and Pose Estimation from RGB and Depth Data for Real-time, Adaptive Robotic Grasping

Real-time multiple human perception with color-depth cameras on a mobile robot

Research in Target Recognition Arithmetic of Robot Based on Color