Abstract:Navigation and autonomous operation in farmland or greenhouse environments present significant challenges for agricultural mobile robots on account of unstructured scene conditions and complex task requirements. The immersive interfaces based on human–robot interaction have been extensively developed, which bridge the high‐level decision of human beings and the precise sensing and high motion ability of robots. In this study, we propose a human–robot interaction framework based on three‐dimensional mapping and virtual reality (VR) visualization. On the construction client, dense three‐dimensional point‐cloud maps are created based on the simultaneous localization and mapping real‐time appearance‐based mapping algorithm with a Kinect‐style depth camera. Subsequently, point‐cloud filtering and mesh creation methods are introduced for postprocessing the map models. A data communication network is employed to transmit the optimized models to the VR‐based exploration client. Utilizing VR visualization, the operator can gain an intuitive and comprehensive understanding of the environments to be explored. The three‐dimensional agricultural scenes are mapped into VR models through the system, which can combine the physical worlds with the VR spaces in a better manner. Compared with the video streaming‐based method for three‐dimensional mapping, the map models with colors and textures are entirely displayed in the VR headset, thus providing robust and global human–robot interaction interfaces and enhancing the immersive exploration for human beings to a considerable extent. Experimental results indicate that the three‐dimensional map models exhibit relatively sufficient construction integrity and yield favorable optimization effects. Additionally, according to a user questionnaire survey, immersive VR‐based interfaces can be more effectively utilized for the three‐dimensional mapping of mobile robots. Research evidence and results demonstrate that our proposed system framework offers positive assistance and support for enhancing human–robot interaction and executing specific robotic tasks in unstructured field agricultural environments. Code implementation and related data sets are shared at the link https://github.com/WangDongBUAA/Mapping_Interfacing.

Research and application of 3-D VR remote monitor based on Web

Study on Remote Monitoring System of 3D Scanning and Measuring Machine

Infrastructure of Web-based VR-form virtual laboratory

Study on monitoring approach of 3D scanning and measuring machine based on Internet/Intranet

Design of the Virtual Reality-based Remote Equipment Monitoring System

Distributed Management of Virtual Reality Digital Assets for Online Experimental Education

Virtual-reality Robot Teleoperation System Based on Web Developed by JAVA 3D

RESEARCH ON REMOTE ROBOT CONTROL SYSTEM BASED ON WEB

An Internet-Based Conference System for Real-Time Distributed Design Evaluation

Design And Implement Of Web-Based Three-Dimensional Coalmine Safety Monitoring Visualization Control

Three‐dimensional mapping and immersive human–robot interfacing utilize Kinect‐style depth cameras and virtual reality for agricultural mobile robots

The Remote Intelligent Virtual Reality Monitoring and Control System for Flower Maintenance Using Internet of Things and Unity 3 D

Internet-Based Remote Control System For Measurement Of 3-D Object Profile

Research on Remote Virtual Assembly Technology of Plug Valve Based on VRML

A VR Simulation Framework Integrated with Multisource CAE Analysis Data for Mechanical Equipment Working Process.

Research and Implementation of Ancient Architecture Roaming System Based on VRML

Design and Realization of a Remote Control System Based on Web

A remote VR operation system for a fully mechanised coal-mining face using real-time data and collaborative network technology

Development and application of VR course resources based on embedded system in open education

A Distributed System Monitoring Tool with Virtual Reality

Virtual Reality Based Robot Teleoperation via Human-Scene Interaction