Abstract:The application of virtual reality technology in science experiment education is a research with practical significance and value in human-computer interaction. However, in some existing education tools based on virtual reality, due to the single interaction mode, the complexity of user intention and the non-physical interaction characteristics brought by virtualization, their experimental teaching ability is limited, resulting in the lack of practical value and popularity. In order to solve these problems, a multimodal interaction model is constructed by fusing gesture, speech and pressure information. Specifically, our tasks include: 1) collecting user input information and time series information to construct basic data input tuples. 2) The basic interaction information is used to identify the user's basic intention, and the correlation degree between the user's intentions is considered to determine the correctness of the current identification intention. 3) It allows users to alternate between multi-channel and single channel interaction. Based on this model, we build a multi-modal intelligent interactive virtual experiment platform (MIIVEP), and design and implement a kind of dropper with strong perception ability, which has been verified, tested, evaluated and applied in the intelligent virtual experiment system. In addition, in order to evaluate this work more effectively, we developed a fair scoring criterion for the virtual experimental system (Evaluation scale of virtual experiment system, ESVES), and invited middle school teachers and students to participate in the verification of the results of this work. Through the user's actual use effect verification and result research, we prove the effectiveness of the proposed model and the corresponding implementation.

Research on Human Engineering Oriented Experimental Intelligent Testing Technology

Humanity Test—EEG Data Mediated Artificial Intelligence Multi-Person Interactive System

Subjective evaluation on human-machine-environment system of automotive driving simulation experiment platform

A Multimodal Emotional Communication Based Humans-Robots Interaction System

Design and Implementation of a Human-Machine Interaction Platform for Analysis on Human Cognitive Behaviors

Research on Intelligent Experimental Equipment and Key Algorithms Based on Multimodal Fusion Perception

The Research on Adaptive Operational Interface for Vehicle Equipments Based on Cognitive Experiments

Design and Construction of Automotive Human-machine Interface Experimental Platform with Multi-parameter and Adjustable Function

Human-computer interaction: Evaluation of trust and emotional

Research of Real-time Interaction Method in Human-machine Interface Design and Evaluation Based on UG/OPEN

Augmented Affective-Cognition for Usability Study of In-Vehicle System User Interface

Research on Scenario-Based User Experience Evaluation Method for Intelligent Cars

Design and Analysis of a Test Stand for Biomechanics of Man-Machine Interface in Automobile Driving

Study on Road Traffic Human-Computer Interface Simulator Experiment System

The Research On Human-Computer Interaction In Ambient Intelligence

Design Of An Integrative Automotive Ergonomics Experiment Platform

Inspiring Real-Time Evaluation and Optimization of Human–Robot Interaction with Psychological Findings from Human–Human Interaction

Evaluating User Experience for Intelligent Connected Vehicles: A Qualitative Study

Design of Human-Computer Interface System Based on Wireless Sensor Technology

Research of Design Evaluation Method Based on the Jack Human Model

Design and Evaluation of a Novel Person Transfer Assist System.