Abstract:Objective. Brain–machine interfaces (BMIs) interpret human intent into machine reactions, and the visual stimulation (VS) paradigm is one of the most widely used of these approaches. Although VS-based BMIs have a relatively high information transfer rate (ITR), it is still difficult for BMIs to control machines in dynamic environments (for example, grabbing a dynamic object or targeting a walking person). Approach. In this study, we utilized a BMI based on augmented reality (AR) VS (AR-VS). The proposed VS was dynamically generated based on machine vision, and human intent was interpreted by a dynamic decision time interval approach. A robot based on the coordination of a task and self-motion system was controlled by the proposed paradigm in a fast and flexible state. Methods. Objects in scenes were first recognized by machine vision and tracked by optical flow. AR-VS was generated based on the objects' parameters. The number and distribution of VS was confirmed by the recognized objects. Electroencephalogram (EEG) features corresponding to VS and human intent were collected by a dry-electrode EEG cap and determined by the filter bank canonical correlation analysis method. Key parameters in the AR-VS, including the effect of VS size, frequency, dynamic object moving speed, ITR and the performance of the BMI-controlled robot, were analyzed. Conclusion and significance. The ITR of the proposed AR-VS paradigm for nine healthy subjects was 36.3 ± 20.1 bits min−1. In the online robot control experiment, brain-controlled hybrid tasks including self-moving and grabbing objects were 64% faster than when using the traditional steady-state visual evoked potential paradigm. The proposed paradigm based on AR-VS could be optimized and adopted in other kinds of VS-based BMIs, such as P300, omitted stimulus potential, and miniature event-related potential paradigms, for better results in dynamic environments.

Brain-Inspired Visual Attention Modeling Based on EEG for Intelligent Robotics

Human Visual Perception Based Image Quality Assessment for Video Prediction

Visual Attention Measurement Based on Electroencephalogram Feature Learning

Video Abstraction Based on Fmri-Driven Visual Attention Model

Achieving More Human Brain-Like Vision via Human EEG Representational Alignment

EEG-based Decoding of Selective Visual Attention in Superimposed Videos

A Visual Brain Chip Based on Selective Attention for Robot Vision Application

Human-centered attention models for video summarization

Coordinating human-robot collaboration by EEG-based human intention prediction and vigilance control

Learning Computational Models of Video Memorability from Fmri Brain Imaging

Brain-Inspired Image Perceptual Quality Assessment Based on EEG: A QoE Perspective

Seeing through the Brain: Image Reconstruction of Visual Perception from Human Brain Signals

Machine-vision fused brain machine interface based on dynamic augmented reality visual stimulation

FMRI-Guided Time-Symmetric Joint Model for Visual Attention Prediction.

Channel Optimized Visual Imagery based Robotic Arm Control under the Online Environment

Brain-Inspired Image Quality Assessment Method Based on Electroencephalography Feature Learning

Unidirectional brain-computer interface: Artificial neural network encoding natural images to fMRI response in the visual cortex

Research on binocular stereo video attention model based on human visual system

Autoencoder-enabled eye-tracking data analytics for objective assessment of user preference in humanoid robot appearance design

A Vision-Based Measure of Environmental Effects on Inferring Human Intention During Human Robot Interaction

A Human-Attention Driven Video Capture Method for Wearable Visual Computing