Abstract:Vision and proprioception regulate motor output during reaching. To study the effects of sensory input on motor control, brain computer interfaces (BCIs) offer particular advantages. As part of a long-term clinical BCI trial, we implanted two 96-channel microelectrode arrays into M1 of a person who was completely paralyzed below the neck but retained intact somatosensation. Neural recordings from M1 were transformed into a 2-dimensional velocity control signal for a robotic arm using an optimal linear estimator decoder that was calibrated while the participant imagined performing movements demonstrated by a virtual arm. Once the decoder was calibrated, we asked the participant to move the robotic arm left and right past a pair of lines as many times as possible in one minute. We examined how visual and proprioceptive feedback were incorporated into BCI control during this task by providing the participant with either visual or proprioceptive feedback, both, or neither. Proprioceptive feedback was provided by moving the participant’s own arm to match the movement of the robotic arm. Task performance with vision or proprioception alone was better than when neither were provided. However, providing proprioceptive feedback impaired performance relative to visual feedback alone, unless the decoder was calibrated with neural data collected while both visual and proprioceptive feedback were provided. Providing proprioceptive feedback during decoder calibration rescued performance because it better captured M1’s neural activity during BCI control with proprioceptive feedback. In general, BCI performance was positively correlated with how well the decoder captured variance in neural activity during the task. In summary, we found that while the BCI participant was able to use proprioceptive feedback regardless of whether the decoder was trained with vision only or vision and proprioception, training the decoder with both visual and proprioceptive feedback made performance more robust to the addition or removal of visual or proprioceptive feedback. This was because training a decoder with proprioceptive feedback allows the decoder to take advantage of proprioception-driven activity in M1. Overall, we demonstrated that natural sensation can be effectively combined with BCI to improve performance in humans.

Intermittent visuomotor processing in the human cerebellum, parietal cortex, and premotor cortex

Corticospinal interaction during isometric compensation for modulated forces with different frequencies

Effect of Visual Feedback on Behavioral Control and Functional Activity During Bilateral Hand Movement

Visual feedback alters the variations in corticospinal excitability that arise from rhythmic movements of the opposite limb

The Strength of the Corticospinal Coherence Depends on the Predictability of Modulated Isometric Forces

Neural, physiological, and behavioral correlates of visuomotor cognitive load

Visuoinertial and visual feedback in online steering control

World Statistics Drive Learning of Cerebellar Internal Models in Adaptive Feedback Control: A Case Study Using the Optokinetic Reflex

Influence of visual feedback on force-EMG curves from spinally innervated versus trigeminally innervated muscles

Visual Feedback Gain Modulates the Activation of Task-Related Networks and the Suppression of Non-Task Networks During Precise Grasping

Cortical, muscular, and kinetic activity underpinning attentional focus strategies during visuomotor control

Corticomuscular Coherence Dependence on Body Side and Visual Feedback

The Amount and Pattern of Reciprocal Compensations Predict Performance Stability in a Visually Guided Finger Force Production Task

Evaluating the Influence of Motor Control on Selective Attention through a Stochastic Model: The Paradigm of Motor Control Dysfunction in Cerebellar Patient

Mini-review: The Role of the Cerebellum in Visuomotor Adaptation

Early modality-specific somatosensory cortical regions are modulated by attended visual stimuli: interaction of vision, touch and behavioral intent

Double dissociation of visuomotor interaction mediated by visual feedback during continuous de novo motor learning

Visual processing of optic flow and motor control in the human posterior cingulate sulcus

Irregularity of instantaneous gamma frequency in the motor control network characterize visuomotor and proprioceptive information processing

Visuomotor feedback tuning in the absence of visual error information

Volitional control of movement interacts with proprioceptive feedback in motor cortex during brain-computer interface control in humans