Abstract:Objective: Customized human-machine interfaces for controlling assistive devices are vital in improving the self-help ability of upper limb amputees and tetraplegic patients. Given that most of them possess residual shoulder mobility, using it to generate commands to operate assistive devices can serve as a complementary approach to brain-computer interfaces. Approach: We propose a hybrid body-machine interface prototype that integrates soft sensors and an inertial measurement unit. This study introduces both a rule-based data decoding method and a user intent inference-based decoding method to map human shoulder movements into continuous commands. Additionally, by incorporating prior knowledge of the user's operational performance into a shared autonomy framework, we implement an adaptive switching command mapping approach. This approach enables seamless transitions between the two decoding methods, enhancing their adaptability across different tasks. Main results: The proposed method has been validated on individuals with cervical spinal cord injury, bilateral arm amputation, and healthy subjects through a series of center-out target reaching tasks and a virtual powered wheelchair driving task. The experimental results show that using both the soft sensors and the gyroscope exhibits the most well-rounded performance in intent inference. Additionally, the rule-based method demonstrates better dynamic performance for wheelchair operation, while the intent inference method is more accurate but has higher latency. Adaptive switching decoding methods offer the best adaptability by seamlessly transitioning between decoding methods for different tasks. Furthermore, we discussed the differences and characteristics among the various types of participants in the experiment Significance: The proposed method has the potential to be integrated into clothing, enabling non-invasive interaction with assistive devices in daily life, and could serve as a tool for rehabilitation assessment in the future.

FootUI: Assisting People with Upper Body Motor Impairments to Use Smartphones with Foot Gestures on the Bed

An IoT-Enabled Telerobotic-Assisted Healthcare System Based on Inertial Motion Capture.

Shoes++: A Smart Detachable Sole for Social Foot-to-foot Interaction.

EarTouch: Facilitating Smartphone Use for Visually Impaired People in Mobile and Public Scenarios

A Portable Wireless Eye Movement-Controlled Human-Computer Interface for the Disabled

Motion-Based Perceptual User Interface

Toucha11y: Making Inaccessible Public Touchscreens Accessible

3D Intuitive Gesture Interaction via Motion Sensing

Perceived User Reachability in Mobile UIs Using Data Analytics and Machine Learning

AccessWear: Making Smartphone Applications Accessible to Blind Users

Eyelid gestures for people with motor impairments

Human body features recognition based adaptive user interface for extra-large touch screens

Mata: An Android Eye-Tracking Based User Interface Control Application

ThumbUp: Secure Smartwatch Controller for Smart Homes using Simple Hand Gestures

SwivelTouch: Boosting Touchscreen Input with 3D Finger Rotation Gesture

Non-verbal Interaction and Interface with a Quadruped Robot using Body and Hand Gestures: Design and User Experience Evaluation

Dynamic UI Adaptations for One-Handed Use of Large Mobile Touchscreen Devices

Finger Based Technique (FBT): An Innovative System for Improved Usability for the Blind Users' Dynamic Interaction with Mobile Touch Screen Devices

Mobile assistant: enhancing desktop interaction using mobile phone

Novel Interaction Techniques by Combining Hand and Foot Gestures on Tabletop Environments

SoftBoMI: a non-invasive wearable body-machine interface for mapping movement of shoulder to commands