Abstract:Background: Few studies have systematically investigated robust controllers for lower limb rehabilitation exoskeletons (LLREs) that can safely and effectively assist users with a variety of neuromuscular disorders to walk with full autonomy. One of the key challenges for developing such a robust controller is to handle different degrees of uncertain human-exoskeleton interaction forces from the patients. Consequently, conventional walking controllers either are patient-condition specific or involve tuning of many control parameters, which could behave unreliably and even fail to maintain balance. Methods: We present a novel, deep neural network, reinforcement learning-based robust controller for a LLRE based on a decoupled offline human-exoskeleton simulation training with three independent networks, which aims to provide reliable walking assistance against various and uncertain human-exoskeleton interaction forces. The exoskeleton controller is driven by a neural network control policy that acts on a stream of the LLRE's proprioceptive signals, including joint kinematic states, and subsequently predicts real-time position control targets for the actuated joints. To handle uncertain human interaction forces, the control policy is trained intentionally with an integrated human musculoskeletal model and realistic human-exoskeleton interaction forces. Two other neural networks are connected with the control policy network to predict the interaction forces and muscle coordination. To further increase the robustness of the control policy to different human conditions, we employ domain randomization during training that includes not only randomization of exoskeleton dynamics properties but, more importantly, randomization of human muscle strength to simulate the variability of the patient's disability. Through this decoupled deep reinforcement learning framework, the trained controller of LLREs is able to provide reliable walking assistance to patients with different degrees of neuromuscular disorders without any control parameter tuning. Results and conclusion: A universal, RL-based walking controller is trained and virtually tested on a LLRE system to verify its effectiveness and robustness in assisting users with different disabilities such as passive muscles (quadriplegic), muscle weakness, or hemiplegic conditions without any control parameter tuning. Analysis of the RMSE for joint tracking, CoP-based stability, and gait symmetry shows the effectiveness of the controller. An ablation study also demonstrates the strong robustness of the control policy under large exoskeleton dynamic property ranges and various human-exoskeleton interaction forces. The decoupled network structure allows us to isolate the LLRE control policy network for testing and sim-to-real transfer since it uses only proprioception information of the LLRE (joint sensory state) as the input. Furthermore, the controller is shown to be able to handle different patient conditions without the need for patient-specific control parameter tuning.

Imposing Healthy Hip Motion Pattern and Range by Exoskeleton Control for Individualized Assistance

A Data-Driven Reinforcement Learning Solution Framework for Optimal and Adaptive Personalization of a Hip Exoskeleton

Non-linear Sliding Mode Control of the Lower Extremity Exoskeleton Based on Human–robot Cooperation

The Control System for Flexible Hip Assistive Exoskeleton

Optimizing Exoskeleton Assistance: Muscle Synergy-Based Actuation for Personalized Hip Exoskeleton Control

Human-in-the-Loop Control of a Wearable Lower Limb Exoskeleton for Stable Dynamic Walking

Reducing the muscle activity of walking using a portable hip exoskeleton based on human-in-the-loop optimization

Iterative Learning Control for a Soft Exoskeleton with Hip and Knee Joint Assistance

Adaptive CPG-Based Gait Planning With Learning-Based Torque Estimation and Control for Exoskeletons

Optimal Energy Shaping Control for a Backdrivable Hip Exoskeleton

Optimally Biomimetic Passivity-Based Control of a Lower-Limb Exoskeleton Over the Primary Activities of Daily Life

Reinforcement Learning and Control of a Lower Extremity Exoskeleton for Squat Assistance

Human-in-the-Loop Trajectory Optimization Based on sEMG Biofeedback for Lower-Limb Exoskeleton

Learning to Assist Different Wearers in Multitasks: Efficient and Individualized Human-In-the-Loop Adaption Framework for Exoskeleton Robots

Exoskeleton active assistance strategy for human muscle activation reduction during linear and circular walking

Experiment-free exoskeleton assistance via learning in simulation

Robust walking control of a lower limb rehabilitation exoskeleton coupled with a musculoskeletal model via deep reinforcement learning

Adaptive Oscillator-Based Robust Control for Flexible Hip Assistive Exoskeleton

Learning to Assist Different Wearers in Multitasks: Efficient and Individualized Human-in-the-Loop Adaptation Framework for Lower-Limb Exoskeleton

Hybrid oscillator-based no-delay hip exoskeleton control for free walking assistance

Adaptive Control of Lower-Limb Exoskeletons for Walking Assistance Based on Inter-Joint Coordination