Abstract:From the computational point of view, musculoskeletal control is the problem of controlling high degrees of freedom and dynamic multi-body system that is driven by redundant muscle units. A critical challenge in the control perspective of skeletal joints with antagonistic muscle pairs is finding methods robust to address this ill-posed nonlinear problem. To address this computational problem, we implemented a twofold optimization and learning framework to be specialized in addressing the redundancies in the muscle control . In the first part, we used model predictive control to obtain energy efficient skeletal trajectories to mimick human movements. The second part is to use deep reinforcement learning to obtain a sequence of stimulus to be given to muscles in order to obtain the skeletal trajectories with muscle control. We observed that the desired stimulus to muscles is only efficiently constructed by integrating the state and control input in a closed-loop setting as it resembles the proprioceptive integration in the spinal cord circuits. In this work, we showed how a variety of different reference trajectories can be obtained with optimal control and how these reference trajectories are mapped to the musculoskeletal control with deep reinforcement learning. Starting from the characteristics of human arm movement to obstacle avoidance experiment, our simulation results confirm the capabilities of our optimization and learning framework for a variety of dynamic movement trajectories. In summary, the proposed framework is offering a pipeline to complement the lack of experiments to record human motion-capture data as well as study the activation range of muscles to replicate the specific trajectory of interest. Using the trajectories from optimal control as a reference signal for reinforcement learning implementation has allowed us to acquire optimum and human-like behaviour of the musculoskeletal system which provides a framework to study human movement in-silico experiments. The present framework can also allow studying upper-arm rehabilitation with assistive robots given that one can use healthy subject movement recordings as reference to work on the control architecture of assistive robotics in order to compensate behavioural deficiencies. Hence, the framework opens to possibility of replicating or complementing labour-intensive, time-consuming and costly experiments with human subjects in the field of movement studies and digital twin of rehabilitation.

Predictive Control of Musculotendon Loads Across Fast and Slow-twitch Muscles in a Simulated System with Parallel Actuation

Predictive Control of Peak Achilles Tendon Force in a Simulated System of the Human Ankle Joint with a Parallel Artificial Actuator During Hopping

Bio-inspired Control of a Rehabilitation Robot Actuated by Pneumatic Artificial Muscles

Simulating Ankle Torque during Walking Using a new Bioinspired Muscle Model with Application for Controlling a Powered Exoskeleton

Neuromechanical model-based adaptive control of bi-lateral ankle exoskeletons: biological joint torque and electromyogram reduction across walking conditions

A Tube-Based Model Predictive Control Method to Regulate a Knee Joint With Functional Electrical Stimulation and Electric Motor Assist

Design of EMG-driven Musculoskeletal Model for Volitional Control of a Robotic Ankle Prosthesis

Real-time Robust Nonlinear Model Predictive Control with Monotonically Increasing Weight for Quadruped Locomotion

Event-Triggered Adaptive Hybrid Torque-Position Control (ET-AHTPC) for Robot-Assisted Ankle Rehabilitation

Model Predictive Control for Human-Centred Lower Limb Robotic Assistance

Adaptive Interaction Torque-Based AAN Control for Lower Limb Rehabilitation Exoskeleton.

A Realistic Model Reference Computed Torque Control Strategy for Human Lower Limb Exoskeletons

A Non-Linear Model Predictive Task-Space Controller Satisfying Shape Constraints for Tendon-Driven Continuum Robots

Neural-network-based Nonlinear Model Predictive Tracking Control of a Pneumatic Muscle Actuator-Driven Exoskeleton

Implementation of controlling strategy in a biomechanical lower limb model with active muscles for coupling multibody dynamics and finite element analysis

Model-Based Dynamic Control Allocation in a Hybrid Neuroprosthesis

Performance Control of Tendon-Driven Endoscopic Surgical Robots With Friction and Hysteresis

Evaluation of a Fused Sonomyography and Electromyography-Based Control on a Cable-Driven Ankle Exoskeleton

Optimum trajectory learning in musculoskeletal systems with model predictive control and deep reinforcement learning

A hybrid approach for dynamically training a torque prediction model for devising a human-machine interface control strategy

Modeling and Sensorless Force Control of Novel Tendon-Sheath Artificial Muscle Based on Hill Muscle Model