Yang An,Yaqi Li,Hongwei Wang,Rob Duffield,Steven W. Su
Abstract:This study introduces a novel approach to robot-assisted ankle rehabilitation by proposing a Dual-Agent Multiple Model Reinforcement Learning (DAMMRL) framework, leveraging multiple model adaptive control (MMAC) and co-adaptive control strategies. In robot-assisted rehabilitation, one of the key challenges is modelling human behaviour due to the complexity of human cognition and physiological systems. Traditional single-model approaches often fail to capture the dynamics of human-machine interactions. Our research employs a multiple model strategy, using simple sub-models to approximate complex human responses during rehabilitation tasks, tailored to varying levels of patient incapacity. The proposed system's versatility is demonstrated in real experiments and simulated environments. Feasibility and potential were evaluated with 13 healthy young subjects, yielding promising results that affirm the anticipated benefits of the approach. This study not only introduces a new paradigm for robot-assisted ankle rehabilitation but also opens the way for future research in adaptive, patient-centred therapeutic interventions.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is how to effectively model the complexity of human behavior during robot - assisted ankle rehabilitation, and how to achieve human - machine co - adaptation through the Dual - Agent Multi - Model Reinforcement Learning (DAMMRL) framework. Traditional single - model methods have limitations in capturing the dynamics of human - machine interaction, while the method proposed in this paper uses multiple simple sub - models to approximate complex human responses, which is suitable for patients with different levels of dysfunction. This method not only improves the flexibility of the system in actual experiments and simulated environments, but also paves the way for future research on adaptive, patient - centered treatment interventions.
### Key Point Analysis
1. **Problem Background**:
- The development of robot - assisted rehabilitation technology has significantly improved the recovery of patients' motor functions.
- Traditional single - model methods are difficult to capture the complex dynamics of human - machine interaction, especially in modeling human behavior during the rehabilitation process.
2. **Research Objectives**:
- Propose a Dual - Agent Multi - Model Reinforcement Learning (DAMMRL) framework, using Multi - Model Adaptive Control (MMAC) and co - adaptation control strategies to solve the problem of human behavior modeling in robot - assisted ankle rehabilitation.
- Through the multi - model strategy, use simple sub - models to approximate complex patient responses and meet the needs of different rehabilitation stages.
3. **Method Innovation**:
- **Multi - Model Strategy**: Use multiple simple sub - models to approximate complex human responses, with each sub - model corresponding to different rehabilitation stages and patient states.
- **Dual - Agent Decision - Making Process**: Introduce a patient - machine dual - agent decision - making process, emphasizing human adaptation to the machine, and achieving human - machine co - adaptation through a co - learning algorithm.
- **Reward Function Design**: Design a reward function that includes expected movement performance and rehabilitation effect indicators, and make personalized adjustments according to different rehabilitation purposes and stages.
4. **Experimental Verification**:
- The feasibility and potential of this method were proven through experiments on 13 healthy young subjects.
- The experimental results show that the DAMMRL framework can effectively improve the performance and efficiency of the human - machine system.
5. **Future Prospects**:
- This research not only provides a new paradigm for robot - assisted ankle rehabilitation, but also lays the foundation for future research on adaptive, patient - centered treatment interventions.
- Future research directions include the application of model simplification techniques, such as pruning, to reduce model complexity and improve interpretability and acceptance.
### Formula Examples
- **Reward Function**:
\[
R = w_1\cdot\text{tracking_error}+w_2\cdot\text{patient_effort}
\]
where \(\text{tracking_error}\) represents the trajectory error, \(\text{patient_effort}\) represents the patient's movement effort, and \(w_1\) and \(w_2\) are weight parameters that can be adjusted according to the rehabilitation stage and objective.
- **Policy Function**:
\[
v_{\text{VR}}=\pi_0(p, e_t, s_m, v_{\text{prev}}, \tau_m)
\]
where \(v_{\text{VR}}\) is the virtual reality indicator, \(p\) is the position, \(e_t\) is the trajectory error, \(s_m\) is the trajectory smoothness, \(v_{\text{prev}}\) is the previous virtual reality indicator value, and \(\tau_m\) is the torque provided by the machine.
Through these innovative methods, this research provides new ideas and solutions for the development of robot - assisted rehabilitation technology.