Terrain-Aware Stride-Level Trajectory Forecasting for a Powered Hip Exoskeleton via Vision and Kinematics Fusion

Ruoqi Zhao,Xingbang Yan,Yubo Fan
2024-04-18
Abstract:Powered hip exoskeletons have shown the ability for locomotion assistance during treadmill walking. However, providing suitable assistance in real-world walking scenarios which involve changing terrain remains challenging. Recent research suggests that forecasting the lower limb joint's angles could provide target trajectories for exoskeletons and prostheses, and the performance could be improved with visual information. In this letter, We share a real-world dataset of 10 healthy subjects walking through five common types of terrain with stride-level label. We design a network called Sandwich Fusion Transformer for Image and Kinematics (SFTIK), which predicts the thigh angle of the ensuing stride given the terrain images at the beginning of the preceding and the ensuing stride and the IMU time series during the preceding stride. We introduce width-level patchify, tailored for egocentric terrain images, to reduce the computational demands. We demonstrate the proposed sandwich input and fusion mechanism could significantly improve the forecasting performance. Overall, the SFTIK outperforms baseline methods, achieving a computational efficiency of 3.31 G Flops, and root mean square error (RMSE) of 3.445 \textpm \ 0.804\textdegree \ and Pearson's correlation coefficient (PCC) of 0.971 \textpm\ 0.025. The results demonstrate that SFTIK could forecast the thigh's angle accurately with low computational cost, which could serve as a terrain adaptive trajectory planning method for hip exoskeletons. Codes and data are available at
Robotics
What problem does this paper attempt to address?
This paper aims to address the problem of providing adaptive motion assistance for powered hip exoskeletons in different terrains. Existing exoskeleton control strategies perform poorly in real walking scenarios because they fail to effectively adapt to continuously changing terrains. The researchers propose a network called "Sandwich Fusion Transformer for Image and Kinematics" (SFTIK), which combines visual images and inertial measurement unit (IMU) data to predict the thigh angle for the next gait cycle. By using a method called width-level partitioning, the computational requirements for first-person terrain images are reduced, resulting in improved prediction performance. The paper presents a dataset consisting of walking data for 10 healthy participants in 5 common terrains, with labels for each gait cycle. The SFTIK network utilizes terrain images of the previous and next steps, as well as the IMU time series of the previous step, to predict the thigh angle. Its design includes a "sandwich" fusion mechanism that significantly improves prediction performance. Experimental results show that SFTIK achieves high prediction accuracy (RMSE of 3.445±0.804°, PCC of 0.971±0.025) while maintaining low computational costs, making it a promising approach for adaptive trajectory planning of hip exoskeletons in different terrains.