Corrigendum: An initial prediction and fine-tuning model based on improving GCN for 3D human motion prediction

Zhiquan He,Lujun Zhang,Hengyou Wang

DOI: https://doi.org/10.3389/fncom.2023.1232765

IF: 3.387

2023-06-13

Frontiers in Computational Neuroscience

Abstract:[This corrects the article DOI: 10.3389/fncom.2023.1145209.].

neurosciences,mathematical & computational biology

What problem does this paper attempt to address?

An initial prediction and fine-tuning model based on improving GCN for 3D human motion prediction

Zhiquan He,Lujun Zhang,Hengyou Wang

DOI: https://doi.org/10.3389/fncom.2023.1145209

IF: 3.387

2023-04-06

Frontiers in Computational Neuroscience

Abstract:Human motion prediction is one of the fundamental studies of computer vision. Much work based on deep learning has shown impressive performance for it in recent years. However, long-term prediction and human skeletal deformation are still challenging tasks for human motion prediction. For accurate prediction, this paper proposes a GCN-based two-stage prediction method. We train a prediction model in the first stage. Using multiple cascaded spatial attention graph convolution layers (SAGCL) to extract features, the prediction model generates an initial motion sequence of future actions based on the observed pose. Since the initial pose generated in the first stage often deviates from natural human body motion, such as a motion sequence in which the length of a bone is changed. So the task of the second stage is to fine-tune the predicted pose and make it closer to natural motion. We present a fine-tuning model including multiple cascaded causally temporal-graph convolution layers (CT-GCL). We apply the spatial coordinate error of joints and bone length error as loss functions to train the fine-tuning model. We validate our model on Human3.6m and CMU-MoCap datasets. Extensive experiments show that the two-stage prediction method outperforms state-of-the-art methods. The limitations of proposed methods are discussed as well, hoping to make a breakthrough in future exploration.

neurosciences,mathematical & computational biology
Forecasting Distillation: Enhancing 3D Human Motion Prediction with Guidance Regularization

Yawen Du,Zhihao Wang,Yinming Li,Xiaosong Yang,Chao Wu,Zhao Wang

DOI: https://doi.org/10.1109/ijcnn60899.2024.10650336

2024-01-01

Abstract:Human motion prediction aims to forecast future body poses from historically observed sequences, which is challenging due to motion’s complex dynamics. Existing methods mainly focus on dedicated network structures to model the spatial and temporal dependencies. The predicted results are required to be strictly similar to the training samples with ℓ 2 loss in the current training pipeline. It needs to be pointed out that most approaches predict the next frame conditioned on the previously predicted sequence, where a small error in the initial frame could be accumulated significantly. In addition, recent work indicated that different stages could play different roles. Hence, this paper considers a new direction by introducing a model learning framework with motion guidance regularization to reduce uncertainty. The guidance information is extracted from a designed Fusion Feature Extraction network (FE-Net) while knowledge distilling is conducted through intermediate supervision to improve the multi-stage prediction network during training. Incorporated with baseline models, our guidance design exhibits clear performance gains in terms of 3D mean per joint position error (MPJPE) on benchmark datasets Human3.6M, CMU Mocap, and 3DPW datasets, respectively. Related code will be available on https://github.com/tempAnonymous2024/MotionPredict-GuidanceReg.
The Correction Of The Distortion Of Human Face Based On Three-Dimensional Modeling Methods

qingmin ye,kuo chen,huajun feng,zhihai xu,qi li

DOI: https://doi.org/10.1117/12.2193273

2015-01-01

Abstract:When the human face is on the edge of field of the camera which has a large view, serious deformation will be captured. To correct the distortion of the human face, we present an approach based on setting up a 3D model. Firstly, we construct 3D target face modeling by using the data and depth information of the standard human face, which is set up by the three-dimensional model with three-dimensional Gaussian function with sectional type. According to the size of the face in the image and the parameters of the camera, we can obtain the information of relative position and depth of the human face. Then by translating the virtual camera axis to the center of the face, we can achieve the goal to correct the distortion of the face based on the theory of three-dimensional imaging. Finally, we have made a lot of experiments, and we study the influence of parameters of the 3D model of human face. The result indicates that the method presented by this paper can play an effective role in correcting the distortion of the face in the edge of the view, and we can get better results if the model appreciates the real human face.
Corrigendum: A progression analysis of motor features in Parkinson's disease based on the mapper algorithm

Ling-Yan Ma,Tao Feng,Chengzhang He,Mujing Li,Kang Ren,Junwu Tu

DOI: https://doi.org/10.3389/fnagi.2023.1174022

IF: 4.8

2023-03-28

Frontiers in Aging Neuroscience

Abstract:[This corrects the article DOI: 10.3389/fnagi.2023.1047017.].

neurosciences,geriatrics & gerontology,gerontology
Corrigendum: Validation of the GALAD model and establishment of a new model for HCC detection in Chinese patients

Ping Chen,Haolin Song,Xu Wang,J. Guo,Jianfei Wang,Jianhua Zhou,Xiaohong Kang,Chaolei Jin,Yubo Cai,Zixuan Feng,Hainv Gao,Fang Lü,Lanjuan Li

DOI: https://doi.org/10.3389/fonc.2023.1170066

IF: 4.7

2023-01-01

Frontiers in Oncology

Abstract:[This corrects the article DOI: 10.3389/fonc.2022.1037742.].
Correction to: Predicting protein inter-residue contacts using composite likelihood maximization and deep learning

Haicang Zhang,Qi Zhang,Fusong Ju,Jianwei Zhu,Yujuan Gao,Ziwei Xie,Minghua Deng,Shiwei Sun,Wei-Mou Zheng,Dongbo Bu

DOI: https://doi.org/10.1186/s12859-019-3198-2

IF: 3.307

2019-01-01

BMC Bioinformatics

Abstract:Following publication of the original article [1], the author explained that there are several errors in the original article
Corrigendum: A novel approach for sports injury risk prediction: based on time-series image encoding and deep learning

Xiaohong Ye,Yuanqi Huang,Zhanshuang Bai,Yukun Wang

DOI: https://doi.org/10.3389/fphys.2024.1441107

2024-07-22

Abstract:[This corrects the article DOI: 10.3389/fphys.2023.1174525.].
Gradient multi-foci networks for 3D skeleton-based human motion prediction

Junyu Shi,Jianqi Zhong,Zhiquan He,Wenming Cao

DOI: https://doi.org/10.1007/s00521-024-09817-5

2024-05-11

Neural Computing and Applications

Abstract:To achieve 3D skeleton-based human motion prediction, attention-based methods have encouraged performance due to the observation that attention parts in the past state influence future actions. The model can identify the most relevant information for motion prediction by introducing an attention mechanism. However, existing methods tend to address the general allocation of attention without further exploiting the precision to pinpoint the exact location of the most relevant information. This oversight subsequently curtails the potential of prediction performance. To solve this problem, we propose a novel Gradient Multi-Foci Network (GMFnet), which leverages two-stage foci: spectral focus and spatial focus, to find the most pertinent information in a manner that emulates natural cognitive processes. The core idea of the proposed GMFnet is based on two aspects: spectral focus to model the repeatability of the observation action sequence by deploying an attention-based Related Sequences Directing Block (RSDB), spatial focus to capture the most valuable parts between motion joints by using Attention Feature Computational Unit (AFCU). Extensive experiments are conducted to reveal that GMFnet can capture the precision to pinpoint the exact attention location, thus enhancing the prediction performance. The proposed GMFnet outperforms state-of-the-art methods by 10.7 and 7.4% of MPJPEs for short-term and long-term prediction in Human 3.6M and by 14.3 and 4.8% of MPJPEs for short-term and long-term forecast in CMU-Mocap. Moreover, GMFnet outperforms even more in short term by 24.9% in AMASS. The code is available at https://github.com/JunyuShi02/GMFNet.

computer science, artificial intelligence
Prediction of Human Motion with Motion Optimization and Neural Networks

Juxing Wang,Zaojun Fang,Linyong Shen,Chen He

DOI: https://doi.org/10.1109/isrimt53730.2021.9596968

2021-01-01

Abstract:Human motion prediction is a crucial foundation for efficient and safe human-robot cooperation. In order to improve the accuracy of long-term motion prediction, an approach combining initial trajectory prediction and motion optimization is proposed. The neural network model is applied to model the human hand motion for initial prediction. Then, we utilize the Gradient Descent method to optimize the motion considering the target set constraints. The motion's target point position is obtained from K-Nearest Neighbor (KNN) classification model. Experiments on real reaching motion data demonstrate that the proposed algorithm achieves a higher prediction accuracy compared with the traditional single neural network model.
AMHGCN: Adaptive multi-level hypergraph convolution network for human motion prediction

Jinkai Li,Jinghua Wang,Lian Wu,Xin Wang,Xiaoling Luo,Yong Xu

DOI: https://doi.org/10.1016/j.neunet.2024.106153

IF: 7.8

2024-02-03

Neural Networks

Abstract:Human motion prediction is the key technology for many real-life applications, e.g., self-driving and human–robot interaction. The recent approaches adopt the unrestricted full-connection graph representation to capture the relationships inside the human skeleton. However, there are two issues to be solved: (i) these unrestricted full-connection graph representation methods neglect the inherent dependencies across the joints of the human body; (ii) these methods represent human motions using the features extracted from a single level and thus can neither fully exploit the various connection relationships among the human body nor guarantee the human motion prediction results to be reasonable. To tackle the above issues, we propose an adaptive multi-level hypergraph convolution network (AMHGCN), which uses the adaptive multi-level hypergraph representation to capture various dependencies among the human body. Our method has four different levels of hypergraph representations, including (i) the joint-level hypergraph representation to capture inherent kinetic dependencies in the human body, (ii) the part-level hypergraph representation to exploit the kinetic characteristics at a higher level (in comparison to the joint-level) by viewing some part of the human body as an entirety, (iii) the component-level hypergraph representation to model the semantic information, and (iv) the global-level hypergraph representation to extract long-distance dependencies in the human body. In addition, to take full advantage of the knowledge carried in the training data, we propose a reverse loss ( i.e., adopting the future human poses to predict the historical poses reversely) to realize data augmentation. Extensive experiments show that our proposed AMHGCN can achieve state-of-the-art performance on three benchmarks, i.e., Human3.6M, CMU-Mocap, and 3DPW.

computer science, artificial intelligence,neurosciences
Adversarial Refinement Network for Human Motion Prediction.

Xianjin Chao,Yanrui Bin,Wenqing Chu,Xuan Cao,Yanhao Ge,Chengjie Wang,Jilin Li,Feiyue Huang,Howard Leung

DOI: https://doi.org/10.1007/978-3-030-69532-3_28

2020-01-01

Abstract: Human motion prediction aims to predict future 3D skeletal sequences by giving a limited human motion as inputs. Two popular methods, recurrent neural networks and feed-forward deep networks, are able to predict rough motion trend, but motion details such as limb movement may be lost. To predict more accurate future human motion, we propose an Adversarial Refinement Network (ARNet) following a simple yet effective coarse-to-fine mechanism with novel adversarial error augmentation. Specifically, we take both the historical motion sequences and coarse prediction as input of our cascaded refinement network to predict refined human motion and strengthen the refinement network with adversarial error augmentation. During training, we deliberately introduce the error distribution by learning through the adversarial mechanism among different subjects. In testing, our cascaded refinement network alleviates the prediction error from the coarse predictor resulting in a finer prediction robustly. This adversarial error augmentation provides rich error cases as input to our refinement network, leading to better generalization performance on the testing dataset. We conduct extensive experiments on three standard benchmark datasets and show that our proposed ARNet outperforms other state-of-the-art methods, especially on challenging aperiodic actions in both short-term and long-term predictions.
DMS-GCN: Dynamic Mutiscale Spatiotemporal Graph Convolutional Networks for Human Motion Prediction

Zigeng Yan,Di-Hua Zhai,Yuanqing Xia

DOI: https://doi.org/10.48550/arXiv.2112.10365

2021-12-20

Computer Vision and Pattern Recognition

Abstract:Human motion prediction is an important and challenging task in many computer vision application domains. Recent work concentrates on utilizing the timing processing ability of recurrent neural networks (RNNs) to achieve smooth and reliable results in short-term prediction. However, as evidenced by previous work, RNNs suffer from errors accumulation, leading to unreliable results. In this paper, we propose a simple feed-forward deep neural network for motion prediction, which takes into account temporal smoothness and spatial dependencies between human body joints. We design a Multi-scale Spatio-temporal graph convolutional networks (GCNs) to implicitly establish the Spatio-temporal dependence in the process of human movement, where different scales fused dynamically during training. The entire model is suitable for all actions and follows a framework of encoder-decoder. The encoder consists of temporal GCNs to capture motion features between frames and semi-autonomous learned spatial GCNs to extract spatial structure among joint trajectories. The decoder uses temporal convolution networks (TCNs) to maintain its extensive ability. Extensive experiments show that our approach outperforms SOTA methods on the datasets of Human3.6M and CMU Mocap while only requiring much lesser parameters. Code will be available at https://github.com/yzg9353/DMSGCN.
Corrigendum: A machine learning model for visualization and dynamic clinical prediction of stroke recurrence in acute ischemic stroke patients: a real-world retrospective study

Kai Wang,Qianqian Shi,Chao Sun,Wencai Liu,Vicky Yau,Chan Xu,Haiyan Liu,Chenyu Sun,Chengliang Yin,Xiu'e Wei,Wenle Li,Liangqun Rong

DOI: https://doi.org/10.3389/fnins.2023.1235340

IF: 4.3

2023-07-10

Frontiers in Neuroscience

Abstract:[This corrects the article DOI: 10.3389/fnins.2023.1130831.].

neurosciences
Learning Constrained Dynamic Correlations in Spatiotemporal Graphs for Motion Prediction

Jiajun Fu,Fuxing Yang,Yonghao Dang,Xiaoli Liu,Jianqin Yin

DOI: https://doi.org/10.1109/tnnls.2023.3277476

IF: 14.255

2023-01-01

IEEE Transactions on Neural Networks and Learning Systems

Abstract:Human motion prediction is challenging due to the complex spatiotemporal feature modeling. Among all methods, graph convolution networks (GCNs) are extensively utilized because of their superiority in explicit connection modeling. Within a GCN, the graph correlation adjacency matrix drives feature aggregation, and thus, is the key to extracting predictive motion features. State-of-the-art methods decompose the spatiotemporal correlation into spatial correlations for each frame and temporal correlations for each joint. Directly parameterizing these correlations introduces redundant parameters to represent common relations shared by all frames and all joints. Besides, the spatiotemporal graph adjacency matrix is the same for different motion samples, and thus, cannot reflect samplewise correspondence variances. To overcome these two bottlenecks, we propose dynamic spatiotemporal decompose GC (DSTD-GC), which only takes 28.6% parameters of the state-of-the-art GC. The key of DSTD-GC is constrained dynamic correlation modeling, which explicitly parameterizes the common static constraints as a spatial/temporal vanilla adjacency matrix shared by all frames/joints and dynamically extracts correspondence variances for each frame/joint with an adjustment modeling function. For each sample, the common constrained adjacency matrices are fixed to represent generic motion patterns, while the extracted variances complete the matrices with specific pattern adjustments. Meanwhile, we mathematically reformulate GCs on spatiotemporal graphs into a unified form and find that DSTD-GC relaxes certain constraints of other GC, which contributes to a better representation capability. Moreover, by combining DSTD-GC with prior knowledge like body connection and temporal context, we propose a powerful spatiotemporal GCN called DSTD-GCN. On the Human3.6M, Carnegie Mellon University (CMU) Mocap, and 3D Poses in the Wild (3DPW) datasets, DSTD-GCN outperforms state-of-the-art methods by 3.9%-8.7% in prediction accuracy with 55.0%-96.9% fewer parameters. Codes are available at https://github.com/Jaakk0F/DSTD-GCN.

computer science, artificial intelligence, theory & methods,engineering, electrical & electronic, hardware & architecture
Investigating Pose Representations and Motion Contexts Modeling for 3D Motion Prediction

Zhenguang Liu,Shuang Wu,Shuyuan Jin,Qi Liu,Shouling Ji,Shijian Lu,Li Cheng

DOI: https://doi.org/10.1109/tpami.2021.3139918

IF: 23.6

2022-01-01

IEEE Transactions on Pattern Analysis and Machine Intelligence

Abstract:Predicting human motion from historical pose sequence is crucial for a machine to succeed in intelligent interactions with humans. One aspect that has been obviated so far, is the fact that how we represent the skeletal pose has a critical impact on the prediction results. Yet there is no effort that investigates across different pose representation schemes. We conduct an indepth study on various pose representations with a focus on their effects on the motion prediction task. Moreover, recent approaches build upon off-the-shelf RNN units for motion prediction. These approaches process input pose sequence sequentially and inherently have difficulties in capturing long-term dependencies. In this paper, we propose a novel RNN architecture termed AHMR (Attentive Hierarchical Motion Recurrent network) for motion prediction which simultaneously models local motion contexts and a global context. We further explore a geodesic loss and a forward kinematics loss for the motion prediction task, which have more geometric significance than the widely employed L2 loss. Interestingly, we applied our method to a range of articulate objects including human, fish, and mouse. Empirical results show that our approach outperforms the state-of-the-art methods in short-term prediction and achieves much enhanced long-term prediction proficiency, such as retaining natural human-like motions over 50 seconds predictions. Our codes are released.

computer science, artificial intelligence,engineering, electrical & electronic
Corrigendum: SCTN: event-based object tracking with energy-efficient deep convolutional spiking neural networks

Mingcheng Ji,Ziling Wang,Rui Yan,Qingjie Liu,Shu Xu,Huajin Tang

DOI: https://doi.org/10.3389/fnins.2023.1204334

IF: 4.3

2023-05-16

Frontiers in Neuroscience

Abstract:[This corrects the article DOI: 10.3389/fnins.2023.1123698.].

neurosciences
Corrigendum: Prediction of EGFR Mutation Status Based on 18F-FDG PET/CT Imaging Using Deep Learning-Based Model in Lung Adenocarcinoma

Guotao Yin,Ziyang Wang,Yingchao Song,Xiaofeng Li,Yiwen Chen,Lei Zhu,Qian Su,Dong Dai,Wengui Xu

DOI: https://doi.org/10.3389/fonc.2021.747316

IF: 4.7

2021-09-07

Frontiers in Oncology

Abstract:[This corrects the article DOI: 10.3389/fonc.2021.709137.].

oncology
April-GCN: Adjacency Position-velocity Relationship Interaction Learning GCN for Human motion prediction

Baoxuan Gu,Jin Tang,Rui Ding,Xiaoli Liu,Jianqin Yin,Zhicheng Zhang

DOI: https://doi.org/10.1016/j.knosys.2024.111613

IF: 8.139

2024-03-14

Knowledge-Based Systems

Abstract:The Human Motion Prediction (HMP) task attempts to model human kinematics, which requires considering both the physical connections between joints and the continuity of the joints' trajectories. To handle this complex task, some recent works have employed Graph Convolutional Networks (GCN) in learning dynamic relations among joints. However, HMP task is essentially a time series task, to model the temporal and spatial features of motion, some works extended the common adjacency matrix formulation to create a large-scale mixed adjacency matrix which introduces redundant connection relationships in single-path GCN. Other works proposed dual-path GCNs and multi-path GCNs to make feature representations more informative, while ignoring the significant differences of features. To tackle these issues, we propose a novel adjacency matrix interaction learning framework called Adjacency Position-velocity Relationship Interaction Learning GCN (April-GCN) which alleviates the differences between dual-path features and improves the feature fusion results. The proposed network utilizes a dual-path GCN structure to exploit the adjacency relationship between physical connections and movement amplitude by interactively learning the GCN adjacency matrices of the two paths. The network excavated closely aligned position-velocity correlations to obtain a more comprehensive and richer representation of joint connections. On three datasets, H3.6M, CMU, and 3DPW, it achieves state-of-the-art performance for both short-term and long-term predictions.

computer science, artificial intelligence
EAN: Error Attenuation Network for Long-term Human Motion Prediction

Jie Xu,Xuguang Lan,Jin Li,Xingyu Chen,Nanning Zheng

DOI: https://doi.org/10.1109/cchi.2019.8901951

2019-01-01

Abstract:Human motion prediction is an important problem in human-robot interaction, computer graphics, and autonomous driving. Currently, there are still some difficulties. Firstly, the error of reasoning about temporal relations accumulates over time. Secondly, the local optimal caused by unbalanced data. For example, leg walking is more common than knee bending or standing. Thirdly, the problem of mean pose is easy to occur in long sequence prediction. In this work, we propose a novel prediction method named Error Attenuation Network (EAN) by taking the Recursive Attenuation Mechanism into consideration with attention model. Firstly, an error attenuation wrapper for optimal function is introduced to alleviate the effect of error accumulation and mean pose. Secondly, the attention model is introduced to restrain incidental actions to balance motions, which aims to mitigate the impact of unbalanced data and mean pose. Experimental results demonstrate that our method predicts the future human motion more accurately, which outperforms the related state-of-the-art approaches on long-term prediction in most cases while having a comparable performance on short-term prediction.
Towards Realistic 3D Human Motion Prediction with A Spatio-temporal Cross-transformer Approach

Hua Yu,Xuanzhe Fan,Yaqing Hou,Wenbin Pei,Hongwei Ge,Xin Yang,Dongsheng Zhou,Qiang Zhang,Mengjie Zhang

DOI: https://doi.org/10.1109/tcsvt.2023.3255186

IF: 5.859

2023-01-01

IEEE Transactions on Circuits and Systems for Video Technology

Abstract:Human motion prediction intends to predict how humans move given a historical sequence of 3D human motions. Recent transformer-based methods have attracted increasing attentions and demonstrated their promising performance in 3D human motion prediction. However, existing methods generally decompose the input of human motion information into spatial and temporal branches in a separate way and seldom consider their inherent coherence between the two branches, hence often failing to register the dynamic spatio-temporal information during the training process. Motivated by these issues, we propose a spatio-temporal cross-transformer network (STCT) for 3D human motion predictions. Specifically, we investigate various types of interaction methods (i.e., Concatenation Interaction, Msg token interaction, and Cross-transformer) to capture the coherence of the spatial and temporal branches. According to the obtained results, the proposed cross-transformer interaction method shows its superiority over other methods. Meanwhile, considering that most existing works treat the human body as a set of 3D human joint positions, the predicted human joints are proportionally less appropriate to the realistic human body due to unreasonable bone length and non-plausible poses as time progresses. We further resort to the bone constraints of human mesh to produce more realistic human motions. By fitting a parametric body model (i.e., SMPL-X model) to the predicted human joints, a reconstruction loss function is proposed to remedy the unreasonable bone length and pose errors. Comprehensive experiments on AMASS and Human3.6M datasets have demonstrated that our method achieves superior performance over compared methods.

engineering, electrical & electronic

Corrigendum: An initial prediction and fine-tuning model based on improving GCN for 3D human motion prediction

An initial prediction and fine-tuning model based on improving GCN for 3D human motion prediction

Forecasting Distillation: Enhancing 3D Human Motion Prediction with Guidance Regularization

The Correction Of The Distortion Of Human Face Based On Three-Dimensional Modeling Methods

Corrigendum: A progression analysis of motor features in Parkinson's disease based on the mapper algorithm

Corrigendum: Validation of the GALAD model and establishment of a new model for HCC detection in Chinese patients

Correction to: Predicting protein inter-residue contacts using composite likelihood maximization and deep learning

Corrigendum: A novel approach for sports injury risk prediction: based on time-series image encoding and deep learning

Gradient multi-foci networks for 3D skeleton-based human motion prediction

Prediction of Human Motion with Motion Optimization and Neural Networks

AMHGCN: Adaptive multi-level hypergraph convolution network for human motion prediction

Adversarial Refinement Network for Human Motion Prediction.

DMS-GCN: Dynamic Mutiscale Spatiotemporal Graph Convolutional Networks for Human Motion Prediction

Corrigendum: A machine learning model for visualization and dynamic clinical prediction of stroke recurrence in acute ischemic stroke patients: a real-world retrospective study

Learning Constrained Dynamic Correlations in Spatiotemporal Graphs for Motion Prediction

Investigating Pose Representations and Motion Contexts Modeling for 3D Motion Prediction

Corrigendum: SCTN: event-based object tracking with energy-efficient deep convolutional spiking neural networks

Corrigendum: Prediction of EGFR Mutation Status Based on 18F-FDG PET/CT Imaging Using Deep Learning-Based Model in Lung Adenocarcinoma

April-GCN: Adjacency Position-velocity Relationship Interaction Learning GCN for Human motion prediction

EAN: Error Attenuation Network for Long-term Human Motion Prediction

Towards Realistic 3D Human Motion Prediction with A Spatio-temporal Cross-transformer Approach