Abstract:The identification of key points in the human body is vital for sports rehabilitation, medical diagnosis, human–computer interaction, and related fields. Currently, depth cameras provide more precise depth information on these crucial points. However, human motion can lead to variations in the positions of these key points. While the Mediapipe algorithm demonstrates effective anti-shake capabilities for these points, its accuracy can be easily affected by changes in lighting conditions. To address these challenges, this study proposes an illumination-adaptive algorithm for detecting human key points through the fusion of multi-source information. By integrating key point data from the depth camera and Mediapipe, an illumination change model is established to simulate environmental lighting variations. Subsequently, the fitting function of the relationship between lighting conditions and adaptive weights is solved to achieve lighting adaptation for human key point detection. Experimental verification and similarity analysis with benchmark data yielded R2 results of 0.96 and 0.93, and cosine similarity results of 0.92 and 0.90. With a threshold range of 8, the joint accuracy rates for the two rehabilitation actions were found to be 89% and 88%. The experimental results demonstrate the stability of the proposed method in detecting key points in the human body under changing illumination conditions, its anti-shake ability for human movement, and its high detection accuracy. This method shows promise for applications in human–computer interaction, sports rehabilitation, and virtual reality.

What problem does this paper attempt to address?

The paper primarily focuses on addressing the accuracy and stability issues of human keypoint detection under varying lighting conditions, especially in applications such as sports rehabilitation and human-computer interaction. Specifically, the paper addresses the following core issues: 1. **Decrease in keypoint detection accuracy due to lighting changes**: When environmental lighting conditions change, such as when the light dims or brightens, the performance of traditional keypoint detection algorithms (e.g., Mediapipe) is affected, thereby reducing detection accuracy. 2. **Keypoint position jitter caused by rapid movement**: When using a Kinect depth camera for dynamic human keypoint detection, rapid human movements can cause jitter in the detection results, affecting the stability of keypoint positioning. To solve the above problems, the paper proposes a lighting-adaptive human keypoint detection algorithm based on multi-source information fusion. This method combines the advantages of the Kinect depth camera and Mediapipe by constructing a lighting change model to simulate different lighting conditions and using the L-BFGS optimization algorithm to find the fitting function between lighting conditions and weights, achieving adaptation to lighting changes. Additionally, the Kalman filter is used to further smooth the fused data, improving the accuracy of keypoint estimation. ### Main Contributions - Achieved multi-source information fusion, fully utilizing human keypoint data detected from depth images and RGB images, thereby improving the accuracy and stability of keypoint detection. - Applied the L-BFGS optimization algorithm to find the fitting function between lighting conditions and weights, achieving lighting adaptation for human keypoint detection. - Compared and analyzed the effects of the proposed method with Kinect and Mediapipe on two rehabilitation actions: gait balance training and center of gravity transfer training. The results show that the proposed lighting-adaptive human keypoint detection algorithm has higher accuracy and stability. ### Conclusion This study effectively addresses the accuracy and stability issues of human keypoint detection under different lighting conditions by proposing a new multi-source information fusion algorithm, providing strong technical support for fields such as sports rehabilitation and virtual reality.

Light-Adaptive Human Body Key Point Detection Algorithm Based on Multi-Source Information Fusion

Human face detection under weak light based on depth information

Motion Parameters Measurement of User-Defined Key Points Using 3D Pose Estimation

Human Body Recognition And Positioning With Multiple Cameras Based On "Vibration Signals" From Skin Surfaces

A Human Movement Detection Algorithm on Mobile Phone Platform

Parametric Human Body Reconstruction Based on Sparse Key Points.

RGB-D Camera based Human Limb Movement Recognition and Tracking in Supine Positions.

High-precision Human Body Acquisition Via Multi-View Binocular Stereopsis

Towards High Performance Human Keypoint Detection

Multi-LiDAR human joint recognition algorithm in hospital wards based on improved V2V-Posenet

Automatic tracking of moving human body based on remote sensing spatial information

Detection of 3D Human Posture Based on Improved Mediapipe

Single-Stage Pose Estimation and Joint Angle Extraction Method for Moving Human Body

Visualization of movements in sports training based on multimedia information processing technology

Automatic livestock body measurement based on keypoint detection with multiple depth cameras

Cloud Computing Image Processing Application in Athlete Training High-Resolution Image Detection

All Keypoints You Need: Detecting Arbitrary Keypoints on the Body of Triple, High, and Long Jump Athletes

Human Motion Tracking by Multiple RGBD Cameras.

Enhancing 3D Human Skeleton Key-point Detection through Weakly Supervised Learning and Multi-level Attention Mechanisms

A High‐Precision Dynamic Movement Recognition Algorithm Using Multimodal Biological Signals for Human–Machine Interaction

An ultra-fast human detection method for color-depth camera