Single Image Depth Map Estimation for Improving Posture Recognition
Jiaqing Liu,Seiju Tsujinaga,Shurong Chai,Hao Sun,Tomoko Tateyama,Yutaro Iwamoto,Xinyin Huang,Lanfen Lin,Yen-Wei Chen
DOI: https://doi.org/10.1109/jsen.2021.3122128
IF: 4.3
2021-12-01
IEEE Sensors Journal
Abstract:Image-based posture recognition is a very challenging problem since it is difficult to acquire rich 3D information from the posture in color image. To address this issue, we present a novel and unified framework for human posture recognition, applying single image depth map estimation from color images. The proposed method includes two stages. The first stage estimates the depth map from the single-color image by an improved Pix2Pix generation module. The generation module is equipped with a hybrid loss function that captures the high-level features and recovers the sharp depth discontinuities, thus improving the depth estimation results. The second stage (the recognition stage) improves the color image-based recognition performance by incorporating the estimated depth map. Thereby, a two-stream CNN architecture that separately processes the color image and its estimated depth image is developed for robust posture recognition. To verify its effectiveness, we first test the proposed method on a novel pose dataset, which contains 13800 samples of paired color-and-depth of 6 subjects with 15 poses. The dataset used in this work is been created and released, is available at http://media.ritsumei.ac.jp/iipl/database/pose/. Extensive experiments are also performed on the public OUHANDS hand gesture dataset. Experiments demonstrate that the proposed method achieves superior performance on both human pose and hand gesture recognition tasks.
engineering, electrical & electronic,instruments & instrumentation,physics, applied