A Scan-Line Forest Growing-Based Hand Segmentation Framework With Multipriority Vertex Stereo Matching for Wearable Devices

Yiyi Ren,Xiang Xie,Guolin Li,Zhihua Wang

DOI: https://doi.org/10.1109/TCYB.2016.2646701

Abstract:A hand segmentation framework is proposed for 3-D hand gesture interaction for wearable devices. In this framework, all the objects in a scene are regarded as directed trees in a forest, and the problem of the hand segmentation can be converted into finding the target tree (called hand tree) in the forest with proper hand properties including color consistency, space consistency, disparity, and hand shape constraints. The forest grows scan-line by scan-line from high reliable regions to low reliable regions. First, the vertices in each tree are generated through boundary detection. To get rid of the interference of skin-colored objects, an under-segment vertex splitting step is added in. Second, a scan-line-based forest growing method is proposed by constructing the maximum spanning forest (MSF) with the edges weighted by the consistencies of color and space. Meanwhile, the high-order history information of the tree construction propagates along each growing tree to the current scan-line to predict the 3-D location of the growing hand tree, so as to achieve semiglobal optimization. At last, the growing hand tree is trimmed and labeled with the disparity and hand shape constraints to correct the misconnection in the MSF. The vertex's disparity is obtained by our multipriority vertex stereo matching algorithm using the 3-D location prediction to overcome the problem of occlusion and high similarity between fingers. Experimental results demonstrate that by using our method, the hand can be well segmented. The score and accuracy of the hand segmentation result is about 96.3% and 92.9%, respectively.