DHand: Hand Pose Estimation From Single Depth Images

Members:   Chi Xu ,   Ashwin Nanjappa ,  Xiaowei Zhang,   Li Cheng (Project Leader)

BII, A*STAR, Singapore

  • Introduction

We tackle the problem of hand pose estimation from single depth images. A dedicated two-step regression forest pipeline is proposed: Given an input hand depth image, step one involves mainly estimation of 3D location and in-plane rotation of the hand using a pixel-wise regression forest. This is utilized in step two which delivers final hand estimation by a similar regression forest model based on the entire hand image patch. Our estimation is driven by internally running a 3D hand kinematic chain model. For an unseen test image, instead of fixing its kinematic model parameters to training examples, we propose to estimate them by a dynamically weighted scheme, which empirically leads to significant error reductions. Our approach consumes only around 288MB main memory, and works at 15.6 frame-per-second (FPS) on an average laptop when implemented in CPU. This is further sped-up to 67.2FPS running on consumer graphics cards or GPU, as the bottleneck object matching component is identified and specifically accelerated using GPU. In addition to empirical studies, the consistency property of our approach is also theoretically analyzed.



  • Videos

  • Our Dataset


• Our Data-glove Annotated Hand-Depth Image Dataset and its Online Performance Evaluation: http://hpes.bii.a-star.edu.sg/

  • Related Projects

Lie-X: Depth Image Based Articulated Object Pose Estimation, Tracking, and Action Recognition on Lie Groups

  • Publications

[1] Chi Xu, Li Cheng. Efficient hand pose estimation from a single depth image. In International Conference on Computer Vision (ICCV), 2013. [ bib ]  [ pdf ]
[2] Chi Xu, Ashwin Nanjappa, Xiaowei Zhang, Li Cheng. Estimate Hand Poses Efficiently from Single Depth Images. In In International Journal of Computer Vision (IJCV), 2015. [ bib ]  [ pdf ]
[3] Chi Xu, Lakshmi Narasimhan Govindarajan, Yu Zhang, Li Cheng. Lie-X: Depth Image Based Articulated Object Pose Estimation, Tracking, and Action Recognition on Lie Groups. In In International Journal of Computer Vision (IJCV), in press, 2016. [ pdf ]