Rodent Pose Estimation From Depth Images


Ashwin Nanjappa, Wei Gao, Chi Xu, Adam Claridge-Chang, Zoe Bichler, Li Cheng (project leader),


mouse
An illustration of our 3D rodent pose estimation system when a top-view input depth image is provided. Its estimated pose in top and side views are shown.

Introduction

Our method provides the 3D pose estimation for rodent with the aid of a commodity depth camera mounted on top of the cage. With the addition of a color camera at the bottom of the cage, our method can estimate a full-body 3D pose that includes limbs and paws.

Important features of our method include:

  • Estimation of the 3D joint positions of our mouse model using a discriminatively trained random forest

  • Use of a custom mouse model with 24 joints, that is complex enough to handle important mouse poses

  • Estimation of many important mouse poses, including walking, running and turning

  • Works with any common cage setup used by neuroscientists

  • Shown to work with both structured light (Primensense) and time-of-flight (SoftKinetic) depth cameras

Video


Results

Lab mouse

real results lab mouse
Results of full-body pose estimation of lab mouse (mus musculus) from image pairs captured from Primesense depth and Basler color cameras.

Left column shows pairs of cropped regions from depth and color input images with the detected paws marked in red circles. Middle column shows a side-view 3D rendering of the estimated mouse pose skeleton while right column is the corresponding top view rendering of the pose. Paws are rendered as yellow spheres. Point cloud of depth pixels are rendered in grey and floor is rendered in pink.

Hamster

real results hamster
Results of full-body pose estimation of hamster (cricetidae) from image pairs captured from Primesense depth and Basler color cameras.

Left column shows pairs of cropped regions from depth and color input images with the detected paws marked in red circles. Middle column shows a side-view 3D rendering of the estimated mouse pose skeleton while right column is the corresponding top view rendering of the pose. Paws are rendered as yellow spheres. Point cloud of depth pixels are rendered in grey and floor is rendered in pink.

Publication

Ashwin Nanjappa, Wei Gao, Chi Xu, Adam Claridge-Chang, Zoe Bichler, Li Cheng, Rodent Pose Estimation From Depth Images, in submission, 2017. An earlier version is also available at arXiv

Software

estimator
An illustration of our implementation running live on Ubuntu Linux. Input RGB-depth images are on top and side and top views of estimated 3D pose are shown in the bottom.

Our rodent 3D pose estimation software has been ported to work on Linux and Windows. Executables of our software for Linux and Windows is available below. Please see readme.txt in the download for usage instructions.

Rodent datasets (coming soon)

Dataset of single rodent depth images will be available for download and evaluation.