HybrIK

This repo contains the code of our paper:

HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation

Jiefeng Li, Chao Xu, Zhicun Chen, Siyuan Bian, Lixin Yang, Cewu Lu

[Paper] [Supplementary Material] [arXiv] [Project Page]

In CVPR 2021

Twist-and-Swing Decomposition

TODO

• Provide pretrained model
• Provide parsed data annotations

Installation instructions

# 1. Create a conda virtual environment.
conda create -n hybrik python=3.6 -y
conda activate hybrik

# 2. Install PyTorch
conda install pytorch==1.2.0 torchvision==0.4.0

# 3. Pull our code
git clone https://github.com/Jeff-sjtu/HybrIK.git
cd HybrIK

# 4. Install
python setup.py develop

Download models

• Download the SMPL model basicModel_neutral_lbs_10_207_0_v1.0.0.pkl from here at common/utils/smplpytorch/smplpytorch/native/models.
• Download our pretrained model from [ Google Drive | Baidu (code: qre2) ].

Fetch data

Download Human3.6M, MPI-INF-3DHP, 3DPW and MSCOCO datasets. You need to follow directory structure of the data as below. Thanks to the great job done by Moon et al., we use the Human3.6M images provided in PoseNet.

|-- data
`-- |-- h36m
    `-- |-- annotations
        `-- images
`-- |-- pw3d
    `-- |-- json
        `-- imageFiles
`-- |-- 3dhp
    `-- |-- annotation_mpi_inf_3dhp_train.json
        |-- annotation_mpi_inf_3dhp_test.json
        |-- mpi_inf_3dhp_train_set
        `-- mpi_inf_3dhp_test_set
`-- |-- coco
    `-- |-- annotations
        |   |-- person_keypoints_train2017.json
        |   `-- person_keypoints_val2017.json
        |-- train2017
        `-- val2017

• Download Human3.6M parsed annotations. [ Google | Baidu ]
• Download 3DPW parsed annotations. [ Google | Baidu ]
• Download MPI-INF-3DHP parsed annotations. [ Google | Baidu ]

Train from scratch

./scripts/train_smpl.sh train_res34 ./configs/256x192_adam_lr1e-3-res34_smpl_3d_base_2x_mix.yaml

Evaluation

Download the pretrained model [Google Drive].

./scripts/validate_smpl.sh ./configs/256x192_adam_lr1e-3-res34_smpl_24_3d_base_2x_mix.yaml ./pretrained_res34.pth

Citing

If our code helps your research, please consider citing the following paper:

@inproceedings{li2020hybrik,
    title={HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation},
    author={Li, Jiefeng and Xu, Chao and Chen, Zhicun and Bian, Siyuan and Yang, Lixin and Lu, Cewu},
    booktitle={CVPR},
    year={2021}
}