Skip to content

Commit f9cb753

Browse files
vchoutasvchoutas
committed
Merge branch 'model_transfer' into master
2 parents 1fc470c + b14a52d commit f9cb753

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

42 files changed

+2449
-0
lines changed

‎README.md

+4
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,10 @@ SMPL-X is defined by a function M(θ, β, ψ), where θ is the pose parameters,
4545

4646
## News
4747

48+
- 3 November 2020: We release the code to transfer between the models in the
49+
SMPL family. For more details on the code, go to this [readme
50+
file](./transfer_model/README.md). A detailed explanation on how the mappings
51+
were extracted can be found [here](./transfer_model/docs/transfer.md).
4852
- 23 September 2020: A UV map is now available for SMPL-X, please check the
4953
Downloads section of the website.
5054
- 20 August 2020: The full shape and expression space of SMPL-X are now available.

‎config_files/smpl2smplh.yaml

+25
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
datasets:
2+
mesh_folder:
3+
data_folder: 'transfer_data/meshes/smpl'
4+
deformation_transfer_path: 'transfer_data/smpl2smplh_def_transfer.pkl'
5+
mask_ids_fname: ''
6+
summary_steps: 100
7+
8+
edge_fitting:
9+
per_part: False
10+
11+
optim:
12+
type: 'trust-ncg'
13+
maxiters: 100
14+
gtol: 1e-06
15+
16+
body_model:
17+
model_type: "smplh"
18+
# SMPL+H has no neutral model, so we have to manually select the gender
19+
gender: "female"
20+
# gender: "male"
21+
folder: "transfer_data/body_models"
22+
use_compressed: False
23+
smplh:
24+
betas:
25+
num: 10

‎config_files/smpl2smplx.yaml

+26
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
datasets:
2+
mesh_folder:
3+
data_folder: 'transfer_data/meshes/smpl'
4+
deformation_transfer_path: 'transfer_data/smpl2smplx_deftrafo_setup.pkl'
5+
mask_ids_fname: 'smplx_mask_ids.npy'
6+
summary_steps: 100
7+
8+
edge_fitting:
9+
per_part: False
10+
11+
optim:
12+
type: 'trust-ncg'
13+
maxiters: 100
14+
gtol: 1e-06
15+
16+
body_model:
17+
model_type: "smplx"
18+
gender: "neutral"
19+
folder: "transfer_data/body_models"
20+
use_compressed: False
21+
use_face_contour: True
22+
smplx:
23+
betas:
24+
num: 10
25+
expression:
26+
num: 10

‎config_files/smplh2smpl.yaml

+24
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
datasets:
2+
mesh_folder:
3+
data_folder: 'transfer_data/meshes/smplh'
4+
deformation_transfer_path: 'transfer_data/smplh2smpl_def_transfer.pkl'
5+
mask_ids_fname: ''
6+
summary_steps: 100
7+
8+
edge_fitting:
9+
per_part: False
10+
11+
optim:
12+
type: 'trust-ncg'
13+
maxiters: 100
14+
gtol: 1e-06
15+
16+
body_model:
17+
model_type: "smpl"
18+
gender: "neutral"
19+
folder: "transfer_data/body_models"
20+
use_compressed: False
21+
use_face_contour: True
22+
smpl:
23+
betas:
24+
num: 10

‎config_files/smplh2smplx.yaml

+26
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
datasets:
2+
mesh_folder:
3+
data_folder: 'transfer_data/meshes/smplh'
4+
deformation_transfer_path: 'transfer_data/smplh2smplx_deftrafo_setup.pkl'
5+
mask_ids_fname: 'smplx_mask_ids.npy'
6+
summary_steps: 100
7+
8+
edge_fitting:
9+
per_part: False
10+
11+
optim:
12+
type: 'trust-ncg'
13+
maxiters: 100
14+
gtol: 1e-06
15+
16+
body_model:
17+
model_type: "smplx"
18+
gender: "neutral"
19+
folder: "transfer_data/body_models"
20+
use_compressed: False
21+
use_face_contour: True
22+
smplx:
23+
betas:
24+
num: 10
25+
expression:
26+
num: 10

‎config_files/smplx2smpl.yaml

+25
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
datasets:
2+
mesh_folder:
3+
data_folder: 'meshes/smplx'
4+
deformation_transfer_path: 'transfer_data/smplx2smpl_deftrafo_setup.pkl'
5+
mask_ids_fname: ''
6+
summary_steps: 100
7+
8+
edge_fitting:
9+
per_part: False
10+
11+
optim:
12+
type: 'lbfgs'
13+
maxiters: 200
14+
gtol: 1e-06
15+
16+
body_model:
17+
model_type: "smpl"
18+
gender: "neutral"
19+
ext: 'pkl'
20+
folder: "transfer_data/body_models"
21+
use_compressed: False
22+
use_face_contour: True
23+
smpl:
24+
betas:
25+
num: 10

‎config_files/smplx2smplh.yaml

+27
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
datasets:
2+
mesh_folder:
3+
data_folder: 'meshes/smplx'
4+
deformation_transfer_path: 'transfer_data/smplx2smplh_deftrafo_setup.pkl'
5+
mask_ids_fname: ''
6+
summary_steps: 100
7+
8+
edge_fitting:
9+
per_part: False
10+
11+
optim:
12+
type: 'lbfgs'
13+
maxiters: 200
14+
gtol: 1e-06
15+
16+
body_model:
17+
model_type: "smplh"
18+
# SMPL+H has no neutral model, so we have to manually select the gender
19+
gender: "female"
20+
# gender: "male"
21+
ext: 'pkl'
22+
folder: "transfer_data/body_models"
23+
use_compressed: False
24+
use_face_contour: True
25+
smplh:
26+
betas:
27+
num: 10

‎transfer_model/README.md

+211
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,211 @@
1+
# Model parameter transfer
2+
3+
## Table of Contents
4+
* [License](#license)
5+
* [Description](#description)
6+
* [Using the code](#using-the-code)
7+
* [Data](#data)
8+
* [SMPL to SMPL-X](#smpl-to-smpl-x)
9+
* [SMPL-X to SMPL](#smpl-x-to-smpl)
10+
* [SMPL+H to SMPL](#smpl%2Bh-to-smpl)
11+
* [SMPL to SMPL+H](#smpl-to-smpl%2Bh)
12+
* [SMPL+H to SMPL-X](#smpl%2Bh-to-smpl-x)
13+
* [SMPL-X to SMPL+H](#smpl-x-to-smpl%2Bh)
14+
* [Visualize correspondences](visualize-correspondences)
15+
* [Citation](#citation)
16+
* [Acknowledgments](#acknowledgments)
17+
* [Contact](#contact)
18+
19+
## License
20+
21+
Software Copyright License for **non-commercial scientific research purposes**.
22+
Please read carefully the [terms and conditions](https://github.com/vchoutas/smplx/blob/master/LICENSE) and any accompanying documentation before you download and/or use the SMPL-X/SMPLify-X model, data and software, (the "Model & Software"), including 3D meshes, blend weights, blend shapes, textures, software, scripts, and animations. By downloading and/or using the Model & Software (including downloading, cloning, installing, and any other use of this github repository), you acknowledge that you have read these terms and conditions, understand them, and agree to be bound by them. If you do not agree with these terms and conditions, you must not download and/or use the Model & Software. Any infringement of the terms of this agreement will automatically terminate your rights under this [License](./LICENSE).
23+
24+
## Description
25+
26+
The repository contains code for converting model parameters of one model to
27+
another. **Never** copy parameters between the models. You will not get the
28+
same poses. SMPL, SMPL+H and SMPL-X shape spaces are **NOT** compatible, since
29+
each model is the result of a different training process.
30+
A more detailed explanation on how we extract correspondences
31+
between the models and the loss function used to estimate the parameters can be
32+
found [here](./docs/transfer.md).
33+
34+
## Requirements
35+
36+
1. Start by cloning the SMPL-X repo:
37+
```Shell
38+
git clone https://github.com/vchoutas/smplx.git
39+
```
40+
2. Run the following command to install all necessary requirements
41+
```Shell
42+
pip install -r requirements.txt
43+
```
44+
3. Install the Torch Trust Region optimizer by following the instructions [here](https://github.com/vchoutas/torch-trust-ncg).
45+
46+
47+
## Using the code
48+
49+
### Data
50+
51+
Register on the [SMPL-X website](http://smpl-x.is.tue.mpg.de/), go to the
52+
downloads section to get the correspondences and sample data,
53+
by clicking on the *Model correspondences* button.
54+
Create a folder
55+
named `transfer_data` and extract the downloaded zip there. You should have the
56+
following folder structure now:
57+
58+
```bash
59+
transfer_data
60+
├── meshes
61+
│   ├── smpl
62+
│   ├── smplx
63+
├── smpl2smplh_def_transfer.pkl
64+
├── smpl2smplx_deftrafo_setup.pkl
65+
├── smplh2smpl_def_transfer.pkl
66+
├── smplh2smplx_deftrafo_setup.pkl
67+
├── smplx2smpl_deftrafo_setup.pkl
68+
├── smplx2smplh_deftrafo_setup.pkl
69+
├── smplx_mask_ids.npy
70+
```
71+
72+
### SMPL to SMPL-X
73+
74+
To run the code to convert SMPL meshes to SMPL-X parameters use the following command:
75+
```Shell
76+
python main.py --exp-cfg config_files/smpl2smplx.yaml
77+
```
78+
79+
The file *smpl2smplx.yaml* contains a sample configuration that reads meshes from a folder,
80+
processes them and returns pkl files with SMPL-X parameters. To run on your own data create a folder
81+
with SMPL meshes, in either ply or obj format, change the path in the config file and run the code.
82+
83+
### SMPL-X to SMPL
84+
85+
To run the code to convert SMPL-X meshes to SMPL parameters use the following command:
86+
```Shell
87+
python main.py --exp-cfg config_files/smplx2smpl.yaml
88+
```
89+
90+
The file *smplx2smpl.yaml* contains a sample configuration that reads meshes from a folder,
91+
processes them and returns pkl files with SMPL parameters. To run on your own data create a folder
92+
with SMPL-X meshes, in either ply or obj format, change the path in the config file and run the code.
93+
When creating the SMPL-X meshes, do not use the hand and face parameters.
94+
Naturally, you will lose all hand and face information if you choose this, since
95+
SMPL cannot model them.
96+
97+
98+
### SMPL+H to SMPL
99+
100+
To run the code to convert SMPL+H meshes to SMPL parameters use the following command:
101+
```Shell
102+
python main.py --exp-cfg config_files/smplh2smpl.yaml
103+
```
104+
105+
The file *smplh2smpl.yaml* contains a sample configuration that reads meshes from a folder,
106+
processes them and returns pkl files with SMPL parameters. To run on your own data create a folder
107+
with SMPL+H meshes, in either ply or obj format, change the path in the config file and run the code.
108+
Note that using this direction means that you will lose information on the
109+
hands.
110+
111+
112+
### SMPL to SMPL+H
113+
114+
To run the code to convert SMPL meshes to SMPL+H parameters use the following command:
115+
```Shell
116+
python main.py --exp-cfg config_files/smpl2smplh.yaml
117+
```
118+
119+
The file *smpl2smplh.yaml* contains a sample configuration that reads meshes from a folder,
120+
processes them and returns pkl files with SMPL parameters. To run on your own data create a folder
121+
with SMPL meshes, in either ply or obj format, change the path in the config file and run the code.
122+
123+
### SMPL+H to SMPL-X
124+
125+
To run the code to convert SMPL+H meshes to SMPL-X parameters use the following command:
126+
```Shell
127+
python main.py --exp-cfg config_files/smplh2smplx.yaml
128+
```
129+
130+
The file *smplh2smplx.yaml* contains a sample configuration that reads meshes from a folder,
131+
processes them and returns pkl files with SMPL-X parameters. To run on your own data create a folder
132+
with SMPL+H meshes, in either ply or obj format, change the path in the config file and run the code.
133+
134+
135+
### SMPL-X to SMPL+H
136+
137+
To run the code to convert SMPL-X meshes to SMPL+H parameters use the following command:
138+
```Shell
139+
python main.py --exp-cfg config_files/smplx2smplh.yaml
140+
```
141+
142+
The file *smplx2smpl.yaml* contains a sample configuration that reads meshes from a folder,
143+
processes them and returns pkl files with SMPL+H parameters. To run on your own data create a folder
144+
with SMPL-X meshes, in either ply or obj format, change the path in the config file and run the code.
145+
Make sure that you do not use the jaw pose and expression parameters to generate
146+
the meshes.
147+
148+
149+
## Visualize correspondences
150+
151+
To visualize correspondences:
152+
```Shell
153+
python vis_correspondences.py --exp-cfg configs/smpl2smplx.yaml --exp-opts colors_path PATH_TO_SMPL_COLORS
154+
```
155+
You should then see the following image. Points with similar color are in
156+
correspondence.
157+
![Correspondence example](./images/smpl_smplx_correspondence.png)
158+
159+
## Citation
160+
161+
Depending on which model is loaded for your project, i.e. SMPL-X or SMPL+H or SMPL, please cite the most relevant work:
162+
163+
```
164+
@article{SMPL:2015,
165+
author = {Loper, Matthew and Mahmood, Naureen and Romero, Javier and Pons-Moll, Gerard and Black, Michael J.},
166+
title = {{SMPL}: A Skinned Multi-Person Linear Model},
167+
journal = {ACM Transactions on Graphics, (Proc. SIGGRAPH Asia)},
168+
month = oct,
169+
number = {6},
170+
pages = {248:1--248:16},
171+
publisher = {ACM},
172+
volume = {34},
173+
year = {2015}
174+
}
175+
```
176+
177+
```
178+
@article{MANO:SIGGRAPHASIA:2017,
179+
title = {Embodied Hands: Modeling and Capturing Hands and Bodies Together},
180+
author = {Romero, Javier and Tzionas, Dimitrios and Black, Michael J.},
181+
journal = {ACM Transactions on Graphics, (Proc. SIGGRAPH Asia)},
182+
volume = {36},
183+
number = {6},
184+
pages = {245:1--245:17},
185+
series = {245:1--245:17},
186+
publisher = {ACM},
187+
month = nov,
188+
year = {2017},
189+
url = {http://doi.acm.org/10.1145/3130800.3130883},
190+
month_numeric = {11}
191+
}
192+
```
193+
194+
195+
```
196+
@inproceedings{SMPL-X:2019,
197+
title = {Expressive Body Capture: 3D Hands, Face, and Body from a Single Image},
198+
author = {Pavlakos, Georgios and Choutas, Vasileios and Ghorbani, Nima and Bolkart, Timo and Osman, Ahmed A. A. and Tzionas, Dimitrios and Black, Michael J.},
199+
booktitle = {Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)},
200+
year = {2019}
201+
}
202+
```
203+
204+
205+
## Acknowledgments
206+
The code of this repository was implemented by [Vassilis Choutas]([email protected]),
207+
based on a Chumpy implementation from [Timo Bolkart]([email protected]).
208+
209+
## Contact
210+
211+
For questions, please contact [[email protected]]([email protected]).

‎transfer_model/__init__.py

+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# -*- coding: utf-8 -*-
2+
3+
# Max-Planck-Gesellschaft zur Förderung der Wissenschaften e.V. (MPG) is
4+
# holder of all proprietary rights on this computer program.
5+
# You can only use this computer program if you have closed
6+
# a license agreement with MPG or you get the right to use the computer
7+
# program from someone who is authorized to grant you that right.
8+
# Any use of the computer program without a valid license is prohibited and
9+
# liable to prosecution.
10+
#
11+
# Copyright©2020 Max-Planck-Gesellschaft zur Förderung
12+
# der Wissenschaften e.V. (MPG). acting on behalf of its Max Planck Institute
13+
# for Intelligent Systems. All rights reserved.
14+
#
15+
# Contact: Vassilis Choutas, [email protected]
16+

0 commit comments

Comments
 (0)