Skip to content

Latest commit

 

History

History
122 lines (95 loc) · 5.38 KB

README.md

File metadata and controls

122 lines (95 loc) · 5.38 KB

TRACE's code for inference & evaluation

To run the inference & evaluation, please first install simple-romp following the guidance.

Please refer to this instruction for training.

Installation

CAUTIONS: simple-romp for ROMP and BEV has been tested on Linux, Mac, and Windows, but its TRACE version only works in Linu x environment with CUDA support. Please make sure that you have CUDA and corresponding pytorch installed before using simple-romp for TRACE.

  1. Installing simple_romp via pip:
pip install --upgrade setuptools numpy cython lap
#download the package and install it from source:
git clone https://github.com/Arthur151/ROMP
cd simple_romp
python setup_trace.py install
  1. Preparing SMPL model files in our format:

Firstly, please register and download:
a. Meta data from this link. Please unzip it, then we get a folder named "smpl_model_data" b. SMPL model file (SMPL_NEUTRAL.pkl) from "Download version 1.1.0 for Python 2.7 (female/male/neutral, 300 shape PCs)" in official website. Please unzip it and move the SMPL_NEUTRAL.pkl from extracted folder into the "smpl_model_data" folder.
c. (Optional) If you use BEV, please also download SMIL model file (DOWNLOAD SMIL) from official website. Please unzip and put it into the "smpl_model_data" folder, so we have "smpl_model_data/smil/smil_web.pkl".
Then we can get a folder in structure like this:

|-- smpl_model_data
|   |-- SMPL_NEUTRAL.pkl
|   |-- J_regressor_extra.npy
|   |-- J_regressor_h36m.npy
|   |-- smpl_kid_template.npy
|   |-- smil
|   |-- |-- smil_web.pkl

Secondly, please convert the SMPL model files to our format via

# please provide the absolute path of the "smpl_model_data" folder to the source_dir 
romp.prepare_smpl -source_dir=/path/to/smpl_model_data
# (Optional) If you use BEV, please also run:
bev.prepare_smil -source_dir=/path/to/smpl_model_data

The converted file would be save to "~/.romp/" in defualt.

Inference

After installation, please run

# current code does not support inference with multiple GPUs.
CUDA_VISIBLE_DEVICES=0 trace2 -i /path/to/video_sequence --subject_num=N

For example, you can download our demo videos1 and videos2. Please unzip them and specify the path to these folders to run, like:

CUDA_VISIBLE_DEVICES=0 trace2 -i /path/to/trace_demo --subject_num=1
CUDA_VISIBLE_DEVICES=0 trace2 -i /path/to/trace_demo2 --subject_num=2

Optional functions:

# By default,  subject_num is set to 1, which means we only track and recover the 3D person with the largest scale in the first frame. To change it, please specify:
--subject_num=N

# By default, dir to save the results would be ~/TRACE_results. To change it, please specify:
--save_path /path/to/save/folder

# To save the rendering results in camera coordinates, please add:
--save_video

Visualization

To visualize the estimated results, please download the codes and run

cd simple_romp
python -m trace2.show --smpl_model_folder /path/to/smpl_model_data --preds_path /path/to/trace_demo.npz --frame_dir /path/to/trace_demo 
#    --smpl_model_folder   Folder contains SMPL_NEUTRAL.pkl, like /path/to/smpl_model_data
#    --preds_path          Path to save the .npz results, like /path/to/trace_demo.npz
#    --frame_dir           Path to folder of input video frames, like /path/to/trace_demo

TRACE Benchmark Evaluation

The evaluation code of TRACE is integrated into simple-romp, trace2/eval.py, trace2/evaluation folder.

To prepare the evaluation datasets, please refer to trace_dataset.md.

Please set the path to datasets in dataset_dir of simple-romp/trace2/eval.py, and then run:

DynaCam

For details of DynaCam dataset, please refer to [DynaCam Dataset], which directly provides predictions, and code for evaluation and visualization.

You may also evaluate on DynaCam via running

cd simple_romp/trace2
python -m eval --eval_dataset=DynaCam

MuPoTS

Please download the MuPoTS dataset from official website and the packed annotations from here. Then please set the mupots's dataset_dir in simple-romp/trace2/eval.py.

cd simple_romp/trace2
python -m eval --eval_dataset=mupots

Dyna3DPW

Please download the Dyna3DPW dataset from here. Then please set the Dyna3DPW's dataset_dir in simple-romp/trace2/eval.py.

cd simple_romp/trace2
python -m eval --eval_dataset=Dyna3DPW

3DPW

Please download the 3DPW dataset from official website and the packed annotations from here. Then please set the 3DPW's dataset_dir in simple-romp/trace2/eval.py.

cd simple_romp/trace2
python -m eval --eval_dataset=3DPW