[ECCV'24] ProGraph：Temporal-alignable Probability Guided Graph Topological Modeling for 3D Human Reconstruction
Current 3D human motion reconstruction methods from monocular videos rely on features within the current reconstruction window, leading to distortion and deformations in the human structure under local occlusions or blurriness in video frames. To estimate realistic 3D human mesh sequences based on incomplete features, we propose Temporally-alignable Probability Guided Graph Topological Modeling for 3D Human Reconstruction (ProGraph). For missing parts recovery, we exploit the explicit topological-aware probability distribution across the entire motion sequence. To restore the complete human, Graph Topological Modeling (GTM) learns the underlying topological structure, focusing on the relationships inherent in the individual parts. Next, to generate blurred motion parts, Temporal-alignable Probability Distribution (TPDist) utilizes the GTM to predict features based on distribution. This interactive mechanism facilitates motion consistency, allowing the restoration of human parts. Furthermore, Hierarchical Human Loss (HHLoss) constrains the probability distribution errors of inter-frame features during topological structure variation. Our Method achieves superior results than other SOTA methods in addressing occlusions and blurriness on 3DPW.
We provide two ways to install conda environments depending on CUDA versions.
git clone https://github.com/3DHumanRehab/ProGraph.git
cd Prograph
pip install -r requirements.txtWe provide guidelines to download pre-trained models.
Download pre-trained model and put it into the models folder Prograph-checkpoint.zip.
Download all models and put them into the models folder Downloads_folder_models
- Model checkpoints were obtained in Conda Environment (CUDA 11.7)
We provide guidelines to run end-to-end inference on test video.
The following command will run ProGraph on video in the specified --video_file_or_path.
python demo.py --video video/video.mp4We provide guidelines to train and evaluate our model on Human3.6M, 3DPW and FreiHAND.
Ablation in TPDist and HHLoss.
Comparison with Fastmetro, GLoT, and PyMAF.
This repository provides several experimental results:
.gif)
.gif)
.gif)
Comparison of intra-frame prediction results
Comparison of inter-frame prediction results
Comparison with 4DHumans.
The reconstruction results and probability distribution of human body parts during the prediction process.
Our repository is modified and adapted from these amazing repositories. If you find their work useful for your research, please also consider citing them: