Skip to content

Commit

Permalink
fix run cmd of musev_referencenet_pose
Browse files Browse the repository at this point in the history
  • Loading branch information
itechmusic committed Mar 30, 2024
1 parent e38b3f8 commit 5252552
Show file tree
Hide file tree
Showing 2 changed files with 8 additions and 4 deletions.
6 changes: 4 additions & 2 deletions README-zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@ Wenjiang Zhou
1. 支持多参考图像技术,包括 `IPAdapter``ReferenceOnly``ReferenceNet``IPAdapterFaceID`
1. 我们后面也会推出训练代码。

# 重要更新
1. `musev_referencenet_pose`: `unet`, `ip_adapter` 的模型名字指定错误,请使用 `musev_referencenet_pose`而不是`musev_referencenet`,请使用最新的main分支。

# 进展
- [2024年3月27日] 发布 `MuseV` 项目和训练好的模型 `musev``muse_referencenet``muse_referencenet_pose`
Expand Down Expand Up @@ -482,9 +484,9 @@ python scripts/inference/video2video.py --sd_model_name majicmixRealv6Fp16 --un

### musev_referencenet_pose
仅用于 `pose2video`
基于 `musev_referencenet`,修复 `referencenet``pose-controlnet``T2I`,训练 `motion` 模块和 `IPAdapter`
基于 `musev_referencenet` 训练,固定 `referencenet``pose-controlnet``T2I`,训练 `motion` 模块和 `IPAdapter`
```bash
python scripts/inference/video2video.py --sd_model_name majicmixRealv6Fp16 --unet_model_name musev_referencenet --referencenet_model_name musev_referencenet --ip_adapter_model_name musev_referencenet -test_data_path ./configs/tasks/example.yaml --vision_clip_extractor_class_name ImageClipVisionFeatureExtractor --vision_clip_model_path ./checkpoints/IP-Adapter/models/image_encoder --output_dir ./output --n_batch 1 --controlnet_name dwpose_body_hand --which2video "video_middle" --target_datas dacne1 --fps 12 --time_size 12
python scripts/inference/video2video.py --sd_model_name majicmixRealv6Fp16 --unet_model_name musev_referencenet_pose --referencenet_model_name musev_referencenet --ip_adapter_model_name musev_referencenet_pose -test_data_path ./configs/tasks/example.yaml --vision_clip_extractor_class_name ImageClipVisionFeatureExtractor --vision_clip_model_path ./checkpoints/IP-Adapter/models/image_encoder --output_dir ./output --n_batch 1 --controlnet_name dwpose_body_hand --which2video "video_middle" --target_datas dacne1 --fps 12 --time_size 12
```

### musev
Expand Down
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@ We will soon release `MuseTalk`, a real-time high quality lip sync model, which
5. supports multi reference image technology, including `IPAdapter`, `ReferenceOnly`, `ReferenceNet`, `IPAdapterFaceID`.
6. training codes (comming very soon).

# Important bug fixes
1. `musev_referencenet_pose`: model_name of `unet`, `ip_adapter` of Command is not correct, please use `musev_referencenet_pose` instead of `musev_referencenet`.

# News
- [03/27/2024] release `MuseV` project and trained model `musev`, `muse_referencenet`.
Expand Down Expand Up @@ -477,9 +479,9 @@ all controlnet_names refer to [mmcm](https://github.com/TMElyralab/MMCM/blob/mai

### musev_referencenet_pose
Only used for `pose2video`
Based on `musev_referencenet`, fix `referencenet`, `pose-controlnet`, and `T2I`, train `motion` module and `IPAdapter`.
train based on `musev_referencenet`, fix `referencenet`, `pose-controlnet`, and `T2I`, train `motion` module and `IPAdapter`.
```bash
python scripts/inference/video2video.py --sd_model_name majicmixRealv6Fp16 --unet_model_name musev_referencenet --referencenet_model_name musev_referencenet --ip_adapter_model_name musev_referencenet -test_data_path ./configs/tasks/example.yaml --vision_clip_extractor_class_name ImageClipVisionFeatureExtractor --vision_clip_model_path ./checkpoints/IP-Adapter/models/image_encoder --output_dir ./output --n_batch 1 --controlnet_name dwpose_body_hand --which2video "video_middle" --target_datas dacne1 --fps 12 --time_size 12
python scripts/inference/video2video.py --sd_model_name majicmixRealv6Fp16 --unet_model_name musev_referencenet_pose --referencenet_model_name musev_referencenet --ip_adapter_model_name musev_referencenet_pose -test_data_path ./configs/tasks/example.yaml --vision_clip_extractor_class_name ImageClipVisionFeatureExtractor --vision_clip_model_path ./checkpoints/IP-Adapter/models/image_encoder --output_dir ./output --n_batch 1 --controlnet_name dwpose_body_hand --which2video "video_middle" --target_datas dacne1 --fps 12 --time_size 12
```

### musev
Expand Down

0 comments on commit 5252552

Please sign in to comment.