-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
1 changed file
with
21 additions
and
19 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,43 +1,45 @@ | ||
# API | ||
# audio2shapes API | ||
|
||
## API名称 | ||
|
||
audio2shapes | ||
从语音预测每一帧的 52 个 blendshape 值,输入语音,返回以 json 格式存储的 blendshape。 | ||
|
||
## 调用方式 | ||
|
||
POST | ||
|
||
## 运行 | ||
|
||
``` | ||
### 实验环境 | ||
|
||
```shell | ||
conda create -n flask python=3.9 | ||
conda activate flask | ||
pip install -r requirements.txt | ||
``` | ||
|
||
然后先运行 `app.py` 再运行 `launch.py` 。 | ||
### 预训练模型 | ||
|
||
![截屏2023-12-13 16.58.35](https://github.com/Symbolzzz/audio2shapes/blob/main/static/images/2023-12-13%2016.58.35.png?raw=true) | ||
下载[transformer_model](https://drive.google.com/file/d/1qXvvbSW_L9mG9K9Mh_HwvoD7y2lbaIPo/view?usp=drive_link) ,并放在 `utils/` 目录下。 | ||
|
||
运行 `app.py` 之后会出现上述界面,将上述地址复制到 `launch.py` 中url字段。 | ||
### 服务端 | ||
|
||
![截屏2023-12-13 16.59.31](https://github.com/Symbolzzz/audio2shapes/blob/main/static/images/2023-12-13%2016.59.31.png?raw=true) | ||
启动服务端 | ||
|
||
然后再运行 `launch.py`。 | ||
```shell | ||
python app.py | ||
``` | ||
|
||
`audios` 文件夹中存储了测试音频。 | ||
### 客户端 | ||
|
||
## 结果 | ||
先在 `config.json` 修改服务端返回的公网地址,如下图中的 `http://172.31.70.115:8888`。务必在 `config.json` 中配置正确的语音文件路径,目前支持格式:`wav`。 | ||
|
||
结果会以json格式写在 `res.json` 文件中。 | ||
![截屏2023-12-13 16.58.35](https://github.com/Symbolzzz/audio2shapes/blob/main/static/images/2023-12-13%2016.58.35.png?raw=true) | ||
|
||
目前使用的模型还不够完善,存在问题: | ||
然后启动客户端请求 | ||
|
||
* 模型准确率不高 | ||
* 只预测了 `jawOpen` 值 | ||
```shell | ||
python launch.py -c ./config.json | ||
``` | ||
|
||
接下来的工作: | ||
## 结果 | ||
|
||
* 改善模型并保存,替换目前的模型 | ||
* 再训练好 `blendshapes` 中的其他参数的模型,加入进来 | ||
结果会以json格式写在 `res.json` 文件中。 |