Skip to content

Commit

Permalink
update style and desc
Browse files Browse the repository at this point in the history
  • Loading branch information
ymcui committed Mar 24, 2020
1 parent 101789b commit b14422a
Show file tree
Hide file tree
Showing 2 changed files with 15 additions and 22 deletions.
21 changes: 8 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang, Shijin Wang, Guoping
## 新闻
2020/3/23 本目录发布的模型已接入[飞桨PaddleHub](https://github.com/PaddlePaddle/PaddleHub),查看[快速加载](#快速加载)

**2020/3/11 为了更好地了解需求邀请您填写[调查问卷](https://wj.qq.com/s2/5637766/6281),以便为大家提供更好的资源**
2020/3/11 为了更好地了解需求邀请您填写[调查问卷](https://wj.qq.com/s2/5637766/6281),以便为大家提供更好的资源

2020/2/26 哈工大讯飞联合实验室发布[知识蒸馏工具TextBrewer](https://github.com/airaria/TextBrewer)

Expand All @@ -45,9 +45,10 @@ Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang, Shijin Wang, Guoping
|-|-|
| [简介](#简介) | 介绍BERT-wwm基本原理 |
| [中文模型下载](#中文模型下载) | 提供了BERT-wwm的下载地址 |
| [快速加载](#快速加载) | 介绍了如何使用[🤗Transformers](https://github.com/huggingface/transformers)、[PaddleHub](https://github.com/PaddlePaddle/PaddleHub)快速加载模型 |
| [模型对比](#模型对比) | 提供了本目录中模型的参数对比 |
| [中文基线系统效果](#中文基线系统效果) | 列举了部分中文基线系统效果 |
| [new 小参数量模型](#小参数量模型) | 列举了小参数量模型(3层Transformer)的效果 |
| [小参数量模型](#小参数量模型) | 列举了小参数量模型(3层Transformer)的效果 |
| [使用建议](#使用建议) | 提供了若干使用中文预训练模型的建议 |
| [英文模型下载](#英文模型下载) | 谷歌官方的英文BERT-wwm下载地址 |
| [FAQ](#FAQ) | 常见问题答疑 |
Expand Down Expand Up @@ -113,8 +114,8 @@ chinese_wwm_L-12_H-768_A-12.zip
PyTorch版本则包含`pytorch_model.bin`, `bert_config.json`, `vocab.txt`文件


### 快速加载
1. 使用Huggingface-Transformers
## 快速加载
### 使用Huggingface-Transformers

依托于[Huggingface-Transformers 2.2.2](https://github.com/huggingface/transformers),可轻松调用以上模型
```
Expand All @@ -134,9 +135,9 @@ model = BertModel.from_pretrained("MODEL_NAME")
| RBT3 | hfl/rbt3 |
| RBTL3 | hfl/rbtl3 |

2. 使用PaddleHub
### 使用PaddleHub

依托[PaddleHub](https://github.com/PaddlePaddle/PaddleHub),我们只需一行代码即可完成模型下载安装十余行代码即可完成文本分类序列标注阅读理解等任务
依托[PaddleHub](https://github.com/PaddlePaddle/PaddleHub),只需一行代码即可完成模型下载安装十余行代码即可完成文本分类序列标注阅读理解等任务

```
import paddlehub as hub
Expand All @@ -155,12 +156,6 @@ module = hub.Module(name=MODULE_NAME)
| RBTL3 | [rbtl3](https://www.paddlepaddle.org.cn/hubdetail?name=rbtl3&en_category=SemanticModel) |


### 测试任务数据
我们提供部分任务数据请查看`data`目录了解
压缩包内包含训练和测试数据同一目录下的`README.md`标明数据来源
由于一部分数据需要原作者授权故我们无法提供下载链接敬请谅解


## 模型对比
针对大家比较关心的一些模型细节进行汇总如下

Expand Down Expand Up @@ -392,7 +387,7 @@ A: 谷歌发布的中文BERT怎么用,这个就怎么用。
A: 很遗憾我不能提供相关代码实现可以参考 [#10](https://github.com/ymcui/Chinese-BERT-wwm/issues/10) 和 [#13](https://github.com/ymcui/Chinese-BERT-wwm/issues/13)。

**Q: 某某数据集在哪里下载**
A: 请查看data目录对于有版权的内容请自行搜索或与原作者联系获取数据
A: 请查看`data`目录任务目录下的`README.md`标明了数据来源对于有版权的内容请自行搜索或与原作者联系获取数据

**Q: 会有计划发布更大模型吗比如BERT-large-wwm版本**
A: 如果我们从实验中得到更好效果会考虑发布更大的版本
Expand Down
16 changes: 7 additions & 9 deletions README_EN.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ For further accelerating Chinese natural language processing, we provide **Chine
## News
2020/3/23 The models in this repository now can be easily accessed through [PaddleHub](https://github.com/PaddlePaddle/PaddleHub), check [Quick Load](#Quick-Load)

**2020/2/26 We release a knowledge distillation toolkit [TextBrewer](https://github.com/airaria/TextBrewer)**
2020/2/26 We release a knowledge distillation toolkit [TextBrewer](https://github.com/airaria/TextBrewer)

2020/1/20 Happy Chinese New Year! We've released RBT3 and RBTL3 (3-layer RoBERTa-wwm-ext-base/large), check [Small Models](#Small-Models)

Expand All @@ -31,6 +31,7 @@ For further accelerating Chinese natural language processing, we provide **Chine
|-|-|
| [Introduction](#Introduction) | Introduction to BERT with Whole Word Masking (WWM) |
| [Download](#Download) | Download links for Chinese BERT-wwm |
| [Quick Load](#Quick-Load) | Learn how to quickly load our models through [🤗Transformers](https://github.com/huggingface/transformers) or [PaddleHub](https://github.com/PaddlePaddle/PaddleHub) |
| [Model Comparison](#Model-Comparison) | Compare the models published in this repository |
| [Baselines](#Baselines) | Baseline results for several Chinese NLP datasets (partial) |
| [Small Models](#Small-Models) | 3-layer Transformer models |
Expand Down Expand Up @@ -98,8 +99,8 @@ chinese_wwm_L-12_H-768_A-12.zip
`bert_config.json` and `vocab.txt` are identical to the original **`BERT-base, Chinese`** by Google


### Quick Load
1. Huggingface-Transformers
## Quick Load
### Huggingface-Transformers

With [Huggingface-Transformers](https://github.com/huggingface/transformers), the models above could be easily accessed and loaded through the following codes.
```
Expand All @@ -119,7 +120,7 @@ The actual model and its `MODEL_NAME` are listed below.
| RBT3 | hfl/rbt3 |
| RBTL3 | hfl/rbtl3 |

2. PaddleHub
### PaddleHub

With [PaddleHub](https://github.com/PaddlePaddle/PaddleHub), we can download and install the model with one line of code.

Expand All @@ -130,7 +131,7 @@ module = hub.Module(name=MODULE_NAME)

The actual model and its `MODULE_NAME` are listed below.

| Original Model| MODULE_NAME |
| Original Model | MODULE_NAME |
| - | - |
| RoBERTa-wwm-ext-large | [chinese-roberta-wwm-ext-large](https://www.paddlepaddle.org.cn/hubdetail?name=chinese-roberta-wwm-ext-large&en_category=SemanticModel) |
| RoBERTa-wwm-ext | [chinese-roberta-wwm-ext](https://www.paddlepaddle.org.cn/hubdetail?name=chinese-roberta-wwm-ext&en_category=SemanticModel) |
Expand All @@ -139,9 +140,6 @@ The actual model and its `MODULE_NAME` are listed below.
| RBT3 | [rbt3](https://www.paddlepaddle.org.cn/hubdetail?name=rbt3&en_category=SemanticModel) |
| RBTL3 | [rbtl3](https://www.paddlepaddle.org.cn/hubdetail?name=rbtl3&en_category=SemanticModel) |

### Task Data
We only provide the data that is publically available, check `data` directory.


## Model Comparison
We list comparisons on the models that were released in this project.
Expand Down Expand Up @@ -361,7 +359,7 @@ A: Use it as if you are using original BERT. Note that, you don't need to do CWS
A: Unfortunately, I am not be able to release the code at the moment. As implementation is quite easy, I would suggest you to read #10 and #13.

**Q: How can I download XXXXX dataset?**
A: Please see `data` directory. For copyright reasons, some of the datasets are not publically available. In that case, please search on GitHub or consult original authors for accessing.
A: We only provide the data that is publically available, check `data` directory. For copyright reasons, some of the datasets are not publically available. In that case, please search on GitHub or consult original authors for accessing.

**Q: How to use this model?**
A: Use it as if you are using original BERT. Note that, you don't need to do CWS for your text, as wwm only change the pre-training input but not the input for down-stream tasks.
Expand Down

0 comments on commit b14422a

Please sign in to comment.