Skip to content

Commit

Permalink
update lr
Browse files Browse the repository at this point in the history
  • Loading branch information
ymcui committed Nov 19, 2019
1 parent f7d7170 commit c5409fd
Show file tree
Hide file tree
Showing 2 changed files with 29 additions and 0 deletions.
15 changes: 15 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,21 @@ PyTorch版本则包含`pytorch_model.bin`, `bert_config.json`, `vocab.txt`文件
为了对比基线效果我们在以下几个中文数据集上进行了测试包括`句子级``篇章级`任务
对于`BERT-wwm-ext``RoBERTa-wwm-ext``RoBERTa-wwm-ext-large`我们**没有进一步调整最佳学习率**而是直接使用了`BERT-wwm`的最佳学习率

最佳学习率
| 模型 | BERT | ERNIE | BERT-wwm* |
| :------- | :---------: | :---------: | :---------: |
| CMRC 2018 | 3e-5 | 8e-5 | 3e-5 |
| DRCD | 3e-5 | 8e-5 | 3e-5 |
| CJRC | 4e-5 | 8e-5 | 4e-5 |
| XNLI | 3e-5 | 5e-5 | 3e-5 |
| ChnSentiCorp | 2e-5 | 5e-5 | 2e-5 |
| LCQMC | 2e-5 | 3e-5 | 2e-5 |
| BQ Corpus | 3e-5 | 5e-5 | 3e-5 |
| THUCNews | 2e-5 | 5e-5 | 2e-5 |

*代表所有wwm系列模型 (BERT-wwm, BERT-wwm-ext, RoBERTa-wwm-ext, RoBERTa-wwm-ext-large)


**下面仅列举部分结果完整结果请查看我们的[技术报告](https://arxiv.org/abs/1906.08101)。**

- [**CMRC 2018**篇章片段抽取型阅读理解简体中文)](https://github.com/ymcui/cmrc2018)
Expand Down
14 changes: 14 additions & 0 deletions README_EN.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,20 @@ We experiment on several Chinese datasets, including sentence-level to document-

**We only list partial results here and kindly advise the readers to read our [technical report](https://arxiv.org/abs/1906.08101).**

Best Learning Rate:
| Model | BERT | ERNIE | BERT-wwm* |
| :------- | :---------: | :---------: | :---------: |
| CMRC 2018 | 3e-5 | 8e-5 | 3e-5 |
| DRCD | 3e-5 | 8e-5 | 3e-5 |
| CJRC | 4e-5 | 8e-5 | 4e-5 |
| XNLI | 3e-5 | 5e-5 | 3e-5 |
| ChnSentiCorp | 2e-5 | 5e-5 | 2e-5 |
| LCQMC | 2e-5 | 3e-5 | 2e-5 |
| BQ Corpus | 3e-5 | 5e-5 | 3e-5 |
| THUCNews | 2e-5 | 5e-5 | 2e-5 |
* represents all related models (BERT-wwm, BERT-wwm-ext, RoBERTa-wwm-ext, RoBERTa-wwm-ext-large)


- [**CMRC 2018**Span-Extraction Machine Reading Comprehension (Simplified Chinese)](https://github.com/ymcui/cmrc2018)
- [**DRCD**Span-Extraction Machine Reading Comprehension (Traditional Chinese)](https://github.com/DRCSolutionService/DRCD)
- [**CJRC**: Chinese Judiciary Reading Comprehension](http://cail.cipsc.org.cn)
Expand Down

0 comments on commit c5409fd

Please sign in to comment.