update news

sidingli · Mar 30, 2022 · 0f5fb5d · 0f5fb5d
1 parent e8e9c59
commit 0f5fb5d
Show file tree

Hide file tree

Showing 2 changed files with 27 additions and 26 deletions.
diff --git a/README.md b/README.md
@@ -10,34 +10,37 @@
         <img alt="GitHub" src="https://img.shields.io/github/license/ymcui/Chinese-BERT-wwm.svg?color=blue&style=flat-square">
     </a>
 </p>
-在自然语言处理领域中，预训练语言模型（Pre-trained Language Models）已成为非常重要的基础技术。为了进一步促进中文信息处理的研究发展，我们发布了基于全词遮罩（Whole Word Masking）技术的中文预训练模型BERT-wwm，以及与此技术密切相关的模型：BERT-wwm-ext，RoBERTa-wwm-ext，RoBERTa-wwm-ext-large, RBT3, RBTL3。  
+在自然语言处理领域中，预训练语言模型（Pre-trained Language Models）已成为非常重要的基础技术。为了进一步促进中文信息处理的研究发展，我们发布了基于全词掩码（Whole Word Masking）技术的中文预训练模型BERT-wwm，以及与此技术密切相关的模型：BERT-wwm-ext，RoBERTa-wwm-ext，RoBERTa-wwm-ext-large, RBT3, RBTL3等。  
 
-**[Pre-Training with Whole Word Masking for Chinese BERT](https://arxiv.org/abs/1906.08101)**  
-Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang, Shijin Wang, Guoping Hu
+- **[Pre-Training with Whole Word Masking for Chinese BERT](https://ieeexplore.ieee.org/document/9599397)**  
+- *Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang*
+- Published in *IEEE/ACM Transactions on Audio, Speech, and Language Processing (TASLP)*
 
 本项目基于谷歌官方BERT：https://github.com/google-research/bert
 
-其他相关资源：
+----
 
 [中文MacBERT](https://github.com/ymcui/MacBERT) | [中文ELECTRA](https://github.com/ymcui/Chinese-ELECTRA) | [中文XLNet](https://github.com/ymcui/Chinese-XLNet) | [知识蒸馏工具TextBrewer](https://github.com/airaria/TextBrewer) | [模型裁剪工具TextPruner](https://github.com/airaria/TextPruner)
 
 查看更多哈工大讯飞联合实验室（HFL）发布的资源：https://github.com/ymcui/HFL-Anthology
 
 ## 新闻
-**2021/12/17 哈工大讯飞联合实验室推出模型裁剪工具包TextPruner。查看：https://github.com/airaria/TextPruner**
+**2022/3/30 我们开源了一种新预训练模型PERT。查看：https://github.com/ymcui/PERT**
+
+2021/12/17 哈工大讯飞联合实验室推出模型裁剪工具包TextPruner。查看：https://github.com/airaria/TextPruner
 
 2021/10/24 哈工大讯飞联合实验室发布面向少数民族语言的预训练模型CINO。查看：https://github.com/ymcui/Chinese-Minority-PLM
 
 2021/7/21 由哈工大SCIR多位学者撰写的[《自然语言处理：基于预训练模型的方法》](https://item.jd.com/13344628.html)已出版，欢迎大家选购。
 
 2021/1/27 所有模型已支持TensorFlow 2，请通过transformers库进行调用或下载。https://huggingface.co/hfl
 
+<details>
+<summary>历史新闻</summary>
 2020/9/15 我们的论文["Revisiting Pre-Trained Models for Chinese Natural Language Processing"](https://arxiv.org/abs/2004.13922)被[Findings of EMNLP](https://2020.emnlp.org)录用为长文。
 
 2020/8/27 哈工大讯飞联合实验室在通用自然语言理解评测GLUE中荣登榜首，查看[GLUE榜单](https://gluebenchmark.com/leaderboard)，[新闻](http://dwz.date/ckrD)。
 
-<details>
-<summary>历史新闻</summary>
 2020/3/23 本目录发布的模型已接入[飞桨PaddleHub](https://github.com/PaddlePaddle/PaddleHub)，查看[快速加载](#快速加载)
 
 2020/3/11 为了更好地了解需求，邀请您填写[调查问卷](https://wj.qq.com/s2/5637766/6281)，以便为大家提供更好的资源。
@@ -100,7 +103,7 @@ Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang, Shijin Wang, Guoping
 * **`BERT-large模型`**：24-layer, 1024-hidden, 16-heads, 330M parameters  
 * **`BERT-base模型`**：12-layer, 768-hidden, 12-heads, 110M parameters  
 
-**注意：开源版本不包含MLM任务的权重；如需做MLM任务，请进行二次预训练。**
+**注意：开源版本不包含MLM任务的权重；如需做MLM任务，请使用额外数据进行二次预训练（和其他下游任务一样）。**
 
 | 模型简称 | 语料 | Google下载 | 百度网盘下载 |
 | :------- | :--------- | :---------: | :---------: |
@@ -471,8 +474,8 @@ A: 我们集成了RoBERTa和BERT-wwm的优点，对两者进行了一个自然
 
 
 ## 引用
-如果本目录中的内容对你的研究工作有所帮助，欢迎在论文中引用下述论文。
-- 首选：https://ieeexplore.ieee.org/document/9599397
+如果本项目中的资源或技术对你的研究工作有所帮助，欢迎在论文中引用下述论文。
+- 首选（期刊扩充版）：https://ieeexplore.ieee.org/document/9599397
 ```
 @journal{cui-etal-2021-pretrain,
   title={Pre-Training with Whole Word Masking for Chinese BERT},
@@ -484,7 +487,7 @@ A: 我们集成了RoBERTa和BERT-wwm的优点，对两者进行了一个自然
  }
 ```
 
-- 或者：https://arxiv.org/abs/2004.13922
+- 或者（会议版本）：https://www.aclweb.org/anthology/2020.findings-emnlp.58
 ```
 @inproceedings{cui-etal-2020-revisiting,
     title = "Revisiting Pre-Trained Models for {C}hinese Natural Language Processing",
@@ -506,7 +509,7 @@ A: 我们集成了RoBERTa和BERT-wwm的优点，对两者进行了一个自然
 
 
 ## 致谢
-第一作者部分受到[**谷歌TensorFlow Research Cloud**](https://www.tensorflow.org/tfrc)计划资助。
+第一作者部分受到[**谷歌TPU Research Cloud**](https://www.tensorflow.org/tfrc)计划资助。
 
 
 ## 免责声明

diff --git a/README_EN.md b/README_EN.md
@@ -3,39 +3,37 @@
 ## Chinese BERT with Whole Word Masking
 For further accelerating Chinese natural language processing, we provide **Chinese pre-trained BERT with Whole Word Masking**. Meanwhile, we also compare the state-of-the-art Chinese pre-trained models in depth, including [BERT](https://github.com/google-research/bert)、[ERNIE](https://github.com/PaddlePaddle/LARK/tree/develop/ERNIE)、[BERT-wwm](https://github.com/ymcui/Chinese-BERT-wwm).
 
-**[Pre-Training with Whole Word Masking for Chinese BERT](https://arxiv.org/abs/1906.08101)**  
-Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang, Shijin Wang, Guoping Hu
+- **[Pre-Training with Whole Word Masking for Chinese BERT](https://ieeexplore.ieee.org/document/9599397)**  
+- Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang
+- Published in *IEEE/ACM Transactions on Audio, Speech, and Language Processing (TASLP)*
 
 This repository is developed based on：https://github.com/google-research/bert
 
-You may also be interested in,
+----
 
-- Chinese MacBERT: https://github.com/ymcui/MacBERT
-- Chinese ELECTRA: https://github.com/ymcui/Chinese-ELECTRA
-- Chinese XLNet: https://github.com/ymcui/Chinese-XLNet
-- Knowledge Distillation Toolkit - TextBrewer: https://github.com/airaria/TextBrewer
-- Model Pruning Toolkit - TextPruner: https://github.com/airaria/TextPruner
+[Chinese MacBERT](https://github.com/ymcui/MacBERT) | [Chinese ELECTRA](https://github.com/ymcui/Chinese-ELECTRA) | [Chinese XLNet](https://github.com /ymcui/Chinese-XLNet) | [Chinese BERT](https://github.com/ymcui/Chinese-BERT-wwm) | [TextBrewer](https://github.com/airaria/TextBrewer) | [TextPruner](https://github.com/airaria/TextPruner)
 
 More resources by HFL: https://github.com/ymcui/HFL-Anthology
 
-
 ## News
-**2021/12/17 We release a model pruning toolkit - TextPruner, check https://github.com/airaria/TextPruner**
+**2022/3/30 We release a new pre-trained model called PERT, check https://github.com/ymcui/PERT **
+
+2021/12/17 We release a model pruning toolkit - TextPruner, check https://github.com/airaria/TextPruner
 
 2021/1/27 All models support TensorFlow 2 now. Please use transformers library to access them or download from https://huggingface.co/hfl
 
 2020/9/15 Our paper ["Revisiting Pre-Trained Models for Chinese Natural Language Processing"](https://arxiv.org/abs/2004.13922) is accepted to [Findings of EMNLP](https://2020.emnlp.org) as a long paper.
 
 2020/8/27 We are happy to announce that our model is on top of GLUE benchmark, check [leaderboard](https://gluebenchmark.com/leaderboard).
 
+<details>
+<summary>Past News</summary>
 2020/3/23 The models in this repository now can be easily accessed through [PaddleHub](https://github.com/PaddlePaddle/PaddleHub), check [Quick Load](#Quick-Load)
 
 2020/2/26 We release a knowledge distillation toolkit [TextBrewer](https://github.com/airaria/TextBrewer)
 
 2020/1/20 Happy Chinese New Year! We've released RBT3 and RBTL3 (3-layer RoBERTa-wwm-ext-base/large), check [Small Models](#Small-Models)
 
-<details>
-<summary>Past News</summary>
 2019/12/19 The models in this repository now can be easily accessed through [Huggingface-Transformers](https://github.com/huggingface/transformers), check [Quick Load](#Quick-Load)
 
 2019/10/14 We release `RoBERTa-wwm-ext-large`, check [Download](#Download)
@@ -439,7 +437,7 @@ A: integrate whole word masking (wwm) into RoBERTa model, specifically:
 
 ## Citation
 If you find the technical report or resource is useful, please cite our work in your paper.
-- Primary: https://ieeexplore.ieee.org/document/9599397  
+- Primary (Journal extension): https://ieeexplore.ieee.org/document/9599397  
 ```
 @journal{cui-etal-2021-pretrain,
   title={Pre-Training with Whole Word Masking for Chinese BERT},
@@ -450,7 +448,7 @@ If you find the technical report or resource is useful, please cite our work in
   doi={10.1109/TASLP.2021.3124365},
  }
 ```
-- Secondary: https://arxiv.org/abs/2004.13922  
+- Secondary (conference paper): https://www.aclweb.org/anthology/2020.findings-emnlp.58
 ```
 @inproceedings{cui-etal-2020-revisiting,
     title = "Revisiting Pre-Trained Models for {C}hinese Natural Language Processing",