Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
gitnlp authored Apr 11, 2020
1 parent 27de2ff commit 10d9393
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion minilm/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ Following [Lewis et al. (2019b)](https://arxiv.org/abs/1910.07475), we adopt SQu
| [XLM-R Base](https://arxiv.org/abs/1911.02116) (Our fine-tuned) | 12 | 768 | 85M | 64.9 | 80.3 | 67.0 | 62.7 | 55.0 | 60.4 | 66.5 | 62.3 |
| **mMiniLM-L12xH384** | 12 | 384 | 21M | 63.2 | 79.4 | 66.1 | 61.2 | 54.9 | 58.5 | 63.1 | 59.0 |

# Pre-trained Models for English
# [English] Pre-trained Models
We release the **uncased** **12**-layer and **6**-layer MiniLM models with **384** hidden size distilled from an in-house pre-trained [UniLM v2](/unilm) model in BERT-Base size. We also release **uncased** **6**-layer MiniLM model with **768** hidden size distilled from [BERT-Base](https://github.com/google-research/bert). The models use the same WordPiece vocabulary as BERT.

The links to the pre-trained models:
Expand Down

0 comments on commit 10d9393

Please sign in to comment.