Skip to content

Commit

Permalink
add METEOR scores
Browse files Browse the repository at this point in the history
  • Loading branch information
Yen Chen committed Jul 8, 2018
1 parent 553a6d1 commit 990636c
Showing 1 changed file with 9 additions and 9 deletions.
18 changes: 9 additions & 9 deletions summarization.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,17 +10,17 @@ The [CNN / Daily Mail dataset](https://arxiv.org/abs/1506.03340) as processed by
for evaluating summarization. The dataset contains online news articles (781 tokens
on average) paired with multi-sentence summaries (3.75 sentences or 56 tokens on average).
The processed version contains 287,226 training pairs, 13,368 validation pairs and 11,490 test pairs.
Models are evaluated based on ROUGE-1, ROUGE-2, and ROUGE-L. * indicates that models
Models are evaluated based on ROUGE-1, ROUGE-2, ROUGE-L, and METEOR (optional). * indicates that models
were trained and evaluated on the anonymized version of the dataset.

| Model | ROUGE-1 | ROUGE-2 | ROUGE-L | Paper / Source |
| ------------- | :-----:| :-----:| :-----:| --- |
| Fast-ABS-RL (Chen and Bansal, 2018) (abstractive) | 41.20 | 18.18 | 38.79 | [Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting](https://arxiv.org/abs/1805.11080) / [code](https://github.com/chenrocks/fast_abs_rl) |
| DCA (Celikyilmaz et al., 2018) | 41.69| 19.47 | 37.92 | [Deep Communicating Agents for Abstractive Summarization](https://arxiv.org/abs/1803.10357) |
| REFRESH - Extractive model (Narayan et al., 2018) | 40.0 | 18.2 | 36.6 | [Ranking Sentences for Extractive Summarization with Reinforcement Learning](https://arxiv.org/pdf/1802.08636.pdf) |
| Pointer-generator + coverage (See et al., 2017) | 39.53| 17.28 | 36.38 | [Get To The Point: Summarization with Pointer-Generator Networks](https://arxiv.org/abs/1704.04368) |
| Extractive model (Nallapati et al., 2017)* | 39.6 | 16.2 | 35.3 | [SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents](https://arxiv.org/abs/1611.04230) |
| Abstractive model (Nallapti et al., 2016)* | 35.46 | 13.30 | 32.65 | [Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond](http://www.aclweb.org/anthology/K16-1028) |
| Model | ROUGE-1 | ROUGE-2 | ROUGE-L | METEOR | Paper / Source |
| --------------- | :-----: | :-----: | :-----: | :----: | -------------- |
| Fast-ABS-RL (Chen and Bansal, 2018) (abstractive) | 41.20 | 18.18 | 38.79 | 20.55 | [Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting](https://arxiv.org/abs/1805.11080) / [code](https://github.com/chenrocks/fast_abs_rl) |
| DCA (Celikyilmaz et al., 2018) | 41.69| 19.47 | 37.92 | - | [Deep Communicating Agents for Abstractive Summarization](https://arxiv.org/abs/1803.10357) |
| REFRESH - Extractive model (Narayan et al., 2018) | 40.0 | 18.2 | 36.6 | - | [Ranking Sentences for Extractive Summarization with Reinforcement Learning](https://arxiv.org/pdf/1802.08636.pdf) |
| Pointer-generator + coverage (See et al., 2017) | 39.53| 17.28 | 36.38 | 18.72 | [Get To The Point: Summarization with Pointer-Generator Networks](https://arxiv.org/abs/1704.04368) |
| Extractive model (Nallapati et al., 2017)* | 39.6 | 16.2 | 35.3 | - | [SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents](https://arxiv.org/abs/1611.04230) |
| Abstractive model (Nallapti et al., 2016)* | 35.46 | 13.30 | 32.65 | - | [Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond](http://www.aclweb.org/anthology/K16-1028) |

## Sentence Compression

Expand Down

0 comments on commit 990636c

Please sign in to comment.