Summarization is the task of producing a shorter version of a document that preserves most of the original document's meaning.
The CNN / Daily Mail dataset as processed by Nallapati et al. (2016) has been used for evaluating summarization. The dataset contains online news articles (781 tokens on average) paired with multi-sentence summaries (3.75 sentences or 56 tokens on average). The processed version contains 287,226 training pairs, 13,368 validation pairs and 11,490 test pairs. Models are evaluated based on ROUGE-1, ROUGE-2, and ROUGE-L. * indicates that models were trained and evaluated on the anonymized version of the dataset.
| Model | ROUGE-1 | ROUGE-2 | ROUGE-L | Paper / Source |
|---|---|---|---|---|
| DCA (Celikyilmaz et al., 2018) | 41.69 | 19.47 | 37.92 | Deep Communicating Agents for Abstractive Summarization |
| Pointer-generator + coverage (See et al., 2017) | 39.53 | 17.28 | 36.38 | Get To The Point: Summarization with Pointer-Generator Networks |
| Extractive model (Nallapati et al., 2017)* | 39.6 | 16.2 | 35.3 | SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents |
| Abstractive model (Nallapti et al., 2016)* | 35.46 | 13.30 | 32.65 | Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond |