Bridging the gap between extractive and abstractive summaries: Creation and evaluation of coherent extracts from heterogeneous sources

Benikova, Darina and Mieskes, Margot and Meyer, Christian M. and Gurevych, Iryna :
Bridging the gap between extractive and abstractive summaries: Creation and evaluation of coherent extracts from heterogeneous sources.
[Online-Edition: http://aclweb.org/anthology/C16-1099]
Proceedings of the 26th International Conference on Computational Linguistics (COLING)
[Conference or Workshop Item] , (2016)

Abstract

In our work, we present a corpus of heterogeneous documents for summarization to address the issue that information seekers usually face a range of different types of information sources. A second issue we address, is the summary type, as most manual summaries are abstractive, whereas automatic methods mainly create extractive summaries, which are hard to compare to each other using standard evaluation methods. Therefore, we suggest a multi-step process for creating \emph{coherent extracts}, which are based on information taken directly from the source documents, but minimally redacted and meaningfully ordered to form a coherent text. Our qualitative and quantitative evaluation show that quantitative results are not sufficient to judge the quality of a summary and that other quality criteria, such as coherence, should also be taken into account. We find that our corpus is of high quality and that it has the potential to bridge the gap between reference corpora of abstracts and automatic methods producing extracts. Our corpus is available to the research community for further development.

Optionen (nur für Redakteure)

View Item