EntSUMv2: Dataset, Models and Evaluation for More Abstractive Entity-Centric Summarization

Dhruv Mehra, Lingjue Xie, Ella Hofmann-Coyle, Mayank Kulkarni, Daniel Preotiuc-Pietro


Abstract
Entity-centric summarization is a form of controllable summarization that aims to generate a summary for a specific entity given a document. Concise summaries are valuable in various real-life applications, as they enable users to quickly grasp the main points of the document focusing on an entity of interest. This paper presents ENTSUMV2, a more abstractive version of the original entity-centric ENTSUM summarization dataset. In ENTSUMV2 the annotated summaries are intentionally made shorter to benefit more specific and useful entity-centric summaries for downstream users. We conduct extensive experiments on this dataset using multiple abstractive summarization approaches that employ supervised fine-tuning or large-scale instruction tuning. Additionally, we perform comprehensive human evaluation that incorporates metrics for measuring crucial facets. These metrics provide a more fine-grained interpretation of the current state-of-the-art systems and highlight areas for future improvement.
Anthology ID:
2023.emnlp-main.337
Volume:
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
Month:
December
Year:
2023
Address:
Singapore
Editors:
Houda Bouamor, Juan Pino, Kalika Bali
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5538–5547
Language:
URL:
https://aclanthology.org/2023.emnlp-main.337
DOI:
10.18653/v1/2023.emnlp-main.337
Bibkey:
Cite (ACL):
Dhruv Mehra, Lingjue Xie, Ella Hofmann-Coyle, Mayank Kulkarni, and Daniel Preotiuc-Pietro. 2023. EntSUMv2: Dataset, Models and Evaluation for More Abstractive Entity-Centric Summarization. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 5538–5547, Singapore. Association for Computational Linguistics.
Cite (Informal):
EntSUMv2: Dataset, Models and Evaluation for More Abstractive Entity-Centric Summarization (Mehra et al., EMNLP 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.emnlp-main.337.pdf
Video:
 https://aclanthology.org/2023.emnlp-main.337.mp4