SUMMARY WORKBENCH: Unifying Application and Evaluation of Text Summarization Models

Shahbaz Syed, Dominik Schwabe, Martin Potthast


Abstract
This paper presents Summary Workbench, a new tool for developing and evaluating text summarization models. New models and evaluation measures can be easily integrated as Docker-based plugins, allowing to examine the quality of their summaries against any input and to evaluate them using various evaluation measures. Visual analyses combining multiple measures provide insights into the models’ strengths and weaknesses. The tool is hosted at https://tldr.demo.webis.de and also supports local deployment for private resources.
Anthology ID:
2022.emnlp-demos.23
Volume:
Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: System Demonstrations
Month:
December
Year:
2022
Address:
Abu Dhabi, UAE
Editors:
Wanxiang Che, Ekaterina Shutova
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
232–241
Language:
URL:
https://aclanthology.org/2022.emnlp-demos.23
DOI:
10.18653/v1/2022.emnlp-demos.23
Bibkey:
Cite (ACL):
Shahbaz Syed, Dominik Schwabe, and Martin Potthast. 2022. SUMMARY WORKBENCH: Unifying Application and Evaluation of Text Summarization Models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 232–241, Abu Dhabi, UAE. Association for Computational Linguistics.
Cite (Informal):
SUMMARY WORKBENCH: Unifying Application and Evaluation of Text Summarization Models (Syed et al., EMNLP 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.emnlp-demos.23.pdf