Sources of Noise in Dialogue and How to Deal with Them

Derek Chen, Zhou Yu


Abstract
Training dialogue systems often entails dealing with noisy training examples and unexpected user inputs. Despite their prevalence, there currently lacks an accurate survey of dialogue noise, nor is there a clear sense of the impact of each noise type on task performance. This paper addresses this gap by first constructing a taxonomy of noise encountered by dialogue systems. In addition, we run a series of experiments to show how different models behave when subjected to varying levels of noise and types of noise. Our results reveal that models are quite robust to label errors commonly tackled by existing denoising algorithms, but that performance suffers from dialogue-specific noise. Driven by these observations, we design a data cleaning algorithm specialized for conversational settings and apply it as a proof-of-concept for targeted dialogue denoising.
Anthology ID:
2023.sigdial-1.1
Volume:
Proceedings of the 24th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Month:
September
Year:
2023
Address:
Prague, Czechia
Editors:
Svetlana Stoyanchev, Shafiq Joty, David Schlangen, Ondrej Dusek, Casey Kennington, Malihe Alikhani
Venue:
SIGDIAL
SIG:
SIGDIAL
Publisher:
Association for Computational Linguistics
Note:
Pages:
1–20
Language:
URL:
https://aclanthology.org/2023.sigdial-1.1
DOI:
10.18653/v1/2023.sigdial-1.1
Bibkey:
Cite (ACL):
Derek Chen and Zhou Yu. 2023. Sources of Noise in Dialogue and How to Deal with Them. In Proceedings of the 24th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 1–20, Prague, Czechia. Association for Computational Linguistics.
Cite (Informal):
Sources of Noise in Dialogue and How to Deal with Them (Chen & Yu, SIGDIAL 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.sigdial-1.1.pdf