Crowdsourcing on Sensitive Data with Privacy-Preserving Text Rewriting

Nina Mouhammad, Johannes Daxenberger, Benjamin Schiller, Ivan Habernal


Abstract
Most tasks in NLP require labeled data. Data labeling is often done on crowdsourcing platforms due to scalability reasons. However, publishing data on public platforms can only be done if no privacy-relevant information is included. Textual data often contains sensitive information like person names or locations. In this work, we investigate how removing personally identifiable information (PII) as well as applying differential privacy (DP) rewriting can enable text with privacy-relevant information to be used for crowdsourcing. We find that DP-rewriting before crowdsourcing can preserve privacy while still leading to good label quality for certain tasks and data. PII-removal led to good label quality in all examined tasks, however, there are no privacy guarantees given.
Anthology ID:
2023.law-1.8
Volume:
Proceedings of the 17th Linguistic Annotation Workshop (LAW-XVII)
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Jakob Prange, Annemarie Friedrich
Venue:
LAW
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
73–84
Language:
URL:
https://aclanthology.org/2023.law-1.8
DOI:
10.18653/v1/2023.law-1.8
Bibkey:
Cite (ACL):
Nina Mouhammad, Johannes Daxenberger, Benjamin Schiller, and Ivan Habernal. 2023. Crowdsourcing on Sensitive Data with Privacy-Preserving Text Rewriting. In Proceedings of the 17th Linguistic Annotation Workshop (LAW-XVII), pages 73–84, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
Crowdsourcing on Sensitive Data with Privacy-Preserving Text Rewriting (Mouhammad et al., LAW 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.law-1.8.pdf