Contrastive Learning of Sociopragmatic Meaning in Social Media

Chiyu Zhang, Muhammad Abdul-Mageed, Ganesh Jawahar


Abstract
Recent progress in representation and contrastive learning in NLP has not widely considered the class of sociopragmatic meaning (i.e., meaning in interaction within different language communities). To bridge this gap, we propose a novel framework for learning task-agnostic representations transferable to a wide range of sociopragmatic tasks (e.g., emotion, hate speech, humor, sarcasm). Our framework outperforms other contrastive learning frameworks for both in-domain and out-of-domain data, across both the general and few-shot settings. For example, compared to two popular pre-trained language models, our model obtains an improvement of 11.66 average F1 on 16 datasets when fine-tuned on only 20 training samples per dataset. We also show that our framework improves uniformity and preserves the semantic structure of representations. Our code is available at: https://github.com/UBC-NLP/infodcl
Anthology ID:
2023.findings-acl.152
Volume:
Findings of the Association for Computational Linguistics: ACL 2023
Month:
July
Year:
2023
Address:
Toronto, Canada
Editors:
Anna Rogers, Jordan Boyd-Graber, Naoaki Okazaki
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2405–2439
Language:
URL:
https://aclanthology.org/2023.findings-acl.152
DOI:
10.18653/v1/2023.findings-acl.152
Bibkey:
Cite (ACL):
Chiyu Zhang, Muhammad Abdul-Mageed, and Ganesh Jawahar. 2023. Contrastive Learning of Sociopragmatic Meaning in Social Media. In Findings of the Association for Computational Linguistics: ACL 2023, pages 2405–2439, Toronto, Canada. Association for Computational Linguistics.
Cite (Informal):
Contrastive Learning of Sociopragmatic Meaning in Social Media (Zhang et al., Findings 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.findings-acl.152.pdf
Video:
 https://aclanthology.org/2023.findings-acl.152.mp4