On the Geometry of Concreteness

Christian Wartena


Abstract
In this paper we investigate how concreteness and abstractness are represented in word embedding spaces. We use data for English and German, and show that concreteness and abstractness can be determined independently and turn out to be completely opposite directions in the embedding space. Various methods can be used to determine the direction of concreteness, always resulting in roughly the same vector. Though concreteness is a central aspect of the meaning of words and can be detected clearly in embedding spaces, it seems not as easy to subtract or add concreteness to words to obtain other words or word senses like e.g. can be done with a semantic property like gender.
Anthology ID:
2022.repl4nlp-1.21
Volume:
Proceedings of the 7th Workshop on Representation Learning for NLP
Month:
May
Year:
2022
Address:
Dublin, Ireland
Editors:
Spandana Gella, He He, Bodhisattwa Prasad Majumder, Burcu Can, Eleonora Giunchiglia, Samuel Cahyawijaya, Sewon Min, Maximilian Mozes, Xiang Lorraine Li, Isabelle Augenstein, Anna Rogers, Kyunghyun Cho, Edward Grefenstette, Laura Rimell, Chris Dyer
Venue:
RepL4NLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
204–212
Language:
URL:
https://aclanthology.org/2022.repl4nlp-1.21
DOI:
10.18653/v1/2022.repl4nlp-1.21
Bibkey:
Cite (ACL):
Christian Wartena. 2022. On the Geometry of Concreteness. In Proceedings of the 7th Workshop on Representation Learning for NLP, pages 204–212, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
On the Geometry of Concreteness (Wartena, RepL4NLP 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.repl4nlp-1.21.pdf
Video:
 https://aclanthology.org/2022.repl4nlp-1.21.mp4