Gregor Engels


2020

pdf bib
Mining Crowdsourcing Problems from Discussion Forums of Workers
Zahra Nouri | Henning Wachsmuth | Gregor Engels
Proceedings of the 28th International Conference on Computational Linguistics

Crowdsourcing is used in academia and industry to solve tasks that are easy for humans but hard for computers, in natural language processing mostly to annotate data. The quality of annotations is affected by problems in the task design, task operation, and task evaluation that workers face with requesters in crowdsourcing processes. To learn about the major problems, we provide a short but comprehensive survey based on two complementary studies: (1) a literature review where we collect and organize problems known from interviews with workers, and (2) an empirical data analysis where we use topic modeling to mine workers’ complaints from a new English corpus of workers’ forum discussions. While literature covers all process phases, problems in the task evaluation are prevalent, including unfair rejections, late payments, and unjustified blockings of workers. According to the data, however, poor task design in terms of malfunctioning environments, bad workload estimation, and privacy violations seems to bother the workers most. Our findings form the basis for future research on how to improve crowdsourcing processes.

2014

pdf bib
Modeling Review Argumentation for Robust Sentiment Analysis
Henning Wachsmuth | Martin Trenkmann | Benno Stein | Gregor Engels
Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers

2013

pdf bib
Learning Efficient Information Extraction on Heterogeneous Texts
Henning Wachsmuth | Benno Stein | Gregor Engels
Proceedings of the Sixth International Joint Conference on Natural Language Processing