Albert Weichselbraun


2023

pdf bib
Orbis Annotator: An Open Source Toolkit for the Efficient Annotation and Refinement of Text
Norman Süsstrunk | Andreas Fraefel | Albert Weichselbraun | Adrian M. P. Brasoveanu
Proceedings of the 4th Conference on Language, Data and Knowledge

2020

pdf bib
In Media Res: A Corpus for Evaluating Named Entity Linking with Creative Works
Adrian M.P. Brasoveanu | Albert Weichselbraun | Lyndon Nixon
Proceedings of the 24th Conference on Computational Natural Language Learning

Annotation styles express guidelines that direct human annotators in what rules to follow when creating gold standard annotations of text corpora. These guidelines not only shape the gold standards they help create, but also influence the training and evaluation of Named Entity Linking (NEL) tools, since different annotation styles correspond to divergent views on the entities present in the same texts. Such divergence is particularly present in texts from the media domain that contain references to creative works. In this work we present a corpus of 1000 annotated documents selected from the media domain. Each document is presented with multiple gold standard annotations representing various annotation styles. This corpus is used to evaluate a series of Named Entity Linking tools in order to understand the impact of the differences in annotation styles on the reported accuracy when processing highly ambiguous entities such as names of creative works. Relaxed annotation guidelines that include overlap styles lead to better results across all tools.

2019

pdf bib
Improving Named Entity Linking Corpora Quality
Albert Weichselbraun | Adrian M.P. Brasoveanu | Philipp Kuntschik | Lyndon J.B. Nixon
Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2019)

Gold standard corpora and competitive evaluations play a key role in benchmarking named entity linking (NEL) performance and driving the development of more sophisticated NEL systems. The quality of the used corpora and the used evaluation metrics are crucial in this process. We, therefore, assess the quality of three popular evaluation corpora, identifying four major issues which affect these gold standards: (i) the use of different annotation styles, (ii) incorrect and missing annotations, (iii) Knowledge Base evolution, (iv) and differences in annotating co-occurrences. This paper addresses these issues by formalizing NEL annotations and corpus versioning which allows standardizing corpus creation, supports corpus evolution, and paves the way for the use of lenses to automatically transform between different corpus configurations. In addition, the use of clearly defined scoring rules and evaluation metrics ensures a better comparability of evaluation results.

2018

pdf bib
Framing Named Entity Linking Error Types
Adrian Braşoveanu | Giuseppe Rizzo | Philipp Kuntschik | Albert Weichselbraun | Lyndon J.B. Nixon
Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)

2016

pdf bib
A Regional News Corpora for Contextualized Entity Discovery and Linking
Adrian Braşoveanu | Lyndon J.B. Nixon | Albert Weichselbraun | Arno Scharl
Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)

This paper presents a German corpus for Named Entity Linking (NEL) and Knowledge Base Population (KBP) tasks. We describe the annotation guideline, the annotation process, NIL clustering techniques and conversion to popular NEL formats such as NIF and TAC that have been used to construct this corpus based on news transcripts from the German regional broadcaster RBB (Rundfunk Berlin Brandenburg). Since creating such language resources requires significant effort, the paper also discusses how to derive additional evaluation resources for tasks like named entity contextualization or ontology enrichment by exploiting the links between named entities from the annotated corpus. The paper concludes with an evaluation that shows how several well-known NEL tools perform on the corpus, a discussion of the evaluation results, and with suggestions on how to keep evaluation corpora and datasets up to date.

2012

pdf bib
Leveraging the Wisdom of the Crowds for the Acquisition of Multilingual Language Resources
Arno Scharl | Marta Sabou | Stefan Gindl | Walter Rafelsberger | Albert Weichselbraun
Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12)

Games with a purpose are an increasingly popular mechanism for leveraging the wisdom of the crowds to address tasks which are trivial for humans but still not solvable by computer algorithms in a satisfying manner. As a novel mechanism for structuring human-computer interactions, a key challenge when creating them is motivating users to participate while generating useful and unbiased results. This paper focuses on important design choices and success factors of effective games with a purpose. Our findings are based on lessons learned while developing and deploying Sentiment Quiz, a crowdsourcing application for creating sentiment lexicons (an essential component of most sentiment detection algorithms). We describe the goals and structure of the game, the underlying application framework, the sentiment lexicons gathered through crowdsourcing, as well as a novel approach to automatically extend the lexicons by means of a bootstrapping process. Such an automated extension further increases the efficiency of the acquisition process by limiting the number of terms that need to be gathered from the game participants.

2006

pdf bib
Web coverage of the 2004 US Presidential election
Arno Scharl | Albert Weichselbraun
Proceedings of the 2nd International Workshop on Web as Corpus