Leonardo Rigutini


2023

pdf bib
BUSTER: a “BUSiness Transaction Entity Recognition” dataset
Andrea Zugarini | Andrew Zamai | Marco Ernandes | Leonardo Rigutini
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track

Albeit Natural Language Processing has seen major breakthroughs in the last few years, transferring such advances into real-world business cases can be challenging. One of the reasons resides in the displacement between popular benchmarks and actual data. Lack of supervision, unbalanced classes, noisy data and long documents often affect real problems in vertical domains such as finance, law and health. To support industry-oriented research, we present BUSTER, a BUSiness Transaction Entity Recognition dataset. The dataset consists of 3779 manually annotated documents on financial transactions. We establish several baselines exploiting both general-purpose and domain-specific language models. The best performing model is also used to automatically annotate 6196 documents, which we release as an additional silver corpus to BUSTER.

pdf bib
Multi-word Tokenization for Sequence Compression
Leonidas Gee | Leonardo Rigutini | Marco Ernandes | Andrea Zugarini
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track

Large Language Models have proven highly successful at modelling a variety of tasks. However, this comes at a steep computational cost that hinders wider industrial uptake. In this paper, we present MWT: a Multi-Word Tokenizer that goes beyond word boundaries by representing frequent multi-word expressions as single tokens. MWTs produce a more compact and efficient tokenization that yields two benefits: (1) Increase in performance due to a greater coverage of input data given a fixed sequence length budget; (2) Faster and lighter inference due to the ability to reduce the sequence length with negligible drops in performance. Our results show that MWT is more robust across shorter sequence lengths, thus allowing for major speedups via early sequence truncation.

2022

pdf bib
Fast Vocabulary Transfer for Language Model Compression
Leonidas Gee | Andrea Zugarini | Leonardo Rigutini | Paolo Torroni
Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: Industry Track

Real-world business applications require a trade-off between language model performance and size. We propose a new method for model compression that relies on vocabulary transfer. We evaluate the method on various vertical domains and downstream tasks. Our results indicate that vocabulary transfer can be effectively used in combination with other compression techniques, yielding a significant reduction in model size and inference time while marginally compromising on performance.

2018

pdf bib
Enhancing Modern Supervised Word Sense Disambiguation Models by Semantic Lexical Resources
Stefano Melacci | Achille Globo | Leonardo Rigutini
Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)