OCR Post Correction for Endangered Language Texts

Shruti Rijhwani, Antonios Anastasopoulos, Graham Neubig


Abstract
There is little to no data available to build natural language processing models for most endangered languages. However, textual data in these languages often exists in formats that are not machine-readable, such as paper books and scanned images. In this work, we address the task of extracting text from these resources. We create a benchmark dataset of transcriptions for scanned books in three critically endangered languages and present a systematic analysis of how general-purpose OCR tools are not robust to the data-scarce setting of endangered languages. We develop an OCR post-correction method tailored to ease training in this data-scarce setting, reducing the recognition error rate by 34% on average across the three languages.
Anthology ID:
2020.emnlp-main.478
Volume:
Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
Month:
November
Year:
2020
Address:
Online
Editors:
Bonnie Webber, Trevor Cohn, Yulan He, Yang Liu
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5931–5942
Language:
URL:
https://aclanthology.org/2020.emnlp-main.478
DOI:
10.18653/v1/2020.emnlp-main.478
Bibkey:
Cite (ACL):
Shruti Rijhwani, Antonios Anastasopoulos, and Graham Neubig. 2020. OCR Post Correction for Endangered Language Texts. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5931–5942, Online. Association for Computational Linguistics.
Cite (Informal):
OCR Post Correction for Endangered Language Texts (Rijhwani et al., EMNLP 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.emnlp-main.478.pdf
Video:
 https://slideslive.com/38939129
Code
 shrutirij/ocr-post-correction +  additional community code