Rohit Kumar


2023

pdf bib
Whispering LLaMA: A Cross-Modal Generative Error Correction Framework for Speech Recognition
Srijith Radhakrishnan | Chao-Han Yang | Sumeer Khan | Rohit Kumar | Narsis Kiani | David Gomez-Cabrero | Jesper Tegnér
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

We introduce a new cross-modal fusion technique designed for generative error correction in automatic speech recognition (ASR). Our methodology leverages both acoustic information and external linguistic representations to generate accurate speech transcription contexts. This marks a step towards a fresh paradigm in generative error correction within the realm of n-best hypotheses. Unlike the existing ranking-based rescoring methods, our approach adeptly uses distinct initialization techniques and parameter-efficient algorithms to boost ASR performance derived from pre-trained speech and text models. Through evaluation across diverse ASR datasets, we assess our fusion technique, demonstrating a 37.66% improvement in word error rate (WER) relative performance compared to the n-best Oracle. To encourage future research, we have made our code and pre-trained models open source at [https://github.com/Srijith-rkr/Whispering-LLaMA](https://github.com/Srijith-rkr/Whispering-LLaMA)

pdf bib
ChatGPT_Powered_Tourist_Aid_Applications__Proficient_in_Hindi__Yet_To_Master_Telugu_and_Kannada
Sanjana Kolar | Rohit Kumar
Proceedings of the Third Workshop on Speech and Language Technologies for Dravidian Languages

This research investigates the effectiveness of Chat- GPT, an AI language model by OpenAI, in translating English into Hindi, Telugu, and Kannada languages, aimed at assisting tourists in India’s linguistically diverse environment. To measure the translation quality, a test set of 50 questions from diverse fields such as general knowledge, food, and travel was used. These were assessed by five volunteers for accuracy and fluency, and the scores were subsequently converted into a BLEU score. The BLEU score evaluates the closeness of a machine-generated translation to a human translation, with a higher score indicating better translation quality. The Hindi translations outperformed others, showcasing superior accuracy and fluency, whereas Telugu translations lagged behind. Human evaluators rated both the accuracy and fluency of translations, offering a comprehensive perspective on the language model’s performance.

2019

pdf bib
Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Industry Papers)
Anastassia Loukina | Michelle Morales | Rohit Kumar
Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Industry Papers)

2018

pdf bib
Phonetically Balanced Code-Mixed Speech Corpus for Hindi-English Automatic Speech Recognition
Ayushi Pandey | Brij Mohan Lal Srivastava | Rohit Kumar | Bhanu Teja Nellore | Kasi Sai Teja | Suryakanth V. Gangashetty
Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)

2015

pdf bib
Error-tolerant speech-to-speech translation
Rohit Kumar | Sanjika Hewavitharana | Nina Zinovieva | Matthew E. Roy | Edward Pattison-Gordon
Proceedings of Machine Translation Summit XV: Papers

2014

pdf bib
Anticipatory translation model adaptation for bilingual conversations
Sanjika Hewavitharana | Dennis Mehay | Sankaranarayanan Ananthakrishnan | Rohit Kumar | John Makhoul
Proceedings of the 11th International Workshop on Spoken Language Translation: Papers

Conversational spoken language translation (CSLT) systems facilitate bilingual conversations in which the two participants speak different languages. Bilingual conversations provide additional contextual information that can be used to improve the underlying machine translation system. In this paper, we describe a novel translation model adaptation method that anticipates a participant’s response in the target language, based on his counterpart’s prior turn in the source language. Our proposed strategy uses the source language utterance to perform cross-language retrieval on a large corpus of bilingual conversations in order to obtain a set of potentially relevant target responses. The responses retrieved are used to bias translation choices towards anticipated responses. On an Iraqi-to-English CSLT task, our method achieves a significant improvement over the baseline system in terms of BLEU, TER and METEOR metrics.

2013

pdf bib
Semi-Supervised Word Sense Disambiguation for Mixed-Initiative Conversational Spoken Language Translation
Sankaranarayanan Ananthakrishnan | Sanjika Hewavitharana | Rohit Kumar | Enoch Kan | Rohit Prasad | Prem Natarajan
Proceedings of Machine Translation Summit XIV: Papers

pdf bib
Interactive Error Resolution Strategies for Speech-to-Speech Translation Systems
Rohit Kumar | Matthew Roy | Sankaranarayanan Ananthakrishnan | Sanjika Hewavitharana | Frederick Choi
Proceedings of the SIGDIAL 2013 Conference

pdf bib
Source aware phrase-based decoding for robust conversational spoken language translation
Sankaranarayanan Ananthakrishnan | Wei Chen | Rohit Kumar | Dennis Mehay
Proceedings of the 10th International Workshop on Spoken Language Translation: Papers

Spoken language translation (SLT) systems typically follow a pipeline architecture, in which the best automatic speech recognition (ASR) hypothesis of an input utterance is fed into a statistical machine translation (SMT) system. Conversational speech often generates unrecoverable ASR errors owing to its rich vocabulary (e.g. out-of-vocabulary (OOV) named entities). In this paper, we study the possibility of alleviating the impact of unrecoverable ASR errors on translation performance by minimizing the contextual effects of incorrect source words in target hypotheses. Our approach is driven by locally-derived penalties applied to bilingual phrase pairs as well as target language model (LM) likelihoods in the vicinity of source errors. With oracle word error labels on an OOV word-rich English-to-Iraqi Arabic translation task, we show statistically significant relative improvements of 3.2% BLEU and 2.0% METEOR over an error-agnostic baseline SMT system. We then investigate the impact of imperfect source error labels on error-aware translation performance. Simulation experiments reveal that modest translation improvements are to be gained with this approach even when the source error labels are noisy.

2012

pdf bib
Active error detection and resolution for speech-to-speech translation
Rohit Prasad | Rohit Kumar | Sankaranarayanan Ananthakrishnan | Wei Chen | Sanjika Hewavitharana | Matthew Roy | Frederick Choi | Aaron Challenner | Enoch Kan | Arvid Neelakantan | Prem Natarajan
Proceedings of the 9th International Workshop on Spoken Language Translation: Papers

We describe a novel two-way speech-to-speech (S2S) translation system that actively detects a wide variety of common error types and resolves them through user-friendly dialog with the user(s). We present algorithms for detecting out-of-vocabulary (OOV) named entities and terms, sense ambiguities, homophones, idioms, ill-formed input, etc. and discuss novel, interactive strategies for recovering from such errors. We also describe our approach for prioritizing different error types and an extensible architecture for implementing these decisions. We demonstrate the efficacy of our system by presenting analysis on live interactions in the English-to-Iraqi Arabic direction that are designed to invoke different error types for spoken language translation. Our analysis shows that the system can successfully resolve 47% of the errors, resulting in a dramatic improvement in the transfer of problematic concepts.

2011

pdf bib
Comparing Triggering Policies for Social Behaviors
Rohit Kumar | Carolyn Rosé
Proceedings of the SIGDIAL 2011 Conference

2010

pdf bib
Engaging learning groups using Social Interaction Strategies
Rohit Kumar | Carolyn P. Rosé
Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics

2009

pdf bib
Building Conversational Agents with Basilica
Rohit Kumar | Carolyn P. Rosé | Michael J. Witbrock
Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Companion Volume: Demonstration Session

2007

pdf bib
Conquest—An Open-Source Dialog System for Conferences
Dan Bohus | Sergio Grau Puerto | David Huggins-Daines | Venkatesh Keri | Gopala Krishna | Rohit Kumar | Antoine Raux | Stefanie Tomko
Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Companion Volume, Short Papers