<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="badola-etal-2023-parameter">
    <titleInfo>
        <title>Parameter-Efficient Finetuning for Robust Continual Multilingual Learning</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Kartikeya</namePart>
        <namePart type="family">Badola</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Shachi</namePart>
        <namePart type="family">Dave</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Partha</namePart>
        <namePart type="family">Talukdar</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: ACL 2023</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Anna</namePart>
            <namePart type="family">Rogers</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jordan</namePart>
            <namePart type="family">Boyd-Graber</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Naoaki</namePart>
            <namePart type="family">Okazaki</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Toronto, Canada</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>We introduce and study the problem of Continual Multilingual Learning (CML) where a previously trained multilingual model is periodically updated using new data arriving in stages. If the new data is present only in a subset of languages, we find that the resulting model shows improved performance only on the languages included in the latest update (and a few closely related languages) while its performance on all the remaining languages degrade significantly. We address this challenge by proposing LAFT-URIEL, a parameter-efficient finetuning strategy which aims to increase the number of languages on which the model improves after an update, while reducing the magnitude of loss in performance for the remaining languages. LAFT-URIEL uses linguistic knowledge to balance overfitting and knowledge sharing across languages, allowing for an additional 25% of task languages to see an improvement in performance after an update, while also reducing the average magnitude of losses on the remaining languages by 78% relative.</abstract>
    <identifier type="citekey">badola-etal-2023-parameter</identifier>
    <identifier type="doi">10.18653/v1/2023.findings-acl.619</identifier>
    <location>
        <url>https://aclanthology.org/2023.findings-acl.619</url>
    </location>
    <part>
        <date>2023-07</date>
        <extent unit="page">
            <start>9763</start>
            <end>9780</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Parameter-Efficient Finetuning for Robust Continual Multilingual Learning
%A Badola, Kartikeya
%A Dave, Shachi
%A Talukdar, Partha
%Y Rogers, Anna
%Y Boyd-Graber, Jordan
%Y Okazaki, Naoaki
%S Findings of the Association for Computational Linguistics: ACL 2023
%D 2023
%8 July
%I Association for Computational Linguistics
%C Toronto, Canada
%F badola-etal-2023-parameter
%X We introduce and study the problem of Continual Multilingual Learning (CML) where a previously trained multilingual model is periodically updated using new data arriving in stages. If the new data is present only in a subset of languages, we find that the resulting model shows improved performance only on the languages included in the latest update (and a few closely related languages) while its performance on all the remaining languages degrade significantly. We address this challenge by proposing LAFT-URIEL, a parameter-efficient finetuning strategy which aims to increase the number of languages on which the model improves after an update, while reducing the magnitude of loss in performance for the remaining languages. LAFT-URIEL uses linguistic knowledge to balance overfitting and knowledge sharing across languages, allowing for an additional 25% of task languages to see an improvement in performance after an update, while also reducing the average magnitude of losses on the remaining languages by 78% relative.
%R 10.18653/v1/2023.findings-acl.619
%U https://aclanthology.org/2023.findings-acl.619
%U https://doi.org/10.18653/v1/2023.findings-acl.619
%P 9763-9780

Markdown (Informal)

[Parameter-Efficient Finetuning for Robust Continual Multilingual Learning](https://aclanthology.org/2023.findings-acl.619) (Badola et al., Findings 2023)

• Parameter-Efficient Finetuning for Robust Continual Multilingual Learning (Badola et al., Findings 2023)

ACL

• Kartikeya Badola, Shachi Dave, and Partha Talukdar. 2023. Parameter-Efficient Finetuning for Robust Continual Multilingual Learning. In Findings of the Association for Computational Linguistics: ACL 2023, pages 9763–9780, Toronto, Canada. Association for Computational Linguistics.