BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//UC Irvine//CML Seminars//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:Breaking the Curse of Multilinguality in Language Models
X-WR-TIMEZONE:America/Los_Angeles
BEGIN:VTIMEZONE
TZID:America/Los_Angeles
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:20070311T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:20071104T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
UID:2024-03-07-terra-blevins@cml.ics.uci.edu
DTSTAMP:20240307T000000Z
SEQUENCE:57690
DTSTART;TZID=America/Los_Angeles:20240307T110000
DTEND;TZID=America/Los_Angeles:20240307T120000
SUMMARY:[CML Seminar] Terra Blevins: Breaking the Curse of Multilinguality
in Language Models
LOCATION:Donald Bren Hall 4011
DESCRIPTION:Terra Blevins\, PhD Student\, School of Computer Science and En
gineering\, University of Washington\n\nTitle: Breaking the Curse of Multi
linguality in Language Models\n\nAbstract: While language models (or LMs\,
à la ChatGPT) have become the predominant tool in natural language proce
ssing\, their performance in non-English languages increasingly lags behin
d. This gap is due to the curse of multilinguality\, which harms individua
l language performance in multilingual models through inter-language compe
tition for model capacity. In this talk\, I examine how current language m
odels do and don't capture different languages and present new methods for
fair modeling of all languages. First\, I demonstrate how LMs become mult
ilingual through their data and training dynamics. I then characterize whe
n multilingual models learn (and forget) languages during training to unco
ver how the curse of multilinguality develops. These analyses provide key
insights into developing more equitable multilingual models\, and I propos
e a new language modeling approach for Cross-Lingual Expert Language Model
s (X-ELM) that explicitly allocates model resources to reduce language com
petition.\n\nhttps://cml.ics.uci.edu/seminars/2024-03-07-terra-blevins
X-ALT-DESC;FMTTYPE=text/html:Terra Blevins\, PhD Student
\, School of Computer Science and Engineering\, University of Washington
Title: Breaking the Curse of Multilinguality in Language Mode
ls
Abstract: While language models (or LMs\, à la ChatGPT)
have become the predominant tool in natural language processing\, their pe
rformance in non-English languages increasingly lags behind. This gap is d
ue to the curse of multilinguality\, which harms individual language perfo
rmance in multilingual models through inter-language competition for model
capacity. In this talk\, I examine how current language models do and don
't capture different languages and present new methods for fair modeling o
f all languages. First\, I demonstrate how LMs become multilingual through
their data and training dynamics. I then characterize when multilingual m
odels learn (and forget) languages during training to uncover how the curs
e of multilinguality develops. These analyses provide key insights into de
veloping more equitable multilingual models\, and I propose a new language
modeling approach for Cross-Lingual Expert Language Models (X-ELM) that e
xplicitly allocates model resources to reduce language competition.
https
://cml.ics.uci.edu/seminars/2024-03-07-terra-blevins
URL:https://cml.ics.uci.edu/seminars/2024-03-07-terra-blevins
END:VEVENT
END:VCALENDAR