Past talk · AI/ML Seminar Series
Out-of-Distribution Evaluation: The How, the Which, and the What?!
Assistant Professor of Computer Science, University of Southern California
- Date & time
- Monday, May 16, 2022 · 1:00 PM
- Location
- Donald Bren Hall 4011
Abstract
NLP models have achieved impressive accuracies on in-distribution benchmarks but are unreliable out-of-distribution (OOD). In this talk, I preview my group's ongoing work on evaluating and improving model performance in OOD settings. First, I propose likelihood splits, a general-purpose way to create challenging non-i.i.d. benchmarks by measuring generalization to the tail of the data distribution. Second, I describe the advantages of neurosymbolic approaches over end-to-end pretrained models for OOD generalization in visual question answering. Finally, I show how synthesized examples can improve open-set recognition.
About the speaker
Robin Jia is an Assistant Professor of Computer Science at USC. He received his Ph.D. from Stanford University, advised by Percy Liang, and has spent time as a visiting researcher at Facebook AI Research. He is interested in natural language processing and machine learning, with a focus on building NLP systems robust to distribution shift. His work has received best paper awards at ACL and EMNLP.