Past talk · AI/ML Seminar Series
Enabling Language Models to Process Information at Scale
PhD Student, Department of Computer Science, Princeton University
- Date & time
- Thursday, March 13, 2025 · 11:00 AM
- Location
- Donald Bren Hall 4011
Abstract
Language models (LMs) can effectively internalize knowledge from vast amounts of pre-training data, enabling them to achieve remarkable performance on exam-style benchmarks. Expanding their ability to compile, synthesize, and reason over large volumes of information on the fly will further unlock transformative applications, ranging from AI literature assistants to generative search engines. In this talk, I will present my research on advancing LMs for processing information at scale. (1) I will present my evaluation framework for LM-based information-seeking systems, emphasizing the importance of providing citations for verifying the model-generated answers. (2) I will then introduce my foundational work on using contrastive learning to produce high-performing text embeddings, which form the cornerstone of effective and scalable search. (3) In addition to building systems that can process large-scale information, I will discuss my contributions to creating efficient pre-training and customization methods for LMs. Finally, I will share my vision for the next generation of autonomous information processing systems.
About the speaker
Tianyu Gao is a fifth-year PhD student in the Department of Computer Science at Princeton University, advised by Danqi Chen. His research focuses on developing principled methods for training and adapting language models, many of which have been widely adopted across academia and industry. He led the first workshop on long-context foundation models at ICML 2024. He won an outstanding paper award at ACL 2022 and received an IBM PhD Fellowship in 2023. Before Princeton, he received his BEng from Tsinghua University in 2020.