Upcoming talk · AI/ML Seminar Series
Prediction, Confidence, and Calibration in the Age of Deep Learning
Distinguished Professor, Hasso Plattner Endowed Chair in Artificial Intelligence, Computer Science, UC Irvine
- Date & time
- Thursday, November 12, 2026 · 11:00 AM
- Location
- Donald Bren Hall 4011
Abstract
A natural question in the deployment of modern machine learning models (such as LLMs) is whether they can accurately self-assess confidence in their predictions or responses: do they know what they don’t know? Using this question as a starting point, this talk will discuss uncertainty quantification and calibration properties of probabilistic predictions for modern deep learning models. We will begin by reviewing classic ideas from the statistical and forecasting literature on calibration. The main section of the talk will then discuss how these ideas have been adapted and applied to machine learning contexts, including classification models, language models, and time-series foundation models. The talk will include discussion of current and recent research projects on calibration in the Smyth research group as well as some thoughts on how calibrated confidence can potentially be used in real-world applications such as agentic AI systems and human-AI interaction.
About the speaker
Padhraic Smyth is a Distinguished Professor and Hasso Plattner Endowed Chair in the Department of Computer Science at UC Irvine, with joint appointments in the Department of Statistics and the School of Education. His research interests include machine learning, artificial intelligence, and statistical modeling. He is a Fellow of ACM, AAAI, IEEE, and AAAS and serves in senior chair positions for conferences such as NeurIPS, ICML, UAI, and AI & Statistics. He regularly consults with industry and was advisor to Netflix for the Netflix Prize. He received his undergraduate degree in electronic engineering from University College Galway in Ireland in 1984, and the MS and PhD degrees (in 1985 and 1988) in Electrical Engineering from the California Institute of Technology.