Winter 2022

Standard

Live Stream for all Winter 2022 CML Seminars

January 3
No Seminar
January 10
Live Stream
1 pm

Roy Fox

Assistant Professor
Department of Computer Science
University of California, Irvine

YouTube Stream: https://youtu.be/ImvsK5CFp0w

Ensemble methods for reinforcement learning have gained attention in recent years, due to their ability to represent model uncertainty and use it to guide exploration and to reduce value estimation bias. We present MeanQ, a very simple ensemble method with improved performance, and show how it reduces estimation variance enough to operate without a stabilizing target network. Curiously, MeanQ is theoretically *almost* equivalent to a non-ensemble state-of-the-art method that it significantly outperforms, raising questions about the interaction between uncertainty estimation, representation, and resampling.
In adversarial environments, where a second agent attempts to minimize the first’s rewards, double-oracle (DO) methods grow a population of policies for both agents by iteratively adding the best response to the current population. DO algorithms are guaranteed to converge when they exhaust all policies, but are only effective when they find a small population sufficient to induce a good agent. We present XDO, a DO algorithm that exploits the game’s sequential structure to exponentially reduce the worst-case population size. Curiously, the small population size that XDO needs to find good agents more than compensates for its increased difficulty to iterate with a given population size.

Bio: Roy Fox is an Assistant Professor and director of the Intelligent Dynamics Lab at the Department of Computer Science at UCI. He was previously a postdoc in UC Berkeley’s BAIR, RISELab, and AUTOLAB, where he developed algorithms and systems that interact with humans to learn structured control policies for robotics and program synthesis. His research interests include theory and applications of reinforcement learning, algorithmic game theory, information theory, and robotics. His current research focuses on structure, exploration, and optimization in deep reinforcement learning and imitation learning of virtual and physical agents and multi-agent systems.
January 17
No Seminar (Martin Luther King, Jr. Day)
January 24
Live Stream
1 pm

Ransalu Senanayake

Postdoctoral Scholar
Department of Computer Science
Stanford University

YouTube Stream: https://youtu.be/3yR8BqBElXw

Autonomous agents such as self-driving cars have already gained the capability to perform individual tasks such as object detection and lane following, especially in simple, static environments. While advancing robots towards full autonomy, it is important to minimize deleterious effects on humans and infrastructure to ensure the trustworthiness of such systems. However, for robots to safely operate in the real world, it is vital for them to quantify the multimodal aleatoric and epistemic uncertainty around them and use that uncertainty for decision-making. In this talk, I will talk about how we can leverage tools from approximate Bayesian inference, kernel methods, and deep neural networks to develop interpretable autonomous systems for high-stakes applications.

Bio: Ransalu Senanayake is a postdoctoral scholar in the Statistical Machine Learning Group at the Department of Computer Science, Stanford University. He focuses on making downstream applications of machine learning trustworthy by quantifying uncertainty and explaining the decisions of such systems. Currently, he works with Prof. Emily Fox and Prof. Carlos Guestrin. He also worked on decision-making under uncertainty with Prof. Mykel Kochenderfer. Prior to joining Stanford, Ransalu obtained a PhD in Computer Science from the University of Sydney, Australia, and an MPhil in Industrial Engineering and Decision Analytics from the Hong Kong University of Science and Technology, Hong Kong.
January 31
Live Stream
1 pm

Dylan Slack

PhD Student
Department of Computer Science
University of California, Irvine

YouTube Stream: TBD

To be announced.

Bio: TBD.
February 7
Live Stream
1 pm

Maja Rudolph

Senior Research Scientist
Bosch Center for AI

YouTube Stream: TBD

To be announced.

Bio: TBD.
February 14
Live Stream
1 pm

Ruiqi Gao

Research Scientist
Google Brain

YouTube Stream: TBD

To be announced.

Bio: TBD.
February 21
No Seminar (Presidents’ Day)
February 28
Live Stream
1 pm
To be announced.
March 7
Live Stream
1 pm
To be announced.
March 14
No Seminar (Finals Week)