SEMINAR

Learning and amortized inference in probabilistic programs

Speaker

Tuan Anh Le

Working
Massachusetts Institute of Technology
Timeline
Mon, Jan 6 2020 - 10:00 am (GMT + 7)
About Speaker

Tuan Anh Le is a postdoctoral associate in Josh Tenenbaum’s Computational Cognitive Science Lab in the Department of Brain and Cognitive Sciences at MIT. Previously, he was a PhD student in Frank Wood’s group in the Department of Engineering Science at the University of Oxford. He’s also interned at DeepMind, where he’s worked with Yee Whye Teh.

Abstract

Probabilistic programming is a powerful abstraction layer for Bayesian inference, separating the modeling and inference. However, approximate inference algorithms for general-purpose probabilistic programming systems are typically slow and must be re-run for each new observation. Amortized inference refers to learning an inference network ahead of time, so that for each new observation, an approximate posterior can be quickly obtained. In the first part of the talk, I’ll talk about reweighted wake-sleep (RWS), which is a simple, yet powerful method for simultaneously learning model parameters and amortizing inference that supports discrete latent variables and doesn’t suffer from the “tighter bounds” effect. In the second part of the talk, I’ll talk about the thermodynamic variational objective (TVO) which is a novel objective for learning and amortized inference based on a connection between thermodynamic integration and variational inference. Both of these methods have traditionally been used for learning deep generative models, however they’re equally applicable to a wider class of models like probabilistic programs. Finally, these methods have a lot of connections with statistics: RWS can be viewed as adaptive importance sampling and the TVO is directly derived from thermodynamic integration which is often used for model selection via Bayes factors.

Related seminars

Coming soon

Tim Baldwin

MBZUAI, The University of Melbourne

Safe, open, locally-aligned language models
Mon, Dec 16 2024 - 02:00 pm (GMT + 7)
Coming soon

Alessio Del Bue

Italian Institute of Technology (IIT)

From Spatial AI to Embodied AI: The Path to Autonomous Systems
Mon, Dec 16 2024 - 10:00 am (GMT + 7)

Dr. Xiaoming Liu

Michigan State University

Person Recognition at a Distance
Mon, Dec 9 2024 - 10:00 am (GMT + 7)

Dr Lan Du

Monash University

Uncertainty Estimation for Multi-view/Multimodal Data
Fri, Dec 6 2024 - 10:00 am (GMT + 7)