Towards calibrated and flexible probabilistic deep learning

Deep learning has achieved great successes in many real-world domains, ranging from vision, language to game playing. Yet, it has been shown to possess many limitations, including: (i) it is not robust to out-of-distribution inputs and (ii) it suffers from catastrophic forgetting when faced with streaming data. In this talk, I will show how we have addressed some of these limitations by combining deep learning with probabilistic modelling. This combination provides desirable test-time uncertainty estimates on out-of-distribution data and allows neural networks to be trained in an incremental way. If time permits, I will show general distributed learning, also known as federated learning, can also be handled by the same algorithmic framework.