Stochastic Gradient Descent Methods with Biased Estimators


Quoc Tran-Dinh

University of North Carolina
Fri, Mar 5 2021 - 10:00 am (GMT + 7)
About Speaker

Quoc Tran-Dinh is currently with the Department of Statistics and Operations Research, The University of North Carolina at Chapel Hill, USA. Previously, he was a faculty member at Faculty of Mathematics, Mechanics, and Informatics, VNU University of Science in Hanoi. He obtained his Bachelor and MSc from Vietnam National University in Hanoi, and his Ph.D. from Department of Electrical Engineering and Optimization in Engineering Center, KU Leuven, Belgium. He was a postdoctoral researcher at Ecole Polytechnique Federale de Lausanne (EPFL), Switzerland, before joining UNC-Chapel Hill in 2015. His research mainly focuses on numerical methods for continuous optimization, including convex, nonconvex, and stochastic optimization and related problems. He currently serves as an associate editor of the Computational Optimization and Applications (COAP) and Mathematical Programming Computation (MPC) journals.


The gradient descent algorithm is perhaps one of the most popular methods in practice, especially in machine learning. In the last two decades, research on this topic has become extremely active, leading to various algorithms and interesting theoretical results. In the first part of this talk, I will take an opportunity to briefly discuss some recent progress on this topic, including algorithms, variants, and practical and theoretical aspects. In the second part of my talk, I will present some of our recent results on stochastic gradient-based methods for large-scale optimization and minimax problems using biased estimators.

These methods can potentially be applied to deep learning, statistical learning, generative adversarial nets, and federated learning. This talk is based on the collaboration with Nhan Pham (UNC), Deyi Liu (UNC), Lam M. Nguyen (IBM), and Dzung Phan (IBM).

Related seminars

Trieu Trinh

Google Deepmind

AlphaGeometry: Solving IMO Geometry without Human Demonstrations
Fri, Jul 5 2024 - 10:00 am (GMT + 7)

Tat-Jun (TJ) Chin

Adelaide University

Quantum Computing in Computer Vision: A Case Study in Robust Geometric Optimisation
Fri, Jun 7 2024 - 11:00 am (GMT + 7)

Fernando De la Torre

Carnegie Mellon University

Human Sensing for AR/VR
Wed, Apr 24 2024 - 07:00 am (GMT + 7)

Anh Nguyen

Microsoft GenAI

The Revolution of Small Language Models
Fri, Mar 8 2024 - 02:30 pm (GMT + 7)