The Revolution of Small Language Models


Anh Nguyen

Microsoft GenAI
Fri, Mar 8 2024 - 02:30 pm (GMT + 7)
About Speaker

Anh Nguyen is a Researcher at Microsoft GenAI, he has been contributing to the Physics of AGI project which aims at understanding how intelligence emerges in large language models (LLMs) and use this understanding to improve that intelligence. Before that, he spent his time as an Applied Scientist at Microsoft Azure AI, where he has the opportunities to work on cutting-edge NLP and Deep Learning techniques to enable new applications and scenarios for Microsoft AI products and services such as GitHub Copilot, Office and Azure OpenAI.


In this talk, we delve into the world of small language models, focusing on the revolutionary Phi-2 model. We begin by exploring the evolution of language models, highlighting the challenges and limitations of large-scale models. We then introduce Phi-2, a small language model that has been making waves in the AI community due to its efficiency and versatility. We discuss the process of curating training data for Phi-2, explaining how it achieves comparable performance to larger models while significantly reducing computational requirements. We also explore the various applications of Phi-2, from natural language processing tasks to more complex problem-solving scenarios. The talk will also cover the ethical considerations of deploying small language models and the potential impact of Phi-2 on future AI developments. We conclude with a discussion on the future of small language models, emphasizing the role of Phi-2 in shaping this landscape. Join us as we unravel the revolution of small language models, brought about by the groundbreaking Phi-2.

Related seminars

Fernando De la Torre

Carnegie Mellon University

Human Sensing for AR/VR
Wed, Apr 24 2024 - 07:00 am (GMT + 7)

Thang D. Bui

Australian National University

Recent Progress on Grokking and Probabilistic Federated Learning
Fri, Jan 26 2024 - 10:00 am (GMT + 7)

Tim Baldwin

Mohamed bin Zayed University of Artificial Intelligence, University of Melbourne

Tue, Jan 9 2024 - 10:30 am (GMT + 7)

Quan Vuong

Google DeepMind

Scaling Robot Learning
Wed, Dec 27 2023 - 10:00 am (GMT + 7)