Enriching Communication between Humans and AI Agents


Khanh Nguyen

Princeton University
Fri, Oct 7 2022 - 10:00 am (GMT + 7)
About Speaker

Khanh Nguyen is a postdoctoral researcher at Princeton university. His research focuses on the topics of grounded-interactive learning, with the goal of building AI agents that extend human capabilities through natural communication. He has published papers at top-tier conferences in various subfields of AI, including computer vision (CVPR), natural language processing (EMNLP), and core machine learning (ICML). Khanh obtained his Ph.D. degree in Computer Science at the University of Maryland, College Park.


In this talk, I will present new training and evaluation frameworks that enrich communication between humans and AI agents. My work focuses on improving two important capabilities of an agent: (1) the ability to learn from humans through language-based communication and (2) the ability to request and interpret information from humans to make better decisions. To improve the first capability, I propose a learning framework that allows agents to learn from descriptive human language. To enhance the second capability, I introduce new benchmarks that simulate and measure an agent’s ability to actively ask for and interpret information from humans to successfully complete tasks. On these benchmarks, I build agents that are capable of requesting rich, contextually useful information and show that they significantly outperform agents without such capability. I will conclude with a discussion on future directions to endow AI agents with more expressive communication capabilities and models of cognition.

Related seminars

Trieu Trinh

Google Deepmind

AlphaGeometry: Solving IMO Geometry without Human Demonstrations
Fri, Jul 5 2024 - 10:00 am (GMT + 7)

Tat-Jun (TJ) Chin

Adelaide University

Quantum Computing in Computer Vision: A Case Study in Robust Geometric Optimisation
Fri, Jun 7 2024 - 11:00 am (GMT + 7)

Fernando De la Torre

Carnegie Mellon University

Human Sensing for AR/VR
Wed, Apr 24 2024 - 07:00 am (GMT + 7)

Anh Nguyen

Microsoft GenAI

The Revolution of Small Language Models
Fri, Mar 8 2024 - 02:30 pm (GMT + 7)