Towards Reliable Multi-step Reasoning in Question Answering


Niranjan Balasubramanian

Stony Brook University
Fri, Nov 3 2023 - 10:00 am (GMT + 7)
About Speaker

Niranjan is an Assistant Professor in the Computer Science department at Stony Brook University, where he heads the Language Understanding and Reasoning lab (LUNR). Prior to joining Stony Brook, he was a post-doctoral researcher in the University of Washington, and was one of the early members of the Allen Institute for Artificial Intelligence. Niranjan completed his PhD in Computer Science from the University of Massachusetts Amherst.


Multi-step has seen much empirical progress on many datasets recently, especially in Question Answering. However, training and evaluating on typical crowdsourced datasets is problematic because of the potential for shortcut reasoning based on artifacts. Even in large language models (LLMs) multi-step reasoning is challenging because of their tendency to hallucinate. What can we do about this? In this three part talk, I will first show how we can formalize and measure disconnected reasoning, a type of bad multihop reasoning. I will then discuss how we can construct new datasets using a bottom-up construction process, which allows us to better control for desired properties in the resulting dataset. In the second part, I will briefly present how synthetically generated data can be used to teach a broad range of multihop skills in a reliable manner and how to improve reliable multi-step reasoning in open-domain QA settings. And last, I will briefly discuss using iterative retrieval and reasoning as a means to reduce hallucination in LLM based QA.

Related seminars

Anh Nguyen

Microsoft GenAI

The Revolution of Small Language Models
Fri, Mar 8 2024 - 02:30 pm (GMT + 7)

Thang D. Bui

Australian National University (ANU)

Recent Progress on Grokking and Probabilistic Federated Learning
Fri, Jan 26 2024 - 10:00 am (GMT + 7)

Tim Baldwin

MBZUAI, The University of Melbourne

Tue, Jan 9 2024 - 10:30 am (GMT + 7)

Quan Vuong

Google DeepMind

Scaling Robot Learning
Wed, Dec 27 2023 - 10:00 am (GMT + 7)