Speaker

Akari Asai is a Ph.D. student in the Paul G. Allen School of Computer Science & Engineering at the University of Washington, advised by Prof. Hannaneh Hajishirzi. Her research lies in natural language processing and machine learning. Her recent research focuses on question answering / IR, multilingual NLP, retrieval-augmented LMs, and efficiency. She received the IBM Fellowship in 2022 and the Nakajima Foundation Fellowship in 2019 and is selected as a 2022 EECS Rising Star. Prior to UW, she obtained a B.E. degree in Electrical Engineering and Computer Science from the University of Tokyo.

Akari’s homepage: https://akariasai.github.io/

Abstract

Large language models (LLMs) have shown great success in various natural language processing (NLP) tasks, inspiring the development of real-world applications of LLMs, such as conversational search systems. However, these LLMs suffer from major issues such as hallucinations, staleness, and the need for prohibitively large models, due to their reliance on memorizing everything within their parameters (parametric memories). To address these limitations, I am developing efficient and trustworthy NLP systems with retrieval, which incorporate non-parametric memories stored as text. In this talk, I will discuss my recent work on analyzing and developing more reliable and efficient retrieval-augmented LMs. First, I will discuss our analysis of the memorization of LLMs and how non-parametric memories can complement the shortcomings of relying on parametric memories. Next, I will introduce our general-purpose retrieval systems, which enable the development of robust and competitive retrieval-augmented LMs. In particular, we introduce the first instruction-following retrieval system that can adapt to a new task with a prompt describing the task. Finally, I will describe our new generative model, which learns to focus on supportive non-parametric memories rather than irrelevant ones. I will conclude with a discussion of the remaining challenges and wider applications.

Video

Coming soon. Stay tuned. :-)