EECS Seminar Spring 2025 - Dr. Kevin Duh, Johns Hopkins University

Welcome to the department of Electrical Engineering and Computer Science Seminar Spring 2025. We would like to invite you to join our upcoming seminar on Friday, March 28, 2022 at 11:00AM (EST). Our speaker for this seminar will be Dr. Kevin Duh from the Johns Hopkins University.

Title: Where does In-context Learning Happen in Large Language Models?

Abstract: In-context learning (ICL) refers to the phenomenon in which large language models (LLMs) perform tasks with no gradient updates when shown task examples in the prompt. Why ICL works is still a mystery; little is known about where the model locates the task with respect to prompt instructions and examples. In this work, we attempt to characterize the region where LLMs transition from recognizing the task to performing the task. Through a series of layer-wise context-masking experiments on LLAMA3 and STARCODER on Machine Translation and Code Generation, we demonstrate evidence of a "task recognition" point where the task is encoded into the input representations and attention to context is no longer necessary. This is joint work with Suzanna Sia and David Mueller.

Biography: Dr. Kevin Duh is a senior research scientist at the Johns Hopkins University Human Language Technology Center of Excellence (JHU HLTCOE). He is also an associate research professor in the Department of Computer Science and a member of the Center for Language and Speech Processing (CLSP). Previously, he was assistant professor at the Nara Institute of Science and Technology (2012-2015) and research associate at NTT CS Labs (2009-2012). He received his B.S. in 2003 from Rice University, and PhD in 2009 from the University of Washington, both in Electrical Engineering. His research interests lie at the intersection of Natural Language Processing and Machine Learning, in particular in areas relating to machine translation and multilingual applications.

Location: Pangborn Building - Scullen room | Virtual Zoom: https://bit.ly/3RiIZkB 
Joining our mailing list (optional): https://bit.ly/3iMe8Lg
EECS Seminar: https://engineering.catholic.edu/eecs/seminar