Speaker:

Weijia Shi

Talk Title:

Breaking the Language Model Monolith

Date and Location:

Tuesday, March 10, 2026

Bahen Centre for Information Technology, BA 3200

This lecture is open to the public. No registration is required, but space is limited.

The grad roundtable that follows the talk is open only to current University of Toronto Department of Computer Science graduate students.

Abstract:

Language models (LMs) are typically monolithic: a single model storing all knowledge and serving every use case. This design presents significant challenges—they often generate factually incorrect statements, require costly retraining to add or remove information, and raise serious privacy and copyright issues. In this talk, I will discuss how to break this monolith by introducing modular architectures and training algorithms that separate capabilities across composable components. I’ll cover two forms of modularity: (1) external modularity, which augments LMs with external tools like retrievers to improve factuality and reasoning; and (2) internal modularity, which builds inherently modular LMs from decentrally trained components to enable flexible composition and an unprecedented level of control.

About Weijia Shi:

Weijia Shi is a Ph.D. candidate at the University of Washington, advised by Luke Zettlemoyer and Noah Smith. Her research focuses on augmented and modular architectures and training algorithms that make language models more adaptable and updatable. She received an Outstanding Paper Award at ACL 2024 and was recognized as a Rising Star in Machine Learning (2023) and in Data Science (2024).