Prompt-based Medical Image Processing
Tuesday, October 8, 2024, 11 a.m.
Bahen Centre for Information Technology, BA 3200
This lecture is open to the public. No registration is required, but space is limited.
Abstract:
Medical image analysis is central to many clinical goals (e.g., diagnosing, monitoring, and treating disease) and research aims (e.g., studying anatomical structure, function, and development). Historically, medical image processing has involved solving complex optimization problems. Machine learning offers the opportunity to develop accurate tools that are far more efficient. However, existing AI-based methods most often focus on building models that address specific image processing objectives while targeting specific regions of interest or diseases. This has led to the creation of many special-purpose models, posing a challenge to users who must navigate an expanding landscape of rigid and brittle tools, often failing to find one that meets their specific needs.
This talk will discuss our approach to building powerful and general ML-based models that can be used “out-of-the-box” by clinicians and researchers who have no machine learning expertise. A key challenge in building such a model is incorporating a way for users to specify exactly what medical imaging task they wish the model to perform. Consider, for example, a model that, given an MRI of a brain, is capable of segmenting any neuroanatomical structure, diagnosing a tumour, correcting for motion artifacts, etc. When a brain MRI is supplied as input to the model, how should the model to decide which task to perform? How should the model handle a task not seen in the training data? In response to this challenge, we have developed networks that exploit various prompting methods. In this talk we will discuss three approaches to prompting in medical image analysis—context-based, graphic-based, and language-based—together with the novel network architectures and training regimes that support them.
Bio:
John Guttag, a U of T graduate, is the Dugald C. Jackson Professor of Computer Science and Electrical Engineering at MIT. He leads MIT’s Computer Science and Artificial Intelligence Laboratory’s Clinical and Applied Machine Learning Group. The group develops and applies advanced machine learning and computer vision techniques to a variety of problems. Current research projects include medical imaging, language/image models, and robust deep learning. In addition to his academic activities, Prof. Guttag is CTO of Health at Scale Technologies. The company’s machine intelligence platforms are used to help manage care for millions of individuals.