Todd Zickler

Toronto Vision Seminar

Todd Zickler, Professor of Electrical Engineering and Computer Science, Harvard University

Date: Wednesday, May 8, 2024

Time: 2 - 3 p.m.

Location: BA 3200 (reception to follow)

Talk Title: "Images as Fields of Junctions"

Abstract: Which pixels belong together, and where are the boundaries between them? My talk revisits these enduring questions about grouping, this time informed by the apparent strengths and weaknesses of modern foundation models. I focus on exactness and generality, aiming to localize boundaries with high spatial precision by exploiting low-level geometric consistencies between boundary curves, corners and junctions. Our approach represents the appearance of each small receptive field by a low-parameter, piece-wise smooth vector-graphics model (a "generalized junction"), and it iteratively decomposes an image into a dense spatial field of such models. This decomposition reveals precise edges, curves, corners, junctions, and boundary-aware smoothing---all at the same time. I present experiments showing this provides unprecedented resilience to image degradations, producing stable output at high noise levels where other methods fail. I also discuss recent work that accelerates the decomposition using a specialized form of spatial self-attention. At the open and close of the talk, I speculate about how these capabilities may help us close gaps between mid-level vision in animals and machines.

Bio: Todd Zickler received B.Eng. and Ph.D. degrees in electrical engineering from McGill University and Yale University. In 2004 he joined Harvard University, where he is a professor of electrical engineering and computer science. He is also a visiting faculty researcher at Google. His research models interactions between light, materials, optics and sensors, and it develops optical and computational systems to extract useful information from visual data. He is motivated by applications in autonomy, robotics, and augmented reality, and he enjoys the intersections of computer vision, computer graphics, signal processing, applied optics, biological vision, and human perception.