Department of Computer Science
Speaker: Alexandros Panagopoulos, Stony Brook University
Title: Illumination Estimation Using Graphical Models
The image formation process is a function of three components: scene geometry, surface reflectance and illumination. Estimating illumination has usually required exact knowledge or strong assumptions about the other two components. In this talk we discuss the estimation of illumination through graphical models. Our goals are to estimate illumination from a single image in real-world scenarios where the knowledge of scene geometry and reflectance are coarse and incomplete, and to enable the use of illumination as a component in larger scene understanding tasks, in order to infer more information about the scene. We estimate illumination parameters based on information in cast shadows, since the latter are a relatively stable cue when knowledge of geometry is inaccurate. We describe a simple image cue and two methods that utilize it to obtain an initial estimate of cast shadows. We then present methods to estimate illumination, first using an Expectation-Maximization approach, and then by modeling the creation of cast shadows as a higher-order Markov Random Field (MRF) that captures the interaction of illumination, geometry and image evidence for shadows. This MRF model provides not only robustness to coarse knowledge of geometry and inaccurate initial shadow estimates, but also allows us to directly incorporate geometry parameters in our model. It therefore allows the joint inference of almost every component of the problem, by estimating cast shadows, illumination and geometry parameters at the same time. Using this approach, as shown in our results, we can obtain convincing illumination estimates using the same geometry and camera parameters to model a whole class of images, and coarse object geometry and pose can be inferred jointly with shadows and illumination based on the 2D position and class of selected objects in the image. Finally, future directions and some extensions to this work are discussed.