A Signal-Processing Framework for Forward and Inverse Rendering

Understanding the nature of reflection and illumination is important in many areas of computer graphics and computer vision. In this talk, I describe a new way of looking at reflection on a curved surface, as a special type of convolution of the incident illumination and the reflective properties of the surface (technically, the bi-directional reflectance distribution function or BRDF). We formalize these notions by deriving a convolution theorem in terms of the spherical harmonic coefficients of the lighting and BRDF. This allows us to introduce a signal-processing framework for reflection, wherein the incident lighting is the signal, the BRDF is the filter, and the reflected light is obtained by filtering the input illumination (signal) using the frequency response of the BRDF filter. I will demonstrate applications in two areas. First, we show how our framework can be used for computing and displaying synthetic images in real-time with natural illumination and physically-based BRDFs. We will call this the "forward rendering" or the convolution problem. Next, we extend and apply our framework to estimating realistic lighting and reflective properties from photographs, and show how this approach can be used to synthesize very realistic images under novel lighting and viewing conditions. We will call this the "inverse rendering" or the deconvolution problem. In my talk, I will first describe the theoretical framework, and then discuss the above two applications.