Image Based Rendering and Illumination Using Spherical Mosaics

Chen Shen,Heung-Yeung Shum,James F. O'Brien

2001-01-01

Abstract:Concentric mosaics represent a scene using a series of sample images captured along a circular path looking outward. Spheri- cal mosaics simply extend this approach by using a set of sam- ple images that are taken from locations distributed over the surface of a sphere. Within an inner sphere, whose radius is de- termined by the sample camera's field of view, any exiting ray can be mapped to a location in one of the sample images. The mapping is accomplished by intersecting the exiting ray with the capture sphere to determine the nearest sample cam- eras. A point along the ray at a constant depth is then projected back to the center of projection for each camera to determine the closest pixels within each image. Linear interpolation of the resulting values yields the value for the exiting ray. If depth es- timates are available, they may be used to improve the accuracy of the projection into the sample images. Because the sample images form a four-dimensional representation of the external light field, the virtual camera is afforded a full six degrees of freedom within the inner sphere. DIFFUSE AND SPECULAR MAPS Once a scene has been sampled, adding additional objects into the environment requires realistically replicating environmen- tal illumination when shading the new object. If the sample images have high dynamic range with pixel values that record incident radiance, then the illumination at any point within the inner sphere can be determined from the sampled data. Un- fortunately, shading calculations at a point on the surface of the new object require expensive summations over all the incoming ray directions. To achieve interactive rendering speeds, we move the summa- tions to a preprocessing step and implement them by filtering the sampled data to form diffuse and specular maps. Both of these maps are stored as spherical mosaics. A mipmap-like structure holds multiple versions of the specular map computed with different sized kernels that can be used for different specu- lar falloff parameters. The diffuse map is indexed according to surface position and normal, while the specular map is indexed by surface position and the reflected viewing direction. Simple