Abstract:Polarisation has proven to be an effective method in the analysis of light reflection in computer vision. It plays an important role in separating reflectance components using cross polarisation settings. Some polarisation based methods have also been presented in the literature for surface orientation estimation, where the incident light is unpolarised. This thesis aims at exploiting the polarising properties of surface reflection for computer vision applications. In particular, we focus on developing a framework for measuring the shape and reflectance information based on polarisation data. The polarised images are acquired using simple and low-cost devices, and the estimates are satisfactory compared with other surface characteristic methods such as shape from shading and stereo approaches. Statistical methods are also used to aid the computations, such as blind source separation (BSS), mutual information estimation, the iterated conditional modes method, etc. Chapter 2 surveys the related literature in the fields of polarisation, reflectance function analysis, shape recovery, and pattern recognition. We also discuss other shape recovery techniques, such as shape from shading, photometric stereo, and geometric stereo. We make comparisons between these approaches to show their advantages and drawbacks. In Chapter 3, we introduce the method of BSS, and show how to incorporate it into polarisation state estimation. The traditional method of using polarisation models for shape recovery and polarisation state estimation usually requires more than three images, normally 10 or more pictures in different polarisation angles. The proposed method solves these problems, and provides a robust way to measure polarisation state and refractive index information using only three images, and without the need to have polarisation angle information. We also demonstrate in the experimental section that the estimates using the proposed method offer significant improvements over existing polarisation based approaches. In Chapter 4, we extend the work in the Chapter 3 in a number of ways. Firstly, we present detailed review of the Fresnel theory in polarisation vision, and propose a novel polarisation model which considers both diffuse and specular reflections. Secondly, we develop a new novel criterion functions to be used in the Chapter 3, which improves its robustness and accuracy. In Chapter 5, we also explore the shape recovery method using the polarised light. When the polarised light is transmitted through the polariser, it becomes fully polarised. The specular reflection part can be fully eliminated using the setting of ``cross-polarisation''. In this case the methodology is changed accordingly. This leads us to develop a new framework which firstly uses BSS for reflectance component separation, and then applies a polarisation model built specifically for this case. The polarisation model is based on Fresnel equation and Malus's law, which describes the reflection of polarised light when transmitting through the polariser in front of the viewer. It is also noted that the proposed framework reduces the noise in the zenith angle estimate, which we demonstrate in the experiments. In Chapter 6 we consider an alternative way to estimate shape and reflectance properties using polarised light. Using the separated diffuse and specular reflection components obtained in Chapter 5, we use parametric reflectance models, that relate the reflectance intensity with the zenith angle when the parameter values are given. The reflectance models are only applied for either diffuse or specular reflections, and the two zenith angle estimates from the two components should be identical as they describe the same object. We use mutual information estimation and Newton's method to find their similarities and their parameter values measured by model fitting. The two estimates then are combined using a mixture model. We choose six reflectance models and compare their performance in this framework.

Variational Specular Separation Using Color and Polarization.

Chromaticity-based separation of reflection components in a single image

Simple and Efficient Method for Specularity Removal in an Image

Polarization Guided Specular Reflection Separation

Reflection Component Separation Using Statistical Analysis and Polarisation.

Saturation-preserving Specular Reflection Separation

Highlight Specular Reflection Separation based on Tensor Low-rank and Sparse Decomposition Using Polarimetric Cues

Separation Of Diffuse And Specular Reflection In Color Images

Polarization-based extracting diffuse reflection from light-field of object surface

Reflection Separation Using a Pair of Unpolarized and Polarized Images.

Snapshot polarimetric diffuse-specular separation

Physics-Guided Reflection Separation from a Pair of Unpolarized and Polarized Images.

Shape and Reflectance Estimation from Dielectric Materials Using Statistical Analysis and Polarisation

Polarimetric Light Transport Analysis for Specular Inter-reflection

Separating Reflections From A Single Image Using Spatial Smoothness And Structure Information

Spin-Weighted Spherical Harmonics for Polarized Light Transport

Specular Highlight Removal In Facial Images

Specular highlight removal and depth estimation based on polarization characteristics of light field

Object Segmentation Based on Refractive Index Estimated by Polarization of Specular Reflection