MSc Dissertation Proposal 2016/2017

Disparity Estimation using Plenoptic Cameras

-- Project Description at Fenix:

Objectives

The appearance of the first commercial versions of plenoptic cameras (Lytro and Raytrix) had raise interest in these cameras by the research community. Also, some players like NVIDIA and Adobe had created prototypes using this technology to create new types of displays: near-eye displays and 3D displays. Recently, Lytro has launched a video camera for cinema that will benefit of the extra information given by the lightfield.

The objectives of this work are:

1. Disparity estimation using anisotropic structure tensor and epipolar volume.

2. Improve disparity estimation using a denoising method.

3. Evaluate and compare the disparity estimation accuracy obtained from the anisotropic structure tensor and the epipolar volume with the disparity estimation obtained from the isotropic structure tensor.

Requirements

N.A.

Place for conducting the work-proposal:

ISR / IST

Observations

Conventional cameras give information about the total amount of light that reaches each position of the sensor. In the process of image formation there is some information of the lightfield that is lost, for example, the direction and contribution of each ray to the total amount of light captured on an image. This information is preserved in plenoptic cameras.

On the other hand, plenoptic cameras make a trade-off between spatial and angular resolution which normally results in images with low spatial resolution. Therefore, plenoptic images are normally considered for superresolution techniques that require the knowledge of the disparity map. Since the disparity map is unknown, it must be recovered from the lightfield captured. Common techniques to estimate disparity are based on the epipolar geometry obtained from the lightfield in a single acquisition. These analyses add noise to the disparity estimation, therefore, regularization schemes are useful to improve the disparity estimations obtained using these methods. The knowledge of this disparity map is not only important for superresolution. The disparity map will aid on other tasks like 3D segmentation and will also allow to obtain 3D features that will aid in classification.

The workplan consists on the implementation of a denoising method and two variants of the existing algorithm for disparity and depth estimation using lightfield data: using an anisotropic structure tensor to obtain the disparity/depth estimates, and using epipolar volumes to obtain disparity/depth information. These algorithms should be applied to synthetic and real datasets and the results should be compared with the disparity/depth estimation using the isotropic structure tensor. The sensibility of these methods to noise should also be evaluated.

-- More Information

More MSc dissertation proposals on Computer and Robot Vision in:

http://omni.isr.ist.utl.pt/~jag