MSc dissertation proposal 2018/2019

Combining Plenoptic Cameras with Depth Sensors for Reconstructing Textureless Surfaces

Objectives:

Conventional cameras give information about the total amount of light that reaches each position of the sensor. During image formation, there is some information of the light field that is lost, for example: (i) the direction of the ray and; (ii) the contribution of each ray to the total amount of light captured in an image.

Plenoptic cameras like Lytro and Raytrix preserve this information. The additional information allows retrieving 3D information from the scene using a single image. Reconstruction methods for plenoptic images are based on identifying line features on epipolar plane images or on shearing the lightfield according to some customized metric for feature matching. These algorithms are computationally demanding and are still not capable of giving accurate depth estimations for low-textured and occluded regions.

On the other hand, depth sensors are active sensors that project a structured light pattern into the scene. This allows obtaining depth measurements regardless of the texture of the scene. Nonetheless, standard depth sensors give depth estimates with lower accuracy in the near range (less than 1.5 m).

References:

Goals:

The objectives of this work are:

- Understanding the plenoptic and depth sensor camera models.

- Acquire a dataset of textureless objects with a depth sensor and plenoptic camera.

- Merge the acquisition of the depth sensors with the depth estimates from the plenoptic cameras.

- Compare depth maps obtained for textureless objects with conventional methods to retrieve depth.

References:

[Dansereau13] Dansereau, Donald G., Oscar Pizarro, and Stefan B. Williams. "Decoding, calibration and rectification for lenselet-based plenoptic cameras." Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on. IEEE, 2013.

[Herrera12] Herrera, Daniel, Juho Kannala, and Janne Heikkilä. "Joint depth and color camera calibration with distortion correction." IEEE Transactions on Pattern Analysis and Machine Intelligence 34.10 (2012): 2058-2064.

Requirements (grades, required courses, etc):

Interest in imaging sensors.

Place for conducting the work-proposal:

ISR / IST

More MSc dissertation proposals on Computer and Robot Vision in:

http://users.isr.tecnico.ulisboa.pt/~jag/msc/msc_2018_2019.html