MSc Dissertation Proposal 2016/2017

Structure Reconstruction using Plenoptic Cameras

-- Project Description at Fenix:

 

Objectives

 

The use of endoscopes in medical diagnosis and in minimal invasive surgery is a common technique adopted by doctors. The endoscopes have several limitations related with weak depth perception. Additionally, organs and tissues are non-rigid bodies which difficult the 3D reconstruction using standard methods. Plenoptic cameras allow to estimate depth from a single image by analysis of their epipolar geometry, which may allow to overcome some of the current endoscope limitations.

The objectives of this work are:

1.       - Plenoptic camera model definition and calibration.

2.       - Depth range limits definition for a plenoptic camera.

3.       - Depth estimation using the camera epipolar geometry.

4.      - Mosaicking and 3D reconstruction from a tubular shape and comparison with standard methods.

Requirements

N.A.

Place for conducting the work-proposal:

ISR / IST

Observations

 

The use of endoscopes in medical diagnosis and in minimal invasive surgery is a common technique adopted by doctors. The endoscopes have several limitations that include limited field of view and reduced capacity to perceive shape. These limitations are related with a weak depth perception.

The monocular endoscope is the most common. The strategies to obtain a 3D reconstruction are based on shape from motion and shape from shading. Nonetheless, the 3D reconstruction is very complex since the scene being observed presents organs and tissues that are highly deformable bodies.

Plenoptic cameras give more information than conventional cameras. Namely, these cameras give information about the direction and contribution of each ray to the total amount of light captured on an image. These cameras allow to estimate depth from a single image by analysis of their epipolar geometry, which may allow to overcome some of the current endoscope limitations.

There are other types of endoscopes, like the stereo endoscope, that has a similar setup to the setup of the plenoptic camera. These stereo endoscopes are recent and not well studied, but their small baseline requires computing image correspondences and still do not give the desired depth range for medical applications. The depth range for plenoptic cameras is unknown for plenoptic cameras. Thus, we want to define the depth range limits for a plenoptic camera.

Plenoptic cameras make a trade-off between spatial and angular resolution which normally results in images with low spatial resolution, similarly to the endoscope images. To overcome this limitation we aim to create a lightfield mosaic and perform a 3D reconstruction from a tubular shape mimicking the conditions that the endoscope camera has to face.

The workplan consists on the study of the plenoptic camera model and calibration, and implementation of a standard mosaicking and depth estimation algorithms for conventional and plenoptic cameras. These methods will be applied to real datasets acquired on a tubular shape structure. The results for the plenoptic camera will be compared to the results from a conventional camera.

 

-- More Information

 

More MSc dissertation proposals on Computer and Robot Vision in:

http://omni.isr.ist.utl.pt/~jag