Navigation with Plenoptic Imaging on
Virtual Reality Scenarios
Description
Light-field representations (LF images) are acquired by
plenoptic or light-field cameras, which can be thought off as arrays of pinhole
cameras. One array of cameras can be represented compactly by an intrinsic
matrix [Dansereau13, Zhang18], like what happens with pinhole cameras. A moving
camera is an alternative way to acquire light-field (LF) images [Mildenhall19].
Considering a set of visual landmarks, metric depth
information can be obtained, provided the LF camera is calibrated. This is an
interesting difference as compared, for example, to do monocular SLAM. One
central objective of the thesis is assessing whether LF information effectively
helps navigation and provides a metric nature to the estimated motion. A specific
objective would be realizing a statistical study about how the LF cameras
baseline effects on depth estimation accuracy.
Detailed steps:
- LF acquisition with a moving pinhole camera
- LF video acquisition and 3D Scene Flow estimation
- LF features integration in a SLAM framework
References:
[Dansereau13] Dansereau, Donald G., Oscar Pizarro, and
Stefan B. Williams. "Decoding, calibration and rectification for lenselet-based plenoptic cameras." Proceedings of the
IEEE conference on computer vision and pattern recognition. 2013.
[Zhang18] Zhang, Qi, et al. "A Generic
Multi-Projection-Center Model and Calibration Method for Light Field
Cameras." IEEE transactions on pattern analysis and machine intelligence
(2018).
[Mildenhall19] Local light field fusion: Practical
view synthesis with prescriptive sampling guidelines. Mildenhall B, Srinivasan
PP, Ortiz-Cayon R, Kalantari NK, Ramamoorthi
R, Ng R, Kar A. ACM Transactions on Graphics (TOG). 2019 Jul 12;38(4):1-4. https://github.com/Fyusion/LLFF
Related previous dissertations:
"Structure Reconstruction
using Plenoptic Cameras", Simão Marto, MSc Thesis, MEAER, IST - 2016/2017,
https://fenix.tecnico.ulisboa.pt/cursos/meaer/dissertacao/1409728525632074
"Deep Depth from Plenoptic
Images", Diogo Portela,
MSc Thesis, MEEC, IST - 2017/2018,
https://fenix.tecnico.ulisboa.pt/cursos/meec/dissertacao/1972678479054436
"Plenoptic Face
Reconstruction", Gonçalo Pedro, MSc Thesis,
MEEC, IST - Oct.2021,
https://fenix.tecnico.ulisboa.pt/cursos/meec/dissertacao/846778572213414
"Visual Anticipation of
Collisions with Moving Objects based on Plenoptic Cameras", Miguel
Rodrigues, MSc Thesis, MEEC, IST - Nov.2021
"Animated Sequences from Image
Based Rendering", António Maia, MSc Thesis, MEEC, IST - started Sep.2021
Place for conducting the work-proposal:
ISR / IST
More MSc dissertation proposals on Computer and Robot
Vision in:
http://users.isr.tecnico.ulisboa.pt/~jag