A Factorization Approach to Structure from Motion with Shape Priors
Partially supported by Pourtugese FCT grant MODI-PTDC/EEA ACR/72201/2006.
| Problem Statement: The problem is to extract a reasonable 3D reconstruction from sequences with strong motion degenrancies. The video on the left shows one of such examples. In this case a normal structure from motion approach would fail in obtaining a correct solution. For instance, it is very complex to understand the depth of the object shape without any prior information over the 3D structure. |
| 3D reconstruction without priors: The video shows the 3D reconstructions obtained using a standard procedure to compute deformable 3D shapes from images given the previous video sequence. The results are poor — the method fail to obtain usable information given the degenerate motion. The video shows front, side and top views of the given reconstruction. |
|
The shape prior: The video on the left shows a different subject performing only rigid motion. From this sequence it is possible to obtain a rigid 3D shape using standard factorization techniques such as Tomasi&Kanade algorithm. The computed shape is shown in the figure on the left. This shape can be then used as a prior even if the subjects’ face have different somatic characteristics. By merging measurements from images and 3D shape prior information we aim to obtained an improved 3D reconstruction.
| 3D reconstruction with priors: This is the final 3D reconstruction using the top sequence with the 3D shape prior. Now the depth estimates are reasonably describing the subject’s face and the mouth deformations are properly modelled. In principle this method can deal with any kind of object’s shape: rigid, articulated and deformable. |
Reference:
More details in the paper: “A. Del Bue. A Factorization Approach to Structure from Motion with Shape Priors”
-
A. Del Bue, "Adaptive Metric Registration of 3D Models to Non-rigid Image Trajectories," in 11th European Conference on Computer Vision (ECCV 2010), Crete, Greece, 2010, pp. 87-100.
@inproceedings{DelBue:2010,
author = {A. {Del Bue}},
title = {Adaptive Metric Registration of 3D Models to Non-rigid Image Trajectories},
editor = {Kostas Daniilidis and Petros Maragos and Nikos Paragios},
booktitle = {11th European Conference on Computer Vision (ECCV 2010), Crete, Greece},
publisher = {Springer},
location = {Heidelberg},
series = {Lecture Notes in Computer Science},
volume = {6313},
year = {2010},
isbn = {978-3-642-15557-4},
pages = {87--100}
}
-
A. Del Bue, "A factorization approach to structure from motion with shape priors," in Proc. IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, Alaska, 2008, pp. 1-8.
@conference{DelBue:2008, title={A factorization approach to structure from motion with shape priors},
author={A. {Del Bue}},
booktitle = {Proc. IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, Alaska},
pages={1--8},
year={2008}
}
[...] SfM with Priors [...]