Alessio Del Bue Homepage

Factorization for Non-Rigid and Articulated Structure using Metric Projections
In collaboration with: M. Paladini, M. Stosic, M. Dodig, J. Xavier, L. Agapito

We propose a new algorithm to recover the 3D shape and motion of deformable and articulated objects from uncalibrated 2D image measurements using an iterative factorization approach. Such problem is solved with an alternate least-squares approach associated with a globally optimal projection step onto the manifold of metric constraints. An important advantage of this new algorithm is its ability to handle missing data.

Structure from Motion problems are strongly bilinear in the sense that the image projections can be represented as the combination of two factors. A motion component representing the camera projectione matrices and the time varying evolution of the shape (i.e. rotation, translation and deformation). A 3D structure component which represent the 3D parametrization of the object shape location in the metric space.

For such reason, it is rather common to solve for these two elements with an Alternation alike procedure where one component is optimized while the other remain fixed. However, each component has a very specific structure given by the studied problem i.e. the parameters of motion and 3D structure belong to a certain manifold.

In this paper we study the properties of two of such manifolds belonging to the motion components of deformable and articulated shapes. By projecting the motion matrices onto the correct motion manifold we enforce the non-linear metric constraints on the motion component and we find a globally optimal solution for the camera matrices. In particular, we impose that the camera matrices lie on the Stiefel manifold. This constraint results in a non-convex problem which can then be solved by semidefinite tight relaxation in the case of deformable shape. In the articulated case, we efficiently compute the joints given the non-linear constraints on the motion of the two bodies.

• M. Paladini, A. Del Bue, S. M. s, M. Dodig, J. Xavier, and L. Agapito, "Factorization for Non-Rigid and Articulated Structure using Metric Projections," in Proc. IEEE Conference on Computer Vision and Pattern Recognition, Miami, Florida, 2009, pp. 2898-2905.
  @conference{paladini:etal:2009, title={{Factorization for Non-Rigid and Articulated Structure using Metric Projections}},
  author={M. Paladini and A. {Del Bue} and M. Sto{\v{s}}ic and M. Dodig and J. Xavier and L. Agapito},
  year={2009},
  pages={2898--2905},
  booktitle={Proc. IEEE Conference on Computer Vision and Pattern Recognition, Miami, Florida}
}
  bibtex

• M. Paladini, A. Del Bue, S. M. s, M. Dodig, J. Xavier, and L. Agapito, "Optimal Metric Projections for Deformable and Articulated Structure-From-Motion," International Journal of Computer Vision (IJCV), vol. 96, pp. 252-276, 2012.
  @Article{DelBue:Agapito:IJCV2011,
  author = {M. Paladini and A. {Del Bue} and M. Sto{\v{s}}ic and M. Dodig and J. Xavier and L. Agapito},
  title = {Optimal Metric Projections for Deformable and Articulated Structure-From-Motion},
  journal = {International Journal of Computer Vision (IJCV)},
  publisher = {Springer Netherlands},
  issn = {0920-5691},
  pages = {252-276},
  year = {2012},
  volume = {96},
  issue = {2},
  url = {http://dx.doi.org/10.1007/s11263-011-0468-5},
  Doi = {10.1007/s11263-011-0468-5}
}
  bibtex