3D face recognition from multiple images:
a shape-from-motion approach
Typical images used in this approach
In this work, we explore the use of methodologies for 3D reconstruction from multiple images to recognize faces. We try to devise a strategy to tackle the problem of recognizing faces from images exhibiting strong pose (rotation and occlusion) and without prior knowledge (uncalibrated cameras, images from different sources). We do so by framing the recognition in the context of 3D structure from motion with missing data problems. In fact, recently, there has been a strong trend towards using 3D information to verify and recognize faces. However most of the state of the art works are developed over 3D sensors (3D range finders, stereo). Here we propose to do recognition by measuring the likelihood that one or more images were generated by a given 3D shape. In other words, given the subject’s 3D shape, we compute the required transformation to match the probe images and measure its deviation from rigidity. This process is not straightforward due to the strong pose: face points are not visible in all images so there is a ”missing data” problem intrinsic to the formulation. One key assumption is that all images (and shape) correspond to neutral expression.
For detailed info, see:
Marques, M., and Costeira, J.. "3D face recognition from multiple images: a shape-from-motion approach", FG2008 - 8th IEEE International Conference on Automatic Face and Gesture Recognition, Amsterdam, The Netherlands, September 2008.
Related work:
Marques, M., and Costeira, J.. "Estimating 3D shape from degenerate sequences with missing data", Computer Vision and Image Understanding, 113:(2):261-272, February 2009.
Database: Zip file