The representation of non-rigid structures is extremely important for visual segmentation applications using shape models. Shape models consist of statistical models that represent deformation of the each region of the shape. The statistical models are generally built using a collection of training data, which is usually composed of the photometric information (i.e., a sequence of images in time or depth that form a 3-D data) and a manual delineation of the visual structure of interest. There are several examples of visual segmentation problems that could benefit from such training data, such as: human body tracking, or 3-D segmentation of organs using medical imaging data (e.g., 3-D ultrasound or Magnetic Resonance Imaging). The most likely annotation available to build shape models is the manual delineation of the structure of interest, which is essentially characterized by a set of points at several locations around the shape. A trivial shape model can be obtained by explicitly determining key locations at the shape, which can be done manually or automatically. These key locations allow for the alignment of all annotated shapes in the training set, which allow us to build a statistical model of the variation of the transformation parameters (translation, scale, rotation, and non-rigid deformations). It is important to mention that generally there is a clear distinction between the rigid parameters (translation, scale, and rotation) and non-rigid parameters, and in this work we are mostly concerned only with the non-rigid parameters. The most common approach for the statistical model of non-rigid deformations is based on a pre-alignment of the shapes (in terms of translation, scale, and rotation), and the construction of the principal components analysis (PCA) space of the non-rigid shape variation. Although appealing in a computational sense, this representation is quite limited in the set of deformations they can represent. For instance, visual objects that are under the effect of large non-rigid deformations will be poorly represented by a linear approximation given by PCA. In computer graphics the problem of non-rigid 3-D representation has been studied for quite a long time, and produced interesting models that are now being used in computer vision (e.g., spherical harmonics and radial basis functions). In this work, we plan to study and implement the most common models being used in computer vision, computer graphics, and medical image analysis and compare their efficacy in typical 3-D computer vision problems.
Objectives:
The objective of this work is the study, implementation and comparison of 3-D representation models of non-rigid visual objects for typical segmentation problems in computer vision. It is important to mention, that we shall consider state-of-the-art models developed in computer vision and computer graphics. This work will be framed in the context of 3D medical image analysis where we expect to work on real data from CT images.
Detailed description:
The statistical representation of non-rigid visual objects is a widely studied topic in computer vision given its applicability in segmentation problems. An important representation developed in the 90s was the active shape and appearance models [Cootes95,Cootes01], which takes into account the appearance and location of the key points, and the non-rigid model is essentially the PCA model described before. An important alternative representation is described by Romeny [Romeny99], which is based on spherical harmonics. This representation seems more appropriate for cases where the visual object of interest suffers extremely large non-rigid deformations. A further interesting alternative is given by the Radial Basis models used in Computer Graphic [Park08] to visualize the motion of the soft-tissues of the human body. These new representations have brought clear advancements in 3D visualization and their effectiveness in the medical and computer vision scenario is still under study. Although an extremely important topic in computer vision, the non-rigid 3-D representation has not been investigated in depth, and we plan to achieve this goal with this work. Specifically, the following steps are planned in the work:
1. Literature review in the area of recurrent visual non-rigid 3-D representation in the areas of computer vision, computer graphics, and medical image analysis.
2. Implementation of the at least three of the most relevant representations (PCA, Spherical Harmonics and Radial Basis).
3. Comparison and applicability of the representations for several computer vision applications in the medical domain.
4. Pubblication of the representations for the computer vision community.
References:
[Cootes95] T.F. Cootes and C.J. Taylor and D.H. Cooper and J. Graham. Active shape models - their training and application. Computer Vision and Image Understanding (61): 38--59.1995.
[Cootes01] T. F. Cootes, G. J. Edwards, and C. J. Taylor. Active appearance models. IEEE TPAMI, 23(6):681-685, 2001.
[Romeny99] B. H. Romeny, B. Titulaer, S. Kalitzin, G. Scheffer, F. Broekmans, J. Staal, and E. te Velde. Computer assisted human follicle analysis for fertility prospects with 3d ultrasound. Proceedings of the International Conference on Information Processing in Medical Imaging, pages 56-69, 1999.
[Park08] Sang Il Park and Jessica K. Hodgins. Data-driven Modeling of Skin and Muscle Deformation. SIGGRAPH 2008.
Requirements (grades, required courses, etc):
Image and signal processing courses. Knowledge of MATLAB scientific language.
Expected results:
At the end of the work, the students will have enriched their experience in computer vision, computer graphics, and medical image analysis. In particular the following goals are expected:
- Deep understanding of current state-of-the-art 3-D representation models present in the literature of computer vision, computer graphics and medical image analysis.
- Implementation and comparison of at least three current state-of-the-art representations in various computer vision applications.
- Leave the implementations available to be downloaded and used by the computer vision community. Place for conducting the work-proposal: