Mobile Cameras: Pan-tilt-zoom cameras and cameras mounted on mobile
robots
MSc dissertation proposal 2011/2012
Mobile
Cameras Calibration
Introduction:
Conventional calibration of video cameras assumes static cameras in front
of which one shows a structured calibration pattern in various poses [Bouguet-WWW]. Nowadays, many cameras have motion degrees of
freedom, e.g. the pan-tilt-zoom cameras, or are simply mounted on top of robot
arms or mobile robots. These motion degrees of freedom can be used to the
advantage of the calibration process by releasing the need of using calibration
patterns and showing them at various poses.
Objectives:
In this work the main objective is to explore the calibration of (i) constrained
motion (e.g. pan-tilt-zoom) cameras, and (ii) unconstrained motion (hand-held) cameras. In both cases, cameras
are expected to be calibrated intrinsically (focal length, scaling, principal
point) by assuming static scenarios.
Detailed description:
Determining the intrinsic parameters of a mobile camera without any
assumptions about the imaged world is called camera self- or auto-calibration
[Hassanpour04]. While capturing a sequence of images, the camera motion can be
either general or restricted, depending on the camera degrees of freedom. For
instance, a pan-tilt camera can rotate but cannot translate, while a camera
mounted on a helicopter can both rotate and translate. In both motion cases,
general or restricted, there are some specific motions, known as critical
motions, where no unique solution can be found for the camera parameters (see
e.g. [Sturm97]). Nevertheless, the non-critical cases are more than enough for
many interesting commercial applications, such as the augmented reality in the
movies industry [2d3-WWW].
Considering the non-critical cases, various calibration methodologies
exist for both general and restricted motion. For example [Hartley94,
Hartley00] gives a practical calibration solution for pan-tilt cameras based on
the rearrangement of the composition of rotation matrices. [Agapito01] further
extends the methodology to the case of pan-tilt cameras that can also zoom
(i.e. vary their intrinsic parameters). In the case of the general motion, one
finds methodologies based on the "absolute quadric", "Kruppa equations", "essential matrix"
decomposition, etc (see more details in [Hassanpour04]).
In this work proposal is expected to test a number of referred
calibration methodologies. The testing methodology will be first based on
simulation and in some cases using real cameras (hand-held or pan-tilt-zoom
mounted on static basis or on mobile robots). The main steps of the work are
therefore the following:
- build a simulated setup (VRML world with a controlled camera /
viewpoint)
- extraction and matching (correspondence) of
image features (e.g. SIFT [Lowe04, Lowe-WWW])
- estimation of fixed intrinsic parameters on a
pan-tilt camera
- estimation of the intrinsic parameters on a
pan-tilt-zoom camera
- estimation of the intrinsic and extrinsic (up-to
a scale factor) parameters of a hand-held camera
References:
[Bouguet-WWW] Jean-Yves Bouguet, "Camera calibration toolbox for
matlab", http://www.vision.caltech.edu/bouguetj/calib_doc/
[Hassanpour04] Camera auto-calibration using a sequence of 2D images
with small rotations, Reza Hassanpour, Volkan Atalay, Pattern
Recognition Letters, Vol.25, Issue 9, 2 July 2004, Pages 989-997
[Sturm97] Sturm, P., 1997. Critical motion sequences for monocular selfcalibration and uncalibrated
Euclidean reconstruction. In: Conference on Computer Vision and Pattern
Recognition. pp. 1100–1105.
[2d3-WWW] "2d3 develop and deliver technology built on unrivalled
vision science expertise", http://www.2d3.com
[Hartley94] Self-calibration from multiple views with a rotating camera,
Richard Hartley, ECCV'94, pp.471-478
[Agapito01] Agapito, L., Hayman, E., Reid,
I.D., 2001. Self calibration of rotating and zooming cameras. Int. J. Comput.
Vision 45(2), 107–127.
[Hartley00] R. I. Hartley and
[Lowe04] David G. Lowe. Distinctive image features from scale-invariant keypoints. In International Journal of Computer Vision,
pages 91–110, 2004.
[Lowe-WWW] David G. Lowe. Demo software: Sift keypoint
detector. http://www.cs.ubc.ca/˜lowe/keypoints/.
Requirements (grades, required courses, etc):
-
Expected results:
At the end of the work, the students will have enriched their experience
in computer vision. In particular are expected to develop and assess:
- geometric models for pan-tilt-zoom cameras;
- algorithms for calibrating cameras having
restricted or general motion.
Place for conducting the work-proposal:
ISR / IST
More MSc dissertation
proposals on Computer and Robot Vision in:
http://omni.isr.ist.utl.pt/~jag