Panoramic
video sequence and the corresponding mosaic in a VRML format (from [Galego10]).
MSc dissertation proposal 2012/2013
Omni-Scene
Modelling using Pan-Tilt Cameras
Introduction:
Panoramic scene representations are actually ubiquitous in various
markets such as real estate promotion, studying architectural and ergonomic
solutions, or virtually visiting tourist and cultural places. Using pan and
tilt cameras to build virtual scene representations in the form of mosaics is
therefore a subject well explored in literature [Sinha04, Vitorino07, Galego11,
Leite11].
Despite being a subject well explored, there are still many limitations
in current virtual scene representations. For example Google Street View contains
excellent panoramic views of streets, but does not include (yet ;) time varying
panoramic scenes. Another example, in the video surveillance domain, camera
manufacturers require holding the camera to detect motion events, and usually
do not offer scene change detections when the camera "looked to another
direction and then came back". One reason is that commercial cameras do not
construct robust background-scene representations, capable of handling e.g.
scene illumination changes due to artificial-lights or daylight changes.
In this project is proposed to explore the subject of robust panoramic-scene
creation, encompassing the prediction and compensation of illumination changes.
Note:
this project is proposed in the framework of two projects, a national project
on camera modelling and calibration, DCCAL, and
an EU financed project (QREN) on modelling and using high-definition
surveillance cameras, HDA.
Objectives:
This project involves three main objectives: (i)
composing a
mosaic from a pan-tilt video sweeping a spherical scene surrounding
the camera, (ii) incorporating daylight change effects, and (iii) detecting
background changes in the current image.
Detailed description:
Building the virtual scenes by composing mosaics involves carefully
registering images having large overlapping among them. The precise
registration either involves precise geometric calibration and the pan and tilt
angular information, or precise estimation of mapping homographies. This is an
important aspect to study in the project. In either of the cases it is
interesting to use just video information to understand whether or not it can
complement or even replace the (control electronics / odometry)
measurements of the camera pose.
In order to understand illumination effects, it is important to realize
that the camera is itself responsible for various artefacts, such as image
darkening towards the borders or brightness compression in order to cover a
larger dynamic ranges of the scene radiance (see e.g. [Galego11] work done in a
previous MSc project). By properly estimating the imaging artefacts, one is
better equipped to model illumination changes such as daylight change.
This project comprises therefore understanding and compensating the
geometric and the radiometric artefacts present in pan-tilt cameras, and
exploring the scene (surface) properties to find illumination changes. Having
these tools, allows one to build simple background representations which enable
event detection, such as objects introduction or removal, comparing images taken
at very disparate times of the day.
The work is organized in the following main parts:
1) acquiring data covering more than one day
2) estimating imaging artefacts and estimating
the daylight
changes
3) incorporating daylight changes in the
detection methodologies based on background subtraction
References:
[Sinha04] "Towards
Calibrating a Pan-Tilt-Zoom Camera Network",
[Rav-Acha05] "Dynamosaics: Video Mosaics with Non-Chronological Time",
Alex Rav-Acha, Yael Pritch, Dani Lischinski, Shmuel Peleg, CVPR'05. See also
http://www.vision.huji.ac.il/dynmos/.
[Vitorino07] "Panoramic Mosaics Minimizing Overlappings
in the Azimuthal Field-of-View", João Vitorino, José Gaspar, Proc.
of RecPad 2007 - 13ª Conferência
Portuguesa de Reconhecimento de Padrões,
[Vicente09] "Assessing Control Modalities Designed for Pan-Tilt
Surveillance Cameras", Diogo Vicente, Jacinto C.
Nascimento, José Gaspar, RecPad
2009.
[Galego10] "Surveillance
with Pan-Tilt Cameras: Background Modeling",
Ricardo Galego, Alexandre
Bernardino, José Gaspar, In 16th Portuguese Conference on Pattern Recognition (RecPad 2010), Vila Real, Portugal, October 2010.
[Galego11] "Vignetting Correction for Pan-Tilt Surveillance
Cameras", Ricardo Galego, Alexandre
Bernardino, José Gaspar, Int. Conf. on Computer Vision Theory and Applications
(Visapp), March 5-7, 2011
[Leite11] "Auto-Calibration
of Pan-Tilt-Zoom Cameras: Estimating Intrinsic and Radial Distortion Parameters",
Diogo Leite, Alexandre Bernardino, José Gaspar , In 17th Portuguese
Conference on Pattern Recognition (RecPad 2011),
Porto, Portugal, October 2011
Expected results:
At the end of the work the students will have enriched their knowledge
in:
* Computer vision
* Virtual scenarios construction
Observations:
--
More MSc dissertation
proposals on Computer and Robot Vision in:
http://omni.isr.ist.utl.pt/~jag