Panoramic
video sequence and the corresponding mosaic in a VRML format (from [Galego10]).
MSc dissertation proposal 2011/2012
Creating Virtual
Scenes with Pan-Tilt Cameras
Introduction:
Panoramic virtual scene representations are actually ubiquitous in
various markets such as real estate promotion, studying architectural and
ergonomic solutions, or virtually visiting tourist and cultural places. Using
pan and tilt cameras to build virtual scene representations in the form of
mosaics is therefore a subject well explored in literature [Sinha04, Vitorino07,
Galego11].
Despite being a subject well explored, there are still many limitations
in current virtual scene representations. For example in the video surveillance
domain, camera manufacturers require holding the camera to detect motion
events, and usually do not offer scene change detections when the camera "looked
to another direction and then came back". One reason is that commercial cameras
do not offer robust background-scene (virtual-scene) construction, capable of
handling e.g. scene illumination changes due to artificial-lights or daylight
changes.
In this project is proposed to explore the subject of robust virtual-scene
creation, encompassing the prediction and compensation of illumination changes.
Objectives:
This project involves three main objectives: (i)
composing a
mosaic from a pan-tilt video sweeping a spherical scene surrounding
the camera, (ii) estimating the camera imaging artefacts and the daylight change effects,
and (iii) detecting
background changes in the current image.
Detailed description:
Building the virtual scenes by composing mosaics involves carefully
registering images having large overlapping among them. The precise
registration either involves precise geometric calibration and the pan and tilt
angular information, or precise estimation of mapping homographies. This is an
important aspect to study in the project. In either of the cases it is
interesting to use just video information to understand whether or not it can
complement or even replace the (control electronics / odometry)
measurements of the camera pose.
In order to understand illumination effects, it is important to realize
that the camera is itself responsible for various artefacts, such as image darkening
towards the borders or brightness compression in order to cover a larger
dynamic ranges of the scene radiance (see e.g. [Galego11] work done in a
previous MSc project). By properly estimating the imaging artefacts, one is
better equipped to model illumination changes such as daylight change.
This project comprises therefore understanding the geometry and the
radiometric artefacts present in pan-tilt cameras, compensating the geometric
and radiometric aspects, and exploring the scene (surface) properties to find
illumination changes. Having these tools, allows one to build simple background
representations which enable static event detection, such as objects
introduction or removal, comparing images taken at very disparate times of the
day.
The work is organized in the following main parts:
1) acquiring data covering more than one day
2) estimating imaging artefacts and estimating
the daylight changes
3) incorporating daylight changes in the
detection methodologies based on background subtraction
References:
[Sinha04] "Towards Calibrating a Pan-Tilt-Zoom Camera
Network",
[Vitorino07] "Panoramic Mosaics Minimizing Overlappings in the Azimuthal
Field-of-View", João Vitorino,
José Gaspar, Proc. of RecPad 2007 - 13ª Conferência Portuguesa de Reconhecimento
de Padrões,
[Vicente09] "Assessing Control Modalities
Designed for Pan-Tilt Surveillance Cameras", Diogo
Vicente, Jacinto C. Nascimento, José Gaspar, RecPad 2009.
[Galego10] "Surveillance with Pan-Tilt Cameras: Background Modeling", Ricardo Galego, Alexandre Bernardino, José Gaspar, In 16th Portuguese
Conference on Pattern Recognition (RecPad 2010), Vila
Real, Portugal, October 2010.
[Galego11] "Vignetting Correction for
Pan-Tilt Surveillance Cameras", Ricardo Galego, Alexandre Bernardino, José Gaspar, Int. Conf. on Computer
Vision Theory and Applications (Visapp), March 5-7,
2011
Expected results:
At the end of the work the students will have enriched their knowledge
in:
* Computer vision
* Virtual scenarios construction
Observations:
--
More MSc dissertation proposals
on Computer and Robot Vision in:
http://omni.isr.ist.utl.pt/~jag