Panoramic video sequence and the corresponding mosaic in a VRML format (from [Galego10]).

 

MSc dissertation proposal 2011/2012

 

Creating Virtual Scenes with Pan-Tilt Cameras

 

 

Introduction:

 

Panoramic virtual scene representations are actually ubiquitous in various markets such as real estate promotion, studying architectural and ergonomic solutions, or virtually visiting tourist and cultural places. Using pan and tilt cameras to build virtual scene representations in the form of mosaics is therefore a subject well explored in literature [Sinha04, Vitorino07, Galego11].

 

Despite being a subject well explored, there are still many limitations in current virtual scene representations. For example in the video surveillance domain, camera manufacturers require holding the camera to detect motion events, and usually do not offer scene change detections when the camera "looked to another direction and then came back". One reason is that commercial cameras do not offer robust background-scene (virtual-scene) construction, capable of handling e.g. scene illumination changes due to artificial-lights or daylight changes.

 

In this project is proposed to explore the subject of robust virtual-scene creation, encompassing the prediction and compensation of illumination changes.

 

 

Objectives:

 

This project involves three main objectives: (i) composing a mosaic from a pan-tilt video sweeping a spherical scene surrounding the camera, (ii) estimating the camera imaging artefacts and the daylight change effects, and (iii) detecting background changes in the current image.

 

 

Detailed description:

 

Building the virtual scenes by composing mosaics involves carefully registering images having large overlapping among them. The precise registration either involves precise geometric calibration and the pan and tilt angular information, or precise estimation of mapping homographies. This is an important aspect to study in the project. In either of the cases it is interesting to use just video information to understand whether or not it can complement or even replace the (control electronics / odometry) measurements of the camera pose.

 

In order to understand illumination effects, it is important to realize that the camera is itself responsible for various artefacts, such as image darkening towards the borders or brightness compression in order to cover a larger dynamic ranges of the scene radiance (see e.g. [Galego11] work done in a previous MSc project). By properly estimating the imaging artefacts, one is better equipped to model illumination changes such as daylight change.

 

This project comprises therefore understanding the geometry and the radiometric artefacts present in pan-tilt cameras, compensating the geometric and radiometric aspects, and exploring the scene (surface) properties to find illumination changes. Having these tools, allows one to build simple background representations which enable static event detection, such as objects introduction or removal, comparing images taken at very disparate times of the day.

 

The work is organized in the following main parts:

1) acquiring data covering more than one day

2) estimating imaging artefacts and estimating the daylight changes

3) incorporating daylight changes in the detection methodologies based on background subtraction

 

 

References:

 

[Sinha04] "Towards Calibrating a Pan-Tilt-Zoom Camera Network", S Sinha, M Pollefeys, OMNIVIS'04 (with ECCV'04)

 

[Vitorino07] "Panoramic Mosaics Minimizing Overlappings in the Azimuthal Field-of-View", João Vitorino, José Gaspar, Proc. of RecPad 2007 - 13ª Conferência Portuguesa de Reconhecimento de Padrões, Lisbon, Portugal, 2007.

 

[Vicente09] "Assessing Control Modalities Designed for Pan-Tilt Surveillance Cameras", Diogo Vicente, Jacinto C. Nascimento, José Gaspar, RecPad 2009.

 

[Galego10] "Surveillance with Pan-Tilt Cameras: Background Modeling", Ricardo Galego, Alexandre Bernardino, José Gaspar, In 16th Portuguese Conference on Pattern Recognition (RecPad 2010), Vila Real, Portugal, October 2010.

 

[Galego11] "Vignetting Correction for Pan-Tilt Surveillance Cameras", Ricardo Galego, Alexandre Bernardino, José Gaspar, Int. Conf. on Computer Vision Theory and Applications (Visapp), March 5-7, 2011

 

 

Expected results:

 

At the end of the work the students will have enriched their knowledge in:

* Computer vision

* Virtual scenarios construction

 

 

Observations:

--

 

 

More MSc dissertation proposals on Computer and Robot Vision in:

 

http://omni.isr.ist.utl.pt/~jag