Omnidirectional and Pan-Tilt-Zoom surveillance

 

MSc dissertation proposal 2007/2008

 

Surveillance of Public Spaces using
Omnidirectional and Pan-Tilt-Zoom Cameras

 

 

 

Introduction:

 

Standard surveillance systems installed in public spaces, often require a multitude of conventional cameras to cover most of the space under observation. Despite the low-cost of nowadays cameras, there are huge amounts of cabling and maintenance costs associated with these systems. Furthermore, in the surveillance operation center, most of the cameras are unattended due to the very high costs of having humans watching the resulting video-imaging. Hence, in this work we propose using novel non-standard types of cameras do deal with this issue: omnidirectional and pan-tilt-zoom cameras. Omnidirectional cameras are able to cover very wide spaces at the cost of reduced resolution. On the contrary, Pan-Tilt-Zoom cameras are able to move to interesting regions in the visual space, thus being able to inspect them with high resolution but a narrow view field. The combination of these complementary types of cameras is able to bring enhanced fields of view and resolutions, and allow constructing effectively the required automatic surveillance needs, with a reduced number of cameras, cabling and computational resources.

 

 

Objectives:

 

The goals of this work proposal are two-fold: (i) modelling pan-tilt-zoom and omnidirectional cameras, (ii) designing and implementing computer vision algorithms for surveillance applications such as detection and tracking of intrusions.

 

 

Detailed description:

 

In this work we propose using novel omnidirectional and pan-tilt-zoom cameras. Omnidirectional cameras see in all the 360deg azimuthal field-of-view and are thus interesting for surveillance because of the true omni-awareness: conventional limited field-of-view cameras observe just part of the scene at a time. Pan-tilt-zoom cameras allow selecting the field-of-view and therefore give the highest quality image resolution. Cooperative combining both types of cameras allows obtaining the best properties of both systems.

 

In this work-proposal we consider two principal aspects: (i) modelling omnidirectional and pan-tilt-zoom cameras (ii) using these cameras combined in a network for surveillance applications. Modelling omnidirectional and pan-tilt-zoom cameras is in general a challenging task due to the wide variety of the existing designs (many of them involving non-linear optics). In this work we consider just cameras having simple geometries as the ones with constant resolution and/or a single effective view-point.

Networking cameras encompasses creating the communications infrastructure and calibrating the complete system. For example, in order to perform target tracking in a camera network, it is necessary to measure, or estimate, the relationships among the fields-of-view of the overlapping cameras. We consider a network of cameras having largely redundant fields-of-view and doing the calibration based on the large superpositions of the background or by cooperatively observing moving objects.

 

Work-proposal detailed steps:

 

(i) Modelling omnidirectional and pan-tilt-zoom cameras - this involves estimating or measuring some camera parameters and building environmental background representations [Sinha04]. Given the geometric model of an omnidirectional camera one can build virtual cameras, i.e. cameras resembling standard pan-tilt-zoom cameras, by re-locating pixels. The modelling of the pan-titl-zoom cameras allows collaborative observation of the same scene point, as the coordinates of the point in one camera can be translated to another one given the camera models and some scene priors. Note that translating observations from an omnidirectional to a pan-tilt-zoom camera is an important service as conventional pan-tilt images, with the correct zooming, are much simpler to interpret by humans.

 

(ii) Surveillance applications - given the background representations, one main purpose is to detect scene intrusions, moving objects or generic events in the scene [Boult99, Hall05, Nascimento06]. Other surveillance services include doing statistics as bidirectional counting of pedestrian passing through a "gate", reports upon request how many people are gathered in its field of view, counts the number of times people have dwelt for longer than a pre-set period, the most frequent path in a shopping mall (average paths, dwell times), etc.

 

Hardware and software details:

The surveillance cameras will encompass one or more omnidirectional and pan-tilt-zoom cameras. The software builds upon networking software [YARP], which provides for example ports for image acquisition from cameras. At the end of the work, the produced applications will have ports reporting events and situation assessment detected at the camera network.

 

 

References:

[Sinha04] S Sinha, M Pollefeys, "Towards Calibrating a Pan-Tilt-Zoom Camera Network", OMNIVIS'04 (with ECCV'04)

[Boult99] "Frame-Rate Omnidirectional Surveillance & Tracking of Camouflaged and Occluded Targets", T.E. Boult R. Micheals, X. Gao, P. Lewis, C.Power, W. Yin and A. Erkan, in Proc. of the IEEE Workshop on Visual Surveillance, June 1999.

[Hall05] "Comparison of target detection algorithms using adaptive background models", D. Hall, J. Nascimento, P. Ribeiro, E. Andrade, P. Moreno, S. Pesnel, T. List, R. Emonet, R.B. Fisher, J. Santos Victor, J. Crowley, Int. workshop on Performance evaluation of Tracking and Surveillance - Oct 2005

[Nascimento06] "Performance Evaluation of Object Detection Algorithms for Video Surveillance", Nascimento, J.C.; Marques, J.S.; IEEE T-Multimedia, V8.4, 2006 pp:761 - 774.

[YARP] "Yet Another Robot Platform", http://eris.liralab.it/yarp/

 

 

Requirements (grades, required courses, etc):

-

 

Expected results:

 

At the end of the work, the students will have enriched their experience in computer vision applied to video surveillance setups. In particular are expected to develop and assess:

- geometric modelling methods for omnidirectional, pan-tilt-zoom, networked cameras;

- algorithms for intrusion detection and tracking.

 

 

Place for conducting the work-proposal:

ISR / IST

 

 

More MSc dissertation proposals on Computer and Robot Vision in:

 

http://omni.isr.ist.utl.pt/~jag