Objectives:
The main objective of the work consists in estimating
the position, the orientation and the intrinsic parameters
(calibration) of each camera within a network of cameras, possibly with
non-overlapping fields of view. Calibration comprises both the intrinsic and
extrinsic parameters. The non-overlapping problem is
considered to be overcome using a portable setup with the capability of
estimating its pose in a global frame. The portable setup is composed of a
calibrated camera which we assume that can be oriented in a manner to observe
world points also seen by the network of cameras, and therefore transfer the
calibration to the networked camera.
Expected Results: At the end of the work, the students
will have enriched their experience in computer vision applied to camera
network setups. In particular are expected to develop
and assess:
- precise image (feature) registration methodologies
- precise calibration methodologies
Detailed description:
The increasing need of surveillance in public spaces
and the recent technological advances on embedded video compression and
communications made camera networks ubiquitous. Typical environments include
single rooms, complete buildings, streets, highways, tunnels, etc. While the
technological advances already allowed such a wide installation of camera
networks, the automatically extracting information from the video streams is
still an active research area.
One of the crucial problems in camera networks is to
obtain a correct calibration of each camera in terms of a unique, global,
reference frame. This condition is a fundamental feature required for further higher level processing (i.e. people/car tracking, event
detection, metrology) and nowadays one of the most active research subjects in
Computer Vision. The problematics generally arise from the lack of overlapping
field of views (FOV) of the camera which does not allow the estimation of a
common reference frame for each camera. In such scenario, exactly geolocating
each sensor is difficult without the aid of special equipment (moving
calibration patterns or GPS) or a priori reference images such as a panorama of
the given environment.
In order to calibrate each camera of the network we propose using
a portable setup that can localize itself with respect to a global coordinate
system. The portable setup is based on a colour-point-depth
camera which can be pointed to a field of view similar to
the one observed by the network (fixed) camera.
Given a number of 3D-points
of the environment and their images we can calibrate one camera using standard
computer vision methodologies [Hartley00, Leite08]. Using 3D lines in the scene
and imaged by the camera, one can obtain more accurate calibrations [Silva12].
Developing automatic correspondences of 3D lines with 2D imaged lines, possibly
using automatic SIFT based matching of 3D points and 2D points [Lowe04], is an
interesting and practical contribution for the calibration of network cameras.
Work-proposal detailed steps:
- Testing the detection and correspondence of SIFT
features in simulated and real scenarios.
- Testing the detection and correspondence of 3D lines
of the scene with imaged 2D lines.
- Testing the camera calibration methodologies based
on the information obtained in the previous steps.
References:
[Gois20] "Calibration of Surveillance Cameras based
on an Auxiliary Color-Depth Camera", Diogo Góis, MSc Thesis, Electrical and Computer Engineering, IST
- 2019/2020.
[Bento19]
Fast Setting of Networks of Surveillance Cameras, Mário Bento, MSc Thesis,
Electrical and Computer Engineering, IST - 2018/2019. More
information.
[Silva12] Camera Calibration using a Color-Depth
Camera: Points and Lines Based DLT including Radial Distortion. M. Silva, R. Ferreira and J. Gaspar. In WS in Color-Depth Camera Fusion
in Robotics, held with IROS 2012.
[Leite08] Calibrating a Network of Cameras Based on
Visual Odometry, Nuno Leite, Alessio Del Bue, José Gaspar, in Proc. of IV Jornadas de Engenharia Electrónica e Telecomunicações e de Computadores,
pp174-179, November 2008, Lisbon, Portugal.
[Hartley00] R. I. Hartley and I. Zisserman. Multiple
view geometry in computer vision. pages 150–152, 2000.
[Lowe04] David G. Lowe. Distinctive image features
from scale-invariant keypoints. In International
Journal of Computer Vision, pages 91–110, 2004.
More information:
http://omni.isr.ist.utl.pt/~jag