Objectives:
The work motivation comes from the problem of
installing a set of surveillance cameras where one needs a common reference
frame to track moving objects inter-cameras even without intersecting fields of
view. One wants to capture data for estimating the position, the orientation
and the intrinsic parameters (calibration) of each camera within a network of
cameras. The non-overlapping problem is overcome by using a mobile point-depth
camera (portable setup) with the capability of estimating its pose in a global
frame.
The portable setup is composed of a calibrated camera
which we assume that can be oriented in a manner to observe world points also
seen by the network of cameras, and therefore transfer the calibration to the
networked camera [Bento19, Gois20]. In this work, we want, in addition, that
the portable setup uses its depth measurement capability to estimate its own
motion, and in that way provide a unified coordinate system to the full set of
surveillance cameras.
Detailed description:
The increasing need of surveillance in public spaces
and the recent technological advances on embedded video compression and
communications made camera networks ubiquitous. Typical environments include
single rooms, complete buildings, streets, highways, tunnels, etc. While the
technological advances already allowed such a wide installation of camera
networks, the automatically extracting information from the video streams is
still an active research area.
One of the crucial problems in camera networks is to
obtain a correct calibration of each camera in terms of a unique, global,
reference frame. This condition is a fundamental feature required for further
higher level processing (i.e. people/car tracking, event detection, metrology)
and nowadays one of the most active research subjects in Computer Vision. The
problematics generally arise from the lack of overlapping field of views (FOV)
of the camera which does not allow the estimation of a common reference frame
for each camera. In such scenario, exactly geolocating each sensor is difficult
without the aid of special equipment (moving calibration patterns or GPS) or a
priori reference images such as a panorama of the given environment.
In order to calibrate each camera of the network we
propose using a portable setup that can localize itself with respect to a
global coordinate system. The portable setup is based on a colour-point-depth
camera which can be pointed to a field of view similar to the one observed by
the network (fixed) camera.
Given a number of 3D-points of the environment and
their images we can calibrate one camera using standard computer vision
methodologies [Hartley00, Leite08]. Using 3D lines in the scene and imaged by
the camera, one can obtain more accurate calibrations [Silva12]. Developing
automatic correspondences of 3D lines with 2D imaged lines, possibly using
automatic SIFT based matching of 3D points and 2D points [Lowe04], is an
interesting and practical contribution for the calibration of network cameras.
These methodologies have been used to calibrate the fixed surveillance cameras [Bento19,
Gois20], and, in this work, they are studied and further developed to provide
also the motion of the portable setup.
Work-proposal detailed steps:
- Testing the detection and correspondence of SIFT
features in simulated and real scenarios.
- Testing the detection and correspondence of 3D lines
of the scene with imaged 2D lines.
- Testing the camera calibration methodologies based
on the information obtained in the previous steps.
References:
[Gois20] "Calibration of Surveillance Cameras
based on an Auxiliary Color-Depth Camera", Diogo
Góis, MSc Thesis, Electrical and Computer
Engineering, IST - 2019/2020.
[Bento19]
Fast Setting of Networks of Surveillance Cameras, Mário Bento, MSc Thesis,
Electrical and Computer Engineering, IST - 2018/2019. More
information.
[Silva12] Camera Calibration using a Color-Depth
Camera: Points and Lines Based DLT including Radial Distortion. M. Silva, R.
Ferreira and J. Gaspar. In WS in Color-Depth Camera Fusion in Robotics, held
with IROS 2012.
[Leite08] Calibrating a Network of Cameras Based on
Visual Odometry, Nuno Leite, Alessio Del Bue, José Gaspar, in Proc. of IV Jornadas de Engenharia Electrónica e Telecomunicações e de Computadores,
pp174-179, November 2008, Lisbon, Portugal.
[Hartley00] R. I. Hartley and I. Zisserman. Multiple
view geometry in computer vision. pages 150–152, 2000.
[Lowe04] David G. Lowe. Distinctive image features
from scale-invariant keypoints. In International
Journal of Computer Vision, pages 91–110, 2004.
More information:
http://omni.isr.ist.utl.pt/~jag