Image from [Galego14].

MSc dissertation proposal 2019/2020

Imaging with Moving Bundles of Optic-fibers

-- Project description at fenix:

Objectives:

A fiberscope carries imaging through a bundle of optic fibers. In a simple description, one may say that each optic fiber transports "one pixel". Pointing the optic-fiber bundle (cable) according to multiple azimuth and elevation angles allows obtaining multiples images which can be fused to form a panoramic image.

The work is to be carried on an existing fiberscope based in a standard video camera and a cable of optic fibers. The opposite to the camera end of the flexible optic-fibers-cable is mounted on top of a pan-tilt-basis.

The main objectives of the work are the following:

- calibrating the fiberscope in order to obtain images readable by humans,

- obtaining panoramic images by fusing multiple (narrow field) images.

Requirements (grades, required courses, etc):

-

Place for conducting the work-proposal:

ISR / IST

Observations:

This work has its motivation in recent research which has show that under certain conditions a cross-cut cable of a fiberscope can be reconnected. The disarrangement of the various fiber tips can be estimated by image processing methodologies. The work is originally inspired by the, still distant, biological goal of reconnecting damaged optic nerves.

Previous works conducted at ISR/IST provide good starting points for the thesis. In particular a number of hardware prototypes already exist, together with a number of datasets.

-- Links to works at ISR / IST:

Journal papers, conference papers and dissertations:

"DCCAL: Discrete Cameras Calibration using Properties of Natural Scenes", 2010-2013, FCT PTDC/EEA-CRO/105413/2008. Project webpage.

"Discrete camera calibration from pixel streams", Etienne Grossmann, José António Gaspar and Francesco Orabona, Computer Vision and Image Understanding (Special issue on Omnidirectional Vision, Camera Networks and Non-conventional Cameras), Volume 114, Issue 2, Pages 198-209, February 2010. PDF file.

"Topological Auto-Calibration of Central Imaging Sensors", R. Galego, R. Ferreira, A. Bernardino, E. Grossmann and J. Gaspar, IbPRIA 2013. PDF file.

"Discrete Camera Autocalibration Consistent with the Frame of the Robotic Pan-Tilt Basis", R. Galego, R. Ferreira, A. Bernardino, E. Grossmann, J. Gaspar, Procedia Technology 17, 186-193, 2014 Elsevier. PDF file.

"Imaging Based on Optic-fiber Bundles: Topological and Geometric Calibration", João Silva, MSc Thesis, Electrical and Computer Engineering, IST - 2017/2018. More information.

-- More information:

Motivation:

Fiberscopes are imaging devices where the images are acquired in one end of a thin and flexible cable of optic fibers, transmitted through the cable to the other end, and finally magnified with an eyepiece. Commercial fiberscopes are based in cables where the fibers are arranged to form a regular grid. Breaking the cable of a fiberscope has traditionally rendered useless the device, as soldering tens of thousands of fibers, without known correspondences and confined in a small diameter (1 to 5 mm), is not a human feasible task.

Recent research, biologically inspired by the remote goal of reconnecting broken optic nerves, has show that under certain conditions the broken cable of a fiberscope can still be used. The disarrangement of the various fiber tips can be estimated by modeling the collection of fiber tips as a discrete camera, and then determining the topology of the discrete camera. Discrete cameras are defined as collections of pixels, photocells, organized as pencils of lines with unknown topologies. One may present to the camera a known pattern, moving with known speed and direction, and therefore estimate progressively the topology. Alternatively, discrete cameras which can be moved freely can be calibrated from natural scenes [Grossmann10].

This MSc project focuses on calibrating a discrete camera. The result is similar to a fiberscope but built from a much lesser expensive bundle of twisted optic-fibers. Given the calibrated discrete camera, then panoramic images can be obtained by fusing multiple (narrow field) images.

Detailed description:

Conventional video cameras are built from CCD or CMOS sensors whose pixels are organized in rectangular grids. Determining the intrinsic parameters of a mobile camera without any assumptions about the imaged world is called camera self- or auto-calibration. More commonly, cameras are static and one shows them a planar structured (chess) calibration pattern in various poses, which is enough to perform the calibration [Bouguet-WWW].

Discrete cameras simply combine pixels in a fixed manner but without a specific arrangement. Discrete cameras are interesting for robotic applications due to allowing designs specific to the tasks at hand [Neumann05], but pose a challenge right from the calibration point. Recent research work has shown that discrete cameras, which can be moved freely and have a central arrangement of the pixels, can be calibrated from natural scenes [Grossmann10, Galego13, Galego14].

The camera to be used in the dissertation work is based on a standard camera and a standard lens. In between the camera and the lens one has a collection of optical fibbers, rigidly glued to each other at both ends of the cable [Neumann05].

This MSc project focuses on calibrating a discrete camera. A number of calibration methodologies are available for discrete cameras. The main idea is that neighbor photocells view approximately the same direction of the world and thus have higher correlations of their time-signal readings (pixel streams). As distinct from many conventional calibration methods in use today, calibrating discrete cameras requires moving them within a diversified (textured) natural world.

The main steps of the work are the following:

- calibrating the camera in order to obtain images readable by humans, by using auxiliary patterns and/or light or laser pointing

- (alternative) calibration based in using the camera motion on textured scenes

- obtaining panoramic images by fusing multiple (narrow field) images

References:

[Bouguet-WWW] Jean-Yves Bouguet, "Camera calibration toolbox for matlab", http://www.vision.caltech.edu/bouguetj/calib_doc/

[Hassanpour04] Camera auto-calibration using a sequence of 2D images with small rotations, Reza Hassanpour, Volkan Atalay, Pattern Recognition Letters, Vol.25, Issue 9, 2 July 2004, Pages 989-997

[Agapito01] Agapito, L., Hayman, E., Reid, I.D., 2001. Self calibration of rotating and zooming cameras. Int. J. Comput. Vision 45(2), 107–127.

[Neumann05] "Compound Eye Sensor for 3D Ego Motion Estimation", Jan Neumann, Cornelia Fermuller, Yiannis Aloimonos, Vladimir Brajovic, IROS 2005, see also http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.147.929

[Grossmann10] "Discrete camera calibration from pixel streams", Etienne Grossmann, José António Gaspar and Francesco Orabona, Computer Vision and Image Understanding (Special issue on Omnidirectional Vision, Camera Networks and Non-conventional Cameras), Volume 114, Issue 2, Pages 198-209, February 2010.

[Galego13] "Topological Auto-Calibration of Central Imaging Sensors", R. Galego, R. Ferreira, A. Bernardino, E. Grossmann and J. Gaspar, IbPRIA 2013

[Galego14] "Discrete Camera Autocalibration Consistent with the Frame of the Robotic Pan-Tilt Basis", R. Galego, R. Ferreira, A. Bernardino, E. Grossmann, J. Gaspar, Procedia Technology 17, 186-193, 2014 Elsevier

Expected results:

At the end of the work, the students will have enriched their experience in computer vision. In particular are expected to develop and assess:

- optics for imaging

- algorithms for calibrating central cameras.

More MSc dissertation proposals on Computer and Robot Vision in:

http://isr.tecnico.ulisboa.pt/~jag