Active Vision with Foveated Sensors

This project aims at exploring the use of foveated sensors on active vision systems.

The Stereo Heads: Medusa and Euryale (4 DOF):


The Imaging System - Log-Polar Retinas:


Foveation: from left to right, the cartesian image, the foveated image and the log-polar image

 


Movies


Zero-disparity based tracking using log-polar images. 

Reference: Binocular Visual Tracking: Integration of Perception and Control, Alexandre Bernardino, Jose' Santos-Victor, VisLab-TR 10/99 IEEE Transactions on Robotics and Automation, (15)6, Dec. 1999.

Tracking. The movies show the view of one of the stereo head cameras while tracking a face and a hand. 

face_u.mpg (363Kb)

hand_u.mpg (252Kb)

Binocular fusion. The red ang green channels represent the left and right cameras. The system cancels disparity in the central object automatically. This is the driver figure-ground segmentation 

verge1_u.mpg (483Kb)

verge2_u.mpg (585Kb)

Segmentation. Tracking sequences showing figure-ground segmentation (yellow points). 

zdf1_u.mpg (549Kb)

zdf2_u.mpg (799Kb)

Attention Shifting. The system shifts attention automatically to the closest object. 

santa_u.mpg (338Kb)


Region Based Tracking with log-polar images

From: Foveated Active Tracking with Redundant 2D Motion Parameters, Alexandre Bernardino, Jose' Santos-Victor and Giulio Sandini, Robotics and Autonomous Systems, 39(3-4), Elsevier Science, June 2002. pp 205-221. 

Region based tracking algorithm with a "translation only" motion model.The movies show the raw and the stabilized sequences. The original template is the inside of the blue circle on the first frame. 

trackshift.m1v (1675Kb)

trackshiftwarp.m1v (1681Kb)

Respectivelly, the raw and stabilized sequences resulting from the region based tracking algorithm with a translation+rotation+scale motion model. The original template is the inside of the blue circle on the first frame. 

tracksimil.m1v (1082Kb)

tracksimilwarp.m1v (1081Kb)

Disparity Computation with log-polar-images 

From: A Binocular Stereo Algorithm for Log-polar Foveated Systems, Alexandre Bernardino, Josť Santos-Victor, VisLab-TR 14/2002, 2nd Workshop on biological motivated Computer Vision, BMCV 2002, Tuebingen, Germany, November 2002, PDF file (308 Kb) 

A person and a hand, in front of the cameras, are segmented from the background by disparity thresholding. 

segment.mpg (1494Kb)

segm_h.mpg (758Kb) 

Disparity maps for the previous examples

depth.mpg (1055Kb)

depth_h.avi (604Kb)