[Inilabs camera, Qualcomm sample image, AER lib G.Orchard]

Human-like Visual Tracking of Targets (id 7252)

Objectives:

Neuromorphic cameras report intensity changes for each pixel independently and asynchronously. These cameras have a time resolution in the order of the µs and have a high dynamic range which makes them interesting for tracking and visual navigation. Nonetheless, the output of these cameras cannot be used directly with computer vision algorithms.

In this work, we intend to focus on target tracking using a stream of events.

The objectives of this work are:

- Neuromorphic camera model definition.

- Develop an intensity image reconstruction algorithm.

- Develop an algorithm for target tracking by optimizing camera resources.

Requirements (grades, required courses, etc):

--

Localization:

ISR / IST

Observations:

Neuromorphic cameras are biologically inspired cameras. The inspiration for their design comes from the transient pathway of primate vision, which processes information due to luminance changes in the scene.

These cameras do not report the intensity values for every pixel on the sensor but report for each pixel, independently and asynchronously, intensity changes above a given threshold. This allows reducing the redundant information for pixels with unchanged intensity values. Furthermore, these cameras have a time resolution in the order of the µs and have a high dynamic range which makes them interesting for tracking and visual navigation.

Nonetheless, these events cannot be used directly with computer vision algorithms that are designed to operate on a frame basis. In the recent years, there has been an effort to adapt computer vision algorithms to neuromorphic vision, namely, optical flow, corner and edge detection, epipolar geometry.

In this work, we want to bridge the gap between computer and neuromorphic vision by focusing on target tracking. Target tracking using neuromorphic sensors is a relatively simple task on an environment with a moving object and a static background. Nonetheless, this task becomes more complex when we consider a moving background. In this situation, we cannot differentiate the events generated by the object being tracked from the ones generated by the background. This task becomes even more complex if we have more targets than observers.

The workplan consists on the study of the neuromorphic camera, and implementation of a tracking algorithm based on a stream of events. The results will be compared with standard computer vision algorithms for target tracking. Since computer vision methods cannot be applied directly to neuromorphic datasets, an intensity image reconstruction algorithm should also be implemented to obtain a frame based video from a stream of events. These methods will be applied to real datasets.

More information about this project:

http://users.isr.ist.utl.pt/~jag/msc/msc_2017_2018.html