[Inilabs camera, Qualcomm sample image, AER lib G.Orchard]

Neuromorphic Vision from Conventional Imaging (id 9296)

Objectives:

Neuromorphic cameras report intensity changes for each pixel independently and asynchronously. These cameras have a time resolution in the order of the microseconds and have a high dynamic range which makes them interesting for tracking and visual navigation. Nonetheless, the output of these cameras cannot be used directly with conventional computer vision algorithms. In this work, we intend to focus on multiple moving targets detection using a stream of events.

The objectives of this work are:

- Neuromorphic camera modeling.

- Develop a conversion method of conventional video into a stream of events based on a public available simulator

- Develop an algorithm for multiple moving targets detection.

Requirements (grades, required courses, etc):

Interest for video processing and visual detection of moving targets.

Localization:

ISR / IST

Observations:

Neuromorphic cameras are biologically inspired cameras. The inspiration for their design comes from the transient pathway of primate vision, which processes information due to luminance changes in the scene.

These cameras do not report the intensity values for every pixel on the sensor but report for each pixel, independently and asynchronously, intensity changes above a given threshold. This allows reducing the redundant information for pixels with unchanged intensity values. Furthermore, these cameras have a time resolution in the order of the µs and have a high dynamic range which makes them interesting for tracking and visual navigation.

Nonetheless, these events cannot be used directly with computer vision algorithms that are designed to operate on a frame basis. In the recent years, there has been an effort to adapt computer vision algorithms to neuromorphic vision, namely, optical flow, corner and edge detection, epipolar geometry.

Much of the current research and development work on neuromorphic vision is based on public datasets, which narrows testing conditions. In this work, we want to create a software basis which allows bridging the gap between computer and neuromorphic vision by generalizing data sources.

In the applications vein, the detection of multiple independent motions, studied in seminal computer vision works, is still under research for neuromorphic vision. It is one application target of the dissertation.

In some more detail, the starting point of this dissertation is proposed to be adapting OpenCV video input methods to work within Matlab. The video inputs are then transformed into an event stream. Finally, multiple independent motions are detected and characterized.

More information in:

http://users.isr.ist.utl.pt/~jag/msc/msc_2017_2018.html