MSc dissertation proposal 2015/2016
Fast Moving
Objects Detection using Neuromorphic Image Processing
Introduction:
In computer vision there is a high demand of real-time applications like
in object detection and recognition. Nonetheless, standard technology limits
machine vision acquisition frequency around 30 to 60 Hz.
The appearing of neuromorphic chips allow to
perform these tasks more rapidly. Nonetheless, these sensors are yet too
expensive to be considered an alternative for robotics. These sensors are
asynchronous in the way they look at a scene, only after some change in the
amplitude of a pixel the sensor creates a spike for that pixel.
During this work, we want to follow another approach. We want to develop
a neuromorphic framework from natural images that
uses slow-motion cameras with higher acquisition frequencies (120 to 1000 Hz
depending on the resolution). This framework will be used to identify moving
objects based on event-based information.
Objectives:
The objectives of this work are: (i)
assembling a slow-motion camera to a mobile robot, (ii) acquire a catalogued
dataset of images in a controlled set, (iii) obtain neuromorphic
like images, and (iv) detect moving objects in the field of view of the camera.
Detailed description:
In the past years there has been a growing effort to transform the way
we process information. Big players of the hardware industry like IBM and
Qualcomm are making efforts for developing new processors that mimic the way we
process information in the brain. Likewise, DARPA (SyNAPSE
project) and the European Commission (Neuromorphic
Computing Platform in the scope of the Human Brain Project) have launched
flagships that are focusing on this line of research.
The main advantages of these processors relatively to the traditional
processors that we use on a daily-basis are (i) fast
processing rate and (ii) lower power consumption. This is accomplished by
changing the paradigm of chip processing: instead of working all the time, neuromorphic chips are event-driven and are only activated
when they need to.
In computer vision there are a lot of applications that require
real-time information like surveillance, robot navigation, and object detection
and recognition. With the advent of more powerful computer
chips and with GPU computing some of these requirements have been met.
Nonetheless, the standard machine vision acquisition frequency is around 30 to
60 Hz [Akolkar2015]. Therefore, neuromorphic chips
are a good alternative to overcome this limitation. The geometry of the images
originated by these chips have been studied recently by Benosman
[Benosman2012A, Benosman2012B, Benosman2013] and these are starting to be used
in research for visual odometry [Censi2014] and pose
tracking [Mueggler2014].
There are two approaches for neuromorphic
processing in computer vision:
- Asynchronous neuromorphic event-based spikes
using neuromorphic chips (Dynamic Vision Sensor or
DVS [Lichtsteiner2008]) that deliver a low-resolution image (128 x 128 pixel
image) at high frequencies.
- Neuromorphic event-based spikes from natural
images (SpikeNet [Delorme1999, Delorme2003]). This
approach creates artificial spikes from gray levels of natural images.
The last approach have been criticized in [Akolkar2015] due to the
redundant information generated and for not capturing the real dynamics of the
scene due to the low sampling frequency (30 to 60 Hz). Therefore, we want to
extend the work of Delorme in order to obtain higher frequency rates using a
slow-motion camera and use it in an application that will allow to identify moving objects.
Main Steps
This work focus on adapting some of the
existing algorithms of computer vision and adapt them to a neuromorphic
like algorithm. In this work we intend to:
- Assemble an high-rate camera to a mobile
robot
- Acquire a catalogued dataset of images in a controlled set
- Develop an algorithm to obtain neuromorphic
like images
- Detect moving objects in the field of view of the camera.
References & Links:
[Akolkar2015] Akolkar, Himanshu, et al. "What Can Neuromorphic
Event-Driven Precise Timing Add to Spike-Based Pattern Recognition?." Neural computation (2015).
[Mueggler2014] Mueggler, Elias, Basil Huber, and Davide Scaramuzza. "Event-based, 6-DOF pose tracking for high-speed maneuvers."
Intelligent Robots and Systems (IROS 2014), 2014 IEEE/RSJ
International Conference on. IEEE, 2014.
[Censi2014] Censi, Andrea, and Davide Scaramuzza.
"Low-latency event-based visual odometry."
Robotics and Automation (ICRA), 2014 IEEE International
Conference on. IEEE, 2014.
[Lichtsteiner2008] Lichtsteiner, Patrick, Christoph Posch, and Tobi Delbruck. "A 128× 128
120 dB 15 μs latency asynchronous temporal
contrast vision sensor." Solid-State Circuits, IEEE Journal of 43.2
(2008): 566-576.
[Delorme2003] Delorme, A., Thorpe,
S. (2003). SpikeNET: An Event-driven
Simulation Package for Modeling Large Networks of
Spiking Neurons,Network: Comput. Neural Syst., 14, 613:627.
[Delorme1999] Delorme, A., Gautrais, J., VanRullen, R.,
& Thorpe, S.J. (1999). SpikeNET: A simulator for modeling large networks
of integrate and fire neurons. Neurocomputing,
26-27, 989-996.
[Benosman2012A] Benosman, R.,
Ieng, S.H., Rogister, P., Posch, C., "Asynchronous Event-based EpipolarGeometry", IEEETransactions
on Neural Networks, vol. 22, no. 1, pp- 1723-1734, 2011.
[Benosman2012B] R. Benosman,
S-H. Ieng
, C. Clercq, C. Bartolozzi,
M. Srinivasan, (2012) Asynchronous frameless
event-based optical flow, Neural Networks. Volume 27, March, Pages 32–37.
[Benosman2013] João Carneiro, Sio-Hoi Ieng, Christoph Posch, Ryad Benosman,
"Asynchronous Event-Based 3D Reconstruction From Neuromorphic Retinas". Neural Networks, 15th march
2013. DOI:10.1016/j.neunet.2013.03.006.
IBM:
http://www.research.ibm.com/cognitive-computing/neurosynaptic-chips.shtml
- fbid=7tKIG9DSnZK
Qualcomm:
http://www.fiercewireless.com/tech/story/qualcomm-demos-robot-neuromorphic-chip/2014-04-27
Human Brain Project:
https://www.humanbrainproject.eu/neuromorphic-computing-platform1
https://www.humanbrainproject.eu/neuromorphic-computing-platform1
Requirements (grades, required courses, etc):
-
Expected results:
At the end of the work, the students will have enriched their experience
in computer vision. In particular are expected to develop and assess the
following topics:
- Dataset acquisition and cataloging.
- Camera calibration.
- Neuromorphic transformation of natural
images.
- Object detection and recognition from images.
Place for conducting the work-proposal:
ISR / IST
More MSc dissertation
proposals on Computer and Robot Vision in:
http://omni.isr.ist.utl.pt/~jag