Research Overview
Keywords: planning under uncertainty, sequential
decision making, autonomous robots, cooperative
multiagent / multi-robot systems, (decentralized) partially
observable Markov decision processes (POMDPs / Dec-POMDPs),
reinforcement learning, machine learning and artificial
intelligence in general.
During my PhD I became interested in formal ways of
modeling robot decision processes, in particular how
to plan under sensing and acting uncertainty. Major
contributions of my PhD work concern approximate POMDP
planning machinery such as the Perseus POMDP solver, a
fast approximate POMDP planner which is easy to
implement (JAIR
2005). We extended Perseus
to continuous action spaces (ICRA
2005) and we
generalized approximate POMDP planning to fully
continuous domains (RSS
2005, JMLR 2006). At that
time, we also considered planning for teams of
communicating agents that optimize their use of
communication primitives (AAMAS 2006).
After my PhD I have been pursuing two lines of research. First, I have
been interested in planning under uncertainty for multiagent and
multi-robot systems, developing theory as well as solution
methods for Dec-POMDPs. Contributions include
- a journal paper which lays the foundations for value-based
planning in Dec-POMDPs (JAIR 2008),
-
a fast optimal planner for general Dec-POMDPs (AAMAS 2009),
- an algorithm for speeding up a key Dec-POMDP operation (the
backup) with up to 10 orders of magnitude speedups on benchmarks
(AAMAS 2010),
- models for local interactions on a task level
(AAMAS
2008a),
- exploiting local interactions for improved scalability (AAMAS
2008b),
- multiagent planning with unreliable communication (ICAPS 2008),
- exploring reinforcement learning for multiagent POMDPs
(IEEE-FUZZ 2010).
A second line of research that I have initiated is on
applying approximate POMDP planning techniques to
robotic applications, for instance in Network Robot
Systems. We have demonstrated successful POMDP-based
cooperation between surveillance cameras and mobile
robots (ICAPS 2009, ICRA 2010, IROS
2010). Surveillance cameras provide an incomplete and
inaccurate global view, which can be enhanced by a
robot's local sensors (IROS 2010). POMDPs form a sound
framework for modelling such active cooperative
perception tasks. Also, we mapped a dynamic sensor
selection problem to a POMDP that selects a subset of
active sensors (ICAPS 2009), which was successfully
tested in our Network Robot Systems testbed (IROS 2009).
|