Research Overview

Keywords: planning under uncertainty, sequential decision making, autonomous robots, cooperative multiagent / multi-robot systems, (decentralized) partially observable Markov decision processes (POMDPs / Dec-POMDPs), reinforcement learning, machine learning and artificial intelligence in general.

During my PhD I became interested in formal ways of modeling robot decision processes, in particular how to plan under sensing and acting uncertainty. Major contributions of my PhD work concern approximate POMDP planning machinery such as the Perseus POMDP solver, a fast approximate POMDP planner which is easy to implement (JAIR 2005). We extended Perseus to continuous action spaces (ICRA 2005) and we generalized approximate POMDP planning to fully continuous domains (RSS 2005, JMLR 2006). At that time, we also considered planning for teams of communicating agents that optimize their use of communication primitives (AAMAS 2006).

After my PhD I have been pursuing two lines of research. First, I have been interested in planning under uncertainty for multiagent and multi-robot systems, developing theory as well as solution methods for Dec-POMDPs. Contributions include

• a journal paper which lays the foundations for value-based planning in Dec-POMDPs (JAIR 2008),
• a fast optimal planner for general Dec-POMDPs (AAMAS 2009),
• an algorithm for speeding up a key Dec-POMDP operation (the backup) with up to 10 orders of magnitude speedups on benchmarks (AAMAS 2010),
• models for local interactions on a task level (AAMAS 2008a),
• exploiting local interactions for improved scalability (AAMAS 2008b),
• multiagent planning with unreliable communication (ICAPS 2008),
• exploring reinforcement learning for multiagent POMDPs (IEEE-FUZZ 2010).

A second line of research that I have initiated is on applying approximate POMDP planning techniques to robotic applications, for instance in Network Robot Systems. We have demonstrated successful POMDP-based cooperation between surveillance cameras and mobile robots (ICAPS 2009, ICRA 2010, IROS 2010). Surveillance cameras provide an incomplete and inaccurate global view, which can be enhanced by a robot's local sensors (IROS 2010). POMDPs form a sound framework for modelling such active cooperative perception tasks. Also, we mapped a dynamic sensor selection problem to a POMDP that selects a subset of active sensors (ICAPS 2009), which was successfully tested in our Network Robot Systems testbed (IROS 2009).