User Tools

Site Tools


research

Research Overview

In the context of human-robot interaction, it is natural to classify humans into two distinct categories, namely users and operators. Human users are those without the specific knowledge regarding the internal operations of a robot but are expected to benefit from using that robot out of the box in a complex environmental setting. On the other hand, human operators are expected to have the basic technical knowledge of a robot's sensory and motor functionalities and be able to manipulate those in order to perform human-robot collective tasks. For both human users and operators, it is essential to have a smooth and effortless interaction with large-scale multi-robot systems operating in the real-world. However, in order to study and develop methods for such human-robot interactions that involve not only complex robotic systems but also a large number of those robots in a huge environment, it becomes impractical to experiment only with real robots. The inhibiting factors include the costs to build a large number of robots and the infeasibility to perform controlled, repeatable experiments in huge, real physical spaces. We aim to address this challenge by using state of the art, ecologically sound and immersive virtual environment and virtual reality (VR) tools and the related expertise available with the department of Human Perception, Cognition and Action in the Max Planck Institute for Biological Cybernetics (MPI-KYB) in Tübingen. It will enable us to provide real human users and operators with sensory stimuli originating from their interaction with a team of large number of robots operating inside a virtual environment. Using this technology, we will investigate and model human user and operator behavior, perception and cognition resulting from their interaction with large-scale multi-robot systems. Eventually, to achieve the final goal of this research project, we will use these models to design integrated and scalable multi-robot functionalities to maximize the collective performance of the robot team while maintaining an intuitive, effortless and natural interaction with both human users and operators.

Scalability Challenge

Prominent applications requiring multi-robot systems include automation in manufacturing industries, search and rescue missions, automation in agriculture, robotic assistants in shopping malls, museums and other large public establishments, underwater/terrestrial/areal exploration, ore extraction and the recently upcoming area of deep-sea mining. In order to be cost-effectively deployed in real world, the key challenge for multi-robot systems in all such applications is to be able to overcome the curse of dimensionality. We identify three prominent contributors to the dimensionality issue:

  • the enormous amount of data generated by the exteroceptive sensors due to the extensive physical expanse and feature-richness of the environment in which the robots operate,
  • the very large number number of robots operating in cooperation with each other, and
  • the increasing number of inter-dependent low level operations (e.g., localization and mapping) needed to be performed in an integrated manner by each individual robot.

To solve the scalability challenge our primary focus is on exploiting the properties of conditional and mutual independence of the involved variables. Using state of the art formulation methods, such as, pose graphs and Bayesian networks, we aim to address the problem of large-scale multi-robot cooperative localization, target tracking, mapping and motion planning. A key focus of our formulation is to optimally capture the sparsity of the system in question.

Fully Integrating Human-in-the-loop Challenge

Human-robot interaction can range from simpler situations such as a human operator monitoring the robot's operation for fault detection to more complex interactions such as a cooperative exploration task performed jointly by robots and humans. The complexity of human-robot interaction depends on both

  • the nature of the task being performed and
  • the number of robots human(s) have to interact with.

In large-scale multi-robot systems the latter is clearly predominant. However, the nature of the task can have profound effects over the resource constraints. The challenge is to maximize the human user comfort, ease and naturalness of interaction; and minimize human operator's time requirements and his/her efficiency based not only on the resource constraints concerning the multi-robot system but also on the individual human user's/operator's perception, reaction and cognitive abilities.

The first steps to address this challenge is to model human user and operator's cognitive responses using virtual reality (VR) environment and tools in addition to real robots. While an operator in reality is usually expected to tele-operate the robots in order to perform a collaborative task, human users are more likely to be in direct contact of the robots (e.g., robotic assistants for the elderly). For the former, experiments will include bilateral shared control of a team of real unmanned areal vehicles (UAVs) in an augmented reality setting. For the latter, we are developing a large-scale multi-robot system within a computer simulation, directly interactable by human users through interfaces such as head mounted displays (HMDs) in a fully virtual environment.

Regarding the nature of the task being performed, we broadly classify them into singular or multiple-task-oriented categories. Singular-task-oriented applications are those where there is a single global task to be achieved by the multi-robot system with human operators and the robots are allowed to swap their own-sub tasks whenever necessary and possible to optimize the utility of the available resources, e.g., as in the case of cooperative exploration. Multiple-task-oriented applications are those where each robot in the system is constrained to a specific sub-task, thus they are usually restricted from swapping it with the other robots in the team, e.g., multiple robotic assistants in a shopping mall where each robot is assisting an individual human user (multi semi-autonomous personal vehicles). In such a scenario, optimizing the usage of the limited available resources becomes even more difficult.

research.txt · Last modified: 2014/11/03 12:50 by aamir