Wii Hand: A Practical Hand Tracking System

Game consoles sales have recently reached an unimaginable peak in the entertainment industry. Among the three biggest competitors, the Nintendo Wii console is the single system that has proposed innovative tools and technologies for gameplaying. Given the console diffusion, every application related to the Wii equipment can have a widespread impact. In such scenario, we want to build a minimally invasive system to interact directly with the console using the user hand and one (or more) standard Wii remotes.

The success of the Wii is not only based on its gaming capabilities but also on its new controller technology which integrates infrared cameras and gyroscopes in a single device (the Wii Remote). The introduction of this item in the consumer electronics market has already brought some innovation which were not clearly envised by Nintendo itself. This popular video on the left for instance shows how the device has inspired a new range of applications for human-computer interaction.

Our aim is to create a new system with a hardware given by a glove with 5 led emitters on each fingertip and one (or more) Wii remote. The software involved in the project is a real-time hand tracking system which can localize and follow the hand leds using the infrared camera inside the Wii remote. In order to strengthen the hand tracking algorithm we will also attempt to learn the hand dynamics using the ground-truth provided by a CyberGlove device available in our laboratories. The final aim is to create a novel interaction tool (the Wii Hand) for application as human-computer interaction (i.e. interactive desktop), remote actuation of robotic hands and a device for the gaming industry.

Instituto de Sistemas e Robotica has already an ongoing experience on interfacing Wii hardware to generic desktop PCs via a Bluetooth link. The work scheduled for this thesis has a first practical stage for learning the software interface with the Wii remote and to calibrate the camera system inside the remote. The chosen software library for such implementation is the Open Source wiiuse C library. A second stage will deal with the problem of tracking the led emitters in real-time. This part has a more theoretical flavour and it is a chance for the candidate to learn different methods for image tracking and modelling of articulated bodies. The final stage will be dedicated to the construction of an application which can be easily integrated in our software framework.

Project Objectives

We expect to obtain a working system for hand tracking using a Wii remote and a glove with 5 leds attached over each fingertip. Such system will have a high diffusion and impact since the system is composed by readily available consumer electronics. The diffusion of the Wii console will also give the chance to advertise our system to a widespread audience.

Prof. Alexandre Bernardino.
Torre Norte, Piso 7, Room 7.19
Phone Number: 2293

Dr. Alessio Del Bue
Torre Norte, Piso 7, Room 7.25
Phone Number: 2299
Email

Dr. Manuel Lopes
Torre Norte, Piso 7, Room 7.25
Phone Number: 2299