Manuel Marques
Manuela M. Veloso
João Paulo Costeira
Our vision: Manuel Marques is having lunch and asks the waiter to open his case, pick his FeedBot, as an embedded small robot arm, camera and computer and clamp it to the table. Next, Manuel chooses his entree and asks to have the food placed in the small containers of the special plate stored in his case. He is a researcher with a PhD in computer vision, who suffers from cerebral palsy in a severe degree.His lack of fine movement control and involuntary movements of his arm and head frustrate his attempts to feed himself. If not for the robot, Manuel would require full human assistance to have his meals. The waiter brings his meal and places the dish on the table while Manuel is explaining to his guests why his arm is so flexible. The robot waits until Manuel rotates his head and fixates the container with rice, peas, and salmon. The robot scoops a full spoon and moves towards Manuel’s mouth. Due to his involuntary movements, the robot cannot just stand still and wait for Manuel to get the food. FeedBot must track Manuel’s head and slowly approach his mouth. At some time, because FeedBot has learned a model of Manuel’s reflexes, in particular knowing that he twists his head right when he wants to move forward, FeedBot pulls back a bit to a safer position until Manuel’s head returns to the forward moving direction.
Manuel Marques
Manuela M. Veloso
João Paulo Costeira
Nuno Nunes
Alexandre Candeias
Travers Rhodes
The act of assisted feeding is a challenging task that requires a good reactive planning strategy to cope with an unpredictable environment. It can be seen as a tracking task, where some end effector must travel to a moving goal. This work builds upon state of the art algorithms, such as Discriminative Optimization, making use of a Kinect camera and a modular robotic arm to implement a closed form system that performs assisted feeding. It presents two different approaches: the use of a variable rate function for updating the trajectory with information on the moving goal, and the definition of different risk regions that will shape a safer trajectory.
Researchers have over time developed robotic feeding assistants to help at meals so that people with disabilities can live more autonomous lives. Current commercial feeding assistant robots acquire food without feedback on acquisition success and move to a preprogrammed location to deliver the food. In this work, we evaluate how vision can be used to improve both food acquisition and delivery. We show that using visual feedback on whether food was captured increases food acquisition efficiency. We also show how Discriminative Optimization (DO) can be used in tracking so that the food can be effectively brought all the way to the user's mouth, rather than to a preprogrammed feeding location.
See also: Alexandre Candeias' MSc Thesis, IST, 2018
We propose a methodology to classify motion of subjects with cerebral palsy based on RGB image sequences and present a new dataset with 2D facial landmark trajectories from RGB images of people with and without disabilities while performing specific types of movements. Depending on these movements, parts of the face can be occluded and we are able to recover the 3D face’s shape and its motion based on the Structure from Motion framework. Using the 3D structure and the motion, we propose two different motion descriptors, one is focused on describing the spatial distribution of the motion and the other on the temporal distribution. Finally, we discuss the physical meaning of these descriptors and show that they are very informative about the degree of the subjects’ disabilities. Our descriptor can classify people with and without cerebral palsy from 2D image sequences.
See also: Mário Macedo's MSc Thesis, IST, 2018
In the era of robotics associated with increasingly intelligent systems, we introduce some of the most exciting 'made in portugal' robot projects, from interaction with children and seniors, to underwater robots. We also talk about the future with the experts.
Soon, patients at the Calouste Gulbenkian Rehabilitation Center for Cerebral Palsy will test a robotic arm developed between Instituto Superior Técnico, Instituto de Tecnologia da Madeira and Carnegie Mellon University. Meet the researcher who created Feedbot to help with daily meals.
The project is led by four Portuguese researchers and is the result of a partnership between the Institute of Systems and Robotics, of the Instituto Superior Técnico, and Carnegie Mellon University.
The awards are attributed by Exame Informática and aim to distinguish researchers still unknown to the general public, ideas put to the test in universities and companies that do more for the environment and sustainability. In addition, they recognize the best products and leading manufacturers.
This robotic arm was developed with the objective of helping people with motor disabilities in their food and uses computer vision algorithms to follow the user's movements.
The robotic arm was developed by a research team from Instituto Superior Técnico, Madeira Interactive Technologies Institute (M-ITI) and Carnegie Mellon University.
