TUM technology endows brain-controlled exoskeleton with sense of touch

Opening ceremony of FIFA World Cup: Paralyzed person kicks the first ball

The Walk Again Project is an international collaboration of more than one hundred scientists, led by Prof. Miguel Nicolelis of Duke University and the International Institute for Neurosciences of Natal, Brazil. Prof. Gordon Cheng, head of the Institute for Cognitive Systems at the Technische Universität München (TUM), is a leading partner.

Eight Brazilian patients, men and women between 20 and 40 years of age who are paralyzed from the waist down, have been training for months to use the exoskeleton. The system works by recording electrical activity in the patient's brain, recognizing his or her intention – such as to take a step or kick a ball – and translating that to action. It also gives the patient tactile feedback using sensitive artificial skin created by Cheng's institute.

The feeling of touching the ground

Inspiration for this so-called CellulARSkin technology – as well as for the Walk Again Project itself – came from a 2008 collaboration. As Cheng sums up that complex and widely reported experiment, "Miguel set up a monkey walking on a treadmill in North Carolina, and then I made my humanoid robot walk with the signal in Kyoto." It was a short step for the researchers to envision a paralyzed person walking with the help of a robotic exoskeleton that could be guided by mental activity alone.

"Our brains are very adaptive in the way that we can extend our embodiment to use tools," Cheng says, "as in driving a car or eating with chopsticks. After the Kyoto experiment, we felt certain that the brain could also liberate a paralyzed person to walk using an external body." It was clear that technical advances would be required to allow a relatively compact, lightweight exoskeleton to be assembled, and that visual feedback would not be enough. A sense of touch would be essential for the patient's emotional comfort as well as control over the exoskeleton. Thus the challenge was to give a paralyzed person, together with the ability to walk, the feeling of touching the ground.

A versatile solution

Upon joining TUM in 2010, Cheng made it a research priority for his institute to improve on the state of the art in tactile sensing for robotic systems. The result, CellulARSkin, provides a framework for a robust and self-organizing surface sensor network. It can be implemented using standard off-the-shelf hardware and thus will benefit from future improvements in miniaturization, performance, and cost.

The basic unit is a flat, six-sided package of electronic components including a low-power-consumption microprocessor as well as sensors that detect pre-touch proximity, pressure, vibration, temperature, and even movement in three-dimensional space. Any number of these individual "cells" can be networked together in a honeycomb pattern, protected in the current prototype by a rubbery skin of molded elastomer.

"It's not just the sensor that's important," Cheng says. "The intelligence of the sensor is even more important." Cooperation among the networked cells, and between the network and a central system, allows CellulARSkin to configure itself for each specific application and to recover automatically from certain kinds of damage. These capabilities offer advantages in enabling smarter, safer interaction of machines with people, and in rapid setup of industrial robots – as is being pursued in the EU-sponsored project "Factory in a Day."

In the Walk Again Project, CellulARSkin is being used in two ways. Integrated with the exoskeleton, for example on the bottoms of the feet, the artificial skin sends signals to tiny motors that vibrate against the patient's arms. Through training with this kind of indirect sensory feedback, a patient can learn to incorporate the robotic legs and feet into his or her own body schema. CellulARSkin is also being wrapped around parts of the patient's own body to help the medical team monitor for any signs of distress or discomfort.

A milestone, but "just the beginning"

"I think some people see the World Cup opening as the end," Cheng says, "but it's really just the beginning. This may be a major milestone, but we have a lot more work to do." He views the event as a public demonstration of what science can do for people. "Also, I see it as a great tribute to all the patients' hard work and their bravery!"

High-resolution photos to download:

mediatum.ub.tum.de

Contact:
Prof. Gordon Cheng
Institute for Cognitive Systems
Technische Universität München
Tel: +49 89 289 26800
gordon@tum.de

Further information:
www.ics.ei.tum.de
www.youtube.com/user/icsTUMunich
www.cellularskin.eu
www.factory-in-a-day.eu