Autonomous vehicle thinks for itself

Robot Jack moves like a human

The little robot weaves its way through crowds of people on its wheels like a human. To make this possible, researchers from TUM Professor Angela Schoellig's Learning Systems and Robotics Lab have combined computing power, sensors and mathematical skills. ‘Our robot models the way people will react to its movements to plan its paths. This is the big difference to other approaches that typically ignore this interaction,’ explains Prof. Schoellig.

New route calculated ten times per second

A lidar system constantly sends laser beams into the surroundings, measures their reflections, and uses them to build a precise 360-degree map of what the robot sees. A special focus is placed on people walking around nearby. At the same time, sensors in the wheels measure the robot's own speed and the distances traveled. A computer processes this information, calculates the estimated distances people will cover in the next two seconds, and simultaneously plans the optimum route to the destination. ‘Our robot adjusts its route ten times a second while simultaneously recognizing people's paths,’ explains TUM researcher Sepehr Samavi.

System learns from data based on people in crowds

The robot, which Sepehr Samavi has named ‘Jack’, learns behavioral patterns from humans so that it does not constantly stop due to the risk of collisions. ‘Our mathematical model, on which the planning algorithm is based, was derived from human movements and translated into equations,’ explains Professor Schoellig. For Jack's decision, this means that he does not stop immediately as soon as a person approaches him. He takes into account that people will adapt to the situation, react and change their route slightly so that they do not collide with him. If someone remains on a collision course contrary to expectations, the robot changes its plans quickly and takes a different route - but does not stop. 

The researchers also incorporate data sets that show people's behavior in crowds. The robot, which has already been used outside of the laboratory, is constantly learning and becoming more human-like: ‘Jack knows his destination, observes people and sees where they are going to constantly optimize his own paths,’ says Prof. Schoellig, ‘almost like a human being.’ 

The TUM researchers have already reached the third evolutionary stage with the new algorithm. Instead of ‘only’ reacting to a situation (stage 1) or ‘merely’ predicting the movements of oncoming people (stage 2), the TUM robot is interactive (stage 3). ‘On the one hand, it predicts other people's movements, but it also manages to influence these people through its behavior and simultaneously avoid collisions,’ explains researcher Sepehr Samavi.   

Development for use in autonomous driving

It is precisely such interactive scenarios that form the bottleneck in autonomous driving, says Prof Schoellig. For example, if a vehicle starts accelerating on a highway on-ramp, many drivers will change lanes or tap the brakes. The new approach makes it possible to consider the reaction of others in such a scenario. However, the researchers are initially looking at applications in delivery robots or with wheelchair users. The advantage: these vehicles can reach their destination independently and reliably. Even humanoid robots could benefit from the new algorithms. However, the intelligent vehicle has one decisive disadvantage: ‘A moving robot can simply stop if necessary - humanoids are still quite wobbly and quickly lose their balance,’ says Prof. Schoellig.