DLR CalLab software receives Manfred Fuchs Prize

Visual measurement: spin-off from space travel

Whether human or machine - visual perception is one of the most important senses for navigating and interacting safely and purposefully in an environment. Unlike us humans, however, robots and machines are programmed to perceive the three-dimensional world geometrically on the basis of two-dimensional image data. This results in distortions in the camera's "view", which must be compensated for in order to process the image data accurately. The DLR CalLab calibration software simplifies this process of visual measurement by enabling a precise geometric description of cameras.
Klaus Strobl has been involved in geometric calibration for around 20 years and has significantly developed the DLR software and made it commercially viable. The software can now precisely determine the geometric information of the respective camera from a single image, such as the center of its projection and the direction in which it was taken. This enables the seamless merging of multiple camera images. "Simplifying the method was very important to us. Not only to achieve more cost-effective production, but also to avoid possible interference factors that could affect accuracy and reliability," says Klaus Strobl, who heads the specialist group for sensor-related algorithms.
DLR CalLab is also part of German space history: the first version of the software was developed back in 1993 to calibrate stereo cameras of the ROTEX gripper during the German D2 mission. Since then, the software has been continuously expanded and is used in a variety of robot systems. One challenge of the application at Liebherr was to enable an all-round view with just a few static cameras. Wolfgang Stürzl was able to contribute his many years of experience with wide-angle and fisheye cameras, which he had used for the simulation of insect eyes, among other things: "I had started to work with camera models and this led to me having calibration software that can also be used for wide-angle cameras." This software was integrated into the CalLab concept together with a fisheye camera model. Using just a few parameters, the software can measure the respective cameras easily and efficiently.

360-degree surround view

Today's construction and agricultural machinery is getting bigger and heavier, but also more confusing. Modern surround view assistance systems help to provide a better overview of the machine's surroundings.
The Liebherr assistance system consists of four cameras that are mounted on each side of the vehicle and whose images can be combined on a display to create an overall bird's eye view. The cameras are calibrated around the vehicle using DLR CalLab. Each individual fisheye camera has a viewing angle of 150 degrees. Thanks to the DLR software, the cameras are adjusted in such a way that individual images are rectified and seamlessly merged into an overall image. Compared to other systems available on the market, neither additional images nor support from a robot arm are required. This means that the cameras can be calibrated more easily, quickly and reliably using the DLR software. "The award winners have developed this calibration tool to such a level of maturity in the long term that it is better than what the market can offer. This makes CalLab interesting for commercial users, as they benefit in their business. A real innovation for practical use," emphasizes Institute Director Prof. Albu-Schäffer.
DLR licensed the CalLab prototype to Liebherr-Components AG in September 2022. The Lindau-based company is part of the Liebherr Group, one of the world's leading manufacturers of construction machinery. The Group uses the calibrated optical sensors in the new LiXplore digital video system for construction machinery. "It's a great feeling when you can see your own work not only in the publications or at the institute, but also on the road," says a delighted Klaus Strobl.
However, the project is not yet complete. "We want to continuously develop the calibration software," says Wolfgang Stürzl. For example, it would be conceivable for CalLab itself to automatically select the appropriate calibration model. This would allow users without specialist knowledge to use the software efficiently.
The Manfred Fuchs Innovation Prize, endowed with 5000 euros, is awarded annually for an outstanding scientific achievement whose technology is also suitable for transfer to industry. Founded in 1972, the Society of Friends of DLR (Gesellschaft von Freunden des DLR e. V.) is a non-profit organization that promotes DLR's scientific and technical activities.

Technology transfer from space travel