Department Unmanned Aircraft

Sensor Fusion and Environmental Perception

Stereo camera at the ARTIS flight test carrier

Real-time 3D Obstacle Mapping during a flight in an urban area
Helicopter with mounted camera and laser range finder
Laser-based point cloud from house obstacle

A safe flight in arbitrary scenarios requires a situational awareness that is able to consider the properties of the environment. This mainly means that the aircraft has to sense and avoid dangers by itself, even during wireless data link losses and beyond the pilot's line of sight. This becomes possible through the environmental perception with cameras, laser range finders, and other sensors and the application of intelligent algorithms that integrate the environmental properties into flight control. The used methods are comparable to those used in the robotics and automation domain, but they have to be adapted to meet the specific requirements of unmanned aircraft. For the evaluation of new methods, the image and sensor data processing system was integrated into the simulation environment and into flight control.

Data fusion of satellite navigation, camera, laser, and inertial sensors is gaining importance as well as the integration of the whole sensor data processing chain into a reliable flying system. Amongst others, the environment sensors are used to detect dangers and the ego-motion of the aircraft. Additionally, the sensors are taken as a payload for further applications like environment cartography.

Recent research investigations which have already been tested with unmanned helicopters are for example:

  • Improved flight state estimation, reduction of the inertial drift and stabilized (hover) flight also in the case of failures and dropout of the satellite navigation, by the determination of the ego-motion and position with monocular and stereo image data,
  • automatic search and tracking of moving ground targets,
  • aerial imagery with intelligent camera control and automatic photo-stitching to an aerial map,
  • real-time detection and mapping of dangers like buildings with stereo and laser range sensing, and avoidance of a-priori unknown obstacles.

Furthermore, there are synergies to manned aviation and to spacecraft topics. Examples are the detection of other aircraft for the improved automatic collision avoidance (sense/see-and-avoid) and the evaluation of optical navigation and mapping techniques for the automated and precise lunar and planetary landing.

URL for this article