Semi-Global Matching (SGM) is a dense stereo matching method that can be used for accurate 3D reconstruction from a pair of calibrated images. SGM tries to find correspondences for every pixel. This is supported by a global cost function, which is optimized in 8 path directions across the image. The method has a regular algorithmic structure and uses simple operations. In the core loop, integer values are compared and added. This enables parallel implementations on graphics cards and FPGAs for real-time applications. Although SGM is not any more the winning method in international benchmarks that only evaluate quality (see Middlebury benchmark and KITTI), it turns out that SGM offers a very good mixture of speed, quality and robustness, which is the reason for its success in practice.
SGM is used in photogrammetry. The development of many commercial photogrammetric software packages has been strongly influenced by its appearance. This fact has been honored by the Carl-Pulfrich Award 2011.
SGM is also used for driver assistance systems. The 6D vision system by Daimler researchers uses a real-time implementation of SGM. Since summer 2013, the 6D vision system is a foundation of several commercially available driver assistance systems in production cars.
At the RMC, SGM is used for environment modeling from satellite, aerial and multicopter images as well as the workspace analysis of robots like ROMO. A real-time FPGA implementation is used for autonomous navigation of flying, walking and crawling robots as well as for planetary rovers.
- Heiko Hirschmüller (2008), Stereo Processing by Semi-Global Matching and Mutual Information, in IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 30(2), February 2008, pp. 328-341.
- Heiko Hirschmüller and Daniel Scharstein (2009), Evaluation of Stereo Matching Costs on Images with Radiometric Differences, in IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 31(9), September 2009, pp. 1582-1599.
- Annett Stelzer, Heiko Hirschmüller and Martin Görner (2012), Stereo-Vision-Based Navigation of a Six-Legged Walking Robot in Unknown Rough Terrain, in the International Journal of Robotics Research, Special Issue on Robot Vision, Volume 31, Issue 4, pp. 381-402.
- Korbinian Schmid, Teodor Tomic, Felix Ruess, Heiko Hirschmüller and Michael Suppa (2013), Stereo Vision based indoor/outdoor Navigation for Flying Robots, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), November 2013, Tokyo Japan, Robocup Best Paper Award!
Computer Vision Toolkit (cvkit)
The cvkit software package is offered for supporting an easy exchange and analysis of computer vision data including images in floating point format and 3D data like point clouds or triangle meshes. The toolkit targets researchers as well as interested groups and individuals. It is also especially useful for working with the Middlebury stereo and multi-view data.
sv is a simple / scientific image viewer that can display monochrome and color images with 8 and 16 bit integer as well as 32 bit float values as data types per color channel. Functions include showing monochrome images with color encoding, defining radiometric ranges, zooming and automatically reloading images (Linux only). For image comparison, settings like zoom, radiometric range, etc, can be kept while switching between images. Depth images (full or parts) with associated camera parameter files can be visualized on-the-fly in 3D. sv natively supports the pgm, ppm and pfm image formats as well as tiff with 8 and 16 bit integer and 32 bit float values. Tiff, jpg, png, gif and many other raster data formats are supported through optional libraries like GDAL.
plyv is a simple but pretty fast viewer for colored point clouds and meshes with per vertex coloring, shading and texture images. It also supports on-the-fly conversion and visualization of depth images and cameras. plyv is based on OpenGL and can cope with big data sets that consist of many million vertices and triangles. Mainly the ply format is supported, which has been invented at Stanford University as an extendable format for storing vertices and polygons together with additional information. It is especially useful for scanned real-world data.
NOTE: The toolkit does not contain SGM or any other stereo method and we do not offer SGM as source code or binary.
Linux / Source: cvkit-1.6.3-src.tgz