Team: 3D and Modeling

 The generation of Digital Surface Models (DSMs) from optical stereo data has a long heritage at IMF. Starting from the first space mission using a digital stereo camera (MOMS) in the early 90s, continuous improvements in stereo image matching have been the focus of our research.  The stereo methods developed by our team already allow the creation of city models from space with the currently highest resolution data, as WorldView-1/2/3/4, IKONOS, GeoEye-1, Pléiades, Cartosat and ZY3. Our strategy is not only to generate the best possible DSMs using the latest computer vision methods, but also to develop new methodologies for high-level semantic modeling.  Of particular interest is the mathematical algorithms development for building footprint extraction, 3D change detection, Classification, and full 3D reconstruction of buildings and trees. The results enable novel observations of urban growth, the progress of large and safety-critical construction infrastructure or the mapping of damage after natural disasters.

Currently the 3D and Modeling Team focuses on the following topics:

  • DSM/DTM generation from space-, air- and UAV borne imagery
    Our team is developing and further improving the DSM/DTM generation approaches from airborne and spaceborne optical stereo data. For operational processing, we developed a novel variant of the well-known Semi-Global Matching (SGM) algorithm. Our modifications to standard SGM include a robust hierarchical search strategy that dynamically reduces the search range for flat areas and results in faster computation and denser DSMs, as well as a combined data term employing cost functions as Census, Mutual information or generated by a convolutional neural network. Our software is licensed to GAF AG, Munich, who very successfully applies it to produce satellite-based stereo DSMs/DTMs for commercial applications.
  • 3D change detection
    Automatic change detection using DSMs for urban and forest applications is a long-term focus of our research. Depending on the DSM quality, multispectral channel availability and change detection requirements we have developed  region-based and fusion-based approaches for automatic change detection for urban and forest areas by fusing changes from DSMs and optical images.  We have further studied the possibility of using high temporal resolution stereo VHR images to enhance remote sensing image interpretation.
  • Building and infrastructure reconstruction
    Although the DSM matching algorithms are highly developed a full reconstruction of 3D building models from, e.g. WorldView 2, satellite imagery is still challenging. This is due to the limited resolution that leads to blurred building edges in the DSM. Using additional multi-spectral and panchromatic images is therefore recommended to remedy the deficiencies of DSMs and to achieve real 3D models of buildings.  Both of the traditional (based on segmentation, line detection) and the fully convolutional networks based approaches have been developed for this purpose.
uav_airborne_cmyk_neu
Comparison of automatically derived DSMs, result using only the aerial DSM (top) and the fused DSM using Aerial and UAV data (bottom). On the right side of the latter the strong improvements in details of the roofs and in terms of smoothness of all areas can be seen.
DownloadDownload
  • 3D Forest monitoring
    Another important DSM and 3D change application is to monitor forest vertical structures over large areas. For quantitative analysis of forest 3D properties we used various spaceborne and airborne sensors: WorldView 2, Cartosat 1, PRISM, RapidEye, aerial stereo and Lidar data. In addition to a statistical comparison, their performance in monitoring forest changes has been assessed. In a recent project we also study close-range 3D reconstruction of trees.

The developed algorithms are integrated in the following tools

  • XDibias
    XDibias is the fourth generation of a generic tool that is used as an ‘image processor development platform’ and for evaluating remote sensing data. It comprises a broad collection of software modules designed to handle raster- and vector-based information within an easy-to-use environment for image processing.
  • CATENA
    Catena (lat. for chain) is a universal, operational infrastructure to set up processing chains for the automatic processing of (optical) satellite and airborne image data. Available chains include automatic orthorectification based on worldwide reference databases, atmospheric correction, DSM generation from single stereo pairs up to large areas consisting of thousands of images.

Links