An object can automatically be reconstructed from images obtaining a triangulated 3d-point cloud, if it is viewed from different view points.
First, image orientations have to be estimated. Therefore, point correspondences are detected in overlapping images. Then, the optimal positions of these points in space and the optimal pose (position and rotation) of the camera for each image are estimated employing bundle adjustment, cf. Figure 1. This very precise approach [4] was developed at the research group for visual computing (Prof. Helmut Mayer, Institute of Applied Computer Science) of the Bundeswehr University Munich. The DLR Institute for Robotics and Mechatronics cooperates with this research group.
Second, depth images are derived using semiglobal matching [2]. For each pixel, we try to determine the distance from the object to the camera. The depth images are merged to 3d-points [3], which are triangulated in order to reconstruct the object's surfaces [1]. It is possible to obtain reconstructed settlements with small details at single buildings, if the images have a high resolution and largely overlap each other, cf. the railing of the stairs in Figure 2.
Third, the 3d-models can be simplified for their integration into simulation environments. This can be done after recognising and vectorising buildings and their parts, such as windows and doors [5,6]. Urban modeling can be used for cultural heritage and for process analysis in cities, e.g., traffic simulations.
Selected Publications
[1] Bodenmüller, T. (2009): Streaming Surface Reconstruction from Real Time 3D Measurements. PhD Thesis, Technische Universität München.
[2] Hirschmüller, H. (2008): Stereo Processing by Semi-Global Matching and Mutu- al Information. IEEE Transactions on Pattern Analysis and Machine Intelligence 30(2): 328–341.
[3] Kuhn, A., Mayer, H., Hirschmüller, H. & Scharstein, D. (2014): A TV Prior for High-Quality Local Multi-View Stereo Reconstruction. International Conference on 3D Vision, pp. 65–72.
[4] Mayer, H., Bartelsen, J., Hirschmüller, H. & Kuhn, A. (2012): Dense 3D Reconstruction from Wide Baseline Image Sets. In: Dellaert et al. (Eds.): Outdoor and Large-Scale Real-World Scene Analysis, 15th International Workshop on Theoretical Foundations of Computer Vision, Lectures Notes in Computer Science (LNCS), Vol. 7474, pp. 285–304.
[5] Nguatem, W., Drauschke, M. & Mayer, H. (2014): Localization of Windows and Doors in 3D Point Clouds of Facades. International Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume II-3, pp. 87–94.
[6] Nguatem, W., Drauschke, M. & Mayer, H. (2016): Automatic Generation of Building Models with Levels of Detail 1–3. International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol. XLI-B3, pp. 649–654.