3D Object Tracking

Humanoid robot David is perceiving a cup using his RGB-D camera; in the following, he will grasp it and fill the dishwasher.
DownloadDownload

Tracking objects and kinematic structures in 3D space and determining their poses and configurations is an essential task in computer vision. Its applications range from augmented reality to robotic perception. Given consecutive image frames, as well as 3D meshes and kinematic information, the goal is to robustly estimate the rotation and translation of all bodies relative to a camera. While the problem has been thoroughly studied, many challenges such as partial occlusions, appearance changes, motion blur, background clutter, object ambiguity, and real-time requirements still exist.

In this repository, we publish algorithms and code of our ongoing research on 3D object tracking. The folders for the different algorithms include everything necessary to reproduce results presented in our papers. Currently, the latest version of our code can be found in the package M3T.

For further information, please have a look on our GitHub-Repository and our publications.

Download link: https://github.com/DLR-RM/3DObjectTracking

License: https://github.com/DLR-RM/3DObjectTracking/blob/master/LICENSE

Publications

  • Manuel Stoiber, Martin Pfanne, Klaus Strobl, Rudolph Triebel, Alin Olimpiu Albu-Schäffer, "A Sparse Gaussian Approach to Region-Based 6DoF Object Tracking", in: 15th Asian Conference on Computer Vision, ACCV 2020, Kyoto, Japan, pp. 666-682, 2020 [elib] (presentation and demonstration videos)
  • Manuel Stoiber, Martin Pfanne, Klaus Strobl, Rudolph Triebel, Alin Olimpiu Albu-Schäffer, "A Sparse Region-Based 3D Object Tracking Approach for the Real World", International Journal of Computer Vision, pp. 1008-1030, April 2022 [elib] (video)
  • Manuel Stoiber, Martin Sundermeyer, Rudolph Triebel, "Iterative Corresponding Geometry: Fusing Region and Depth for Highly Efficient 3D Tracking of Textureless Objects", in: IEEE. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, pp. 6845-6855, 18-24 Jun 2022 [elib] (presentation and demonstration videos)
  • Manuel Stoiber, Mariam Elsayed, Anne E. Reichert, Florian Steidle, Dongheui Lee, Rudolph Triebel, "Fusing Visual Appearance and Geometry for Multi-modality 6DoF Object Tracking" https://arxiv.org/abs/2302.11458 (video)
  • Manuel Stoiber, Martin Sundermeyer, Wout Boerdijk, Rudolph Triebel "A Multi-body Tracking Framework - From Rigid Objects to Kinematic Structures"  https://arxiv.org/abs/2208.01502 (video)