The Problem
Smooth surfaces, as polished, coated, or wet surfaces, produce specular reflections under unfavourable lighting conditions. Image processing algorithms, such as any correlation-based computation, may completely fail under specular reflection. In order to make texture-based tracking more robust, a reconstruction method was devised which estimates the image structure of the specular region from the surrounding image structure.
Methods
Specular image regions are detected by histogram thresholding. The eigenvector of the smaller eigenvalue of the structure matrix J, the "structure vector", gives the orientation of the local image structure. The intersections of the structure vector with the boundaries of the specular region define the interpolation interval [p1, p2]. Specular regions are filled by linear image interpolation and final Gaussian smoothing.
Results
Application of the algorithm to a metallic surface, the Orbital Removable Unit (ORU), is shown in this sequence of images: a detail of the ORU with specular reflections, with detected specular regions in black, and with reconstructed image regions.
Specular reflections are also a severe problem in medical image processing when dealing with wet and glossy tissue. Our reconstruction method was hence applied to the problem of robustly tracking natural landmarks on the beating heart surface. This image sequence shows a section of a heart surface with a mechanical stabilizer, the same section with specular reflections suppressed, and a larger section with three landmarks and associated tracking areas.
The plots show the dominant degrees of freedom of an affine model of the tracked heart-surface motion. The effects of heartbeat (short waves) and respiration (long waves) can easily be observed.
The tracking performance is strongly improved when specularities are suppressed [1, 2]. Moreover, tracking is more robust when augmented by a prediction algorithm of the heart-surface motion. The tracker is part of a project on robotically assisted surgery; see the section on medical technologies for a description of the complete system.
Publications
[1] M. Groeger, T. Ortmaier, W. Sepp, and G. Hirzinger, Tracking local motion on the beating heart, in Medical Imaging 2002, volume 4681 of SPIE, pp. 233-241, San Diego, USA, February 2002.
[2] M. Groeger, W. Sepp, T. Ortmaier, and G. Hirzinger, Reconstruction of image structure in presence of specular reflections, in Pattern Recognition, 23rd DAGM Symposium, volume 2191 of LNCS (Springer), pp. 53-60, Munich, Germany, September 2001.