Advanced robotic systems require visual perception capabilities beyond plain 2D images or proximity sensors. They instead rely on 3D perception in order to enable the holistic perception of their surroundings that is eventually needed for high-level, task-related reasoning. 3D perception encompasses the acquisition of data based on commercially available or self-developed sensors, the creation of 3D models and formats in different representations, up to the use of these models/data for object recognition. The robotic applications of 3D perception include exploration, navigation, object manipulation, and telepresence.
Visual Sensing is a classical approach in robotics for non-contact sensing. At RMC by default we deploy off-the-shelf visual sensors (digital cameras, stereo heads, Kinect, etc.). Due to the rising demands of complex robotic applications, however, we oftentimes reach their limits. Then we recur to high-end sensors or we develop custom-built sensor systems as well as the required sensor-oriented computational methods.
3D models are required in a variety of applications ranging from small objects for pose estimation and grasping to large buildings or environments for navigation and localization. Depending on the application different representations of 3D models such as surface, volumetric, or feature point models are required. The data acquisition and modeling should be carried out in a real-time stream for instant application. Further, the 3D model needs to be segmented to identify objects of interest e.g. for identifying walls, tables, objects in an indoor environment.
Whenever concrete models and specific knowledge are not available for objects or events in the robot's work environment, a robot system has to rely on more generalized modes of inference to arrive at the semantic content of the situation. This is a common scenario, e.g., for robots in the human living or working environment, and for systems that need to interact closely with humans. Adequate models and knowledge may then describe broad categories of objects or events, acquired through training on sets of numerous examples. Knowledge may also be derived from similarities and correspondences discovered between novel and known cases.