Position for working/master student: Semantic shape completion of objects for robotic manipulation
Background:
In the domain of assistive robotics, i.e., for support and service in elderly care homes or in hospitals, one of the main challenges is the manipulation of unknown every-day objects. For example, proper handling of a mug and a pitcher that the robot has not seen before and has no specific grasp specification for picking up and pouring a beverage poses a challenge with perceptual and planning aspects. In this project the perceptual problem will be addressed. It will be treated as one of analyzing the shape of objects in terms of geometry and functionality. Specifically, the goal is the reconstruction of the full 3D geometry from the partial point cloud data that can be acquired with a depth camera from a single viewpoint on the object, as well as the segmentation of it into the object’s semantic or functional parts. There are two components of a solution that need to be considered: the generation of a large training dataset and the optimization of a deep neural network architecture. There is prior work done at our institute, as well as in the community outside, that this project work has to be based on.
Your tasks:
- Training of deep neural networks on existing datasets for shape completion and part segmentation
- Extending the existing datasets through training of networks in a self-supervised fashion on unlabeled data
- Optimization of the neural network architecture for improving shape completion and part segmentation
- Optimization of the neural network architecture for improving shape completion and part segmentation
- Comparison to known methods for shape completion and part segmentation
- Investigating and comparing to methods based on foundation models
Your qualifications:
- Final year master student in computer science, robotics, or related
- Strong background in machine learning, in particular deep learning
- Knowledge of, preferably even experience with methods for 3D point cloud processing
- Proficiency in Python programming, in particular using TensorFlow or PyTorch
- Experience in scientific data analysis
Suggested reading:
https://arxiv.org/abs/2006.07733
https://arxiv.org/abs/2104.14294
https://arxiv.org/abs/2401.10891
https://arxiv.org/abs/2305.14335
https://arxiv.org/abs/2411.07184
Start at the earliest possible date. Contract duration depending on your availability: between 2 and 6 months as working student, followed by 6 months for the thesis project.
For application, first make contact (see below) to indicate your interest. We will then get back to you.