BlenderProc
BlenderProc is a software to generate realistic images for the training of neural networks. Under the hood it uses Blender to generate photo-realistic synthetic datasets and can be controlled via an intuitive python API. We focus on the realism of the final images instead of only on their generation speed. This realism is achieved by using a path tracer that follows the path of virtual light beams from a virtual light source to the virtual camera. Physical material properties then determine how the light interacts with the 3D scene and appears in the image. We try to minimize the difference between the generated synthetical images and real images (sim-to-real gap).
Our pipeline can be employed in various use cases, including segmentation, depth, normal and pose estimation, and many others. A key feature of our tool is the simple-to-use python API, designed to be easily extendable. Furthermore, many public datasets, such as 3D FRONT or ShapeNet, are already supported, making it easier to clutter synthetic scenes with additional objects. The most significant advantage of BlenderProc is however its large toolbox and the provided examples. Finally, BlenderProc does not only support the rendering of color, depth, distance, surface normals, and semantic segmentation, but is also capable of rendering optical flow and normalized object coordinates (NOCS) and then save the data either in hdf5 containers or in the BOP or COCO formats.
For further information, please have a look at our GitHub repository and our publications.
Download link: https://github.com/DLR-RM/BlenderProc
If you use BlenderProc in a research project or publication, please cite as follows:
@article{Denninger2023, doi = {10.21105/joss.04901}, url = {https://doi.org/10.21105/joss.04901}, year = {2023}, publisher = {The Open Journal}, volume = {8}, number = {82}, pages = {4901}, author = {Maximilian Denninger and Dominik Winkelbauer and Martin Sundermeyer and Wout Boerdijk and Markus Knauer and Klaus~H. Strobl and Matthias Humt and Rudolph Triebel}, title = {BlenderProc2: A Procedural Pipeline for Photorealistic Rendering}, journal = {Journal of Open Source Software} }
Publications
- Maximilian Denninger, Dominik Winkelbauer, Martin Sundermeyer, Wout Boerdijk, Markus Knauer, Markus Wendelin, Klaus H. Strobl, Matthias Humt, Rudolph Triebel,"BlenderProc2: A Procedural Pipeline for Photorealistic Rendering", 8 (82), p. 4901, Februar, 2023. https://elib.dlr.de/193977/
- Maximilian Denninger, Martin Sundermeyer, Dominik Winkelbauer, Dmitry Olefir, Tomas Hodan, Youssef Zidan, Mohamad Elbadrawy, Markus Knauer, Harinandan Katam, Ahsan Lodhi "BlenderProc: Reducing the Reality Gap with Photorealistic Rendering" in: Proc. RSS 2020. Robotics: Science and Systems (RSS), Virtuell, Juli 2020. https://elib.dlr.de/139317/
- Maximilian Denninger, Martin Sundermeyer, Dominik Winkelbauer, Youssef Zidan, Dmitry Olefir, Mohamad Elbadrawy, Ahsan Lodhi, and Harinandan Katam. October, 2019. arXiv:1911.01911.https://arxiv.org/abs/1911.01911
