Publication
Physically based rendering is one of the cornerstones of computer graphics, and has evolved in the last decades into a mature technology, with applications ranging from hyperrealistic movies to product design. The recent efficiency of traditional algorithms, combined with the widespread use of automatic differentiation framework in machine learning has led to the emergence of differentiable rendering as a powerful tool for solving inverse problems involving radiative transfer through gradient-based optimization.
This thesis investigates issues related to the use of such methods in practical scenarios. We highlight that merely differentiating a traditional rendering algorithm is not sufficient to guarantee acceptable results, and therefore new representations and algorithms are needed.
In the first part of the thesis, we show that one crucial aspect driving result quality is the choice of representation for the optimized parameters. In particular, classical triangle mesh representations are not well-suited for optimization tasks, and conventional workarounds like regularization are suboptimal. We introduce a novel parameterization of the geometry that incorporates a notion of neighborhood, which provides smoother gradient steps, and improves the quality of image-based 3D reconstruction drastically.
In a second part, we focus on the efficiency of differentiable rendering algorithms. We show that the iterative nature of the optimization process leads to redundancy in the rendering phases, which can be exploited for variance reduction. We introduce a simple and efficient method to reuse information across iterations, which can drastically increase the accuracy of gradients for a given time budget.
Finally, we discuss how specific applications can benefit from tailored algorithms. We design a differentiable rendering system that is specialized for tomographic volumetric additive manufacturing, a light-based 3D printing technique. By designing a custom algorithm, we can obtain high-quality gradients that allow to use advanced optimization techniques. Our framework is more general than prior work in this field, which expands the exploration space for this technology.