This project aims to develop novel learning-based methods for image restoration and motion estimation that do not require collecting ground truth data. Learning-based methods have demonstrated the capability of achieving a very high performance not only on semantic tasks, but also on low-level computer vision tasks, such as image restoration and motion estimation tasks. These tasks typically require a large data set of input-output pairs, where, for example, in motion estimation the input is a pair of subsequent video frames and the output is the latent optical flow. While this data enables a very effective training of neural network models, collecting such pairs is very often costly (e.g., it requires specialized hardware) or does not scale well (e.g., parallel capture requires multiple replicas of the same specialized hardware). One way to address this limitation is to generate synthetic data either with a rendering engine and then adapt the domain gap with real data or by directly learning to generate realistic data. We aim to use this framework to address image restoration tasks such as denoising, deblurring, super-resolution, demosaicing, reflection separation, and motion estimation tasks such as optical flow estimation and tracking. Because of the broad applicability of this framework to problems where the image-formation model is (at least partly) known, we expect this project to make a substantial impact in both academia and the industry.
Last updated:17.07.2023
