Large-scale diffusion models and, similarly, flow models are strong natural image priors [1,2]. This capability is enabled by both architectural advancements, such as Transformers [3], and internet-scale training.

This capability of the diffusion models enabled perceptual image restoration where non-generative or small-scale models failed to produce realistic restorations [4]. However, these methods rely on external feature extractors to condition the diffusion backbones on the degraded inputs. We showed that this is unnecessary, as the backbone itself exhibits a degree of robustness to degradation [5]. However, this requires processing both the degraded input and the diffused image simultaneously, resulting in high computational complexity.

This project will focus on improving the performance of this family of methods.

[1] Rombach, Robin, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. "High-resolution image synthesis with latent diffusion models." In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10684-10695. 2022.

[2] Liu, Xingchao, Chengyue Gong, and Qiang Liu. "Flow straight and fast: Learning to generate and transfer data with rectified flow." arXiv preprint arXiv:2209.03003 (2022).

[3] Peebles, William, and Saining Xie. "Scalable diffusion models with transformers." In Proceedings of the IEEE/CVF international conference on computer vision, pp. 4195-4205. 2023.

[4] Li, Xin, Yulin Ren, Xin Jin, Cuiling Lan, Xingrui Wang, Wenjun Zeng, Xinchao Wang, and Zhibo Chen. "Diffusion models for image restoration and enhancement: A comprehensive survey." International Journal of Computer Vision 133, no. 11 (2025): 8078-8108.

[5] Eteke, Cem, Alexander Griessel, Wolfgang Kellerer, and Eckehard Steinbach. "BIR-Adapter: A parameter-efficient diffusion adapter for blind image restoration." Pattern Recognition (2026): 113824.

Voraussetzungen