Available Projects – Spring 2026

Diffusion and flow-matching models generate images by iteratively denoising an initial noise sample, typically sampled from pure independent white Gaussian noise. Despite their generalization ability (they seem most of the time to generate images that are new), recent scientific works [1, 2, 3, 4] have shown that these models occasionally replicate images from their training data. This is undesirable (e.g., copyright infringement, ethical/legal concerns).

 

Existing works have primarily focused on detecting and mitigating this memorization through indirect mechanisms. For instance, [2] relies on the magnitude of the classifier-free-guidance term used in generation for detecting memorization, [3] introduces a variant of classifier-free-guidance to reduce memorization, [4] finds out neurons with specific activations that triggers memorization in the diffusion model. These approaches do not directly analyze memorization from the core principle of diffusion/flow-matching models itself.

 

The core principle of diffusion and flow-matching models is to train a denoiser model, which, given a noisy image and a noise level, outputs the average of all possible clean images that could have produced this noisy image.

 

In this research project, we want to analyze the issue of memorization from this core principle. Specifically, the following questions are interesting:

Can we identify and fix memorization by checking properties of the denoising prediction? 

Does the denoising prediction indeed look like “the average of all possible clean images that could have produced this noisy image”?

if not, is that because of memorization?

Can we modify the prediction accordingly to remove the memorization?

 

[1] Chen, Y., Wang, S., Zou, D., & Ma, X. (2024). Extracting training data from unconditional diffusion models. arXiv preprint arXiv:2410.02467.

[2] Wen, Y., Liu, Y., Chen, C., & Lyu, L. (2024). Detecting, explaining, and mitigating memorization in diffusion models. In The Twelfth International Conference on Learning Representations.

[3] Chen, C., Liu, D., & Xu, C. (2024). Towards memorization-free diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 8425-8434).

[4] Hintersdorf, D., Struppek, L., Kersting, K., Dziedzic, A., & Boenisch, F. (2024). Finding nemo: Localizing neurons responsible for memorization in diffusion models. Advances in Neural Information Processing Systems, 37, 88236-88278.

 

Deliverables: Deliverables should include code, well cleaned up and easily reproducible, as well as a written report, explaining the experiments and the steps taken for the project.

 

Prerequisites: Python and PyTorch.

 

Level:  Ideally MS research project (semester project), potentially BS research project (semester project)

 

Number of students: 1

 

Supervisor: Martin Nicolas Everaert (martin.everaert [at] epfl.ch)

Diffusion and flow-matching models generate images by iteratively denoising an initial noise sample, typically sampled from pure independent white Gaussian noise.

 

Diffusion inversion refers to the process of recovering an initial noise sample that, when passed through a diffusion model (Stable Diffusion), generates a specific desired image. Inversion is useful because it lets us map images back into the model’s latent space [space of initial noise samples], enabling applications such as image editing [slightly modifying the initial noise for controlled variation of the image] and model interpretability (understanding how the model encodes visual features in its latent space).

 

Most existing inversion techniques (like DDIM inversion [1]) achieve this by reversing the generation process, starting from a clean image, obtaining the model’s denoising prediction (in which direction to move to denoise the image), and move in the opposite direction. This approach assumes detailed access to the model internals, and produces only one initial noise sample.

 

In this project, you will implement and analyze a black-box diffusion inversion algorithm that does not rely on reversing the denoising trajectory. Instead, the diffusion model will be treated purely as a generator: we can input initial noise vectors and observe the resulting images, without access to intermediate predictions or weights.

 

The algorithm iteratively searches and constrains the components of the initial noise sample based on how they influence the generated image. The starting step of the algorithm would be the constraint the lowest-frequency components (ie, the average color) of the initial noise sample to generate images with the desired average color [2]. The following steps would constraint additional components, until the model generates the desired image.

 

Steps of the project: Literature review, implementation of the algorithm (details of the algorithm from the project supervisor), evaluation of the algorithm (failure cases, hyperparameters, etc), potential applications of the algorithm, comparison with other methods, improvement of the algorithm.

 

[1] Song, J., Meng, C., & Ermon, S (2021). Denoising Diffusion Implicit Models. In International Conference on Learning Representations.

[2] Everaert, M. N., Fitsios, A., Bocchio, M., Arpa, S., Süsstrunk, S., & Achanta, R. (2024). Exploiting the signal-leak bias in diffusion models. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (pp. 4025-4034).

 

Deliverables: Deliverables should include code, well cleaned up and easily reproducible, as well as a written report, explaining the experiments and the steps taken for the project.

Prerequisites: Python and PyTorch.

Level:  Ideally MS research project (semester project), potentially BS research project (semester project)

Number of students: 1

Supervisor: Martin Nicolas Everaert (martin.everaert [at] epfl.ch)

Startup company Innoview has developed a software framework to create hidden watermarks printed on paper and to acquire and decode them by a smartphone. The acquisition by smartphone comprises many separate parametrizable parts. The project consists in improving some of the parts of the acquisition pipeline in order to optimize the recognition rate of the hidden watermarks (under Android).

Deliverables:

• Report and running prototype.

Prerequisites:

• basic knowledge of image processing and computer vision,
• Coding skills in Java Android, C#, and/or Matlab

 

Level: BS or MS semester project

 

Supervisors:

Dr. Romain Rossier, Innoview Sàrl, [email protected], tel 078 664 36 44

Prof. Roger D. Hersch, BC320, [email protected], cell: 077 406 27 09

Startup company Innoview has developed arrangements of lenslets that can be used to create document security features. The goal is to improve these security features and to optimize them by simulating the interaction of light with these 3D lenslet structures, using the Blender software.

 

Deliverables:

• Report and running prototype (Matlab). Blender lenslet simulations.

 

Prerequisites:

• knowledge of computer graphics, interaction of light with 3D mesh objects,
• basic knowledge of Blender,
• Coding skills in Matlab

 

Level: BS or MS semester project

 

Supervisors:

Prof. Roger D. Hersch, BC320, [email protected], cell: 077 406 27 09

Dr. Romain Rossier, Innoview Sàrl, [email protected], tel

078 664 36 44

Startup company Innoview has developed arrangements of transparent lenslets and of opaque structures that yield interesting moiré effects.

The goal is to create plastic objects composed of a revealing layer made of transparent lenses and of a base layer made of partly opaque structures. The superposition of the two layers shows interesting moiré evolutions. Once created as 3D volumes, their aspect can be simulated in Blender. After simulation and verification, these objects are to be printed by a 3D printer.

Deliverables:

    Report and running prototype (Matlab). Blender lenslet simulations. Fabricated 3D objects showing the moiré evolutions.

Prerequisites:

1. Good knowledge of computer graphics, especially the construction
of 3D mesh objects,
2. Basic knowledge of Blender,
3. Good coding skills in Matlab

Level: BS or MS semester project, master’s project

Supervisors:

Prof. Roger D. Hersch, BC320, [email protected], cell: 077 406 27 09

Dr. Romain Rossier, Innoview Sàrl, [email protected], tel 078 664 36 44