Open Projects

This project investigates whether graph foundation model embeddings can provide a more transferable interface between graphs and large language models than conventional GNN or text-encoder-based representations. A lightweight projector maps frozen graph embeddings into the LLM latent space, enabling graph-conditioned instruction tasks such as link prediction and node classification.

Project Creator:

Adam Hazimeh ([email protected])

Units Involved:

Laboratoire de traitement des signaux 4 (LTS4)

Tags:

AI, Master,

Memberships:

Nour Guermazi – Role: Student, Adam Hazimeh – Role: Owner,

This project studies methods for detecting and mitigating backdoors in open-weight LLM checkpoints by inspecting the model’s internal structure rather than relying only on behavioral prompting. Recent work has shown that sleeper-agent-style backdoors can persist through standard safety training [1], while complementary work suggests that backdoors may leave detectable signatures in parameter space, for example in LoRA weight spectra [2]. Building on this, the project asks whether backdoored checkpoints can be distinguished from clean ones using weight-space statistics, activation-space probes, or mechanistic interpretability tools, and whether the detected signatures can be used to localize, edit, or remove the backdoor while preserving model utility. The broader goal is to develop practical auditing tools for untrusted or third-party LLM checkpoints.

Note: This project is ongoing, and one possible outcome is a demo that showcases the final developed detection mechanism.

[1] Hubinger, Evan, et al. “Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training.”
[2] Merenciano, David Puertolas, et al. “Weight Space Detection of Backdoors in LoRA Adapters.”

Project Creator:

Adam Hazimeh ([email protected])

Units Involved:

Laboratoire de traitement des signaux 4 (LTS4)

Tags:

PhD, AI, Master,

Memberships:

Adam Hazimeh – Role: Owner, Amel Abdelraheem – Role: Supervisor, Michele Smaldone – Role: Student,

This project explores concept-based representations in pretrained vision models, focusing on concept bottleneck models (CBMs), sparse autoencoders (SAEs), and hybrids of the two. Recent work suggests that separately trained CBMs and SAEs can recover similar concept-oriented representations under certain conditions [1]. Building on this, the project asks how these methods compare in terms of interpretability, disentanglement, information leakage, and downstream usefulness, with the broader aim of understanding how to build more interpretable and robust concept-based vision models. 

[1] Fel, Thomas, and Gianni Franchi. "A Geometric Unification of Concept Learning with Concept Cones." arXiv preprint arXiv:2512.07355 (2025)

Project Creator:

Amel Abdelraheem ([email protected])

Units Involved:

Laboratoire de traitement des signaux 4 (LTS4)

Tags:

AI,

Memberships:

Amel Abdelraheem – Role: Owner,

Description: Can we learn to read the inner experience of a person through their art? Art Brut — the raw, unmediated artistic expression produced by people with psychiatric diagnoses, outside of any communicative intent — may encode cognitive signatures specific to mental disorders. Working with digitised collections from the Collection de l'Art Brut (Lausanne), the Psychiatriemuseum Bern, and the Prinzhorn Collection (Heidelberg), the student will apply deep learning and computer vision methods to extract and cluster symbolic patterns across thousands of artworks, in dialogue with experts in art history and symbolism. This project is part of a broader transdisciplinary research programme combining AI, psychiatry, and molecular biology.

Tasks:
Build structured representations of artworks using unsupervised/self-supervised vision models (e.g. CLIP, ViTs)Extract and cluster recurrent visual and compositional patterns across diagnoses and artistsDevelop interpretable visualisations of discovered patterns in collaboration with domain experts
Profile: MSc student in Computer Science, Electrical Engineering, or Data Science. Strong background in deep learning and computer vision (PyTorch/JAX). Genuine curiosity for interdisciplinary applications. Experience with NLP or multimodal models is a plus.

Contact: [email protected] (CV + short motivation)

Project Creator:

Pascal Frossard ([email protected])

Units Involved:

Laboratoire de traitement des signaux 4 (LTS4)

Tags:

SEL, SIN, SSC, Data & Network Sciences, Imaging & Vision, Learning & Decision Systems, Bachelor, Master,

Memberships:

Pascal Frossard – Role: Supervisor, Raphaelle Luisier – Role: Owner,

Creating RNAs with desired properties such as folding in a specific structure or binding with a given protein requires an in-depth understanding of sequence-structure dependencies and how these relate to functional properties. Most frameworks for RNA design sequence and structure sequentially, and optimize for one single property at a time. Representing RNA as a graph is a promising path towards designing both RNA sequence and structure at the same time, especially to optimize objectives that require specific structure-sequence patterns. 

We will explore the suitability of graph diffusion frameworks for RNA design, using existing methods developed in the LTS4 lab or elsewhere, with special consideration for how methods scale with sequence length. 

Candidates should have strong mathematical and computational skills and should be familiar with the pytorch framework. Candidates do not necessarily have to have a biological background but should have a strong desire to directly work with experimental biologists.

Project Creator:

Vincent Jung ([email protected])

Units Involved:

Laboratoire de traitement des signaux 4 (LTS4)

Tags:

AI, Saturna, Master,

Memberships:

Vincent Jung – Role: Owner,

A growing body of work has shown that incorporating domain knowledge into diffusion models by constraining their trajectories can significantly improve sample quality across various data modalities [1,2]. While similar ideas have been explored for graph generation, where discrete diffusion-based models have achieved state-of-the-art performance [3], existing approaches still exhibit notable shortcomings. For instance, PRODIGY [4] constrains graph generation by relaxing the adjacency matrix to a continuous space, which fails to fully capture the categorical nature of graphs. ConStruct [5], in contrast, operates entirely in discrete spaces but only imposes constraints at the structural level, without enforcing joint conditions between node and edge attributes. Such joint constraints are critical in many applications—for example, in molecular generation, where validity requires consistency between node types and their incident edges (e.g., a carbon atom must have degree at most 4). In this project, we aim to develop a graph diffusion method that natively enforces joint node-edge constraints during generation, with the goal of advancing state-of-the-art performance in real-world tasks such as molecular design.

Requirements:
Mandatory: At least one deep learning course + prior experience with PyTorch.Plus: Familiarity with graph deep learning and/or diffusion models.
[1] Lou et al., Reflected Diffusion Models, ICML 2023.
[2] Fishman et al., Diffusion Models for Constrained Domains, TMLR 2023.
[3] Vignac et al., DiGress: Discrete Denoising Diffusion for Graph Generation, ICLR 2023.
[4] Sharma et al., Plug-and-Play Controllable Graph Generation with Diffusion Models, ICML 2024.
[5] Madeira et al., Generative Modelling of Structurally Constrained Graphs, NeurIPS 2024.

Project Creator:

Manuel Madeira ([email protected])

Units Involved:

Laboratoire de traitement des signaux 4 (LTS4)

Tags:

EDEE, EDIC, Data & Network Sciences, AI, Master,

Memberships:

Alba Carballo Castro – Role: Supervisor, Manuel Madeira – Role: Owner,

Language Models are inherently black-box models. While interpreting LLMs trained on natural language is an active area of research, particularly in the field of mechanistic interpretability, little work has been done in biological language models, let alone RNA Language models (RNA-LMs). Yet, having a strong understanding of what RNA-LMs capture is important for multiple reasons: a lack of interpretability is one of the main bottlenecks to widespread adoption, our lack of knowledge of what models do and do not capture is limiting our ability to improve them, and it holds the possibility of finding new biology. 

The goal of this project is to study the applicability of existing MechInt method in RNA-LMs, identify the most promising framework for our use case, train an interpretability model on existing RNA-LMs and use extensive annotation datasets to relate sparse features to known biological knowledge. 

This project can be a Master semester project or a Master's thesis. Candidates should have strong mathematical and computational skills and should be familiar with the pytorch framework. Knowledge in Mechanistic interpretability is an advantage. Candidates do not necessarily have to have a biological background but should have a strong desire to directly work with biologists, bioinformaticians and computer scientists. 

 

Project Creator:

Vincent Jung ([email protected])

Units Involved:

Laboratoire de traitement des signaux 4 (LTS4)

Tags:

AI, Saturna, Master,

Memberships:

Vincent Jung – Role: Owner,

The advent of foundation models has revolutionized the landscape of machine learning, introducing a new paradigm where practitioners can access pre-trained checkpoints, such as those available on platforms like Huggingface, tailored for specific tasks. These models are derived from the same initial checkpoints but are fine-tuned on specific tasks, such as CIFAR or MNIST. Task arithmetic techniques [1,2] merge these different models into one multi-task model, i.e., a model that showcases good performance on all involved tasks, without needing additional training.
While the task arithmetic literature has focused on merging of classification models fine-tuned on different inputs, more traditional multi-task learning settings remain unexplored. An important case, for instance, is of a model solving the tasks of semantic segmentation, instance segmentation and depth regression from a single image input [3]. The goal of this project is to leverage existing model merging techniques for the single-input multiple-output case.

References:[1] G. Ilharco, M. T. Ribeiro, M. Wortsman, L. Schmidt, H. Hajishirzi, and A. Farhadi, “Editing models with task arithmetic,” in ICLR 2023. [2] P. Yadav, D. Tam, L. Choshen, C. A. Raffel, and M. Bansal, “TIES-Merging: Resolving interference when merging models,” in NeurIPS 2023.
[3] A. Kendall, Y. Gal, and R. Cipolla, “Multi-task learning using uncertainty to weigh losses for scene geometry and semantics,” in CVPR 2018

Requirements:Applicants must have completed at least one deep learning course and have experience with PyTorch. Familiarity with multi-task learning and model merging techniques is preferred.

Contact: [email protected], [email protected]

Project Creator:

Nikolaos Dimitriadis ([email protected])

Units Involved:

Laboratoire de traitement des signaux 4 (LTS4)

Tags:

EDEE, AI, Master,

Memberships:

Ke Wang – Role: Supervisor, Nikolaos Dimitriadis – Role: Owner,

This project tackles the challenge of integrating valuable knowledge from specialized models trained on sensitive or proprietary datasets into larger, generalist models. In fields like medicine, where data privacy and regulatory constraints limit data sharing, a small specialized model might excel at diagnosing rare diseases based on limited data. This project aims to leverage such specialized models to improve the performance of broad, generalist models. By applying innovative techniques like unsupervised reverse knowledge distillation [1][2] and model merging[3][4], this approach aims to enhance generalist models’ capabilities while adhering to privacy concerns. Engaging in this project will provide hands-on experience with cutting-edge methods in machine learning and offer a chance to contribute to practical solutions in data-sensitive domains.
References:
[1] Sahar Almahfouz Nasser, Nihar Gupte, and Amit Sethi. Reverse knowledge distillation: Training a large model using a small one for retinal image matching on limited data. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 7778–7787, 2024.
[2] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015.
[3] G. Ilharco, M. T. Ribeiro, M. Wortsman, L. Schmidt, H. Hajishirzi, and A. Farhadi, “Editing models with task arithmetic,” in ICLR 2023.
[4] Guillermo Ortiz-Jimenez, Alessandro Favero, and Pascal Frossard. Task arithmetic in the tangent space: Improved editing of pre-trained models. Advances in Neural Information Processing Systems, 36, 2024.

Project Creator:

Ortal Yona Senouf ([email protected])

Units Involved:

Laboratoire de traitement des signaux 4 (LTS4)

Tags:

Learning & Decision Systems, AI, Master,

Memberships:

Ortal Yona Senouf – Role: Owner, Nikolaos Dimitriadis – Role: Supervisor,

Graph Neural Networks (GNNs) have been established as a very competitive modeling in a wide range of applications due to their inductive bias and expressivity. However, their extension to temporal data becomes very challenging as the spatial and temporal updates are generally entangled. In this project we try to investigate how Neural ODE formulation helps us to learn latent graph trajectories, enabling interpretabilty of the flow field and learning in scarce data setting.

Project Creator:

Jérémy Jean Philippe Baffou ([email protected])

Units Involved:

Laboratoire de traitement des signaux 4 (LTS4)

Tags:

EDIC, SIN, SMA, SSC, SSIE, Data & Network Sciences, Learning & Decision Systems, Theory and Simulation, AI, Master, SV,

Memberships:

Jérémy Jean Philippe Baffou – Role: Owner,

Test-Time Adaptation (TTA) allows models to dynamically adjust to unfamiliar domains, specific prompt formats, or new reasoning tasks at inference time without needing a labeled dataset for fine-tuning. Instead of computationally expensive retraining, TTA in LLMs typically works by rapidly updating lightweight parameters (like LoRA adapters), tuning small components of the model such as BatchNorms to the distribution of the test set, or by using self-labeling of test samples. This enables the LLM to self-calibrate and align its internal representations to the user's specific context on the fly, bridging the gap between general pre-training and specialized deployment. 

This project aims to explore the use of TTA in RNA Language Models, in particular for their use in structure prediction for rare RNAs on which the model might not have been trained, or has seen very few similar examples. Structure prediction for these RNAs often fails with traditional methods as there is little to no co-evolutionary signal in homologs, or simply a lack of homologs. 

Candidates should be familiar with pytorch. 

1.Li, Yanghao, et al. "Revisiting batch normalization for practical domain adaptation." arXiv preprint arXiv:1603.04779 (2016).

Project Creator:

Vincent Jung ([email protected])

Units Involved:

Laboratoire de traitement des signaux 4 (LTS4)

Tags:

AI, Saturna, Master,

Memberships:

Vincent Jung – Role: Owner,

Tokenization in most RNA-Language Models (RNA-LMs) is done at the nucleotide-level. Attempts to adopt methods from the NLP literature such as Byte-Pair Encoding have been shown to lead to worse performance on downstream tasks [1]. Yet, moving away from nucleotide-level tokenization can greatly increase the effective input length of transformer-based RNA-LMs as well as create better inductive biases for generalization where the model can learn to decompose new samples into previously seen semantic units that are more meaningful than sole nucleotides. 

The goal of this project is to develop a new dynamic tokenization method specifically designed with RNA in mind, which incorporates known biological priors.

This project can be a Master semester project or a Master's thesis. Candidates should have strong mathematical and computational skills and should be familiar with the pytorch framework. Candidates do not necessarily have to have a biological background but should be willing to learn some fundamentals of RNA biology and work with bioinformaticians.

[1] Morales-Pastor, Adrián, et al. "Character-level Tokenizations as Powerful Inductive Biases for RNA Foundational Models." arXiv preprint arXiv:2411.11808 (2024).

Project Creator:

Vincent Jung ([email protected])

Units Involved:

Laboratoire de traitement des signaux 4 (LTS4)

Tags:

AI, Saturna, Master,

Memberships:

Vincent Jung – Role: Owner,

Topoformer: A Spatial-Informed Foundation Model for Spatial-Biology
Can a single model learn to read the architecture of a tissue? Modern tissue imaging (IMC, CODEX, MIBI) now measures dozens of protein markers at single cell resolution across millions of cells, yet most models read this data as chemistry alone and ignore its shape. 

In this project you'll help build a foundation model for spatial biology. Using topological data analysis (TDA),  you'll turn each tissue into a compact set of spatial tokens where structure is explicit and provable, something today's spatial biology models struggle to capture. Then we will develop and train a novel transformer architectures on top of them. 

You'll work hands on with self supervised pretraining across large public cohorts (20+ datasets, ~10M cells), attention mechanisms, and zero shot transfer, running large scale experiments toward tasks like patient stratification, survival prediction, and biomarker discovery. The work blends mathematical framing with modern deep learning and is aimed at a publication in a top-tier venue (ICML/NeurIPS/…) 

Project Creator:

Jérémy Jean Philippe Baffou ([email protected])

Units Involved:

Laboratoire de traitement des signaux 4 (LTS4)

Tags:

EDIC, EDPY, IF, MTE, SEL, SIN, SMA, SODH, SPH, SSC, STI-SNX, SV-SNX, IC-SSIQ, SB-SSIQ, STI-SSIQ, Data & Network Sciences, Imaging & Vision, Learning & Decision Systems, Personalized Health, AI, Master, Short project, Master thesis, Spatial transcriptomics, spatial transcriptomics, AI-classifier, machine learning, vision, mathematical modeling, Machine learning, Computer vision, proteomics, SV,

Memberships:

Pascal Frossard – Role: Supervisor, Dorina Thanou – Role: Supervisor, Jérémy Jean Philippe Baffou – Role: Owner,