Open Student Projects

Generative Workflow for Navigable 3D Worlds from a Large-Scale Panoramic Painting

Master thesis (DH/IC)

Supervisor: Tsz Kin Chau and Prof. Sarah Kenderdine

Project description:

Generative AI can now turn images into navigable 3D scenes, which opens new ways to experience large-scale panoramic paintings in VR/XR. Existing work has shown that a single painting can be converted into an explorable 3D scene. It is not known how to do this for a painting that depicts a dense, continuous and extensive scene, such as a 360° battle panorama with thousands of figures spread across a whole landscape.

The difficulty goes beyond the domain gap between photographic training data and painted input. Four problems compound it:

• Scale. Current models keep spatial consistency and detail only within a limited area, far smaller than a panoramic landscape.
• Density. Crowds, overlapping figures and layered action exceed what a single generation can resolve.
• Continuity. Scenes generated separately have to join into one coherent space.
• Conditioning. The painting must condition the generation throughout, so that the result stays faithful to the artwork rather than drifting toward the model’s own idea of a plausible landscape.

The project therefore takes a bottom-up approach. The student will first generate a single, localized scene and build a reproducible workflow around it. Once the workflow is established, it can be repeated across the painting, and the resulting scenes placed on a geolocated map of the depicted site, following the approach of the lab’s PLACE and ALATS projects. The primary case study is the Murten Panorama, a 10×100 m oil painting (1893) of the 1476 Battle of Murten, digitized at 1.6 terapixels.

No single model covers the whole problem today. Two complementary families of technology handle different parts of 3D, namely world models and 3D asset generation. Both families are highly sensitive to their inputs. Prompts engineering, style transfer and the use of intermediate images and generated novel views all shape the result. The core challenge is to combine these technologies into a reproducible workflow conditioned on the painting. The result should be not only plausible but convincing in the context of the painting and its history.

The approach is deliberatly open. The following interconnected directions are examples of viable paths, and the student is free to combine, adapt or replace them:

• Input preparation and conditioning. Turn painting segments into inputs that generative models handle well, for example through restyling, completion and intermediate image generation, while keeping them anchored to the painting.
• Workflow design. Combine world-model environments with generated 3D assets into a single scene, and make every step reproducible.
• Scene composition and placement. Assemble the results into a navigable scene, join multiple scenes into a continuous space on a terrain map.
• Agentic methods. Use AI agents to orchestrate the above process.

The emphasis across these directions is flexible and should be agreed with the student based on their skills and background.

Prerequisite: 

• Must have:

  • Proven 3D modelling skills, mastered before starting (a portfolio is expected)
  • Hands-on experience with generative AI tools for images and 3D
  • A good artistic sense
  • The ability to work independently

  Should have:

  • Machine learning and computer vision experience
  • Python skills, to script and automate generation steps

  A plus:

  • Experience with AI agents or tool orchestration
  • Interest in documentary or historical research

What the project offers:

The student will work at the intersection of generative AI, 3D production and digital humanities, on the largest digitized paintings in the world. The project offers hands-on experience with state-of-the-art world models, 3D asset generators and agentic workflows, and the chance to define a workflow that does not yet exist. A working workflow would make 3D VR/XR heritage production technique more accssible and could apply to panoramic and narrative paintings well beyond Murten.

Beyond the Strings: LLM-Based Knowledge Extraction for Historical Puppetry Knowledge Graphs

Master thesis (DH/IC)

Supervisor: Heden Yan and Prof. Sarah Kenderdine

Project Description

A puppet can be a theatrical character, a heritage object, an avant-garde artwork, and a museum object. Yet the historical connections between these identities are often scattered across theatrical scripts, museum archives, and scholarly publications. How can computational methods help us uncover these relationships while preserving the complexity of historical knowledge and interpretation?

This project takes inspiration from Sophie Taeuber-Arp’s extraordinary König Hirsch (1918) marionette ensemble, a landmark of early modernist puppetry and avant-garde design. Adapted from Carlo Gozzi’s Italian play Il re cervo (1762), the production connects traditional theatre with modernist experimentation and intellectual debates (e.g., Dadaism, psychoanalysis). The Laboratory for Experimental Museology (eM+) previously brought these puppets to life through gesture-based interaction in Animating the Avant-Garde: Sophie Taeuber-Arp’s Puppets, presented at Museum für Gestaltung Zürich in 2025.

Building on this remarkable case, the project will focuse on evaluating LLM-based extraction of historical relationships and scholarly interpretations from a small selection of theatrical and archival texts. Working with a predefined critical ontology, the student will investigate and compare how language models can distinguish historical facts from scholarly interpretations, preserve their source references, and produce structured data suitable for knowledge graph integration.

Main Activities

1. Structured Knowledge Extraction: Implement and test extraction strategies using locally deployed LLMs, focusing on historical entities, relationships, and attributed scholarly interpretations.
2. Evaluation & Error Analysis: Compare selected models or prompting strategies against manually annotated examples. Evaluate extraction accuracy, source attribution, and common errors, particularly when dealing with ambiguous or competing interpretations.
3. Knowledge Graph Compatibility: Map the extracted information to a predefined ontology and demonstrate how a small set of verified relationships and interpretations can be represented as connected knowledge.

A small, authorised research corpus, an initial ontology, and reference annotations will be prepared. The project focuses on developing and evaluating the extraction method, rather than building a complete historical knowledge system.

Prerequisites

• Good Python programming skills
• Basic knowledge of machine learning and NLP
• Experience with LLMs, information extraction, or knowledge graphs is a plus
• Interest in digital humanities, cultural heritage, and critical approaches to knowledge representation
• Knowledge of Italian and/or German would be an advantage for working with the original theatrical text

What the Project Offers

This project offers an opportunity to explore the rich history of Sophie Taeuber-Arp’s remarkable marionette collection, connecting early modernist art, theatrical traditions, and intellectual history.

Alongside practical experience with local LLMs, structured extraction, and evaluation methods, the project offers an opportunity to develop critical approaches to historical data: understanding how different sources relate, why scholarly interpretations may differ, and how computational systems can preserve these distinctions.

Working closely with an interdisciplinary research team at eM+, the student will contribute to a reproducible extraction and evaluation prototype that can support further research into historical knowledge graphs and cultural heritage interpretation.

References

Schimmenti, A., Pasqual, V., Vitali, F., & van Erp, M. (2026). Knowledge graphs generation from cultural heritage texts: Combining LLMs and ontological engineering for scholarly debates. Journal of Documentation.

CIDOC CRM Special Interest Group. (2024). Definition of the CIDOC Conceptual Reference Model (Version 7.1.3).

Canning, E. (2026). The CAAPT ontologies: An ontological framework for the representation of museum-based critical cataloging. Semantic Web, 17(4).

Evaluation Framework for Cross-Domain Visual Retrieval in Cultural Heritage Image Collections

Master thesis (DH/IC)

Supervisor: Tsz Kin Chau and Prof. Sarah Kenderdine

Project description:

We have developed a Human-in-the-Loop (HITL) toolset for exploring and comparing large visual heritage collections. The primary case study is the Murten Panorama, a 10×100m oil painting (1893) digitized at 1.6 terapixels, studied alongside 15th-century illuminated manuscripts. The toolset supports correspondence detection and prototype discovery at scale, combining vision-language embedding models (e.g., SigLIP 2, CLIP) with VLM embedding models (e.g., Qwen3-VL). It includes interactive UMAP-based visualization of embedding spaces, enabling scholars to explore visual similarity across collections.

However, we currently lack a rigorous, reproducible evaluation framework to assess the performance of these tools under their various configurations.

The core objective is to develop an evaluation framework for assessing the performance of cross-domain visual retrieval and visualization tools in the context of cultural heritage image collections. The student will work on three interconnected components:

• Dataset survey and repurposing. Identify existing datasets that can be adapted as benchmarks for cross-domain visual similarity tasks. The goal is to assemble a portfolio of evaluation sets, each testing a different facet of the retrieval pipeline.
• Targeted evaluation set construction. For aspects not covered by existing datasets, the student will construct targeted evaluation sets with generative AI.
• Evaluation protocol and benchmarking. Define and implement evaluation metrics for two complementary tasks: (1) retrieval accuracy and (2) visualization fidelity.

The scope and emphasis across these three components is flexible and should be discussed with the student based on their interests and background. The project does not require model training. It is fundamentally about evaluation methodology.

Prerequisite: 

• Excellent Python skills
• Experience in AI image generator
• Interest in evaluation methodology and experimental design
• Experience with computer vision or digital humanities is a plus but not required

What the project offers:

The student will work at the intersection of computer science and digital humanities. The evaluation framework produced will have direct impact on how cross-domain visual retrieval tools are developed and assessed in the cultural heritage domain. The project offers hands-on experience with state-of-the-art vision-language models, embedding pipelines, and dimensionality reduction techniques, applied to a unique and visually rich dataset.

Reference:

Tschirschwitz, D., Klemstein, F., Schmidgen, H., & Rodehorst, V. (2023). Drawing the Line: A Dual Evaluation Approach for Shaping Ground Truth in Image Retrieval Using Rich Visual Embeddings of Historical Images. Proceedings of the 7th International Workshop on Historical Document Imaging and Processing, 13–18. https://doi.org/10.1145/3604951.3605524

Jiang, Z., Meng, R., Yang, X., Yavuz, S., Zhou, Y., & Chen, W. (2024, October 4). VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks. The Thirteenth International Conference on Learning Representations.