Student Projects

LTS5 OPEN SEMESTER AND MASTER PROJECTS – Spring 2020

SEMESTER PROJECT PROPOSALS

1. Facial Attributes Modeling

Human face has always been of particular interest in the computer graphics community. Because of its complexity, modeling lifelike synthetic objects is challenging. A variety of approaches have been proposed such as statistical models (i.e. principal component analysis models) or blendshapes models. Moreover, tackling the variation in terms of population (*i.e. identity*) and expression at the same time in a generic 3D model increases the difficulty.

With conventional modelling technics, the detailed facial attributes such as the wrinkles are lost in the process. Moreover these mid-frequencies informations are important for photo-realistic expressions generation. Therefore an explicit model can be jointly used to augment the original face model to recover them. Furthermore, photo-realistic rendering shading rely on depth cue given by ambient occlusion (AO). The ambient occlusions define how the light is attenuated at a specific location and is therefore dependent on the geometry of the face.

The goals of this project are in two folds. Build a model of the detailed facial attributes distribtion from high resolution meshes. Then explore the relation between the geometry and its corresponding ambient occlusions map in order to generate them it (i.e. regresse ambient occlusions from shape coefficients).

Requirements: The project will be implemented in Python / C++ so good knowledge is required. Previous experience in one or several of the following topics would be a plus: image processing, computer vision or machine learning.

Assistant: Christophe Ecabert ([email protected])

Supervisor: Prof. Jean-Philippe Thiran

2. Procedural generation of synthetic Brain tissue: Modeling vs Reinforcement Learning.

 Procedural generation is a method for creating data algorithmically — using a IA agent for example. Typically,  this requires a combination of human-generated assets coupled with computer-generated randomness and processing power. A interesting application in video-games of such algorithms are the so-called procedural generated maps, which are able to create entire random levels or environments for the player.  

In this project, we will use such algorithms with a twist: We will use them to generated realistic 3d mesh models (maps) of the Brain’s the matter microstructure. Such 3d environments are a fundamental part of the study of the brains microstructure via simulations. 

The task of the student will be to develop and test novel algorithms for this task; from model based algorithms (using stochastic properties and modeling)  to the training of self-supervised algorithms (Deep Reinforce Learning)

Plus: Use of Deep Learning Platforms (Tensorflow, (or/and) Pytorch, Keras)

Supervisor: Prof. Jean-Philippe Thiran
Assistant: Jonathan Rafael-Patiño

3. Apizoom – deep learning to quantify the Verroa parasite in honey bee hive images

Varroa mites are recognized as the biggest pest to honey bees worldwide, and are believed to be the single largest contributing factor in the modern-day decline of honey bees due to their ability to transmit diseases, resulting in death or severe deformity of the pupae.

Verroa on honey bees.

Detecting and quantifying the presence of Verroa in a beehive is therefore crucial to treat the infection appropriately and as early as possible, and image analysis appears very useful in this problem.

In this project, we propose to develop an image analysis to detect and count Verroa cadavers who felt down on a plate below the beehive, as a non-intrusive way to quantify the presence of the Verroa. High definition images will be capture and Deep Learning techniques will be investigated here, to design and train a Convolutional Neural Network (CNN) to detect the Verroa and distinguish it from other wastes. See the following video for more details on the project (in French).

Depending on the evolution of the project, several steps will be investigated:

  • Development of the Deep Learning method
  • Training on a collection of annotated images
  • Test and improve

And possibly:

  • Study integration in mobile phones
  • Study the development of a web-based analysis system.

This project is jointly proposed with the company Apizoom (Fribourg, Switzerland).

Responsible: Prof. J.-Ph. Thiran

4. CleanCityIndex – A Deep Learning based system to localize and classify wastes on the streets

A review of major European cities places “urban cleanliness” as a top priority for the authorities, as it directly impacts the concern and satisfaction of their citizens and the attractiveness of their economy and tourism. Littering quantification is an important step toward improving urban cleanliness. When human interpretation is too cumbersome or in some cases impossible, an objective index of cleanliness could reduce the littering by awareness actions.

The goal of this project is to propose a fully automated computer vision application for littering quantification, based on images taken from the streets and sidewalks. We employ a deep learning based framework to localize and classify different types of wastes.

In this project, the student will be involved into study and develop new detection algorithms and investigates deep learning based object tracking techniques for this specific case. The student is expected to be familiar with Python and Tensorflow.

Assistant: Saeed Rad ([email protected])
Supervisor: Prof. Jean-Philippe Thiran

5. Abnormality Detection for X-Ray Radiographic Images

The X-ray radiograph (XR) such as chest X-ray is frequently acquired for detecting and diagnosing pathologies and it is widely used in cancer prevention and treatment planning. However, the radiologist reporting of every image is a tedious and time-consuming task. Having said that, abnormal cases are usually much rarer than normal cases . In addition, it is very time-consuming to collect a reasonable amount for abnormal cases, while on the other hand, normal XRs are much easier to obtain. Therefore, fully supervised deep neural networks’ (DNNs) effectiveness is reduced. Consequently, an automatic system of XR abnormality detection would be advantageous, allowing reducing time for annotation and paying more attention on pathology analysis of abnormal XRs. The goal of this project is to develop a computer-aided diagnosis (CAD) system that can automatically identify abnormal XRs by learning only from normal ones during training phase.

In this project, the student will be involved in study and develop DNN-based one-class classifier for abnormal X-ray detection. The student is expected to be familiar with Python and Pytorch.

Assistant: Dr Behzad Bozorgtabar ([email protected])

Supervisor: Prof. Jean-Philippe Thiran

6. Image Modality Conversion

 Recently, interest in using image modality conversion for radiotherapy applications have grown rapidly. For example, for replacing Computed tomography (CT) with magnetic resonance imaging (MRI) for diagnostic and therapeutic purposes. This is mainly because MRI delivers superior contrast of soft tissue compared with the CT scans and is free of ionizing radiation. Therefore, modality conversion is desirable for many applications such as CT-based radiotherapy. Recent studies have shown remarkable success in image-to-image translation for modality conversion. This task has experienced significant improve- ments following the introduction of generative adversarial networks (GANs).

In this project, the student will be involved in study and develop a DNN-based image synthesis approach for wide range of modality conversion with the focus on radiotherapy applications. The student is expected to be familiar with Python and Pytorch.

Assistant: Dr Behzad Bozorgtabar ([email protected])

Supervisor: Prof. Jean-Philippe Thiran

 

7. Deep learning for speed of sound ultrasound imaging

The last years have seen the development of a new modality for ultrasound imaging, namely speed of sound imaging in pulse-echo mode. Its basic principle is to compute locally the speed of sound inside the soft tissue of the body based solely on the wave reflected back by those tissues. Such a modality allows the extraction of meaningful informations for medical diagnoses. Being able to distinguish malignant and benign liver tumors is a typical example of application of such a technique.

The basic principle of most of the state-of-the art methods is to compare  the ultrasound images reconstructed trough delay-and-sum under different conditions (typically the angle of the plane-wave we send) and deduce from the local spatial differences between the image the map of speed of sound. This comparison is usually performed with traditional image processing technique suffering from several drawbacks.

Our idea is to replace the traditional image processing techniques by a neural network specifically trained for this task, a CNN inspired by what is done in the image processing community. The student will be involved in the development and the training of such a network. Depending on the availability of the data, different type of techniques requiring less data can be investigated as well.

Requirements: Python (PyTorch / Tensorflow), a good knowledge in signal processing and machine learning is preferable.

Supervisor: Jean-Philippe Thiran

Assistant: Samuel Beuret ([email protected])

 

8. Quantification of the intensity of immune reponses in plant roots by image anaylsis – project in collaboration with Prof. Niko Geldner’s lab – Plant Molecular Biology Department at UNIL

Plants, as animals, are in contact with different types of microbes and can be subjected to infection. Nevertheless, they developed in the course of evolution an innate immune system allowing them to defend themselves. The group Prof. Geldner at UNIL is interested in visualizing these defenses in the root of the plant model system Arabidopsis thaliana. Indeed, roots of this small plants are very thin and completely transparent, making it the perfect tool for confocal microscopy. The Geldner lab possesses different defense reporter transgenic lines that permit to highlight the nucleus of a cell when this one activates defense. They can use these high-resolution tools to test whether defense responses are spatially compartmented in the roots of plants. The current hypothesis is that plants can regulate their defenses by inducing them only at strategic points, in specific tissues, to avoid a constitutive activation in the presence of the natural soil microbiome. The understanding of how plants can distinguish microbial friends from foes in their native soil is still scarce and such knowledge could help better protecting crop plants on which humanity relies to get food.

Thanks to the reporter lines, le lab demonstrated that not all genes are activated in the same cells, and that the compartmentation of defenses can be altered by modifying the expression of a receptor of bacterial molecular pattern. They yet would like to quantify this response in a tissue specific fashion. Using Fiji software, they managed to automatize the process of rough segmentation in 3D of all the nuclei in the root and to quantify the intensity of their markers in each of these regions. Nevertheless, the attribution of each signal to a specific tissue is done so far manually, which is extremely time consuming and is a major bottleneck to the analysis.

Therefore, they are looking for a student with an image processing background (EE, I&C or SV) to complete this task, which they cannot perform themselves by lack of sufficient coding skills. They have nevertheless different strategies in mind to categorize each nucleus, from relative positioning in the root, shape of the nuclei to machine learning. No previous knowledge in biology is required, they will teach to the student how to recognize the different tissues, the process of imaging and how to use the software. They hope to obtain a standardized protocol that would automatize the detection of signal in any cell type and be attributed to the right tissue. Such tools would be of great help for the current project as well as to the full lab in the future. It must be noted that the lab does not have the skills to help the student with the programming per se, but supervision will be provided to guide the student towards a successful approach. LTS5 will provide the additional technical support in image analysis. Programming was so far done using FiJi Macro IJ1 language, but the software also accepts Python, Java, JavaScript, R, Scala, and other languages.

In summary, we expect from the student to provide a macro for Fiji with the following criteria:

  • The macro can integrate the work previously done for 3D segmentation and quantification of signal in nuclei
  • The macro can distinguish different cell type based on the nuclei position/nuclei size/cell shape
  • The macro will assign each ROI identified with a cell type
  • The macro give as output an excel file (or .csv)

Supervision : Aurélia Emonet (PhD candidate – Niko Geldner’s lab) and Rémy Gardier (EPFL-LTS5)

 

MASTER PROJECT PROPOSALS 

MEDICAL IMAGING PROJECTS

1. Semi-supervised learning for the segmentation of echocardiography images using GANs – joint project with the CREATIS lab in Lyon, France

The analysis of 2D echocardiographic images plays a crucial role in clinical routine to measure morphology and cardiac function as well as to establish a diagnosis. This analysis is based on the interpretation of clinical indices extracted from low-level image processing such as segmentation. Thus, the estimation of indices such as the volume of the left ventricle (main cavity involved in the expulsion of oxygenated blood throughout the body) requires a precise delineation of the endocardial wall for two key moments of the cardiac cycle, such as illustrated in Figure below. In clinical routine, the semi-automatic or manual annotation remains a daily work because of the lack of precision and reproducibility of results provided by automatic software. This leads to time-consuming tasks that are subject to inter / intra variability problems. Thus, having automatic segmentation algorithms, accurate and reproducible is a major issue in the field of health in general, and echocardiography in particular.

The objective of this project is to improve the segmentation results produced by the U-Net method in order to produce segmentation scores lower than the intra-expert variability. To do this, the project will focus on integrating the U-Net model into a semi-supervised learning scheme exploiting the formalism of generative adversarial networks (GANs).

GANs have been successfully applied in many areas such as image generation, super-resolution imaging, semantic segmentation, motion estimation, transfer from one imaging modality to another. This formalism is currently being studied in order to improve the performance of the supervised deep learning algorithm (that is to say having learned from a referenced learning base) by effectively integrating the possibility of continuing to learn from non-referenced data, easier access data requiring no expertise from a practitioner. An example of a recently developed GAN architecture for semi-supervised learning motion estimation is given below [1].

Thus, the methodological objective of this project will be in a first to appropriate the formalism recently proposed for motion estimation [1] and then to adapt to the issue of segmentation of echocardiographic images. The performance of the proposed algorithm will be quantified via the CAMUS evaluation platform.

[1] W. S.Lai, J. B.Huang, and M. H. Yang.Semi-supervised learning for optical flow with generative adversarial networks.  Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, pp. 353-363, 2017.

This project will be done in collaboration with the CREATIS lab in Lyon, France (Prof. Olivier Bernard – https://www.creatis.insa-lyon.fr/~bernard/)

2. Abnormality Detection for X-Ray Radiographic Images

The X-ray radiograph (XR) such as chest X-ray is frequently acquired for detecting and diagnosing pathologies and it is widely used in cancer prevention and treatment planning. However, the radiologist reporting of every image is a tedious and time-consuming task. Having said that, abnormal cases are usually much rarer than normal cases . In addition, it is very time-consuming to collect a reasonable amount for abnormal cases, while on the other hand, normal XRs are much easier to obtain. Therefore, fully supervised deep neural networks’ (DNNs) effectiveness is reduced. Consequently, an automatic system of XR abnormality detection would be advantageous, allowing reducing time for annotation and paying more attention on pathology analysis of abnormal XRs. The goal of this project is to develop a computer-aided diagnosis (CAD) system that can automatically identify abnormal XRs by learning only from normal ones during training phase.

In this project, the student will be involved in study and develop DNN-based one-class classifier for abnormal X-ray detection. The student is expected to be familiar with Python and Pytorch.

Assistant: Dr Behzad Bozorgtabar ([email protected])

Supervisor: Prof. Jean-Philippe Thiran

3. Image Modality Conversion

 Recently, interest in using image modality conversion for radiotherapy applications have grown rapidly. For example, for replacing Computed tomography (CT) with magnetic resonance imaging (MRI) for diagnostic and therapeutic purposes. This is mainly because MRI delivers superior contrast of soft tissue compared with the CT scans and is free of ionizing radiation. Therefore, modality conversion is desirable for many applications such as CT-based radiotherapy. Recent studies have shown remarkable success in image-to-image translation for modality conversion. This task has experienced significant improve- ments following the introduction of generative adversarial networks (GANs).

In this project, the student will be involved in study and develop a DNN-based image synthesis approach for wide range of modality conversion with the focus on radiotherapy applications. The student is expected to be familiar with Python and Pytorch.

Assistant: Dr Behzad Bozorgtabar ([email protected])

Supervisor: Prof. Jean-Philippe Thiran

4. Deep learning for speed of sound ultrasound imaging

The last years have seen the development of a new modality for ultrasound imaging, namely speed of sound imaging in pulse-echo mode. Its basic principle is to compute locally the speed of sound inside the soft tissue of the body based solely on the wave reflected back by those tissues. Such a modality allows the extraction of meaningful informations for medical diagnoses. Being able to distinguish malignant and benign liver tumors is a typical example of application of such a technique.

The basic principle of most of the state-of-the art methods is to compare  the ultrasound images reconstructed trough delay-and-sum under different conditions (typically the angle of the plane-wave we send) and deduce from the local spatial differences between the image the map of speed of sound. This comparison is usually performed with traditional image processing technique suffering from several drawbacks.

Our idea is to replace the traditional image processing techniques by a neural network specifically trained for this task, a CNN inspired by what is done in the image processing community. The student will be involved in the development and the training of such a network. Depending on the availability of the data, different type of techniques requiring less data can be investigated as well.

Requirements: Python (PyTorch / Tensorflow), a good knowledge in signal processing and machine learning is preferable.

Supervisor: Jean-Philippe Thiran

Assistant: Samuel Beuret ([email protected])

5. Procedural generation of synthetic Brain tissue: Modeling vs Reinforcement Learning.

 Procedural generation is a method for creating data algorithmically — using a IA agent for example. Typically,  this requires a combination of human-generated assets coupled with computer-generated randomness and processing power. A interesting application in video-games of such algorithms are the so-called procedural generated maps, which are able to create entire random levels or environments for the player.  

In this project, we will use such algorithms with a twist: We will use them to generated realistic 3d mesh models (maps) of the Brain’s the matter microstructure. Such 3d environments are a fundamental part of the study of the brains microstructure via simulations. 

The task of the student will be to develop and test novel algorithms for this task; from model based algorithms (using stochastic properties and modeling)  to the training of self-supervised algorithms (Deep Reinforce Learning)

Plus: Use of Deep Learning Platforms (Tensorflow, (or/and) Pytorch, Keras)

Supervisor: Prof. Jean-Philippe Thiran
Assistant: Jonathan Rafael-Patiño

6. Dynamic image analysis for Cervical Cancer Detection: methods and Android demonstrator – analyse d’images dynamiques pour la détction du cancer du col de l’uterus: méthodes et démonstreteur Android

Cervical cancer is a major concern in public health, both in developed and developing countries. Especially in this later context, the availability of well-trained experts is limited, and computer-aided diagnosis is clearly needed for large-scale screening.

This project will focus on the analysis of dynamic image sequences (videos) of the cervix under a contrast agent: the visual inspection with acetic acid (VIA) is known as one of the reference methods to detect cervical cancer. However the human eye has limited capabilities in assessing the time evolution of the appearance of the cervix after administration of the contrast agent. Therefore, in this phase of the project, the intensity curves of each pixel of the video will be analyzed by a machine-learning algorithm. A synthetic image displaying the results will be produced as a help for the diagnosis. Moreover, as a simple and portable tool is needed, especially in developing countries, an Android application will be developed to perform this analysis and display the results on mobile platforms.

This project will be realized in close collaboration with the Department of Gynecology of the Geneva University Hospital.

Requirements: C++, image processing, skills in development for Android would be ideal.

Supervisor: Prof. Jean-Philippe Thiran

7. Computational pathology for automated analysis of histopathologic scans

Computational pathology is a state-of-the-art technology that aims to diagnose cancer and distinguish tissue components (e.g. nuclei, tumour) which has seen great improvements in recent years due to the advancement of convolutional neural networks (CNN) based diagnosis systems. However, automated analysis of histopathology whole-slide images is impeded by the scanner-dependent variance such as stain inconsistency introduced in the slide scanning process. In addition, CNNs are not the best suited for large scale (i.e. millions of pixels) multi-resolution histopathology whole slide images. Finding computationally efficient solutions to automatically analyze these images remains an open challenge. The goal of this project is to develop a computer-aided diagnosis (CAD) system, which can be used  for a number of applications where histopathologic images are captured from different scanners.

In this project, the student will be involved into study and develop CNN-based algorithms for histopathology-related applications including cancer detection and classification of the subtype from histopathologic scans . The student is expected to be familiar with Python and Tensorflow and/or Pytorch.

Assistant: Dr Behzad Bozorgtabar ([email protected])

Supervisor: Prof. Jean-Philippe Thiran

COMPUTER VISION PROJECTS

​​​​8. Facial Attributes Modeling

Human face has always been of particular interest in the computer graphics community. Because of its complexity, modeling lifelike synthetic objects is challenging. A variety of approaches have been proposed such as statistical models (i.e. principal component analysis models) or blendshapes models. Moreover, tackling the variation in terms of population (*i.e. identity*) and expression at the same time in a generic 3D model increases the difficulty.

With conventional modelling technics, the detailed facial attributes such as the wrinkles are lost in the process. Moreover these mid-frequencies informations are important for photo-realistic expressions generation. Therefore an explicit model can be jointly used to augment the original face model to recover them. Furthermore, photo-realistic rendering shading rely on depth cue given by ambient occlusion (AO). The ambient occlusions define how the light is attenuated at a specific location and is therefore dependent on the geometry of the face.

The goals of this project are in two folds. Build a model of the detailed facial attributes distribtion from high resolution meshes. Then explore the relation between the geometry and its corresponding ambient occlusions map in order to generate them it (i.e. regresse ambient occlusions from shape coefficients).

Requirements: The project will be implemented in Python / C++ so good knowledge is required. Previous experience in one or several of the following topics would be a plus: image processing, computer vision or machine learning.

Assistant: Christophe Ecabert ([email protected])

Supervisor: Prof. Jean-Philippe Thiran

9. Apizoom – deep learning to quantify the Verroa parasite in honey bee hive images (can be considered as a Master project in industry)

Varroa mites are recognized as the biggest pest to honey bees worldwide, and are believed to be the single largest contributing factor in the modern-day decline of honey bees due to their ability to transmit diseases, resulting in death or severe deformity of the pupae.

Verroa on honey bees.

Detecting and quantifying the presence of Verroa in a beehive is therefore crucial to treat the infection appropriately and as early as possible, and image analysis appears very useful in this problem.

In this project, we propose to develop an image analysis to detect and count Verroa cadavers who felt down on a plate below the beehive, as a non-intrusive way to quantify the presence of the Verroa. High definition images will be capture and Deep Learning techniques will be investigated here, to design and train a Convolutional Neural Network (CNN) to detect the Verroa and distinguish it from other wastes. See the following video for more details on the project (in French).

Depending on the evolution of the project, several steps will be investigated:

  • Development of the Deep Learning method
  • Training on a collection of annotated images
  • Test and improve

And possibly:

  • Study integration in mobile phones
  • Study the development of a web-based analysis system.

This project is jointly proposed with the company Apizoom (Fribourg, Switzerland).

Responsible: Prof. J.-Ph. Thiran

10. CleanCityIndex – A Deep Learning based system to localize and classify wastes on the streets

A review of major European cities places “urban cleanliness” as a top priority for the authorities, as it directly impacts the concern and satisfaction of their citizens and the attractiveness of their economy and tourism. Littering quantification is an important step toward improving urban cleanliness. When human interpretation is too cumbersome or in some cases impossible, an objective index of cleanliness could reduce the littering by awareness actions.

The goal of this project is to propose a fully automated computer vision application for littering quantification, based on images taken from the streets and sidewalks. We employ a deep learning based framework to localize and classify different types of wastes.

In this project, the student will be involved into study and develop new detection algorithms and investigates deep learning based object tracking techniques for this specific case. The student is expected to be familiar with Python and Tensorflow.

Assistant: Saeed Rad ([email protected])
Supervisor: Prof. Jean-Philippe Thiran

11. Quantification of the intensity of immune reponses in plant roots by image anaylsis – project in collaboration with Prof. Niko Geldner’s lab – Plant Molecular Biology Department at UNIL

Plants, as animals, are in contact with different types of microbes and can be subjected to infection. Nevertheless, they developed in the course of evolution an innate immune system allowing them to defend themselves. The group Prof. Geldner at UNIL is interested in visualizing these defenses in the root of the plant model system Arabidopsis thaliana. Indeed, roots of this small plants are very thin and completely transparent, making it the perfect tool for confocal microscopy. The Geldner lab possesses different defense reporter transgenic lines that permit to highlight the nucleus of a cell when this one activates defense. They can use these high-resolution tools to test whether defense responses are spatially compartmented in the roots of plants. The current hypothesis is that plants can regulate their defenses by inducing them only at strategic points, in specific tissues, to avoid a constitutive activation in the presence of the natural soil microbiome. The understanding of how plants can distinguish microbial friends from foes in their native soil is still scarce and such knowledge could help better protecting crop plants on which humanity relies to get food.

Thanks to the reporter lines, le lab demonstrated that not all genes are activated in the same cells, and that the compartmentation of defenses can be altered by modifying the expression of a receptor of bacterial molecular pattern. They yet would like to quantify this response in a tissue specific fashion. Using Fiji software, they managed to automatize the process of rough segmentation in 3D of all the nuclei in the root and to quantify the intensity of their markers in each of these regions. Nevertheless, the attribution of each signal to a specific tissue is done so far manually, which is extremely time consuming and is a major bottleneck to the analysis.

Therefore, they are looking for a student with an image processing background (EE, I&C or SV) to complete this task, which they cannot perform themselves by lack of sufficient coding skills. They have nevertheless different strategies in mind to categorize each nucleus, from relative positioning in the root, shape of the nuclei to machine learning. No previous knowledge in biology is required, they will teach to the student how to recognize the different tissues, the process of imaging and how to use the software. They hope to obtain a standardized protocol that would automatize the detection of signal in any cell type and be attributed to the right tissue. Such tools would be of great help for the current project as well as to the full lab in the future. It must be noted that the lab does not have the skills to help the student with the programming per se, but supervision will be provided to guide the student towards a successful approach. LTS5 will provide the additional technical support in image analysis. Programming was so far done using FiJi Macro IJ1 language, but the software also accepts Python, Java, JavaScript, R, Scala, and other languages.

In summary, we expect from the student to provide a macro for Fiji with the following criteria:

  • The macro can integrate the work previously done for 3D segmentation and quantification of signal in nuclei
  • The macro can distinguish different cell type based on the nuclei position/nuclei size/cell shape
  • The macro will assign each ROI identified with a cell type
  • The macro give as output an excel file (or .csv)

Supervision : Aurélia Emonet (PhD candidate – Niko Geldner’s lab) and Rémy Gardier (EPFL-LTS5)

 

12. Efficient Deep Architectures for Hand Pose Estimation from Color Images – master thesis at the CSEM in Neuchâtel

The project aims to develop efficient deep learning algorithms for human hand pose estimation from standard RGB camera images. State-of-the art approaches for hand pose estimation rely on depth cameras due to the inherent high-information content of depth images for the pose estimation problem as well as the availability of large-scale annotated depth datasets with ground-truth hand joint locations. Despite their recent success these algorithms are neither cost effective nor power efficient, which pose a barrier in application areas which need low-power, small-size and low-cost requirements.

The project’s objective is to overcome this barrier by developing efficient deep learning architectures that would work solely on standard color image sequences. Due to the lack of large annotated RGB datasets in the public domain we plan to tackle this problem with a teacher-student learning paradigm, which involves training on an RGB-only deep network from a pre-trained state-of-the-art depth-based one using an RGBD sensor to collect synchronized RGB and depth image pairs.

Your mission:

To analyse the state-of-the-art in hand pose estimation from various data modalities, and develop as well as train an RGB image-based deep network model entirely from public resources. Along with accuracy, computational and power efficiency of the network are major factors that will be considered from the beginning of the study. The candidate is expected to present their results at the end of the project, which will last a minimum of 6 months.

Your profile:

You have a background in machine learning, especially deep learning techniques, computer vision, and programming in python. You are interested in understanding as well as designing intelligent algorithms and are a good team player.

The offer:

CSEM offers a stimulating and multidisciplinary work environment which promotes an employee-driven culture and diversity.

For further information on this position, please contact Dr. Engin Türetken, Machine Learning Expert in the Vision Embedded Systems group (+41 32 720 5237, [email protected]).

CSEM is a private company which transfers recent academic achievements to commercial products by carrying out applied research work, product development and prototype production. CSEM anticipates and fulfils the industrial needs by offering its high-tech knowhow and expertise in the fields of micro, nano and information technologies.

 

13. Human Rendering System for Realistic Video Data Generation – master thesis at the CSEM in Neuchâtel

One of the most important factors for the success seen in deep learning applications is the availability of large-scale datasets with ground truth annotations. This is partly due to the fact that today’s most successful approaches are fully supervised. Deep learning algorithms, particularly, fall short of generalizing well to unseen cases from only a few data samples, which is due to their large capacity to model patterns in the data, and hence, higher chance of overfitting under weak regularization. As a result, one of the major challenges in many modern artificial intelligence (AI) projects today is the requirement for collecting a large enough dataset with ground truth labels. Obtaining such a dataset manually using only human annotators is not only time consuming and costly, but also not scalable.

An attractive approach to tackle this problem involves synthetically rendering realistic scenes with moving synthetic humans and other objects in them. Augmenting a small real dataset with such a large synthetic one has proven to improve classification accuracy in several application domains including eye gaze estimation from still images. The project aims at developing such a rendering system entirely from publicly available resources to generate realistic video sequences with high levels of variation in them (i.e., various actions, objects, backgrounds etc).

Your mission:

To analyse the state-of-the-art in realistic scene rendering techniques involving humans as well as a large collection of inanimate objects. The candidate is expected to present their results at the end of the project, which will last a minimum of 6 months.

Your profile:

You have a background in computer graphics or a similar domain. Experience in scene and/or human body rendering techniques and tools is a plus. You are skilled in programming in at least the python language. You are a good team player ready to take technical challenges.

The offer:

CSEM offers a stimulating and multidisciplinary work environment which promotes an employee-driven culture and diversity.

For further information on this position, please contact Dr. Engin Türetken, Machine Learning Expert in the Vision Embedded Systems group (+41 32 720 5237, [email protected]).

CSEM is a private company which transfers recent academic achievements to commercial products by carrying out applied research work, product development and prototype production. CSEM anticipates and fulfils the industrial needs by offering its high-tech knowhow and expertise in the fields of micro, nano and information technologies.