Student Projects

Modern districts are commonly equipped with PV panels, electric batteries, variable power heat pump and electric vehicles. This means local energy systems are becoming complex energy hubs which require an efficient energy management system (EMS) that can be configured in a simple way while satisfying complex needs such as improving self-consumption and reducing cost.

The energy systems group at CSEM has developed the software NRGMaestro(TM) a next-generation energy management system (EMS) that is based on Model Predictive Control (MPC) and which can be easily reconfigured through an automated controller synthesis process. This energy manager has been embedded in the commercial product Soleco Optimizer [https://soleco-optimizer.ch/] and is currently in operation in several buildings in Switzerland.

In order to scale-up to districts and eventually beyond that, centralizing all the information in one location to calculate an optimal energy dispatch is problematic for multiple reasons (privacy, computational issues, reliability). Therefore, a distributed management approach is largely preferable. The goal of this project is to design a mechanism to network local optimizing EMS in multiple buildings in order to achieve a collectively satisfactory outcome in terms of energy costs for an energy community. To this end, the intern is expected to

– Review academic literature on multi-agent systems, distributed optimization and game theory, including distributed Nash equilibrium seeking to identify promising directions,

– Implement prototypes of strategies

– Evaluate in simulation on synthetic scenarios using real-world data from different buildings equipped with modern energy devices such as electric vehicles, heat pumps, photovoltaic production, etc

While the objective of the work is clearly industrial, there will be scope to present the results in scientific journals or at conferences. The student will be part of a team of twelve experienced engineers and researchers and will face theoretical and practical challenges.

Comment

Requirements:

– Interest in energy systems and energy management problematics.

– Solid understanding of algorithms, in particular optimization methods

– Familiarity with some of the following topics is a plus:

o Game-theory

o Distributed algorithms

o Model predictive control

– Interest and experience in programming ( Python )

Salary: 2000 CHF/month

Type of project: Master in industry or internship, minimum 5 months. Location: Neuchatel

Professor(s)

Maryam Kamgarpour (Recherche en systèmes), Maryam Kamgarpour

Administration

Sandra de Best

External

Tomasz Gorecki, [email protected]

The EPFL babyfoot is under continuous improvement.

While the babyfoot can easily intercept the ball, it has difficulty to capture it and make a pass. Juggling with the ball is another challenge. This project aims at improving the ball handling to permit capture and juggling, and then shoot toward opponents goal.

Students suggested improvements are also welcome.

Professor(s)

Christophe Salzmann (Laboratoire d’automatique 3)

Site

https://www.epfl.ch/labs/la/studentprojects/babyfoot/

Background

The data capacity of the hard disk drive (HDD) must increase to meet the demands for data storage. As a result, we must improve the positioning accuracy of the magnetic head in the HDD for a better future.

Objective

To encourage research about magnetic-head positioning control, a benchmark problem which works with MATLAB has been released for the IFAC World Congress 2023 (see https://www.ifac2023.org/media-download/79/8d46c77a8e65ee38/). A model is given to simulate the magnetic-head positioning system used in the latest HDDs. The objective for students is to apply control synthesis techniques seen in the Advanced Control system course to beat the benchmark performance.

Requirements: Advanced Control Systems

Comment

Contact: [email protected]

Professor(s)

Alireza Karimi

Administration

Philippe Louis Schuchert

Commercial buildings consume around 40% of the world’s energy. Many research papers have explored different advanced methods to save energy. For example, model predictive control (MPC) does the job and guarantees occupancy comfort. However, there is still a large gap between the promising research results and real-world deployment. Thus, this project explores a more implement-friendly building control framework – data-driven control methods.

Current industrial practice is typically simple controllers such as bang-bang and PI controllers. To update these simple controllers to more advanced optimization/ learning-based control, such as predictive model control and reinforcement learning, additional investment in computation infrastructure is needed. One low-cost alternative solution is directly tuning those parameters of existing simple controllers to optimize the building performance. For example, Bayesian optimization has been demonstrated to be effective in tuning building controllers. This project could explore more experimental work on Bayesian optimization-based building controller tuning. A building testbed in the EPFL campus, Polydome (shown in the image below), is ready for actual experiments.

Requirements:

1. Proficient in Matlab or Python

2. Knowledge in optimal control or machine learning is preferred

3. Knowledge in Bayesian optimization or derivative-free optimization is a plus

Comment

Assistants: Jicheng Shi ([email protected]), Wenjie Xu ([email protected])

Professor(s)

Colin Neil Jones (Laboratoire d’automatique 3), Christophe Salzmann

Administration

Nicole Anne Bouendin

The large-scale integration of distributed power-electronic devices has rendered modern power systems difficult to be explicitly and accurately modeled through first principle or system identification. Meanwhile, the ubiquitous smart meters and smart sensors in power systems give us the access to a substantial amount of data. The behavioral approach, representing the system dynamics with trajectory data, lend itself to the data-driven analysis and control of complex power systems owing to non-parametric representation.

This project aims to design output-feedback controller for complex microgrids using behavioral approach. On top of some reasonable assumptions about noise, robust controllers and (or) optimal controllers are supposed to be designed. The performance of controllers will be validated and compared by simulation.

This project can either be a semester project or a master project.

Professor(s)

Alireza Karimi

Administration

Zhaoming Qin

The proposed project aims to analyse the dynamics of a microgrid with multiple sources, including renewables (such as solar and wind) and backup sources (such as batteries or generators). The primary objective is to develop a comprehensive understanding of the microgrid’s behaviour under varying conditions and propose an efficient control algorithm to optimise its operation. This will be achieved through real-time simulations using Simulink and MATLAB.

Comment

Assistant: Vaibhav Gupta

Professor(s)

Alireza Karimi, Vaibhav Gupta

Administration

Sandra de Best

Site

ddmac.epfl.ch, la.epfl.ch

In this project, the objective is first identify a model for a 3-Degree-Of-Freedom (3DOF) hover based on conventional techniques in system identification. To do so, first, a code to run the system using LABVIEW should be written and deployed on MyRIO.

Moreover, all the electrical connections should also become compatible for running the system.

The code for system identification should be flexible to select different excitation such as PRBS, white noise, single tone sinusoidal and sin-sweep. The second part of the project includes designing a controller using the methods of advanced control such as H-infinity and data-driven method and then applying on the device using an appropriate labview code and validate the performance.

Comment

Assistant: Vaibhav Gupta

Professor(s)

Alireza Karimi, Vaibhav Gupta

Administration

Isabelle Stoudmann Schmutz

Site

la.epfl.ch

Motivation:

In the previous semesters, we build a lightweight hovercraft platform capable of hovering on an air-hockey table. The long-term goal of the project would be the autonomous play against humans, or having multiple hovercrafts playing air hockey autonomously against each other.

Description:

In this project, we want to develop and deploy sensor fusion (Kalman filter) and control algorithms onboard the microprocessor of the hovercraft (ESP32). For this, you will build upon our existing micro-ROS implementation running on FreeRTOS. We have data from the onboard sensors (accelerometer, velocity sensor, and gyroscope) which have to be fused with delayed external position measurements (Optitrack) to obtain a high-frequency state estimate. Using this state estimate, the aim is to develop an onboard low-level controller tracking trajectories given from an offboard computer.

Skills needed:

– Excellent knowledge of C/C++

– Knowledge about Control Systems is a plus

– Familiarity with ROS/ROS2 is a plus

If interested, please contact the responsible assistant directly ([email protected]) and attach your transcripts and CV.

Professor(s)

Colin Neil Jones (Laboratoire d’automatique 3), Roland Schwan

Administration

Nicole Anne Bouendin

Motivation

Mini race car systems provide a compact yet realistic platform for testing and refining autonomous driving algorithms. In this project, we aim to control these vehicles using neural networks (NN).

The student will be able to improve the following skills:

* Neural Network Integration (Incorporating neural networks into control real systems).

* Sensor Fusion (Integrating various sensors for data processing).

* Safety Implementation (Implementing safety measures in autonomous systems).

* Project Management (Planning, executing, and managing progress).

* Innovation (Exploring creative solutions for advanced technology).

Description

This project aims to use NN for mini race car control, focusing on autonomy and safety. Through advanced algorithms and model training, we aspire to create an adaptive system that not only optimizes speed and agility but also ensures safety guarantees.

Skills needed (in decreasing order of importance)

* Mechanical understanding, ideally knowledge about car dynamics

* Proficiency in ROS/ROS2

* Familiarity with Python/PyTorch (for Machine Learning)

* Familiarity with Matlab (for data analysis)

* Significant experience in coding projects

* Eigen, Casadi knowledge is a plus (for modeling/parameter optimization)

* Optimal Control (e.g., MPC) is a plus

Professor(s)

Giancarlo Ferrari Trecate, Daniele Martinelli

Administration

Nicole Anne Bouendin

Deep neural networks have proved successful in many application domains, such as image recognition, language comprehension, and sequential decision-making. However, recently, neural networks have been successfully employed for the control of non-linear dynamical systems.

In this project, our goal is to employ a class of distributed neural networks called recurrent equilibrium networks (RENs) for the control of autonomous systems while guaranteeing closed-loop stability. For instance, one possible application is learning to navigate a fleet of robots through a cluttered environment while avoiding obstacles. We will test our neural network controllers on the state-of-the-art platform called MuJoCo. MuJoCo stands for Multi-Joint dynamics with Contact. It is a general-purpose physics engine that aims to facilitate research and development in robotics, biomechanics, graphics and animation, machine learning, and other areas that demand fast and accurate simulation of articulated structures interacting with their environment.

Comment

TA : Muhammad Zakwan ( [email protected])

Professor(s)

Giancarlo Ferrari Trecate, Christophe Salzmann

Administration

Nicole Anne Bouendin

Site

https://www.epfl.ch/labs/la/pi/

Motivation:

The EPFL Rocket Team aims for building thrust-vector-controlled rockets that can be reused by controlling them to land upright at a specific location. We develop algorithms that guide and control the rocket during the descent phase, approach, and touchdown.

Description:

In this project, we would like to develop algorithms for safely adapting drone and rocket controllers online, i.e., during the flight.

First, we will establish a baseline method, i.e., an adaptation approach based on conventional model identification/controller design. One approach could be to recursively fit a linear model and update controller gains based on it.

Second, we will develop a Machine Learning method that tunes the existing controller, and/or models a physical submodule, and/or directly approximates a feedforward control law. Eventually, we will compare the performance of the two developed methods.

Skills needed:

– Understanding of flight mechanics, modeling, and identification

– Classical and/or Optimal Control (LQR/MPC)

– Machine Learning methods (Neural Networks/Gaussian Processes)

– Familiarity with Python/PyTorch (for Machine Learning)

– Experience in C++ (for operating the drone)

– Significant experience in coding projects

– Familiarity with ROS (Robot Operating System) is a plus

Comment

Please contact Johannes Waibel ([email protected]) if interested.

Professor(s)

Colin Neil Jones (Laboratoire d’automatique 3), Johannes Christian Karl Waibel

Administration

Nicole Anne Bouendin

Reinforcement Learning (RL) involves studying sequential decision-making problems, where an agent aims to maximize an expected cumulative reward by interacting with an unknown environment. While RL has achieved impressive success in domains like video games and board games, safety concerns arise when applying RL to real-world problems, such as autonomous driving, robotics, power systems, and cyber-security. This project proposal aims to explore the field of safe RL, addressing the challenges of sample complexity, stability, and theoretical guarantees.

Outline: This project contains the following potential directions:

Designing Safe RL Algorithms: Explore and propose approaches for safe reinforcement learning that address the challenges of sample complexity, stability, and safety guarantees.

Theoretical Advancements: Advance the theoretical foundations of safe RL algorithms, focusing on a specific class of RL methods, such as policy-based methods, actor-critic methods, or other relevant approaches. Develop provable guarantees for the safety and stability of these algorithms.

Implementation and Evaluation: Implement the proposed safe RL algorithms in a realistic simulation environment. Evaluate their performance and compare them against existing methods using safety benchmarks.

Moving from single agent setting to multi-agent setting.

Comment

Requirement: We seek for motivated students with a strong mathematical, or computer science background. We do have some concrete ideas on how to tackle the above challenges, but we are always open for different suggestions. If you are interested, please send an email containing 1. one paragraph on your background and fit for the project, 2. your BS and MS transcripts to [email protected]. The students who have suitable track record will be contacted.This project will be supervised by Prof. Maryam Kamgarpour and Tingting Ni ([email protected]).

Professor(s)

Maryam Kamgarpour (Recherche en systèmes), Tingting Ni

Administration

Sandra de Best

Site

https://www.epfl.ch/labs/sycamore/safe-reinforcement-learning-from-single-agent-to-multiagent/

Motivation:

The Automatic Control Lab develops Model Predictive Control schemes for a miniature race car system. Innovative methods will have a wide scope of applications in driving safety and/or autonomous driving.

Modeling the tire properties (longitudinal and side slip forces) turns out to be challenging for a 1/27-scale. This is due to difficulties collecting informative and consistent experimental data but also uncertainty about the physical effects that should be covered by the model structure.

Description:

The goal of this project is to establish a high-fidelity model of the mini race car by developing new modeling approaches, conducting experiments for data collection, and fitting the models to the data.

One way to improve the data quality will be to integrate an IMU and/or wheel speed sensors into the cars. Given the small size, this requires a careful component selection and mechanical design of attachments/wiring/etc.

A second approach can be to use Machine Learning to describe additional physical effects.

The obtained model will be used in simulation and ideally for model-based control that seizes the newly-gained tire information. Promising controllers will be tested on the real racetrack in return.

Skills needed

– Mechanical understanding, ideally knowledge about car dynamics

– Proficiency in C++ (for controlling the car and sensor interface)

– Proficiency in Matlab (for data analysis)

– Familiarity with Python/PyTorch (for Machine Learning)

– Significant experience in coding projects

– Familiarity with ROS/ROS2 (Robot Operating System) is a plus

– Familiarity with Eigen, Casadi is a plus (for modeling/parameter optimization)

– Optimal Control (e.g., MPC) is a plus

Comment

Please contact Johannes Waibel ([email protected]) if interested.

Professor(s)

Colin Neil Jones (Laboratoire d’automatique 3), Johannes Christian Karl Waibel

Administration

Nicole Anne Bouendin

The first goal of this project is to identify a model of the Quanser Aero 2 – 2 DOF Hover using conventional system identification methods. For this, different input signals and model structures will be considered. Matlab/Simulink environment will be used for the data collection and system identification.

The second goal of the project is to design a controller and implement it on the real system. After the identification process the nonlinearity of the system will be evaluated and a proper controller design strategy will be determined accordingly. If the effect of nonlinearities are observed to be small, an LTI controller with a model based or data-driven approach can be designed. Otherwise, a nonlinear controller using the Koopman operator approach can be synthesized. The designed controllers will be implemented in Simulink and tested on the real system.

Comment

Contact: [email protected]

Professor(s)

Alireza Karimi

Administration

Mert Eyuboglu

Site

https://www.epfl.ch/labs/ddmac/student-projects/

Motivation:

In the previous semesters, we build a lightweight hovercraft platform capable of hovering on an air-hockey table. The long-term goal of the project would be the autonomous play against humans, or having multiple hovercrafts playing air hockey autonomously against each other.

Description:

In this project, we want to steer the hovercraft time-optimally to a given state (position, velocity) to intercept the puck and play it back optimally. For this, we aim to use advanced control algorithms like non-linear time optimal model predictive control to steer the hovercraft at its physical limits. For this, you will develop and deploy high-performance numerical optimization algorithms capable of running in real-time, building upon our existing ROS2 software stack.

Skills needed:

– Excellent knowledge of C/C++

– Knowledge about Optimal Control (MPC)

– Familiarity with ROS/ROS2 is a plus

If interested, please contact the responsible assistant directly ([email protected]) and attach your transcripts and CV.

Professor(s)

Colin Neil Jones (Laboratoire d’automatique 3), Roland Schwan

Administration

Nicole Anne Bouendin

We recently developed and released a high-performance quadratic program (QP) solver based on a proximal interior-point method [1]. While performing well with default settings/hyperparameters, we are interested in finding more optimal and robust default hyperparameters and problem/domain specific hyperparameters to achieve better performance.

Bayesian optimization (BO) is a sample-efficient black-box optimization method, which has been successfully applied to tune the hyperparameters of machine learning models [2]. This project aims to apply the recently proposed BO (e.g., constrained BO [3]) methods to tune the hyperparameters of a QP solver.

Requirements:

– Proficient in Matlab or Python

– Knowledge in optimization or machine learning is preferred

– Knowledge in Bayesian optimization or quadratic programming is a plus

If interested, please contact the responsible assistants directly (roland.schwan@epfl.ch, [email protected]) and attach your transcripts and CV.

References:

[1]: Schwan, R., Jiang, Y., Kuhn, D., & Jones, C.N. (2023). PIQP: A Proximal Interior-Point Quadratic Programming Solver. In IEEE Conference on Decision and Control.

[2]: Snoek, J., Larochelle, H., & Adams, R. P. (2012). Practical Bayesian optimization of machine learning algorithms. Advances in Neural Information Processing Systems, 25.

[3]: Xu, W., Jiang, Y., Svetozarevic, B., & Jones, C.N. (2023, July). Constrained efficient global optimization of expensive black-box functions. In International Conference on Machine Learning (pp. 38485-38498). PMLR.

Comment

Assistants: Roland Schwan ([email protected]), Wenjie Xu ([email protected])

Professor(s)

Colin Neil Jones (Laboratoire d’automatique 3), Roland Schwan

Administration

Nicole Anne Bouendin