Online Optimization for the locomotion of Roombots


    This semester project focuses on the problem of online locomotion learning in Roombots, a modular robot developed at the BioRobotic Laboratory of EPFL- École Polytechnique Fédérale de Lausanne.

    The online learning locomotion controller includes two components: a Central Pattern Generator(CPG)
and an optimization algorithm, Particle Swarm Optimization (PSO). The CPG is
implemented as a system of coupled nonlinear oscillators in Roombots, which has 3 degrees of freedom,
each of which supports two type of movements: rotation and oscillation. Online learning involves
optimization of the fitness value which is the speed of a certain locomotion or the travelling distance in a
period of time. A stochastic and gradient-free PSO optimization method was used to find a set of CPG parameters of a gait that maximizes the fitness value. The fitness value is the distance that Roombots can move in a period of time from an initial position. CPG and PSO were implemented directly on Roombots and the optimization process was performed by themselves.The work continued the previous work of Wilhelm Frédéric and Soha Pouya.
    The main objective of the project is to fully make online locomotion learning with CPG and PSO
optimization are implemented in a distributed fashion on physical Roombots in the real environment with
fitness values calculated by a tracking system. The tracking system was implemented with input from Microsoft Kinect and background substraction algorithm to detect Roombots’ locations.
    During the experiments, we used the simplest structure, called meta-module which is made of two modules.
We used 10 PSO particles to find the best solution for 6 parameters of CPG model. The result shows that a
good locomotion gait can be obtained in a few iterations and the results started to converge to the best one
after a large number of iterations, which is larger than 10. The explored good gaits are similar to the locomotion patterns of animals in nature.
    After obtaining good gaits, a number of experiments were conducted to evaluate the gaits’ robustness
against changes in initial states and environments such as friction, slope and obstacles. The results reveal
that initial state is the most affective factor to the gait’s performance.



The purpose of the project is to investigate Plug-and-play robots that can learn locomotion on their own
and can adapt to different environment conditions with online learning. 

Experimental Software

    The software is developed in C++ and use OpenFramework toolkit. The development environment is CodeBlock IDE integrated with Open-Framework version 007 on Ubuntu 11. The below figure illustrates the GUI of the software which contains 6 main sections:

1. RED section: Kinect Settings.

2. YELLOW section: Bluetooth Settings.


4. WHITE section: Visual Monitor.

5. ORANGE section: Program status and messages.

6. BLUE section: Bluetooth Send/ Receive data.

 Fig.1: The Graphical User Interface of the experimental software

   Tracking algorithm

Fig.2: The diagram illustrating the tracking algorithm which based on the background subtraction algorithm on IR-Images data.

Experiment Setup

   The robots are tested in a square area of 2mx2m. The distance travelled in one experiment time (30s) is measured by tracking the centroid of the blob of Roombots by a Kinect fixed to the ceiling. The distance travelled is fed to PSO algorithm as the fitness value. 

Fig.3: Experiment setup


Optimization process result


Fig.4: Fitness value of 10 PSO particles along 8 iterations. The fitness values are generally increasing along the iterations. A high fitness value means a good result. The fitness values are coded by the color bar with red as the highest value and blue as the lowest value. At the first iteration, every particle has a fitness value of 0.


Gait results

A good gait:

Fig.5: A good gait with a fitness value of 324 in average or 121.5cm (i.e a speed of 4.05cm/s).


The best gait:

Fig.6: The best gait with a fitness value of 395 in average or 150cm (i.e a speed of 5cm/s).


Gait videos:





Gait evaluation videos: 







Midterm Presentation: Midterm presentation_Nguyen The Anh.pdf

Final Presentation: Final presentation_Nguyen The Anh.pdf

Final Report: Final_Report_Nguyen The Anh.pdf

Experimental Software Manual: Experimental Software Manual_Nguyen The Anh.pdf