What is the project about?
Recently, the Laboratory for Timber Construction, IBOIS, raised the question of how to take advantage of new technologies such as robotic vision for workers in the timber construction industry. Robotic technologies have the potential of bridging skill gaps and thus help unexperienced carpenters. Prof. Yves Weinand (head of IBOIS) and myself together with Dr. Julien Gamerro developed an Augmented Reality (AR) platform to guide users with manual tools in performing wood fabrication tasks. AR can be made accessible for everyone since a sensor or a display can be used in a simple manner. So, we developed in Linux an augmented reality framework, and started from nothing with OpenGL and C++.
Other commercial devices in this area already exist, such as Hololens from Microsoft. Hololens is an augmented reality device where you see holograms, but you need to use development platforms which are protected.
At this moment we don’t know how the technology might look like in the future. Big companies may require specialized support like client assistance and custom scenarios, but we believe there could be potential projects with low income areas or countries that would benefit from those technologies directly.
Why Open?
This project should benefit the end user as the general public from findings that are produced in a public funded institution such as EPFL.
Also, this new software culture is bringing open science to everyone: the importance of sharing data is now starting to grow also in the construction sector.
Who benefits from it?
The potential of this kind of tools is very wide. The general public could benefit from them, but also the digital fabrication sector as well as the medical sector if the resolution refinement one day would allow for it. Solutions already exists, but none of them is fully open.
Initially, deep learning algorithms and computer vision resources were developed by the automotive industry, who admit for much higher tolerances than we do. They use simultaneous localization and mapping (SLAM), which have tolerances in the range of centimeters. For carpentry and the medical sector, we need lower tolerances and we are on the brink to achieve resolutions of 1-2 millimeter.
How did you make it Open Software?
At IBOIS architects, computer scientists and engineers look at the same code. We must be very explicit in the way we work, so that everything is visible and clear within each domain. We did a pilot study, highlighting the workflows and improvements by building the new open software. We use version-control to check the code quality. Students are working on it, and we have a detailed guide for contributors. For example, we show the structure of the code, how to write a variable, how to define headers, etc. This allows us to have a very well-structured code. This helps communication!
We publish in real time as we work. A good example is our latest journal article in the Automation and Construction Journal on Augmented-reality-assisted timber drilling with smart retrofitted tools. Each component of the software is released as a standalone piece of software and publication.
However, we believe that the real value in open-science is maintenance of software. Another project developed at the lab with my colleague Petras Vestartas is called Cockroach. Here, we have seen that the best is to keep the library alive. Doing so, you can justify the work schedule for the lab and implementing new features, not only code refactoring or bug fixing.
Contact: Andrea Settimi