Implicit Surfaces for Surface Reconstruction ‒ CVLAB ‐ EPFL

Implicit Meshes For 3-D Reconstruction And Tracking

The novel ways, both to represent the static surfaces, and to parameterize their deformations, are developed. This can be used both by automated algorithms for efficient 3–D shape reconstruction, and by graphics designers for editing and animation

We propose a method that combines the strength of explicit and implicit surface representations to avoid their drawbacks, and in this way build robust surface representation, called implicit mesh, suitable for automated shape recovery from video sequences.

We also advocate the use of Dirichlet Free Form Deformation (DFFD) as generic surface deformation technique that can be used to parameterize objects of arbitrary geometry defined as explicit meshes.

Combining DFFD as a generic parameterization method for explicit surfaces and implicit meshes as a generic surface representation we obtained a powerfull tool for automated shape recovery from images. However,we also argue that any other available surface parameterization can be used.

We demonstrate the applicability of our technique in case of 3–D reconstruction from images and tracking of deformable objects in monocular scenes. The examples of those reconstruction and tracking results are depicted in Figure 1. bellow.





Figure 1. Reconstruction and tracking results obtained using implicit mesh formalism.Top row: 3D reconstruction of the human upper body, parametrized in terms of DFFD transformation, from uncalibrated video sequence using stereo and silhouette information. Original images and textured reconstructed models are shown together with overlaid silhouettes. Model alignment with the silhouettes and correct texture map indicate quality of the results. Second row: Face reconstruction using PCA face model. First and third images are two input images with overlaid silhouettes automatically detected using PCA face model turned into implicit mesh. The images next to them depict textured reconstruction result. Note that the camera motion is recovered together with the shape. Third row: Shoulders and head tracking in a cluttered scene using implicit meshes highlights its ability to handle occluding contours in a robust way. Bottom row: Tracking of deformable piece of paper in presence of the changing background and partial occlusion. Silhouette information is again handled using implicit meshes.

Building Implicit Meshes

The implicit mesh model is a surface representation that unifies explicit (polygonal) and implicit surface representations in such a way that it preserves the strengths of both representations and eliminates their drawbacks. To create an implicit mesh model we must address two problems:

Creating an implicit surface that closely approximates the shape of the initial explicit mesh
Creating an implicit surface that closely approximates the shape of the initial explicit mesh

Our approach is depicted by Figure 2. We now discuss its components. To create implicit surface of the explicit triangular mesh we convert each mesh’ facets to a metaball primitive. Metaball primitives we consider here are:

spherical metaballs
triangular metaballs

Spherical metaball is an isosurface of the scalar field generated from the skeleton which is the center of the circle circumscribed around the facet as shown in Figure 2(b). Triangular metaball is an isosurface of the scalar field generated from the skeleton which is now whole facet instead of the point as in the case of spherical metaball. The parameters for the metaball include the position of the skeleton, and the potential filed function depending on the distance from the skeleton. Triangular metaball is depicted in Figure 2 (c).

Figure 2: Converting an explicit surface into an implicit shell. Upper row: Single triangle as the part of the explicit mesh (a), converted to the spherical metaball shown transparent (b) and to the triangular metaball (c). Middle row: Initial explicit surface (d), converted to the transparent implicit shell with the spherical metaballs (e) and converted to the implicit shell using triangular metaballs (f). Bottom row: Deformed explicit mesh (g), and corresponding deformed implicit shells shown transparent around the explicit one with spherical metaballs (h) and with the triangular materials(i).

Videos of Reconstruction and Tracking results

Here you can see and download some more results. Results are short movies in avi format. First we present results of 3D reconstruction and then tracking of deformable objects.

3D Reconstruction form uncalibrated video sequences


Original video.	Textured reconstructed model.

The above images and corresponding downloadable movies demonstrate 3D reconstruction from uncalibrated video sequences of the upper body. We used the generic upper body surface model, represented as triangular implicit mesh and parameterized in terms of DFFD control points. The sequence was first calibrated using model-based bundle adjustment technique, and then stereo data cloud is extracted using simple correlation based algorithm. Silhouettes are obtained using snakes starting from manually outlined initial contours. In the left video the images form the original video sequence together with the outlined silhouettes are depicted. The depicted video is composed of 6 images, we used for the reconstruction, out of much longer sequence. Note that the small number of images did not influence quality and precision of the reconstruction shown by the video on the right. The result is textured model reprojected in the same view as the originals. The correct model alignment with the depicted silhouettes indicates quality of the reconstruction.

These images and movies depict our use of the morphable PCA face model of Blanz and Vetter to reconstruct the faces and recover the camera motion. First we used feature points on the face and the explicit triangulated mesh representing morhable face model. However, it was not sufficient to obtain the precise reconstruction on the occluding contours (silhouettes) as depicted by the middle image of the given video. To improve the reconstruction result we used our implicit mesh formalism. We then automatically detected silhouettes in the images, which are depicted in yellow in the given video, and used them together with the feature points to improve the recovered shape as shown in the right end image of the video. The implicit mesh surface representation was actually used for robust automatic silhouette handling, which is important when one wants to track deformable objects in the scenes with clutter, changing background and partial occlusions.

Tracking of 3D deformable objects in monocular videos

Here we use single video sequences to track upper body motion and the deformation of a piece of paper. The results are shown in the videos bellow.