Click on the images to jump to corresponding results.
Starting from our earlier approach to fast keypoint matching, we developed real-time techniques to detect deformable surfaces. Using wide-baseline correspondences between an input image and a model image, we recover quickly and accurately the 2D or 3D shape of a deforming surface. In addition, we also model the lighting and detect occlusions.
Results
Painting a Platypus on a Deforming TShirt [top]
Given the TShirt and platypus images shown below, we recover the deformations of the tshirt in the video. We then deform the platypus image accordingly, relight it, and add the result to the original images while taking occlusions into account.
Magic Book [top]
The same technology was used by Camille Scherrer, an designer, to create a magic book.
Augmenting a 3D Surface [top]
In the examples above, we only recover 2D surface deformations. We can also use our techniques for monocular 3D shape recovery to recover the deformations of the paper and augment it with fully 3D objects.