Multimedia Security


Duration of project

36 months

Funding source

Swiss National Science Foundation

Funding number

SNSF contract nb: 200020-113709


Replica detection aims at discovering replicas, or copies, of  originals. By replica, we refer not only to a bit exact copy of a given original, but also to modified versions of the original after minor manipulations, as long as these manipulations do not change its perceptual content. In the particular cases of images, replicas could include all variants of the original image obtained after common image processing manipulations such as compression, blurring, contrast adjustment, change in color space, etc. In the case of objects replica detection, the modification caused by view point  or other contextual changes.

The problem of image and object replica detection can be seen as a particular subset of the more general problem of content-based search and retrieval of multimedia content. For instance, in a search and retrieval system based on query by example, the system returns objects similar to the example, where the exact definition of ‘similar’ is often semantic. A query using the picture of a flower would return pictures of all sorts of flowers. Although, in general, a generic search and retrieval by example can be also used for replica detection, the problem can be resolved more efficiently by optimizing the solution to specific query examples. The goal of this project is to extend the work on image replica detection towards object replica detection for images and video.

Systems for object replica detection can be used for a wide variety of tasks, including:

  • Detect copyright infringement by identifying variations of a given copyrighted object.
  • Cluster retrieved images based on visible objects to reduce reduce redundant information for the user.
  • Propagate tags to new images based on the detection of the same object in previously annotated images.
  • Detect occurrence of objects of interest in large surveillance databases.
  • Provide information about an object captured by a mobile device.

Research activities

  • Research and development of a novel image duplicate detection based on machine learning.
  • Video duplicate detection using small video signature.
  • Object replica detection based on local features and graph matching.
  • Detection of specific object(s) that appear within video sequences.

Object duplicate detection

In this section, we present an efficient solution for 3D object duplicate detection in static images. The goal is to detect the presence of a target object and to predict its bounding box, based on a set of images containing that object. A small number of training images, typically one to four, containing different views of the target object, are sufficient enough to achieve good performance.

The block diagram of the developed object duplicate detection algorithm is depicted in the following figure:

During the training phase, a set of images of the target object (from different views, and with eventual deformations) is processed. First, features are extracted from the training images. To resolve the localization problem efficiently, we use accurate sparse features detector and descriptor. Hessian affine detector and SIFT affine invariant image descriptors are extracted. Hierarchical clustering is then applied in the feature space to increase the efficiency of the matching process.

In the testing phase, we retrieve images according to a query by applying two-direction nearest neighbor feature matching is applied. K-nearest neighbors in the query image are calculated and combined into the spatial neighborhood graph, considering the spatial distance of the features and similarity with the features in the model image. This graph contains spatial information from the potentially matching objects, hence making our algorithm more robust. The position is considered by applying General Hough Transform on the nodes of the graph.  This method results in several bounding boxes with overlapping and duplicated bounding boxes. These are then rejected by applying Non-maximum suppression algorithm.

Results can be seen in the following figure:

  • P. Vajda, L. Goldmann and T. Ebrahimi, “Analysis of the Limits of Graph-Based Object Duplicate Detection”, Proceedings of IEEE International Symposium on Multimedia, 2009.
  • P. Vajda, F. Dufaux, T. H. Minh, T. Ebrahimi, “Graph-Based Approach for 3D Object Duplicate Detection”, 10th International Workshop on Image Analysis for Multimedia Interactive Services, 2009.
  • Y. Maret, “Efficient image duplicate detection based on image analysis”, Thèse EPFL, no 3797 (2007). Dir.: Touradj Ebrahimi.
  • Y. Maret, S. Nikolopoulos, F. Dufaux, T. Ebrahimi and N. Nikolaidis, “A Novel Replica Detection System using Binary Classifiers, R-trees, and PCA”, ICIP 2006.
  • Y. Maret, F. Dufaux, T. Ebrahimi, Adaptive image replica detection based on support vector classifiers, in: Signal Processing: Image Communication, vol. 21, num. 8 (2006), p. 688-703, 2006.
  • Y. Maret, F. Dufaux, T. Ebrahimi, Image Replica Detection based on Support Vector Classifier, in: Proc. SPIE Applications of Digital Image Processing XXVIII, Santa Barbara, USA, 2005.
  • Y. Maret, G. Molina, T. Ebrahimi, Identification of Image Variations based on Equivalence Classes, in: Proc. SPIE Visual Communications and Image Processing, SPIE, Beijing, China, 2005.
  • Y. Maret, G. Molina, T. Ebrahimi, Images Identification Based on Equivalence Classes, in: Proc. Workshop on Image Analysis for Multimedia Interactive Services, Montreux, Switzerland, 2005.