In an earlier project, we had developed four algorithms based on deep-learning models to enhance the retrieval of images from Tamedia’s photo archive. Based on those algorithms, it is now our objective to increase the usage of videos from Tamedia’s archives by enabling advanced search options.
Online news articles carry both images and videos. Since videos are actually a series of images, it is possible to use the current models based on pixel for also retrieving videos. However, extracting all images from videos would clutter the archive and affect the search experience. To address that challenge, the first aim of the project is to effectively extract video key-frames before indexing the videos. To achieve that, we intend to use face tracking and video abstraction.
A second objective of the project is to leverage the transcriptions of audio contained in the videos. Once the videos are translated into textual descriptions, it would be possible to build semantic search using standard text-to-text retrieval models. The challenges foreseen in this approach is multilingualism of Switzerland and semantic text summarization.
Tamedia is a leading private media group in Switzerland. In the mid-1990s, the group launched new digital business segments. With online participants in and beyond Switzerland, the company aims to become one of Europe’s leading digital media enterprises by providing strong journalism, attractive advertising options, and other digital services. Its photo and video archives, driven by our algorithms, are a critical component in that envisaged journey.