Publications

Transhistorical Urban Landscape as Hypermap

D. N. del Castillo; I. Neri; P. Guhennec; J. A. Sanchez-Vaquerizo; L. Schaerf et al. 

This article explores the conception, design, and implementation of a hypertextual map that we call hypermap. Using Giovanni Nolli’s 1748 map of the city of Rome as a backbone, we conducted an experiment based on one of the routes defined by Giuseppe Vasi’s Grand Tour of Rome to collect various types of urban and environmental information, thus aiming to connect a multiplicity of data from different nature and times periods to enhance the serendipitous elaboration of new narratives, interpretations, and data (namely “unfolding”) not implicitly enacted by the pure analytical and mechanistic overlapping of gathered data (“folding”). This experiment is part of the research project entitled Datathink that was conducted at the Bibliotheca Hertziana – Max Planck Institute for Art History in Rome, the experiment serves as a proof of concept for an augmented database of the urban landscape in the city of Rome and new ways to facilitate the access and enhancement of cultural artifacts and knowledge.

34Th Acm Conference On Hypertext And Social Media, Ht 2023

2023-01-01

34th ACM Conference on Hypertext and Social Media (HT), Rome, ITALY, SEP 04-08, 2023.

DOI : 10.1145/3603163.3609083

Yes but.. Can ChatGPT Identify Entities in Historical Documents?

C-E. Gonzalez-Gallardo; E. Boros; N. Girdhar; A. Hamdi; J. G. Moreno et al. 

Large language models (LLMs) have been leveraged for several years now, obtaining state-of-the-art performance in recognizing entities from modern documents. For the last few months, the conversational agent ChatGPT has “prompted” a lot of interest in the scientific community and public due to its capacity of generating plausible-sounding answers. In this paper, we explore this ability by probing it in the named entity recognition and classification (NERC) task in primary sources (e.g., historical newspapers and classical commentaries) in a zero-shot manner and by comparing it with state-of-the-art LM-based systems. Our findings indicate several shortcomings in identifying entities in historical text that range from the consistency of entity annotation guidelines, entity complexity, and code-switching, to the specificity of prompting. Moreover, as expected, the inaccessibility of historical archives to the public (and thus on the Internet) also impacts its performance.

2023 Acm/Ieee Joint Conference On Digital Libraries, Jcdl

2023-01-01

23rd ACM/IEEE Joint Conference on Digital Libraries (JCDL), Santa Fe, NM, JUN 26-30, 2023.

p. 184-189

DOI : 10.1109/JCDL57899.2023.00034

Post-correction of Historical Text Transcripts with Large Language Models: An Exploratory Study

E. Boros; M. Ehrmann; Matteo Romanello; S. Najem-Meyer; F. Kaplan 

The quality of automatic transcription of heritage documents, whether from printed, manuscripts or audio sources, has a decisive impact on the ability to search and process historical texts. Although significant progress has been made in text recognition (OCR, HTR, ASR), textual materials derived from library and archive collections remain largely erroneous and noisy. Effective post-transcription correction methods are therefore necessary and have been intensively researched for many years. As large language models (LLMs) have recently shown exceptional performances in a variety of text-related tasks, we investigate their ability to amend poor historical transcriptions. We evaluate fourteen foundation language models against various post-correction benchmarks comprising different languages, time periods and document types, as well as different transcription quality and origins. We compare the performance of different model sizes and different prompts of increasing complexity in zero and few-shot settings. Our evaluation shows that LLMs are anything but efficient at this task. Quantitative and qualitative analyses of results allow us to share valuable insights for future work on post-correcting historical texts with LLMs.

LaTeCH-CLfL 2024

2024-02-18

The 8th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature, St Julian’s, Malta, March 22, 2024.

p. 133-159

impresso Text Reuse at Scale. An interface for the exploration of text reuse data in semantically enriched historical newspapers

M. Düring; M. Romanello; M. Ehrmann; K. Beelen; D. Guido et al. 

Text Reuse reveals meaningful reiterations of text in large corpora. Humanities researchers use text reuse to study, e.g., the posterior reception of influential texts or to reveal evolving publication practices of historical media. This research is often supported by interactive visualizations which highlight relations and differences between text segments. In this paper, we build on earlier work in this domain. We present impresso Text Reuse at Scale, the to our knowledge first interface which integrates text reuse data with other forms of semantic enrichment to enable a versatile and scalable exploration of intertextual relations in historical newspaper corpora. The Text Reuse at Scale interface was developed as part of the impresso project and combines powerful search and filter operations with close and distant reading perspectives. We integrate text reuse data with enrichments derived from topic modeling, named entity recognition and classification, language and document type detection as well as a rich set of newspaper metadata. We report on historical research objectives and common user tasks for the analysis of historical text reuse data and present the prototype interface together with the results of a user evaluation.

Frontiers in Big Data

2023-11-03

Vol. 6 , num. Visualizing Big Culture and History Data , p. 1-16.

DOI : 10.3389/fdata.2023.1249469

Digitization of the Inscriptions on the Monuments of Armenian Cultural Heritage in Nagorno-Karabakh Region

H. Tamrazyan 

This article discusses the efforts of the DH LAB (EPFL) to preserve Armenian cultural heritage in Artsakh, particularly inscriptions on monuments in the Nagorno-Karabakh region. Using Digital Humanities (DH) technology, the project aims to collect, systematize, and digitize these inscriptions, creating 3D models where possible. The initiative addresses the lack of digital epigraphy applied to Armenian inscriptions, previously underrepresented in digital heritage efforts. The project has established a network of partners in Armenia and Artsakh, and has begun creating a searchable inscriptions database, integrating essential information and translations. Challenges include editing texts, transcription accuracy, and developing a critical apparatus. The project also focuses on creating Linked Open Data and expanding to include Armenian epigraphic heritage in Ukraine, aiming to build a comprehensive corpus of Armenian inscriptions globally.

2023-07-01

Digital Humanities 2023. Collaboration as Opportunity. (DH2023), Graz, Austria, 10-14 July 2023.

DOI : 10.5281/zenodo.8108026

The Skin of Venice: Automatic Facade Extraction from Point Clouds

P. Guhennec; I. Di Lenardo 

We propose a method to extract orthogonal views of facades from photogrammetric models of cities. This method was applied to extract all facades of the city of Venice. The result images open up new areas of research in architectural history.

Digital Humanities 2023 : Book of Abstracts

2023-06-30

ADHO Digital Humanities Conference 2023 (DH2023), Graz, Austria, July 10-14 2023.

DOI : 10.5281/zenodo.8107943

Tracking Transmission of Details in Paintings

B. L. A. Seguin; I. Di Lenardo; F. Kaplan 

In previous articles (di Lenardo et al, 2016; Seguin et al, 2016), we explored how efficient visual search engines operating not on the basis of textual metadata but directly through visual queries, could fundamen- tally change the navigation in large databases of work of arts. In the present work, we extended our search engine in order to be able to search not only for global similarity between paintings, but also for matching de- tails. This feature is of crucial importance for retriev- ing the visual genealogy of a painting, as it is often the case that one composition simply reuses a few ele- ments of other works. For instance, some workshops of the 16th century had repertoires of specific charac- ters (a peasant smoking a pipe, a couple of dancing, etc.) and anatomical parts (head poses, hands, etc.) ,that they reused in many compositions (van den Brink, 2001; Tagliaferro et al, 2009). In some cases it is possible to track the circulation of these visual pat- terns over long spatial and temporal migrations, as they are progressively copied by several generations of painters. Identifying these links permits to recon- struct the production context of a painting, and the connections between workshops and artists. In addi- tion, it permits a fine-grained study of taste evolution in the history of collections, following specific motives successfully reused in a large number of paintings. Tracking these graphical replicators is challenging as they can vary in texture and medium. For instance, a particular character or a head pose of a painting may have been copied from a drawing, an engraving or a tapestry. It is therefore important that the search for matching details still detects visual reuse even across such different media and styles. In the rest of the pa- per, we describe the matching method and discuss some results obtained using this approach.

27-01-2017

Alliance of Digital Humanities Organizations (ADHO), Montréal, Canada, 8-11 ; 08 ; 2017.

Can Venice be saved?

S. Settis; I. Di Lenardo; F. Kaplan 

Will Venice be inhabitable in 2100? What kinds of policies can we develop to navigate the best scenarios for this floating city? In 2012, the École Polytechnique Fédérale de Lausanne (EPFL) and the University Ca’Foscari launched a programme called the Venice Time Machine to create a large-scale digitisation project transforming Venice’s heritage into ‘big data’. Thanks to the support of the Lombard Odier Foundation, millions of pages and photographs have been scanned at the state archive in Venice and at the Fondazione Giorgio Cini. While commercial robotic scanners were used at the archives, a new typology of robotised circular table was developed by Adam Lowe and his team at Factum Arte to process the million photographs of Fondazione Giorgio Cini. The documents were analysed using deep-learning artificial-intelligence methods to extract their textual and iconographic content and to make the data accessible via a search engine. Also during this time, thousands of primary and secondary sources were compiled to create the first 4D model (3D + time) of the city, showing the evolution of its urban fabric. This model and the other data compiled by the Venice Time Machine were part of an exhibition at the Venice Pavilion of the Biennale of Architecture in 2018, shown side-by-side with potential projects for Venice’s future. Having reached an important milestone in convincing not only the Venetian stakeholders but also a growing number of partners around the world that care about Venice’s future, the Venice Time Machine is now raising funds for the most ambitious simulation of the city that has ever been developed. Its planned activities include a high-resolution digitisation campaign of the entire city at centimetre scale, a crucial step on which to base a future simulation of the city’s evolution, while also creating a digital model that can be used for preservation regardless of what occurs in the coming decades. On the island of San Giorgio Maggiore, a digitisation centre called ARCHiVe (Analysis and Recording of Cultural Heritage in Venice) opened in 2018 to process a large variety of Venetian artefacts. This is a joint effort of Factum Foundation, the École Polytechnique Fédérale de Lausanne and the Fondazione Giorgio Cini, along with philanthropic support from the Helen Hamlyn Trust. The centre aims to become a training centre for future cultural heritage professionals who would like to learn how they can use artificial intelligence and robotics to preserve documents, objects and sites. These operations will work together to create a multiscale digital model of Venice, combining the most precise 4D information on the evolution of the city and its population with all the available documentation of its past. The project aims to demonstrate how this ‘digital double’ can be achieved by using robotic technology to scan the city and its archives on a massive scale, using artificial intelligence techniques to process documents and collecting the efforts of thousands of enthusiastic Venetians. In a project called ‘Venice 2100’, the Venice Time Machine team’s ambition is to show how a collectively built information system can be used to build realistic future scenarios, blending ecological and social data into large-scale simulations. The Venice Time Machine’s ‘hypermodel’ will also create economic opportunities. If its hypotheses are valid, Venice could host the first incubators for start-ups using big data of the past to develop services for smart cities, creative industries, education, academic scholarship and policy making. This could be the beginning of a renewal of Venice’s economic life, encouraging younger generations to pursue activities in the historic city, at the heart of what may become one of the first AI-monitored cities of the world. Venice can reinvent itself as the city that put the most advanced information technology and cultural heritage at the core of its survival and its strategy for development. Artificial intelligence can not only save Venice, but Venice can be the place to invent a new form of artificial intelligence.

Apollo, The International Art Magazine

02-01-2019

Where Did the News Come From? Detection of News Agency Releases in Historical Newspapers

L. Marxen 

Since their beginnings in the 1830s and 1840s, news agencies have played an important role in the national and international news market, aiming to deliver news as fast and as reliable as possible. While we know that newspapers have been using agency content for a long time to produce their stories, the amount to which the agencies shape our news often remains unclear. Although researchers have already addressed this question, recently by using computational methods to assess the influence of news agencies at present, large-scale studies on the role of news agencies in the past continue to be rare. This thesis aims to bridge this gap by detecting news agencies in a large corpus of Swiss and Luxembourgish newspaper articles (the impresso corpus) for the years 1840-2000 using deep learning methods. For this, we first build and annotate a multilingual dataset with news agency mentions, which we then use to train and evaluate several BERT-based agency detection and classification models. Based on these experiments, we choose two models (for French and German) for the inference on the impresso corpus. Results show that ca. 10% of the articles explicitly reference news agencies, with the greatest share of agency content after 1940, although systematic citation of agencies already started slowly in the 1910s. Differences in the usage of agency content across time, countries and languages as well as between newspapers reveal a complex network of news flows, whose exploration provides many opportunities for future work.

2023-08-18

Advisor(s): M. Ehrmann; E. Boros; M. Duering; F. Kaplan

From Automated Bootstrapping to Collaborative Editing: A Framework for 4D City Reconstruction

B. Vaienti; P. Guhennec; D. Dupertuis; R. Petitpierre 

We propose a framework for the construction of 4D scientific models for cities of the past. It leverages both the use of computational methods for bootstrapping, and a collaborative interface for visualisation and edition. This permits to continuously enrich the model and dynamically update it through procedural modelling.

Digital Humanities 2023: Book of Abstracts

2023

Digital Humanities 2023: Collaboration as Opportunity (DH2023), Graz, Austria, July 10-14, 2023.

p. 310-313

DOI : 10.5281/zenodo.8107906

Overview of HIPE-2022: Named Entity Recognition and Linking in Multilingual Historical Documents

M. Ehrmann; M. Romanello; S. Najem-Meyer; A. Doucet; S. Clematide 

This paper presents an overview of the second edition of HIPE (Identifying Historical People, Places and other Entities), a shared task on named entity recognition and linking in multilingual historical documents. Following the success of the first CLEF-HIPE-2020 evaluation lab, HIPE-2022 confronts systems with the challenges of dealing with more languages, learning domain-specific entities, and adapting to diverse annotation tag sets. This shared task is part of the ongoing efforts of the natural language processing and digital humanities communities to adapt and develop appropriate technologies to efficiently retrieve and explore information from historical texts. On such material, however, named entity processing techniques face the challenges of domain heterogeneity, input noisiness, dynamics of language, and lack of resources. In this context, the main objective of HIPE-2022, run as an evaluation lab of the CLEF 2022 conference, is to gain new insights into the transferability of named entity processing approaches across languages, time periods, document types, and annotation tag sets. Tasks, corpora, and results of participating teams are presented.

Experimental IR Meets Multilinguality, Multimodality, and Interaction. 13th International Conference of the CLEF Association, CLEF 2022, Bologna, Italy, September 5–8, 2022, Proceedings

2022

13th Conference and Labs of the Evaluation Forum (CLEF 2022), Bologna, Italy, 5-8 September 2022.

p. 423-446

DOI : 10.1007/978-3-031-13643-6_26

Machine-Learning-Enhanced Procedural Modeling for 4D Historical Cities Reconstruction

B. Vaienti; R. G. Petitpierre; I. Di Lenardo; F. Kaplan 

The generation of 3D models depicting cities in the past holds great potential for documentation and educational purposes. However, it is often hindered by incomplete historical data and the specialized expertise required. To address these challenges, we propose a framework for historical city reconstruction. By integrating procedural modeling techniques and machine learning models within a Geographic Information System (GIS) framework, our pipeline allows for effective management of spatial data and the generation of detailed 3D models. We developed an open-source Python module that fills gaps in 2D GIS datasets and directly generates 3D models up to LOD 2.1 from GIS files. The use of the CityJSON format ensures interoperability and accommodates the specific needs of historical models. A practical case study using footprints of the Old City of Jerusalem between 1840 and 1940 demonstrates the creation, completion, and 3D representation of the dataset, highlighting the versatility and effectiveness of our approach. This research contributes to the accessibility and accuracy of historical city models, providing tools for the generation of informative 3D models. By incorporating machine learning models and maintaining the dynamic nature of the models, we ensure the possibility of supporting ongoing updates and refinement based on newly acquired data. Our procedural modeling methodology offers a streamlined and open-source solution for historical city reconstruction, eliminating the need for additional software and increasing the usability and practicality of the process.

Remote Sensing

2023

Vol. 15 , num. 13, p. 3352.

DOI : 10.3390/rs15133352

Ce que les machines ont vu et que nous ne savons pas encore

I. Di Lenardo; F. Kaplan 

Cet article conceptualise l’idée qu’il existe une « matière noire » composée des structurations latentes identifiées par le regard machinique sur de grandes collections photographiques patrimoniales. Les campagnes photographiques de l’histoire de l’art, au xxe siècle, avaient pour ambition implicite de transformer toutes les œuvres d’art en documents plus facilement étudiables. Au fil du temps, la création de ces collections visuelles a permis de produire un corpus d’informations potentiellement plus dense et plus riche que ce que ses créateurs avaient initialement imaginé. En effet, la conversion numérique de ces immenses corpus visuels permet aujourd’hui de réanalyser ces images avec des techniques de vision par ordinateur, l’intelligence artificielle ouvrant ainsi la voie à des perspectives d’études bien différentes de celles envisageables au siècle dernier. Nous pourrions ainsi dire qu’il y a dans ces images un immense potentiel latent de connaissance, un réseau dense de relations qui n’a pas encore été mis en lumière. Qu’est-ce que les machines ont vu ou vont pouvoir voir dans ces collections d’images que l’homme n’a pas encore identifié ? Quelle étendue la connaissance visuelle de l’homme couvre-t-elle par rapport à ce que la machine a pu analyser ? Les nouvelles techniques d’indexation des images et des motifs qui les constituent nous rapprochent d’une révolution copernicienne du visuel dans laquelle l’homme peut, grâce à la machine-prothèse, analyser beaucoup plus d’images qu’il ne pouvait le faire par une simple activité mnémonique et sélectionner des perspectives spécifiques en comparant des ensembles de motifs les uns par rapport aux autres. Cette vision augmentée est fondée sur une pré-analyse conduite par la machine sur l’ensemble de ces corpus visuels, un entraînement qui permet de retrouver la structure sous-jacente du système d’images. La vision humaine est ainsi étendue par le regard artificiel préalable de la machine. Pour comprendre les enjeux de cette nouvelle alliance, il faut étudier la nature de ce regard artificiel, comprendre son potentiel pour découvrir des structures jusqu’à présent inconnues et anticiper les nouvelles formes de connaissances humaines auxquelles il pourra donner naissance. L’enjeu sera donc, pour les prochaines années, de comprendre ce que les machines ont vu et que nous ne savons pas encore.

Sociétés & Représentations

2023

num. 1, p. 249-267.

DOI : 10.3917/sr.055.0249

Les vingt premières années du capitalisme linguistique : Enjeux globaux de la médiation algorithmique des langues

F. Kaplan 

La médiation des flux linguistiques par une poignée d’acteurs mondiaux a permis la constitution de modèles de la langue dont les performances sont aujourd’hui sans précédents. Ce texte revient sur les thèses principales qui expliquent des dynamiques économiques et technologique à l’origine de ce phénomène, prolongeant les réflexions préalablement entamées (thèses 1 et 2), pour ensuite développer la question du développement et de l’usage des nouveaux modèles basé sur le capital linguistique accumulé (thèses 3 et 4) et les conséquences sur les nouvelles formes de médiation algorithmiques qui en découlent (thèse 5 et 6). L’enjeu est de comprendre le basculement qui s’opère dans l’économie de l’expression quand les algorithmes produisent des écrits de performativité supérieure à la langue courante et sont donc naturellement utilisés pour produire du texte par l’intermédiaire de prothèses. Nous assistons à ce que l’on pourrait qualifier de “second régime du capitalisme linguistique” allant au-delà de la vente aux enchères des mots pour proposer additionnellement la vente de services linguistiques se substituant à l’écriture même. Il devient alors possible, à l’image des services de traduction automatique, mais cette fois-ci pour sa propre langue, de s’exprimer avec plus d’efficacité et de produire de l’écrit adapté aux diverses exigences des situations de la vie professionnelle ou privée. Nous discutons comment la généralisation de ces prothèses linguistique pourrait conduire à une atrophie de la capacité expressive, pour l’écriture dans un premier temps et, peut-être, à terme pour l’oralité.

Prendre soin des l’informatique et des générations; Limoges: FYP Editions, 2021.

ISBN : 978-2-36405-212-3

Effective annotation for the automatic vectorization of cadastral maps

R. Petitpierre; P. Guhennec 

The great potential brought by large-scale data in the humanities is still hindered by the time and technicality required for making documents digitally intelligible. Within urban studies, historical cadasters have been hitherto largely under-explored despite their informative value. Powerful and generic technologies, based on neural networks, to automate the vectorization of historical maps have recently become available. However, the transfer of these technologies is hampered by the scarcity of interdisciplinary exchanges and a lack of practical literature destinated to humanities scholars, especially on the key step of the pipeline: the annotation. In this article, we propose a set of practical recommendations based on empirical findings on document annotation and automatic vectorization, focusing on the example case of historical cadasters. Our recommendations are generic and easily applicable, based on a solid experience on concrete and diverse projects.

Digital Scholarship In The Humanities

2023-03-09

DOI : 10.1093/llc/fqad006

From Archival Sources to Structured Historical Information: Annotating and Exploring the “Accordi dei Garzoni”

M. Ehrmann; O. Topalov; F. Kaplan 

If automatic document processing techniques have achieved a certain maturity for present time documents, the transformation of hand-written documents into well-represented, structured and connected data which can satisfactorily be used for historical study purposes is not straightforward and still presents major challenges. Transitioning from documents to structured data was one of the key challenges faced by the Garzoni project and this chapter details the techniques and the steps taken to represent, extract, enhance and exploit the information contained in the archival material.

Apprenticeship, Work, Society in Early Modern Venice; Abingdon: Routledge, Taylor & Francis Group, 2023-02-10. p. 304.

ISBN : 978-1-003197-19-5

DOI : 10.4324/9781003197195-6

Automatic Content Curation of Visual Heritage

D. Ribes Lemay; V. Bernasconi; A. Andrade; L. Défayes; M. Salzmann et al. 

Digitization and preservation of large heritage induce high maintenance costs to keep up with the technical standards and ensure sustainable access. Creating impactful usage is instrumental to justify the resources for long-term preservation. The Museum für Gestaltung of Zurich holds one of the biggest poster collections of the world from which 52’000 were digitised. In the process of building a digital installation to valorize the collection, one objective was to develop an algorithm capable of predicting the next poster to show according to the ones already displayed. The work presented here describes the steps to build an algorithm able to automatically create sequences of posters reflecting associations performed by curator and professional designers. The exposed challenge finds similarities with the domain of song playlist algorithms. Recently, artificial intelligence techniques and more specifically, deep-learning algorithms have been used to facilitate their generations. Promising results were found thanks to Recurrent Neural Networks (RNN) trained on manually generated playlist and paired with clusters of extracted features from songs. We used the same principles to create the proposed algorithm but applied to a challenging medium, posters. First, a convolutional autoencoder was trained to extract features of the posters. The 52’000 digital posters were used as a training set. Poster features were then clustered. Next, an RNN learned to predict the next cluster according to the previous ones. RNN training set was composed of poster sequences extracted from a collection of books from the Gestaltung Museum of Zurich dedicated to displaying posters. Finally, within the predicted cluster, the poster with the best proximity compared to the previous poster is selected. The mean square distance between features of posters was used to compute the proximity. To validate the predictive model, we compared sequences of 15 posters produced by our model to randomly and manually generated sequences. Manual sequences were created by a professional graphic designer. We asked 21 participants working as professional graphic designers to sort the sequences from the one with the strongest graphic line to the one with the weakest and to motivate their answer with a short description. The sequences produced by the designer were ranked first 60%, second 25% and third 15% of the time. The sequences produced by our predictive model were ranked first 25%, second 45% and third 30% of the time. The sequences produced randomly were ranked first 15%, second 29%, and third 55% of the time. Compared to designer sequences, and as reported by participants, model and random sequences lacked thematic continuity. According to the results, the proposed model is able to generate better poster sequencing compared to random sampling. Eventually, our algorithm is sometimes able to outperform a professional designer. As a next step, the proposed algorithm should include a possibility to create sequences according to a selected theme. To conclude, this work shows the potentiality of artificial intelligence techniques to learn from existing content and provide a tool to curate large sets of data, with a permanent renewal of the presented content.

World Academy of Science, Engineering and Technology International Journal of Humanities and Social Sciences

2021-11-18

ICDH 2021 : International Conference on Digital Heritage, London, United Kingdom, November 18-19, 2021.

Digitised Historical Newspapers: A Changing Research Landscape (Introduction)

M. Ehrmann; E. Bunout; F. Clavert 

The application of digital technologies to newspaper archives is transforming the way historians engage with these sources. The digital evolution not only affects how scholars access historical newspapers, but also, increasingly, how they search, explore and study them. Two developments have been driving this transformation: massive digitisation, which facilitates access to remote holdings and, more recently, improved search capabilities, which alleviate the tedious exploration of vast collections, opens up new prospects and transforms research practices. The volume “Digitised newspapers – A New Eldorado for Historians?” brings together the contributions of a workshop held in 2020 on tools, methods and epistemological reflections on the use of digitised newspapers and offers three perspectives: how digitisation is transforming access to and exploration of historical newspaper collections; how automatic content processing allows for the creation of new layers of information; and, finally, what analyses this enhanced material opens up. This introductory chapter reviews recent developments that have influenced the research landscape of digitized newspapers in recent years and introduces the eighteen articles that comprise this volume.

Digitised Newspapers – A New Eldorado for Historians?; Berlin, Boston: De Gruyter Oldenbourg, 2022-12-31. p. 439.

ISBN : 978-3-110729-21-4

Computational Approaches to Digitised Historical Newspapers (Dagstuhl Seminar 22292)

M. Ehrmann; M. Düring; C. Neudecker; A. Doucet 

Historical newspapers are mirrors of past societies, keeping track of the small and great history and reflecting the political, moral, and economic environments in which they were produced. Highly valued as primary sources by historians and humanities scholars, newspaper archives have been massively digitised in libraries, resulting in large collections of machine-readable documents and, over the past half-decade, in numerous academic research initiatives on their automatic processing. The Dagstuhl Seminar 22292 “Computational Approaches to Digitised Historical Newspaper” gathered researchers and practitioners with backgrounds in natural language processing, computer vision, digital history and digital library involved in computational approaches to historical newspapers with the objectives to share experiences, analyse successes and shortcomings, deepen our understanding of the interplay between computational aspects and digital scholarship, and discuss future challenges. This report documents the program and the outcomes of the seminar. DagRep, Volume 12, Issue 7, pages 112-179

2023

p. 69.

Page Layout Analysis of Text-heavy Historical Documents: a Comparison of Textual and Visual Approaches

S. Najem-Meyer; M. Romanello 

Page layout analysis is a fundamental step in document processing which enables to segment a page into regions of interest. With highly complex layouts and mixed scripts, scholarly commentaries are text-heavy documents which remain challenging for state-of-the-art models. Their layout considerably varies across editions and their most important regions are mainly defined by semantic rather than graphical characteristics such as position or appearance. This setting calls for a comparison between textual, visual and hybrid approaches. We therefore assess the performances of two transformers (LayoutLMv3 and RoBERTa) and an objection-detection network (YOLOv5). If results show a clear advantage in favor of the latter, we also list several caveats to this finding. In addition to our experiments, we release a dataset of ca. 300 annotated pages sampled from 19th century commentaries.

Proceedings of the Computational Humanities Research Conference 2022 Antwerp, Belgium, December 12-14, 2022

2022-12-12

Third Conference on Computational Humanities Research (CHR 2022), Antwerp, Belgium, December 12-14, 2022.

p. 36-54

DOI : 10.48550/arXiv.2212.13924

Digitised Newspapers – A New Eldorado for Historians? Reflections on Tools, Methods and Epistemology

 

The application of digital technologies to historical newspapers has changed the research landscape historians were used to. An Eldorado? Despite undeniable advantages, the new digital affordance of historical newspapers also transforms research practices and confronts historians with new challenges. Drawing on a growing community of practices, the impresso project invited scholars experienced with digitised newspaper collections with the aim of encouraging a discussion on heuristics, source criticism and interpretation of digitized newspapers. This volume provides a snapshot of current research on the subject and offers three perspectives: how digitisation is transforming access to and exploration of historical newspaper collections; how automatic content processing allows for the creation of new layers of information; and, finally, what analyses this enhanced material opens up. ‘impresso – Media Monitoring of the Past’ is an interdisciplinary research project that applies text mining tools to digitised historical newspapers and integrates the resulting data into historical research workflows by means of a newly developed user interface. The question of how best to adapt text mining tools and their use by humanities researchers is at the heart of the impresso enterprise.

Berlin: De Gruyter, 2022-12-31.

ISBN : 978-3-110729-21-4

DOI : 10.1515/9783110729214

Optical Character Recognition of 19th Century Classical Commentaries: the Current State of Affairs

S. Najem-Meyer; B. Robertson; M. Romanello 

Together with critical editions and translations, commentaries are one of the main genres of publication in literary and textual scholarship, and have a century-long tradition. Yet, the exploitation of thousands of digitized historical commentaries was hitherto hindered by the poor quality of Optical Character Recognition (OCR), especially on commentaries to Greek texts. In this paper, we evaluate the performances of two pipelines suitable for the OCR of historical classical commentaries. Our results show that Kraken + Ciaconna reaches a substantially lower character error rate (CER) than Tesseract/OCR-D on commentary sections with high density of polytonic Greek text (average CER 7% vs. 13%), while Tesseract/OCR-D is slightly more accurate than Kraken + Ciaconna on text sections written predominantly in Latin script (average CER 8.2% vs. 8.4%). As part of this paper, we also release GT4HistComment, a small dataset with OCR ground truth for 19th classical commentaries and Pogretra, a large collection of training data and pre-trained models for a wide variety of ancient Greek typefaces.

2021-10-13

DOI : 10.48550/arXiv.2110.06817

A data structure for scientific models of historical cities: extending the CityJSON format

B. Vaienti; P. Guhennec; I. di Lenardo 

In the field of the 3D reconstruction of cities in the past there is a raising interest in the creation of models that are not just geometrical, but also informative, semantic and georeferenced. Despite the advancements that were done in the historical reconstruction of architecture and archaeology, the solutions designed for larger scale models are still very limited. On the other hand, research on the digitisation of current-day cities provides useful instruments. In particular, CityJSON – a JSON encoding of CityGML – represents an easy-to-use and lightweight solution for storing 3D models of cities that are geolocated, semantic and that contain additional information in the form of attributes. This contribution proposes (1) to extend the schema to the needs of a historical representation; and (2) to incorporate the newly created model in a continuous flow pipeline, in which the geometry is dynamically updated each time an attribute is changed, as a means to foster collaboration.

Proceedings of the 6th ACM SIGSPATIAL International Workshop on Geospatial Humanities

2022-11-11

6th ACM SIGSPATIAL International Workshop on Geospatial Humanities, Seattle, Washington, November 1, 2022.

p. 20-23

DOI : 10.1145/3557919.3565813

Generic Semantic Segmentation of Historical Maps

R. G. Petitpierre; F. Kaplan; I. Di Lenardo 

Research in automatic map processing is largely focused on homogeneous corpora or even individual maps, leading to inflexible models. Based on two new corpora, the first one centered on maps of Paris and the second one gathering maps of cities from all over the world, we present a method for computing the figurative diversity of cartographic collections. In a second step, we discuss the actual opportunities for CNN-based semantic segmentation of historical city maps. Through several experiments, we analyze the impact of figurative and cultural diversity on the segmentation performance. Finally, we highlight the potential for large-scale and generic algorithms. Training data and code of the described algorithms are made open-source and published with this article.

CEUR Workshop Proceedings

2021-11-17

CHR 2021: Computational Humanities Research Conference, Amsterdam, The Netherlands, November 17-19, 2021.

p. 228-248

Conditional Synthetic Financial Time Series with Generative Adversarial Networks

A. Rusnak 

The creation of high fidelity synthetic data has long been an important goal in machine learning, particularly in fields like finance where the lack of available training and test data make it impossible to utilize many of the deep learning techniques which have proven so powerful in other domains. Despite ample research into different types of synthetic generation techniques, which in recent years have largely focused on generative adversarial networks, there remain key holes in many of the architectures and techniques being utilized. In particular, there are currently no techniques available which can generate multiple series concurrently while capturing the specific stylized facts of financial time series and which incorporate extra information that effect the series such as macroeconomic factors. In this thesis, we propose the Conditional Market Transformer-Encoder Generative Adversarial Network (C-MTE-GAN), a novel generative adversarial neural network architecture that satisfies the aforementioned challenges. C-MTE-GAN is able to capture the relevant univariate stylized facts such as lack of autocorrelation of returns, volatility clustering, fat tails, and the leverage effect. It is also able to capture the multivariate interactions between multiple concurrently generated series such as correlation and tail dependence. Lastly, we are able to condition the generated series both on a prior series of returns as well as on different types of relevant information that typically effect both the characteristics of the market and factor into asset allocation decision making. Furthermore, we demonstrate the effectiveness of data generated by C-MTE-GAN to augment training of a statistical arbitrage model and improve its performance in realistic portfolio allocation scenarios. The abilities of this architecture represent a substantial step forward in financial time series generation which will hopefully unlock many new applications of synthetic data within the realm of finance.

2022

Advisor(s): F. Kaplan; S. Daul

Optical Character Recognition of 19th Century Classical Commentaries: the Current State of Affairs

M. Romanello; S. Najem-Meyer; B. Robertson 

Together with critical editions and translations, commentaries are one of the main genres of publication in literary and textual scholarship, and have a century-long tradition. Yet, the exploitation of thousands of digitized historical commentaries was hitherto hindered by the poor quality of Optical Character Recognition (OCR), especially on commentaries to Greek texts. In this paper, we evaluate the performances of two pipelines suitable for the OCR of historical classical commentaries. Our results show that Kraken + Ciaconna reaches a substantially lower character error rate (CER) than Tesseract/OCR-D on commentary sections with high density of polytonic Greek text (average CER 7% vs. 13%), while Tesseract/OCR-D is slightly more accurate than Kraken + Ciaconna on text sections written predominantly in Latin script (average CER 8.2% vs. 8.4%). As part of this paper, we also release GT4HistComment, a small dataset with OCR ground truth for 19th classical commentaries and Pogretra, a large collection of training data and pre-trained models for a wide variety of ancient Greek typefaces.

HIP ’21: The 6th International Workshop on Historical Document Imaging and Processing

2021-10-31

HIP ’21: The 6th International Workshop on Historical Document Imaging and Processing, Lausanne, Switzerland, September 5-6, 2021.

p. 1-6

DOI : 10.1145/3476887.3476911

Named Entity Recognition and Classification in Historical Documents: A Survey

M. Ehrmann; A. Hamdi; E. Linhares Pontes; M. Romanello; A. Doucet 

After decades of massive digitisation, an unprecedented amount of historical documents is available in digital format, along with their machine-readable texts. While this represents a major step forward with respect to preservation and accessibility, it also opens up new opportunities in terms of content mining and the next fundamental challenge is to develop appropriate technologies to efficiently search, retrieve and explore information from this ‘big data of the past’. Among semantic indexing opportunities, the recognition and classification of named entities are in great demand among humanities scholars. Yet, named entity recognition (NER) systems are heavily challenged with diverse, historical and noisy inputs. In this survey, we present the array of challenges posed by historical documents to NER, inventory existing resources, describe the main approaches deployed so far, and identify key priorities for future developments.

ACM Computing Survey

2021-09-21

Extended Overview of HIPE-2022: Named Entity Recognition and Linking in Multilingual Historical Documents

M. Ehrmann; M. Romanello; S. Najem-Meyer; A. Doucet; S. Clematide 

This paper presents an overview of the second edition of HIPE (Identifying Historical People, Places and other Entities), a shared task on named entity recognition and linking in multilingual historical documents. Following the success of the first CLEF-HIPE-2020 evaluation lab, HIPE-2022 confronts systems with the challenges of dealing with more languages, learning domain-specific entities, and adapting to diverse annotation tag sets. This shared task is part of the ongoing efforts of the natural language processing and digital humanities communities to adapt and develop appropriate technologies to efficiently retrieve and explore information from historical texts. On such material, however, named entity processing techniques face the challenges of domain heterogeneity, input noisiness, dynamics of language, and lack of resources. In this context, the main objective of HIPE-2022, run as an evaluation lab of the CLEF 2022 conference, is to gain new insights into the transferability of named entity processing approaches across languages, time periods, document types, and annotation tag sets. Tasks, corpora, and results of participating teams are presented. Compared to the condensed overview, this paper contains more refined statistics on the datasets, a break down of the results per type of entity, and a discussion of the ‘challenges’ proposed in the shared task.

Proceedings of the Working Notes of CLEF 2022 – Conference and Labs of the Evaluation Forum

2022

13th Conference and Labs of the Evaluation Forum (CLEF 2022), Bologna, Italy, 5-8 Sept 2022.

DOI : 10.5281/zenodo.6979577

Searching for visual patterns in a children’s drawings collection

R. Annapureddy 

The success of large-scale digitization projects at museums, archives, and libraries is pushing other cultural institutions to embrace digitization to preserve their collections. By juxtaposing digital tools with digitized collections, it is now possible to study these cultural objects at a previously unknown scale. This thesis is the first attempt to explore a recently digitized children’s drawings collection while developing a system to identify patterns in them linked with popular cultural objects. Artists, as young as three and as old as 25, created nearly 90,000 drawings in the span of three decades from most countries in the world. The preliminary examination unveils that these drawings mirror a solid cultural ethos by using specific iconographic subjects, objects, and colors, and the distinction between children of different parts of the globe is visible in their works. These factors not only make the dataset distinct from other sketch datasets but place it distantly from them in terms of size and multifariousness of creations and the creators. The essential and another dimension of the project is matching the drawings and the popular cultural objects they represent. A deep learning model that learns a metric to rank the visual similarity between the images is used to identify the drawing-artwork pairs. Though the networks developed for image classification perform inadequately for the matching task, networks used for pattern matching in paintings show good performance. Fine-tuning the models increases the performance drastically. The primary outcomes of this work are (1) systems trained with a few methodically chosen examples perform comparably to the systems trained on thousands of generic samples and (2) using drawings enriched by adding generic effects of watercolor, oil painting, pencil sketch, and texturizing mitigates the situation of network learning examples by heart.

2022-07-08

Advisor(s): F. Kaplan; J. R. Fageot

Opacité et transparence dans le design d’un dispositif de surveillance urbain : le cas de l’IMSI catcher

F. M. M. Goguey / Director(s) : F. Kaplan; N. Nova 

This thesis assesses the surveillance operated on the mobile phone network by governmental actors (intelligence agencies, police, army) and the relationship between monitored spaces and their users. Indeed, some new surveillance devices used by intelligence services redefine surveillance spatiality raising new questions in this field of research. More specifically this research focuses on one specific object: the IMSI catcher, a monitoring apparatus of the cellular network that intercepts cellphones’ identity and some communications in a given area by mimicking the activity of a cell tower. While this kind of device seems to offer a tactical and a security interest in the fight against terrorism and against crime, many civil liberties organisations such as the Electronic Frontier Foundation, Privacy International and _La Commission nationale de l’informatique et des libertés are concerned about the potential of an uncontrolled surveillance ; indeed, the controversial nature of the device could endanger certain individual and public rights. What is this technical object and which new issues comes with its use in surveillance? How and from which perspective is it problematic What does the IMSI catcher teaches us on the potential future of surveillance regimes? I look into this specific device case in a research framework at the intersection of design research practices, science and technology studies (STS) and surveillance studies. First, I deal with this surveillance apparatus as a technical object, from a perspective fed by the theoretical framework of _concretization_ and _technical lines_ proposed by Gilbert Simondon and Yves Deforge, through the analysis of a visual and technical documentation. Second, I use a research by design approach to explore certain assumptions regarding the nature of the object itself, its functioning and its “concrete” aspect – or rather “non-concrete” in the present case – with the help of approaches borrowed to reverse engineering and reconstitution, close to media archeology. Then, I explore possible opposition and protest trajectories with the help of prototypes designed with critical design and speculative design methods. Finally, through the writing of prospective scenarios, I build a design fiction that offers a synthesis, potentially subject to debate, around the IMSI catcher’s uses, present and to come, and more broadly on the potential future of surveillance regimes.

Lausanne: EPFL

2022

p. 230.

DOI : 10.5075/epfl-thesis-8838

ECCE: Entity-centric Corpus Exploration Using Contextual Implicit Networks

J. Schelb; M. Ehrmann; M. Romanello; A. O. Spitz 

In the Digital Age, the analysis and exploration of unstructured document collections is of central importance to members of investigative professions, whether they might be scholars, journalists, paralegals, or analysts. In many of their domains, entities play a key role in the discovery of implicit relations between the contents of documents and thus serve as natural entry points to a detailed manual analysis, such as the prototypical 5Ws in journalism or stock symbols in finance. To assist in these analyses, entity-centric networks have been proposed as a language model that represents document collections as a cooccurrence graph of entities and terms, and thereby enables the visual exploration of corpora. Here, we present ECCE, a web-based application that implements entitycentric networks, augments them with contextual language models, and provides users with the ability to upload, manage, and explore document collections. Our application is available as a web-based service at http://dimtools.uni.kn/ecce.

WWW ’22 Companion

2022

The Web Conference (WWW’22), Lyon, France, April 25-29, 2022.

p. 1-4

DOI : 10.1145/3487553.3524237

Introducing the HIPE 2022 Shared Task: Named Entity Recognition and Linking in Multilingual Historical Documents

M. Ehrmann; M. Romanello; A. Doucet; S. Clematide 

We present the HIPE-2022 shared task on named entity processing in multilingual historical documents. Following the success of the first CLEF-HIPE-2020 evaluation lab, this edition confronts systems with the challenges of dealing with more languages, learning domain-specific entities, and adapting to diverse annotation tag sets. HIPE-2022 is part of the ongoing efforts of the natural language processing and digital humanities communities to adapt and develop appropriate technologies to efficiently retrieve and explore information from historical texts. On such material, however, named entity processing techniques face the challenges of domain heterogeneity, input noisiness, dynamics of language, and lack of resources. In this context, the main objective of the evaluation lab is to gain new insights into the transferability of named entity processing approaches across languages, time periods, document types, and annotation tag sets.

Advances in Information Retrieval

2022-04-05

44th European Conference on IR Research, ECIR 2022, Stavanger, Norway, April 10-14, 2022.

p. 347-354

DOI : 10.1007/978-3-030-99739-7_44

Catch Me If You Can Designing a Disobedient Object to Protest Against GSM Surveillance

F. Goguey 

In this article, I discuss the process of designing an object to protest against a specific surveillance device: the IMSI catcher, a controversial object used to monitor GSM networks. Being widely used in protests, I develop a tactical approach based on obfuscation to be adopted collectively to counteract IMSI catchers. In this case study, (1) I present how can remaking an IMSI catcher allow to re-appropriate the technology and create a basis for designing a disobedient object; (2) I introduce some examples of tactics to defeat surveillance based on obfuscation and the potential of inflatables; (3) I conceptualize a possible design of an object to defeat IMSI catchers and show the types of interactions it might generate in protests.

Extended Abstracts Of The 2021 Chi Conference On Human Factors In Computing Systems (Chi’21)

2021-01-01

CHI Conference on Human Factors in Computing Systems, ELECTR NETWORK, May 08-13, 2021.

DOI : 10.1145/3411763.3450363

CLEF-HIPE-2020 Shared Task Named Entity Datasets

M. Ehrmann; M. Romanello; S. Clematide; A. Flückiger 

CLEF-HIPE-2020 (Identifying Historical People, Places and other Entities) is a evaluation campaign on named entity processing on historical newspapers in French, German and English, which was organized in the context of the impresso project and run as a CLEF 2020 Evaluation Lab. Data consists of manually annotated historical newspapers in French, German and English.

2020

HIPE-2022 Shared Task Named Entity Datasets

M. Ehrmann; M. Romanello; A. Doucet; S. Clematide 

HIPE-2022 datasets used for the HIPE 2022 shared task on named entity recognition and classification (NERC) and entity linking (EL) in multilingual historical documents. HIPE-2022 datasets are based on six primary datasets assembled and prepared for the shared task. Primary datasets are composed of historical newspapers and classic commentaries covering ca. 200 years, feature several languages and different entity tag sets and annotation schemes. They originate from several European cultural heritage projects, from HIPE organizers’ previous research project, and from the previous HIPE-2020 campaign. Some are already published, others are released for the first time for HIPE-2022. The HIPE-2022 shared task assembles and prepares these primary datasets in HIPE-2022 release(s), which correspond to a single package composed of neatly structured and homogeneously formatted files.

2022

Explorer la presse numérisée : le projet Impresso

M. Ehrmann 

« Impresso – Media Monitoring of the Past » est un projet de recherche interdisciplinaire dans lequel une équipe d’historiens, de linguistes informaticiens et de designers collabore à la mise en données d’un corpus d’archives de presse numérisées. Les principaux objectifs du projet sont d’améliorer les outils d’extraction d’information pour les textes historiques, d’indexer sémantiquement des journaux historiques, et d’intégrer les enrichissements obtenus dans les pratiques de recherche des historiens au moyen d’une interface nouvellement développée.

Revue Historique Vaudoise

2021-11-27

Vol. 129/2021 , p. 159-173.

Automatic table detection and classification in large-scale newspaper archives

A. Vernet 

In recent decades, major efforts to digitize historical documents led to the creation of large machine readable corpora, including newspapers, which are waiting to be processed and analyzed. Newspapers are a valuable historical source, notably because of the plurality of subjects and points of view they cover; however their heterogeneity due to their diachronic properties and their visual richness makes them difficult to deal with. Certain recurring elements, such as tables, which are powerful layout objects because of their ability to easily convey a large amount of information through their logical visual arrangement, play a role in the difficulty of processing them. This thesis focuses on automatic table processing in large-scale newspaper archives. Starting from a large corpus of Luxembourgish newspapers annotated with tables, we propose a statistical exploration of this dataset as well as strategies to address its annotation inconsistencies and to automatically bootstrap a training dataset for table classification. We also explore the ability of deep learning methods to detect and semantically classify tables. The performance of image segmentation models are compared in a series of experiments around their ability to learn under challenging conditions, while classifiers based on different combinations of data modalities are evaluated on the task of table classification. Results show that visual models are able to detect tables by learning on an inconsistent ground truth, and that adding further modalities increases classification performance.

2022-02-08

Advisor(s): M. Ehrmann; S. Clematide; F. Kaplan

Method and system for generating a three-dimensional model based on spherical photogrammetry

F. Kaplan; N. Hamel; A. B. Descombes; P-A. Mudry; C. Papon 

A computer-implemented method is proposed for creating a three-dimensional model of the environment. The method comprises the steps of planning (105) a trajectory for a moving system carrying an omnidirectional camera comprising a first image sensor facing in a first direction for capturing first images, and a second image sensor facing in a second, different direction for capturing second images; advancing (107) the moving system along the trajectory; triggering (111) the omnidirectional camera at given time instants depending on the speed of the moving system along the trajectory to capture first and second images; obtaining (117) spherical images by selectively combining the first and second images; and creating (119) the three-dimensional model from the spherical images.

Patent number(s) :

  • EP4165597 (A1)
  • WO2021255495 (A1)

2021

Boosting named entity recognition in domain-specific and low-resource settings

S. Najem-Meyer 

Recent researches in natural language processing have leveraged attention-based models to produce state-of-the-art results in a wide variety of tasks. Using transfer learning, generic models like BERT can be fine-tuned for domain-specific tasks using little annotated data. In the field of digital humanities and classics, bibliographical reference extraction counts among the domain-specific tasks where few annotated datasets have been made available. It therefore remains a highly challenging Named Entity Recognition (NER) problem which has not been addressed by the aforementioned approaches yet. In this study, we try to boost bibliographical reference extraction with various transfer learning strategies. We compare three transformers to a Conditional Random Fields (CRF) developed by Romanello, using both generic and domain-specific pre-training. Experiments show that transformers consistently improve on CRF baselines. However, domain-specific pre-training yields no significant benefits. We discuss and compare these results in light of comparable researches in domain-specific NER.

2022-01-13

p. 21.

Nouveau centre culturel et éducatif de La Chaux-de-Fonds

T. Cristea 

La ville de La Chaux-de-Fonds a été un important moteur économique pour l’industrie manufacturière suisse, avec l’arrivée de nombreuses entreprises horlogères au début du 20e siècle. La population de la ville a atteint son maximum dand les années 1970, en décroissant depuis, alors que la fabrication de produits de précision est devenue obsolète durant les dernières décénies. Par conséquent, bien que les prix des appartements aient baissé, beaucoup d’entre eux restent tout de même vacants, en raison d’un manque de proximité et d’accès au reste des villes francophones. Le projet que je propose vise à créer un point d’intérêt pour la région, à travers un complexe de programmes éducatifs et culturels à proximité de la gare. Le secteur industriel ayant quitté le centre-ville, les terrains appartenant aux Chemins de Fer Fédéraux suisses ont été privatisés à des promoteurs immobiliers. Le projet est situé sur le site des anciens dépôts et hangars de maintenance des opérateurs de fret ferroviaire CFF Cargo, au sud de la ville. Le programme utilise les bâtiments existants afin de conserver l’identité culturelle locale. De plus, deux nouveaux bâtiments accueillent les espaces pédagogiques qui utilisent l’infrastructure de la gare de triage. Le projet vise à créer un nouveau hub pour la ville, qui relie trois points centraux: la gare qui est en cours de rénovation, le parc historique des Crêtets et le Grand-Pont, qui relie les moitiés nord et sud de la ville.

2021

Advisor(s): M. Fröhlich; A. Fröhlich; F. Pardini

Neural networks for semantic segmentation of historical city maps: Cross-cultural performance and the impact of figurative diversity

R. G. Petitpierre 

In this work, we present a new semantic segmentation model for historical city maps that surpasses the state of the art in terms of flexibility and performance. Research in automatic map processing is largely focused on homogeneous corpora or even individual maps, leading to inflexible algorithms. Recently, convolutional neural networks have opened new perspectives for the development of more generic tools. Based on two new maps corpora, the first one centered on Paris and the second one gathering cities from all over the world, we propose a method for operationalizing the figuration based on traditional computer vision algorithms that allows large-scale quantitative analysis. In a second step, we propose a semantic segmentation model based on neural networks and implement several improvements. Finally, we analyze the impact of map figuration on segmentation performance and evaluate future ways to improve the representational flexibility of neural networks. To conclude, we show that these networks are able to semantically segment map data of a very large figurative diversity with efficiency.

2020

Advisor(s): R. Barman; N. Hamel; F. Kaplan

New Techniques for the Digitization of Art Historical Photographic Archives – the Case of the Cini Foundation in Venice

B. L. A. Seguin; L. Costiner; I. Di Lenardo; F. Kaplan 

Numerous libraries and museums hold large art historical photographic collections, numbering millions of images. Because of their non-standard format, these collections pose special challenges for digitization. This paper address these difficulties by proposing new techniques developed for the digitization of the photographic archive of the Cini Foundation. This included the creation of a custom-built circular, rotating scanner. The resulting digital images were then automatically indexed, while artificial intelligence techniques were employed in information extraction. Combined, these tools vastly sped processes which were traditionally undertaken manually, paving the way for new ways of exploring the collections.

Archiving Conference

2018-02-01

Vol. 2018 , num. 1, p. 1-5.

DOI : 10.2352/issn.2168-3204.2018.1.0.2

Aux portes du monde miroir

I. Di Lenardo; F. Kaplan 

The Mirror World is no longer an imaginary device, a mirage in a distant future, it is a reality under construction. In Europe, Asia and on the American continent, large companies and the best universities are working to build the infrastructures, to define their functionalities, to specify their logistics. The Mirror World, in its asymptotic form, presents a quasi-continuous representation of the world in motion, integrating, virtually, all photographic perspectives. It is a new giant computational object, opening the way to new research methods or even probably to a new type of science. The economic and cultural stakes of this third platform are immense. If the Mirror World transforms access to knowledge for new generations, as the Web and Social Networks did in their time, it is our responsibility to understand and, if need be, bend its technological trajectory to make this new platform an environment for the critical knowledge of the past and the creative imagination of the future.

Revue Histoire de l’art : Humanités numériques

2021-06-29

Vol. 87 .

Citation Mining of Humanities Journals: The Progress to Date and the Challenges Ahead

G. Colavizza; M. Romanello 

Even large citation indexes such as the Web of Science, Scopus or Google Scholar cover only a small fraction of the literature in the humanities. This coverage sensibly decreases going backwards in time. Citation mining of humanities publications — defined as an instance of bibliometric data mining and as a means to the end of building comprehensive citation indexes — remains an open problem. In this contribution we discuss the results of two recent projects in this area: Cited Loci and Linked Books. The former focused on the domain of classics, using journal articles in JSTOR as a corpus; the latter considered the historiography on Venice and a novel corpus of journals and monographs. Both projects attempted to mine citations of all kinds — abbreviated and not, to all types of sources, including primary sources — and considered a wide time span (19th to 21st century). We first discuss the current state of research in citation mining of humanities publications. We then present the various steps involved into this process, from corpus selection to data publication, discussing the peculiarities of the humanities. The approaches taken by the two projects are compared, allowing us to highlight disciplinary differences and commonalities, as well as shared challenges between historiography and classics on this respect. The resulting picture portrays humanities citation mining as a field with a great, yet mostly untapped potential, and a few still open challenges. The potential lies in using citations as a means to interconnect digitized collections at a large scale, by making explicit the linking function of bibliographic citations. As for the open challenges, a key issue is the existing need for an integrated metadata infrastructure and an appropriate legal framework to facilitate citation mining in the humanities.

Journal of European Periodical Studies

2019-06-30

Vol. 4 , num. 1, p. 36-53.

DOI : 10.21825/jeps.v4i1.10120

Une approche computationnelle du cadastre napoléonien de Venise

I. Di Lenardo; R. Barman; F. Pardini; F. Kaplan 

At the beginning of the 19th century, the Napoleonic administration introduced a new standardised description system to give an objective account of the form and functions of the city of Venice. The cadastre, deployed on a European scale, was offering for the first time an articulated and precise view of the structure of the city and its activities, through a methodical approach and standardised categories. With the use of digital techniques, based in particular on deep learning, it is now possible to extract from these documents an accurate and dense representation of the city and its inhabitants. By systematically checking the consistency of the extracted information, these techniques also evaluate the precision and systematicity of the surveyors’ work and therefore indirectly qualify the trust to be placed in the extracted information. This article reviews the history of this computational protosystem and describes how digital techniques offer not only systematic documentation, but also extraction perspectives for latent information, as yet uncharted, but implicitly present in this information system of the past.

Humanités numériques

2021-05-01

Vol. 3/2021 , num. 3.

DOI : 10.4000/revuehn.1786

Detecting Text Reuse with Passim

M. Romanello; S. Hengchen 

In this lesson you will learn about text reuse detection – the automatic identification of reused passages in texts – and why you might want to use it in your research. Through a detailed installation guide and two case studies, this lesson will teach you the ropes of Passim, an open source and scalable tool for text reuse detection.

2021-05-16

Historical Newspaper Content Mining: Revisiting the impresso Project’s Challenges in Text and Image Processing, Design and Historical Scholarship

M. Ehrmann; E. Bunout; S. Clematide; M. Düring; A. Fickers et al. 

Long abstract for a presentation at DH2020 (online).

DH2020 Book of Abstracts

2020

Digital Humanities Conference (DH), Ottawa, Canada, July 20-24, 2020.

DOI : 10.5281/zenodo.4641894

The impresso system architecture in a nutshell

M. Romanello; M. Ehrmann; S. Clematide; D. Guido 

This post describes the impresso application architecture and processing in a nutshell. The text was published in October 2020 in issue number 16 of the EuropeanaTech Insights dedicated to digitized newspapers and edited by Gregory Markus and Clemens Neudecker: https://pro.europeana.eu/page/issue-16-newspapers#the-impresso-system-architecture-in-a-nutshell

2020

p. 10.

Dürer tra Norimberga e Venezia, 1506-1507

I. Di Lenardo 

Dürer e il Rinascimento, tra Germania e Italia; 24 Ore Cultura, 2018.

ISBN : 10 : 8866483842

Impresso Named Entity Annotation Guidelines (CLEF-HIPE-2020)

M. Ehrmann; C. Watter; M. Romanello; C. Simon; A. Flückiger 

Impresso annotation guidelines used in the context of corpus annotation for the HIPE shared task (CLEF 2020 Evaluation Lab). CLEF-HIPE-2020 shared task: https://impresso.github.io/CLEF-HIPE-2020/ Impresso project: https://impresso-project.ch

2020

p. 29.

//zenodo.org/deposit/3706857.

CLEF-HIPE-2020 – Shared Task Participation Guidelines

M. Ehrmann; M. Romanello; S. Clematide; A. Flückiger 

This document summarizes instructions for participants to the CLEF-HIPE-2020 shared task. HIPE (Identifying Historical People, Places and other Entities) is a named entity processing evaluation campaign on historical newspapers in French, German and English, organized in the context of the impresso project and run as a CLEF 2020 Evaluation Lab. More information on the website: https://impresso.github.io/CLEF-HIPE-2020/

2020

p. 19.

Survey of digitized newspaper interfaces (dataset and notebooks)

M. Ehrmann; E. Bunout; M. Duering 

This record contains the datasets and jupyter notebooks which support the analysis presented in the paper “Historical Newspaper User Interfaces: A Review”. Please refer to the paper or the github repository for more information (see links below), or do not hesitate to contact us!

2019

Datasets and Models for Historical Newspaper Article Segmentation

R. Barman; M. Ehrmann; S. Clematide; S. Ares Oliveira 

Dataset and models used and produced in the work described in the paper “Combining Visual and Textual Features for Semantic Segmentation of Historical Newspapers”: https://infoscience.epfl.ch/record/282863?ln=en

2021

Combining Visual and Textual Features for Semantic Segmentation of Historical Newspapers

R. Barman; M. Ehrmann; S. Clematide; S. Ares Oliveira; F. Kaplan 

The massive amounts of digitized historical documents acquired over the last decades naturally lend themselves to automatic processing and exploration. Research work seeking to automatically process facsimiles and extract information thereby are multiplying with, as a first essential step, document layout analysis. Although the identification and categorization of segments of interest in document images have seen significant progress over the last years thanks to deep learning techniques, many challenges remain with, among others, the use of more fine-grained segmentation typologies and the consideration of complex, heterogeneous documents such as historical newspapers. Besides, most approaches consider visual features only, ignoring textual signal. We introduce a multimodal neural model for the semantic segmentation of historical newspapers that directly combines visual features at pixel level with text embedding maps derived from, potentially noisy, OCR output. Based on a series of experiments on diachronic Swiss and Luxembourgish newspapers, we investigate the predictive power of visual and textual features and their capacity to generalize across time and sources. Results show consistent improvement of multimodal models in comparison to a strong visual baseline, as well as better robustness to the wide variety of our material.

Journal of Data Mining & Digital Humanities

2021

Vol. 2021 , num. Special Issue on HistoInformatics: Computational Approaches to History, p. 1-26.

DOI : 10.5281/zenodo.4065271

I sistemi di immagini nell’archivio digitale di Vico Magistretti

F. Kaplan; I. di Lenardo 

La messa a disposizione in linea dell’archivio digitalizzato di Vico Magistretti che raggruppa decine di migliaia di disegni preparatori, disegni tecnici e fotografie prodotte tra 1946 e il 2006, apre la strada a un grande rinnovamento delle ricerche sul designer e architetto italiano. L’apertura di questo archivio così speciale ci invita a immaginare diverse prospettive che possono essere considerate per esplorare, visualizzare e studiare un tale insieme di documenti.

Narrare con l’Archivio. Forum internazionale, 19 novembre 2020, Milan, Italy, Novembre 19, 2020.

Extracting And Aligning Artist Names in Digitized Art Historical Archives

B. Seguin; L. Costiner; I. di Lenardo; F. Kaplan 

The largest collections of art historical images are not found online but are safeguarded by museums and other cultural institutions in photographic libraries. These collections can encompass millions of reproductions of paintings, drawings, engravings and sculptures. The 14 largest institutions hold together an estimated 31 million images (Pharos). Manual digitization and extraction of image metadata undertaken over the years has succeeded in placing less than 100,000 of these items for search online. Given the sheer size of the corpus, it is pressing to devise new ways for the automatic digitization of these art historical archives and the extraction of their descriptive information (metadata which can contain artist names, image titles, and holding collection). This paper focuses on the crucial pre-processing steps that permit the extraction of information directly from scans of a digitized photo collection. Taking the photographic library of the Giorgio Cini Foundation in Venice as a case study, this paper presents a technical pipeline which can be employed in the automatic digitization and information extraction of large collections of art historical images. In particular, it details the automatic extraction and alignment of artist names to known databases, which opens a window into a collection whose contents are unknown. Numbering nearing one million images, the art history library of the Cini Foundation was established in the mid-twentieth century to collect and record the history of Venetian art. The current study examines the corpus of the 330’000+ digitized images.

Book of Abstracts of Digital Humanities Conference 2018 Puentes-Bridges

2018

Digital Humanities Conference 2018 Puentes-Bridges, Mexico City, June 26-29, 2018.

Swiss in motion : Analyser et visualiser les rythmes quotidiens. Une première approche à partir du dispositif Time-Machine.

Y. Dubois; N. Hamel; G. Drevon; P. Rérat; V. Kaufmann et al. 

Au cours des 50 dernières années, les développements technologiques dans le domaine des transports et des télécommunications ont contribué à reconfigurer les comportements spatio-temporels (Kaufmann, 2008). Les individus bénéficient ainsi d’un large univers de choix en matière de modes de transport et de lieux accessibles pour réaliser leurs activités. Cette configuration influence en particulier les comportements de mobilité quotidienne qui tendent à se complexifier tant dans leur dimension spatiale que temporelle impliquant ainsi l’émergence de rythmes quotidiens intenses et complexes (Drevon, Gwiazdzinski, & Klein, 2017; Gutiérrez & García-Palomares, 2007). Des recherches récentes menées sur la Suisse (Drevon, Gumy, & Kaufmann, 2020) suggèrent que les rythmes quotidiens sont marqués par une importante diversité en matière de configuration spatio-temporelle et de densité d’activités (Drevon, Gumy, Kaufmann, & Hausser, 2019). La part des rythmes quotidiens qui correspond à la figure du métro-boulot-dodo est finalement relativement modeste. Cette diversité de rythmes quotidiens se déploie entre d’un côté des comportements très complexes et d’autres peu complexes qui se matérialisent à différentes échelles spatiales. Force est de constater que les outils d’analyse actuels en sciences sociales et en socio-économie des transports peinent encore à rendre compte des formes complexes de rythmes quotidiens au niveau individuel et territorial. Face à cet enjeu épistémologique et méthodologique, la communication propose une approche innovante et interdisciplinaire qui associe la Sociologie, la Géographie et les Sciences computationnelle. Il s’agit concrètement de proposer un outil de géo-visualisation des rythmes quotidiens au échelles individuelles et territoriales à partir des comportements spatio-temporels des habitants de la Suisse. L’objectif de cette démarche est de mettre en perspective les différentiels d’intensité en matière d’activité entre les situations sociales et les territoires. Les analyses s’appuient sur l’enquête Microrecensement Mobilité et Transports (MRMT) réalisée tous les 5 ans à l’échelle nationale par l’Office fédéral de la statistique et l’Office fédéral du développement territorial réalisé en 2015. Cette enquête est composée d’un échantillon 57 090 personnes qui ont été interrogées sur l’ensemble de leurs déplacements effectués la veille du jour d’enquête (protocole d’enquête CATI). La visualisation est réalisée à partir du dispositif Time-Machine (Kaplan, 2013; di Lenardo & Kaplan, 2015) qui permet de modéliser un environnement virtuel en 4D (Figure 1 : https://youtu.be/41-klvXLCqM) et de simuler le déploiement des activités et des déplacements quotidiens. Les premières simulations révèlent des régimes rythmiques contrastés à l’échelle individuelle qui se différencient selon les allures, la fréquence d’actions, l’échelle spatiale et la position sociale. Au niveau territoriales les visualisations laissent apparaitre des différentiels importants dans l’intensité d’usage du territoire par les individus et des spécificités spatiales constitutives des activités qui y sont réalisées. Ces premières visualisations permettent d’abord de révéler des inégalités sociales (genre, classe) face aux injonctions à l’activité (Viry, Ravalet, & Kaufmann, 2015; Drevon, 2019; Drevon & Kaufmann, 2020). Elle permettent aussi de rediscuter des modalités de catégorisation des territoires (Rérat, 2008; Schuler et al., 2007) à partir d’une approche dynamique qui témoigne de la réalité des activités temporaires remettant par ailleurs en perspective les principes de l’écologie urbaine factorielle (Pruvot & Weber-Klein, 1984) et en renforçant également l’intérêt de l’économie présentielle (Lejoux, 2009).

Swiss Mobility Conference, Lausanne, October 29-30, 2020.

A digital reconstruction of the 1630–1631 large plague outbreak in Venice

G. Lazzari; G. Colavizza; F. Bortoluzzi; D. Drago; A. Erboso et al. 

The plague, an infectious disease caused by the bacterium Yersinia pestis, is widely considered to be responsible for the most devastating and deadly pandemics in human history. Starting with the infamous Black Death, plague outbreaks are estimated to have killed around 100 million people over multiple centuries, with local mortality rates as high as 60%. However, detailed pictures of the disease dynamics of these outbreaks centuries ago remain scarce, mainly due to the lack of high-quality historical data in digital form. Here, we present an analysis of the 1630–1631 plague outbreak in the city of Venice, using newly collected daily death records. We identify the presence of a two-peak pattern, for which we present two possible explanations based on computational models of disease dynamics. Systematically digitized historical records like the ones presented here promise to enrich our understanding of historical phenomena of enduring importance. This work contributes to the recently renewed interdisciplinary foray into the epidemiological and societal impact of pre-modern epidemics.

Scientific Reports

2020-10-20

Vol. 10 , num. 1, p. 17849.

DOI : 10.1038/s41598-020-74775-6

Extended Overview of CLEF HIPE 2020: Named Entity Processing on Historical Newspapers

M. Ehrmann; M. Romanello; A. Flückiger; S. Clematide 

This paper presents an extended overview of the first edition of HIPE (Identifying Historical People, Places and other Entities), a pioneering shared task dedicated to the evaluation of named entity processing on historical newspapers in French, German and English. Since its introduction some twenty years ago, named entity (NE) processing has become an essential component of virtually any text mining application and has undergone major changes. Recently, two main trends characterise its developments: the adoption of deep learning architectures and the consideration of textual material originating from historical and cultural heritage collections. While the former opens up new opportunities, the latter introduces new challenges with heterogeneous, historical and noisy inputs. In this context, the objective of HIPE, run as part of the CLEF 2020 conference, is threefold: strengthening the robustness of existing approaches on non-standard inputs, enabling performance comparison of NE processing on historical texts, and, in the long run, fostering efficient semantic indexing of historical documents. Tasks, corpora, and results of 13 participating teams are presented. Compared to the condensed overview [31], this paper includes further details about data generation and statistics, additional information on participating systems, and the presentation of complementary results.

CLEF 2020 Working Notes. Conference and Labs of the Evaluation Forum

2020-10-21

11th Conference and Labs of the Evaluation Forum (CLEF 2020), [Online event], 22-25 September, 2020.

DOI : 10.5281/zenodo.4117566

Language Resources for Historical Newspapers: the Impresso Collection

M. Ehrmann; M. Romanello; S. Clematide; P. B. Ströbel; R. Barman 

Following decades of massive digitization, an unprecedented amount of historical document facsimiles can now be retrieved and accessed via cultural heritage online portals. If this represents a huge step forward in terms of preservation and accessibility, the next fundamental challenge– and real promise of digitization– is to exploit the contents of these digital assets, and therefore to adapt and develop appropriate language technologies to search and retrieve information from this `Big Data of the Past’. Yet, the application of text processing tools on historical documents in general, and historical newspapers in particular, poses new challenges, and crucially requires appropriate language resources. In this context, this paper presents a collection of historical newspaper data sets composed of text and image resources, curated and published within the context of the `impresso – Media Monitoring of the Past’ project. With corpora, benchmarks, semantic annotations and language models in French, German and Luxembourgish covering ca. 200 years, the objective of the impresso resource collection is to contribute to historical language resources, and thereby strengthen the robustness of approaches to non-standard inputs and foster efficient processing of historical documents.

Proceedings of the 12th Language Resources and Evaluation Conference

2020-05-11

12th International Conference on Language Resources and Evaluation (LREC), Marseille, France, May 11-16 2020.

p. 958-968

DOI : 10.5281/zenodo.4641902

Overview of CLEF HIPE 2020: Named Entity Recognition and Linking on Historical Newspapers

M. Ehrmann; M. Romanello; A. Flückiger; S. Clematide 

This paper presents an overview of the first edition of HIPE (Identifying Historical People, Places and other Entities), a pioneering shared task dedicated to the evaluation of named entity processing on historical newspapers in French, German and English. Since its introduction some twenty years ago, named entity (NE) processing has become an essential component of virtually any text mining application and has undergone major changes. Recently, two main trends characterise its developments: the adoption of deep learning architectures and the consideration of textual material originating from historical and cultural heritage collections. While the former opens up new opportunities, the latter introduces new challenges with heterogeneous, historical and noisy inputs. In this context, the objective of HIPE, run as part of the CLEF 2020 conference, is threefold: strengthening the robustness of existing approaches on non-standard inputs, enabling performance comparison of NE processing on historical texts, and, in the long run, fostering efficient semantic indexing of historical documents. Tasks, corpora, and results of 13 participating teams are presented.

Experimental IR meets multilinguality, multimodality, and interaction. 11th International Conference of the CLEF Association, CLEF 2020, Thessaloniki, Greece, September 22–25, 2020, Proceedings

2020-09-15

11th International Conference of the CLEF Association – CLEF 2020, Thessaloniki, Greece, September 22–25, 2020.

p. 288–310

DOI : 10.1007/978-3-030-58219-7_21

Repopulating Paris: massive extraction of 4 Million addresses from city directories between 1839 and 1922

I. di Lenardo; R. Barman; A. B. Descombes; F. Kaplan 

In 1839, in Paris, the Maison Didot bought the Bottin company. Sébastien Bottin trained as a statistician was the initiator of a high impact yearly publication, called “Almanachs” containing the listing of residents, businesses and institutions, arranged geographically, alphabetically and by activity typologies (Fig. 1). These regular publications encountered a great success. In 1820, the Parisian Bottin Almanach contained more than 50 000 addresses and until the end of the 20th century the word “Bottin” was the colloquial term to designate a city directory in France. The publication of the “Didot-Bottin” continued at an annual rhythm, mapping the evolution of the active population of Paris and other cities in France.The relevance of automatically mining city directories for historical reconstruction has already been argued by several authors (e.g Osborne, N., Hamilton, G. and Macdonald, S. 2014 or Berenbaum, D. et al. (2016). This article reports on the extraction and analysis of the data contained in “Didot-Bottin” covering the period 1839-1922 for Paris, digitized by the Bibliotheque nationale de France. We process more than 27 500 pages to create a database of 4,2 Million entries linking addresses, person mention and activities.

Abstracts and Posters from the Digital Humanities 2019 conference

2019-07-02

Digital Humanities Conference 2019 (DH2019), Utrecht , the Netherlands, July 9-12, 2019.

DOI : 10.34894/MNF5VQ

The Advent of the 4D Mirror World

F. Kaplan; I. di Lenardo 

The 4D Mirror World is considered to be the next planetary-scale information platform. This commentary gives an overview of the history of the converging trends that have progressively shaped this concept. It retraces how large-scale photographic surveys served to build the first 3D models of buildings, cities, and territories, how these models got shaped into physical and virtual globes, and how eventually the temporal dimension was introduced as an additional way for navigating not only through space but also through time. The underlying assumption of the early large-scale photographic campaign was that image archives had deeper depths of latent knowledge still to be mined. The technology that currently permits the advent of the 4D World through new articulations of dense photographic material combining aerial imagery, historic photo archives, huge video libraries, and crowd-sourced photo documentation precisely exploits this latent potential. Through the automatic recognition of “homologous points,” the photographic material gets connected in time and space, enabling the geometrical computation of hypothetical reconstructions accounting for a perpetually evolving reality. The 4D world emerges as a series of sparse spatiotemporal zones that are progressively connected, forming a denser fabric of representations. On this 4D skeleton, information of cadastral maps, BIM data, or any other specific layers of a geographical information system can be easily articulated. Most of our future planning activities will use it as a way not only to have smooth access to the past but also to plan collectively shared scenarios for the future.

Urban Planning

2020-06-30

Vol. 5 , num. 2, p. 307.

DOI : 10.17645/up.v5i2.3133

Rhythmanalysis of Urban Events: Empirical Elements from the Montreux Jazz Festival

G. Drevon; L. Pattaroni; L. Delley; F. Jacot-Descombes; N. Hamel 

This article proposes an original approach to urban events mapping. At the theoretical level, the article is based on rhythmanalysis and recent research on urban rhythms. It contrasts with previous research by departing from everyday rhythms to tackle the specific rhythms of urban events. Drawing on this theoretical framework, the article proposes to analyse the rhythms of the Montreux Jazz Festival. The article proposes two main types of rhythmic scales, linked with the historical development of the Festival and its annual performance. The methodology is based on a mixed method of data collection and an original analysis framework. The analysis of the historical rhythm is carried out based on the analysis of the festival archives and interviews with experts. The analysis uses the Time Machine visualisation device that reveals three processes of urban resonance: the spread, which shows how the festival is integrated into the existing urban fabric; the openness, which shows accessibility; and the grip, which seeks to evaluate the urban sphere of influence of the event. These different visualisations are enriched by the addition of other data, including ticket scanning and commercial transactions that show the alternance between high and low-intensity periods. These allowed us to not only confirm the impact of programming on flows, but also the effects of the wider organisation of the leisure system. The results of the analysis show that the intertwining of the two rhythmic scales produces a hyper-place that resonates both internationally and locally.

Urban Planning

2020-06-30

Vol. 5 , num. 2, p. 280-295.

DOI : 10.17645/up.v5i2.2940

Building a Mirror World for Venice

F. Kaplan; I. di Lenardo 

Between 2012 and 2019, ‘TheVeniceTime Machine Project’ developed a new methodology for modelling the past, present, and future of a city. This methodology is based on two pillars: (a) the vast digitisation and processing of the selected city’s historical records, (b) the digitisation of the city itself, another vast undertaking. The combination of these two processes has the potential to create a new kind of historical information system organised around a diachronic digital twin of a city.

The Aura in the Age of Digital Materiality : Rethinking Preservation in the Shadow of an Uncertain Future; Milan: SilvanaEditoriale, 2020.

ISBN : 9788836645480

Introducing the CLEF 2020 HIPE Shared Task: Named Entity Recognition and Linking on Historical Newspapers

M. Ehrmann; M. Romanello; S. Bircher; S. Clematide 

Since its introduction some twenty years ago, named entity (NE) processing has become an essential component of virtually any text mining application and has undergone major changes. Recently, two main trends characterise its developments: the adoption of deep learning architectures and the consideration of textual material originating from historical and cultural heritage collections. While the former opens up new opportunities, the latter introduces new challenges with heterogeneous, historical and noisy inputs. If NE processing tools are increasingly being used in the context of historical documents, performance values are below the ones on contemporary data and are hardly comparable. In this context, this paper introduces the CLEF 2020 Evaluation Lab HIPE (Identifying Historical People, Places and other Entities) on named entity recognition and linking on diachronic historical newspaper material in French, German and English. Our objective is threefold: strengthening the robustness of existing approaches on non-standard inputs, enabling performance comparison of NE processing on historical texts, and, in the long run, fostering efficient semantic indexing of historical documents in order to support scholarship on digital cultural heritage collections.

Advances in Information Retrieval. ECIR 2020

2020-04-08

ECIR 2020 : 42nd European Conference on Information Retrieval, Lisbon, Portugal, April 14-17, 2020.

p. 524-532

DOI : 10.1007/978-3-030-45442-5_68

Experiments in digital publishing: creating a digital compendium

M. Romanello 

This chapter introduces the readers and users to the goals of the digitally provided index of the compendium Structures of Epic Poetry and the methods used for it. It also expands on the broader applicability of digital methods in view of electronic publishing, and to the problems involved. The chapter focuses on two aspects of my work for the compendium, where digital tools played a central role: the creation of the index locorum and the development of a digital compendium to the printed volumes.

Structures of Epic Poetry; Berlin, Boston: De Gruyter, 2019-12-15.

ISBN : 9783110492590

DOI : 10.1515/9783110492590-074

Linked Books: un indice citazionale per la storia di Venezia

G. Colavizza; M. Romanello; D. Grandin; M. C. Mataloni; A. Giuliano 

We present the outcomes of the Linked Books project, resulting in a prototype citation index interlinking the Italian national library catalog (Opac SBN) with the information system of the State Archive of Venice and international authority records or “metaengines” such as VIAF.org and Europeana. Our prototype includes 3.850.581 citations extracted from a corpus of 2.475 volumes, of which 1.905 monographs, and 552 journal volumes, or 5.496 articles therein. The corpus is focused on the history of Venice. The Linked Books project allowed us to explore the feasibility and desirability of a citation index for the humanities, and to face and solve technical challenges including: the selection of a thematically representative corpus from bibliographic resources and expertise, the digitization of these materials within the bounds of copyright, the automatic extraction of citations and the development of public search interfaces.

DigItalia

2019-06-01

Vol. 14 , num. 1, p. 132-146.

The Past, Present and Future of Digital Scholarship with Newspaper Collections

M. Ridge; G. Colavizza; L. Brake; M. Ehrmann; J-P. Moreux et al. 

Historical newspapers are of interest to many humanities scholars as sources of information and language closely tied to a particular time, social context and place. Digitised newspapers are also of interest to many data-driven researchers who seek large bodies of text on which they can try new methods and tools. Recently, large consortia projects applying data science and computational methods to historical newspapers at scale have emerged, including NewsEye, \textit{impresso}, Oceanic Exchanges and Living with Machines. This multi-paper panel draws on the work of a range of interdisciplinary newspaper-based digital humanities and/or data science projects, alongside ‘provocations’ from two senior scholars who will provide context for current ambitions. As a unique opportunity for stakeholders to engage in dialogue, for the DH2019 community to ask their own questions of newspaper-based projects, and for researchers to map methodological similarities between projects, it aims to have a significant impact on the field.

DH 2019 Book of Abstracts

2019-07-09

DIgital Humanities Conference, Utrecht, July 2019.

Historical newspaper semantic segmentation using visual and textual features

R. Barman 

Mass digitization and the opening of digital libraries gave access to a huge amount of historical newspapers. In order to bring structure into these documents, current techniques generally proceed in two distinct steps. First, they segment the digitized images into generic articles and then classify the text of the articles into finer-grained categories. Unfortunately, by losing the link between layout and text, these two steps are not able to account for the fact that newspaper content items have distinctive visual features. This project proposes two main novelties. Firstly, it introduces the idea of merging the segmentation and classification steps, resulting in a fine- grained semantic segmentation of newspapers images. Secondly, it proposes to use textual features under the form of embeddings maps at segmentation step. The semantic segmentation with four categories (feuilleton, weather forecast, obituary, and stock exchange table) is done using a fully convolutional neural network and reaches a mIoU of 79.3%. The introduction of embeddings maps improves the overall performances by 3% and the generalization across time and newspapers by 8% and 12%, respectively. This shows a strong potential to consider the semantic aspect in the segmentation of newspapers and to use textual features to improve generalization.

2019-06-21

Advisor(s): M. Ehrmann; S. Ares Oliveira; S. Clematide

Beyond Keyword Search: Semantic Indexing and Exploration of Large Collections of Historical Newspapers

M. Ehrmann 

For long held on library and archive shelving, historical newspapers are currently undergoing mass digitization and millions of facsimiles, along with their machine-readable content acquired via Optical Character Recognition, are becoming accessible via a variety of online portals. If this represents a major step forward in terms of preservation of and access to documents, much remains to be done in order to provide an extensive and sophisticated access to the content of these digital resources. We believe that the promise of newspaper digitization lies in their semantic indexation, closely tied with the development of co-designed interfaces that accommodate text analysis research tools and their usage by humanities scholars. How to go beyond keyword search? How to explore complex and vast amounts of data? Based on the on-going project ‘impresso – Media Monitoring of the Past’, in this talk I will present our interdisciplinary approach and share hands-on experience in going from facsimiles to enhanced search and visualization capacities supporting historical research.

Digital Humanitites in the Nordic Countries, Copenhagen, Denmark, March 2019.

Transforming scholarship in the archives through handwritten text recognition Transkribus as a case study

G. Muehlberger; L. Seaward; M. Terras; S. Ares Oliveira; V. Bosch et al. 

Purpose An overview of the current use of handwritten text recognition (HTR) on archival manuscript material, as provided by the EU H2020 funded Transkribus platform. It explains HTR, demonstrates Transkribus, gives examples of use cases, highlights the affect HTR may have on scholarship, and evidences this turning point of the advanced use of digitised heritage content. The paper aims to discuss these issues. Design/methodology/approach This paper adopts a case study approach, using the development and delivery of the one openly available HTR platform for manuscript material. Findings Transkribus has demonstrated that HTR is now a useable technology that can be employed in conjunction with mass digitisation to generate accurate transcripts of archival material. Use cases are demonstrated, and a cooperative model is suggested as a way to ensure sustainability and scaling of the platform. However, funding and resourcing issues are identified. Research limitations/implications – The paper presents results from projects: further user studies could be undertaken involving interviews, surveys, etc. Practical implications – Only HTR provided via Transkribus is covered: however, this is the only publicly available platform for HTR on individual collections of historical documents at time of writing and it represents the current state-of-the-art in this field. Social implications The increased access to information contained within historical texts has the potential to be transformational for both institutions and individuals. Originality/value This is the first published overview of how HTR is used by a wide archival studies community, reporting and showcasing current application of handwriting technology in the cultural heritage sector.

Journal Of Documentation

2019-09-09

Vol. 75 , num. 5, p. 954-976.

DOI : 10.1108/JD-07-2018-0114

Index-Driven Digitization and Indexation of Historical Archives

G. Colavizza; M. Ehrmann; F. Bortoluzzi 

The promise of digitization of historical archives lies in their indexation at the level of contents. Unfortunately, this kind of indexation does not scale, if done manually. In this article we present a method to bootstrap the deployment of a content-based information system for digitized historical archives, relying on historical indexing tools. Commonly prepared to search within homogeneous records when the archive was still current, such indexes were as widespread as they were disconnected, that is to say situated in the very records they were meant to index. We first present a conceptual model to describe and manipulate historical indexing tools. We then introduce a methodological framework for their use in order to guide digitization campaigns and index digitized historical records. Finally, we exemplify the approach with a case study on the indexation system of the X Savi alle Decime in Rialto, a Venetian magistracy in charge for the exaction—and related record keeping—of a tax on real estate in early modern Venice.

Frontiers in Digital Humanities

2019-03-11

Vol. 6 , num. 1-16, p. 1-16.

DOI : 10.3389/fdigh.2019.00004

Historical Newspaper User Interfaces: A Review

M. Ehrmann; E. Bunout; M. Düring 

After decades of large-scale digitization, many historical newspaper collections are just one click away via online portals developed and supported by various public or private stakeholders. Initially offering access to full text search and facsimiles visualization only, historic newspaper user interfaces are increasingly integrating advanced exploration features based on the application of text mining tools to digitized sources. As gateways to enriched material, such interfaces are however not neutral and play a fundamental role in how users perceive historical sources, understand potential biases of upstream processes and benefit from the opportunities of datafication. What features can be found in current interfaces, and to what degree do interfaces adopt novel technologies? This paper presents a survey of interfaces for digitized historical newspapers with the aim of mapping the current state of the art and identifying recent trends with regard to content presentation, enrichment and user interaction. We devised 6 interface assessment criteria and reviewed twenty-four interfaces based on ca. 140 predefined features.

[Proceedings of the 85th IFLA General Conference and Assembly]

2019-09-02

85th IFLA General Conference and Assembly, Athens, Greece, 24-30 August 2019.

p. 1-24

DOI : 10.5281/zenodo.3404155

Self-Recognition in Data Visualization: How Individuals See Themselves in Visual Representations

D. Rodighiero; L. Cellard 

This article explores how readers recognize their personal identities represented through data visualizations. The recognition is investigated starting from three definitions captured by the philosopher Paul Ricoeur: the identification with the visualization, the recognition of someone in the visualization, and the mutual recognition that happens between readers. Whereas these notions were initially applied to study the role of the book reader, two further concepts complete the shift to data visualization: the digital identity stays for the present-day passport of human actions and the promise is the intimate reflection that projects readers towards their own future. This article reflects on the delicate meaning of digital identity and the way of representing it according to this structure: From Personal Identity to Media is a historical introduction to self-recognition, Data Visualization for Representing Identities moves the focus to visual representation, and The Course of Recognition breaks the self-recognition in through the five concepts above just before the Conclusion.

EspacesTemps.net

2019-08-08

DOI : 10.26151/espacestemps.net-wztp-cc46

Named Entity Processing for Historical Texts

M. Ehrmann; M. Romanello; S. Clematide 

Recognition and identification of real-world entities is at the core of virtually any text mining application. As a matter of fact, referential units such as names of persons, locations and organizations underlie the semantics of texts and guide their interpretation. Around since the seminal Message Understanding Conference (MUC) evaluation cycle in the 1990s, named entity-related tasks have undergone major evolutions until now, from entity recognition and classification to entity disambiguation and linking. Recently, NE processing has been called upon to contribute to the domain of digital humanities, where massive digitization of historical documents is producing huge amounts of texts. De facto, NE processing tools are increasingly being used in the context of historical documents. Research activities in this domain target texts of different nature (e.g., publications by cultural institutions, state-related documents, genealogical data, historical newspapers) and different tasks (NE recognition and classification, entity linking, or both). Experiments involve different time periods (from 16th to 20th c.), focus on different domains, and use different typologies. This great variety demonstrates how many and varied the needs – and the challenges – are, but makes performance comparison difficult, not to say impossible. The objective of this tutorial is to provide the participants with essential knowledge with respect to a) NE processing in general and in DH, and b) how to apply NE recognition approaches.

2019-07-17

A deep learning approach to Cadastral Computing

S. Ares Oliveira; I. di Lenardo; B. Tourenc; F. Kaplan 

This article presents a fully automatic pipeline to transform the Napoleonic Cadastres into an information system. The cadastres established during the first years of the 19th century cover a large part of Europe. For many cities they give one of the first geometrical surveys, linking precise parcels with identification numbers. These identification numbers points to registers where the names of the proprietary. As the Napoleonic cadastres include millions of parcels , it therefore offers a detailed snapshot of large part of Europe’s population at the beginning of the 19th century. As many kinds of computation can be done on such a large object, we use the neologism “cadastral computing” to refer to the operations performed on such datasets. This approach is the first fully automatic pipeline to transform the Napoleonic Cadastres into an information system.

2019-07-11

Digital Humanities Conference, Utrecht, Netherlands, July 8-12, 2019.

Spherical Network Visualizations

D. Rodighiero 

Data visualization is a recent domain that founds his roots in the eighties, but its history dates back to more ancient times in other representations such as diagrams, drawings, or maps. In particular, data visualization is hard to take advantage of the heritage offered by cartography, a discipline with established theoretical and mathematical theories. Over more than two thousand years, cartography stimulated a discussion between critical thinking and spatial projections, with a keen interest for orientation and decision-making. This article picks up the cartographic technique of globe projection to apply it to network visualization. If the primary interest of globe projection in cartography is the flattened representation of the earth, in data visualization the focus is moved to the space continuity. In world maps usually the left and right sides are connected, the gaze can follow a trajectory that goes on the opposite side; while in data visualization the space of drawing is framed in all directions. Network visualizations rely on a relational logic applied on a limited flatland. We guess that drawing networks in a non-continuous space is a habit that can be changed. The hypothesis that drawing networks are less reductive on a spherical surface is supported by an example of travel distances between cities, which is mapped in two and three dimensions. Lastly, we argue that adopting a spherical projection eliminates the bias given by centrality in favor of spatial measure based on density.

Challenges in Design: Imagination, Aesthetics, and New Technology, Porto, Portugal, 26 June 2019.

“Een Italische Keucken van Dirick de Vriese” The Commercialisation of the Artistic Identity between Venice and the ‘North’

I. di Lenardo 

The essay examines in particular the subjects of Kitchens, domestic interiors with various characterizations, and the Carnival, another subject matter peculiar of the lagoon, which was exported and spread rapidly to the North. The presence in the collections of wealthy Antwerp merchants of these subjects, codified as ‘Venetian’, even though produced by artists of Flemish origin, is an important element defining the perception of this production identified as a ‘Venetianity’ and carefully managed by foreign artists.

Artibus Et Historiae

2018-01-01

num. 78, p. 83-93.

The Daily Design of the Quantified Self

D. Rodighiero; A. Rigal 

This article argues how the digital traces that are collected day by day can be used to reshape and improve our personal self. If in the past, we were used to collecting data through diaries, today the task of producing inscriptions is delegated to technological devices such as mobile phones. The article, then, discusses how technology can shape athletes using the Sky Team as an example of personal and collective design.

Swiss Informatics Digital Magazine (SIDM)

2019

DOI : 10.5281/zenodo.3463586

Negentropic linguistic evolution: A comparison of seven languages

V. Buntinx; F. Kaplan 

The relationship between the entropy of language and its complexity has been the subject of much speculation – some seeing the increase of linguistic entropy as a sign of linguistic complexification or interpreting entropy drop as a marker of greater regularity. Some evolutionary explanations, like the learning bottleneck hypothesis, argues that communication systems having more regular structures tend to have evolutionary advantages over more complex structures. Other structural effects of communication networks, like globalization of exchanges or algorithmic mediation, have been hypothesized to have a regularization effect on language. Longer-term studies are now possible thanks to the arrival of large-scale diachronic corpora, like newspaper archives or digitized libraries. However, simple analyses of such datasets are prone to misinterpretations due to significant variations of corpus size over the years and the indirect effect this can have on various measures of language change and linguistic complexity. In particular, it is important not to misinterpret the arrival of new words as an increase in complexity as this variation is intrinsical, as is the variation of corpus size. This paper is an attempt to conduct an unbiased diachronic study of linguistic complexity over seven different languages using the Google Books corpus. The paper uses a simple entropy measure on a closed, but nevertheless large, subset of words, called kernels. The kernel contains only the words that are present without interruption for the whole length of the study. This excludes all the words that arrived or disappeared during the period. We argue that this method is robust towards variations of corpus size and permits to study change in complexity despite possible (and in the case of Google Books unknown) change in the composition of the corpus. Indeed, the evolution observed on the seven different languages shows rather different patterns that are not directly correlated with the evolution of the size of the respective corpora. The rest of the paper presents the methods followed, the results obtained and the next steps we envision.

2018

Digital Humanities 2018, Mexico City, Mexico, June 26-29, 2018.

Layout Analysis on Newspaper Archives

V. Buntinx; F. Kaplan; A. Xanthos 

The study of newspaper layout evolution through historical corpora has been addressed by diverse qualitative and quantitative methods in the past few years. The recent availability of large corpora of newspapers is now making the quantitative analysis of layout evolution ever more popular. This research investigates a method for the automatic detection of layout evolution on scanned images with a factorial analysis approach. The notion of eigenpages is defined by analogy with eigenfaces used in face recognition processes. The corpus of scanned newspapers that was used contains 4 million press articles, covering about 200 years of archives. This method can automatically detect layout changes of a given newspaper over time, rebuilding a part of its past publishing strategy and retracing major changes in its history in terms of layout. Besides these advantages, it also makes it possible to compare several newspapers at the same time and therefore to compare the layout changes of multiple newspapers based only on scans of their issues.

2017

Digital Humanities 2017, Montreal, Canada, August 8-11, 2017.

Frederic Kaplan Isabella di Lenardo

F. Kaplan; I. di Lenardo 

Apollo-The International Art Magazine

2019-01-01

Vol. 189 , num. 671, p. 21-21.

The Hermeneutic Circle of Data Visualization: the Case Study of the Affinity Map

D. Rodighiero; A. Romele 

In this article, we show how postphenomenology can be used to analyze a visual method that reveals the hidden dynamics that exist between individuals within large organizations. We make use of the Affinity Map to expand the classic postphenomenology that privileges a ‘linear’ understanding of technological mediations introducing the notions of ‘iterativity’ and ‘collectivity.’ In the first section, both classic and more recent descriptions of human-technology-world relations are discussed to transcendentally approach the discipline of data visualization. In the second section, the Affinity Map case study is used to stress three elements: 1) the collection of data and the design process; 2) the visual grammar of the data visualization, and 3) the process of self-recognition for the map ‘reader.’ In the third section, we introduce the hermeneutic circle of data visualization. Finally, in the concluding section, we put forth how the Affinity Map might be seen as the material encounter between postphenomenology, actor-network theory (ANT), and hermeneutics, through ethical and political multistability.

Techné: Research in Philosophy and Technology

2020

Vol. 24 , num. 3, p. 357-375.

DOI : 10.5840/techne202081126

Traduire les données en images

D. Rodighiero 

Journée d’étude : Imagination, imaginaire et images des (big) data, Université de Lille, France, 24 January 2019.

dhSegment: A generic deep-learning approach for document segmentation

S. A. Oliveira; B. Seguin; F. Kaplan 

In recent years there have been multiple successful attempts tackling document processing problems separately by designing task specific hand-tuned strategies. We argue that the diversity of historical document processing tasks prohibits to solve them one at a time and shows a need for designing generic approaches in order to handle the variability of historical series. In this paper, we address multiple tasks simultaneously such as page extraction, baseline extraction, layout analysis or multiple typologies of illustrations and photograph extraction. We propose an open-source implementation of a CNN-based pixel-wise predictor coupled with task dependent post-processing blocks. We show that a single CNN-architecture can be used across tasks with competitive results. Moreover most of the task-specific post-precessing steps can be decomposed in a small number of simple and standard reusable operations, adding to the flexibility of our approach.

Proceedings 2018 16Th International Conference On Frontiers In Handwriting Recognition (Icfhr)

2018-01-01

16th International Conference on Frontiers in Handwriting Recognition (ICFHR), Niagara Falls, NY, Aug 05-08, 2018.

p. 7-12

DOI : 10.1109/ICFHR-2018.2018.00011

Traduire les données en images

D. Rodighiero 

Séminaire d’écritures numériques et éditorialisation, CNAM Paris et Université de Montreal, January 17th 2019.

dhSegment : A generic deep-learning approach for document segmentation

S. Ares Oliveira; B. L. A. Seguin; F. Kaplan 

The 16th International Conference on Frontiers in Handwriting Recognition, Niagara Falls, USA, 5-8 August 2018.

Translating Data into Images

D. Rodighiero 

Séminaire du médialab, Sciences Po, Paris, France, January 15th 2019.

Imaging, Vision, and Pattern Recognition

M. Zaied; I. Bouchrika; A. Kumar; F. Slimane; R. Ejbali 

Applied Computational Intelligence And Soft Computing

2018-01-01

p. 1070183.

DOI : 10.1155/2018/1070183

Deep Learning for Logic Optimization Algorithms

W. Haaswijk; E. Collins; B. Seguin; M. Soeken; F. Kaplan et al. 

The slowing down of Moore’s law and the emergence of new technologies puts an increasing pressure on the field of EDA. There is a constant need to improve optimization algorithms. However, finding and implementing such algorithms is a difficult task, especially with the novel logic primitives and potentially unconventional requirements of emerging technologies. In this paper, we cast logic optimization as a deterministic Markov decision process (MDP). We then take advantage of recent advances in deep reinforcement learning to build a system that learns how to navigate this process. Our design has a number of desirable properties. It is autonomous because it learns automatically and does not require human intervention. It generalizes to large functions after training on small examples. Additionally, it intrinsically supports both single-and multioutput functions, without the need to handle special cases. Finally, it is generic because the same algorithm can be used to achieve different optimization objectives, e. g., size and depth.

2018 Ieee International Symposium On Circuits And Systems (Iscas)

2018-01-01

IEEE International Symposium on Circuits and Systems (ISCAS), Florence, ITALY, May 27-30, 2018.

DOI : 10.1109/ISCAS.2018.8351885

Informatica per Umanisti: da Venezia al mondo intero attraverso l’Europa

D. Rodighiero 

In un momento di apertura del mondo scientifico verso un pubblico più ampio, questa conferenza vuole essere una facile introduzione alle digital humanities. L’argomento del conferenza è infatti l’informatica per umanisti, un nuovo ambito di ricerca che arricchisce le discipline umanistiche attraverso l’uso di nuove tecnologie. La mia esperienza personale sarà il filo conduttore di questa introduzione e la conferenza sarà l’occasione per parlare dei progetti ai quali ho contribuito nel corso degli ultimi cinque anni. Da Parigi a Venezia, da Losanna a Boston, fare ricerca vuol dire fare esperienze in tutto il mondo. Parlerò di Bruno Latour e dei suoi modi d’esistenza, di Frédéric Kaplan e della sua macchina del tempo, di Franco Moretti e della sua lettura a distanza, e di Marilyne Andersen e della sua cartografia delle affinità, tutte persone che ho avuto il piacere di incontrare e hanno arricchito il mio percorso accademico. Attraverso un racconto visuale fatto di immagini e video, vi spiegherò come le Digital Humanities possono rendere archivi, musei e biblioteche luoghi più interessanti per tutti.

Conferenza per la Società Dante Alighieri, University of Bern, Switzerland, December 10, 2018.

The Value of Concepts in the Shared Design Practice

D. Rodighiero 

Gilles Deleuze was used to say that philosophers are creators of concepts, extracted by a continuous flux of thinking (Deleuze 1980). These concepts, during a period of intense exchange between disciplines in the 20th century, have not been limited to philosophy as they were employed in Computer Science and Design differently. On one hand, Computer Science uses concepts to stabilize technology language and design architectures (Ciborra 2004); on the other hand, Design makes use of concepts as a creative method in the design process (MoMA 1972). This presentation aims to bring together a common ground in order to establish a shared view that might be useful for the creation of design objects (visualizations, books, and web sites) within the project. Biography – Ciborra, Claudio. 2004. The Labyrinths of Information: Challenging the Wisdom of Systems. Oxford University Press. – Deleuze, Gilles. 1980. “Dernière Année, Vincennes.” Les Courses De Gilles Deleuze. https://www.webdeleuze.com/textes/48. – Museum of Modern Art (MoMA). 1972. Italy: the New Domestic Landscape Achievements and Problems of Italian Design. Edited by Emilio Ambasz. Greenwich, CT: New York Graphic Society. www.moma.org/calendar/exhibitions/1783.

Visio et cognitio. Representations of Knowledge, from Medieval to Digital, Porto, Portugal, November 26–27, 2018.

Translating Data into Images: The Design Process for the Creation of Visualizations

D. Rodighiero 

The design process is a series of endeavors aimed to solve a problem, which is extensively defined as a need, a task, a situation, etc. Visualizations are objects that result from a design process that solves a problem of unreadable data. They are technical mediators that transform data in graphics through a precise authorship that defines their social and political context. One way to understand visualizations is to analyze the process of design: this approach, applied to a case study called Affinity Map, unveils the complexity of an object and reveals the reasons of its creation.

Séminaire du LaDHUL: faire des SHS avec le numérique, University of Lausanne (UNIL), Switzerland, November 21, 2018.

Making large art historical photo archives searchable

B. L. A. Seguin / Director(s) : F. Kaplan; I. di Lenardo 

In recent years, museums, archives and other cultural institutions have initiated important programs to digitize their collections. Millions of artefacts (paintings, engravings, drawings, ancient photographs) are now represented in digital photographic format. Furthermore, through progress in standardization, a growing portion of these images are now available online, in an easily accessible manner. This thesis studies how such large-scale art history collection can be made searchable using new deep learning approaches for processing and comparing images. It takes as a case study the processing of the photo archive of the Foundation Giorgio Cini, where more than 300’000 images have been digitized. We demonstrate how a generic processing pipeline can reliably extract the visual and textual content of scanned images, opening up ways to efficiently digitize large photo-collections. Then, by leveraging an annotated graph of visual connections, a metric is learnt that allows clustering and searching through artwork reproductions independently of their medium, effectively solving a difficult problem of cross-domain image search. Finally, the thesis studies how a complex Web Interface allows users to perform different searches based on this metric. We also evaluate the process by which users can annotate elements of interest during their navigation to be added to the database, allowing the system to be trained further and give better results. By documenting a complete approach on how to go from a physical photo-archive to a state-of-the-art navigation system, this thesis paves the way for a global search engine across the world’s photo archives.

Lausanne: EPFL

2018

p. 169.

DOI : 10.5075/epfl-thesis-8857

Hexagons, Satellites and Semantic Background

D. Rodighiero 

The presentation is focused on a visual method that allows for a hexagonal arrangement in network visualization. Hexagonal tilling is a way to enrich the betweenness of nodes in order to enrich the information that a network visualization can convey. What is usually employed as a background is used to show node context and semantic information. This visual method wants to bring a reflection about the visual representation of networks, which needs further developments and ideas.

Micro Meso Macro, École normale supérieure de Lyon, France, November 15–16, 2018.

Affinity Map

D. Rodighiero; O. Maitre 

This is the final presentation the Affinity Map, a data visualization at the disposal of ENAC members to visualize their own scientific community.

ENAC General Assembly, EPFL, Switzerland, November 8, 2018.

Introduction to Data Visualization

D. Rodighiero 

This would be an introduction to the domain of data visualization. The presentation core is composed of a series of examples organized in three time-based sections: classic visualizations (18th and 19th centuries), modern visualizations (20th century), and contemporary visualizations (21st century). This presentation is not intended to be exhaustive at a historical level, the intent is rather to introduce the subjects of discussion that make these examples interesting, according to a temporal sequence.

Kit de survie en milieu numérique pour l’étudiant en SHS, Institut d’histoire de l’art (INHA), Paris, October 3, 2018.

The Intellectual Organisation of History

G. Colavizza / Director(s) : F. Kaplan; M. Franceschet 

A tradition of scholarship discusses the characteristics of different areas of knowledge, in particular after modern academia compartmentalized them into disciplines. The academic approach is often put to question: are there two or more cultures? Is an ever-increasing specialization the only way to cope with information abundance or are holistic approaches helpful too? What is happening with the digital turn? If these questions are well studied for the sciences, our understanding of how the humanities might differ in their own respect is far less advanced. In particular, modern academia might foster specific patterns of specialization in the humanities. Eventually, the recent rise in the application of digital methods to research, known as the digital humanities, might be introducing structural adaptations through the development of shared research technologies and the advent of organizational practices such as the laboratory. It therefore seems timely and urgent to map the intellectual organization of the humanities. This investigation depends on few traits such as the level of codification, the degree of agreement among scholars, the level of coordination of their efforts. These characteristics can be studied by measuring their influence on the outcomes of scientific communication. In particular, this thesis focuses on history as a discipline using bibliometric methods. In order to explore history in its complexity, an approach to create collaborative citation indexes in the humanities is proposed, resulting in a new dataset comprising monographs, journal articles and citations to primary sources. Historians’ publications were found to organize thematically and chronologically, sharing a limited set of core sources across small communities. Core sources act in two ways with respect to the intellectual organization: locally, by adding connectivity within communities, or globally as weak ties across communities. Over recent decades, fragmentation is on the rise in the intellectual networks of historians, and a comparison across a variety of specialisms from the human, natural and mathematical sciences revealed the fragility of such networks across the axes of citation and textual similarities. Humanists organize into more, smaller and scattered topical communities than scientists. A characterisation of history is eventually proposed. Historians produce new historiographical knowledge with a focus on evidence or interpretation. The former aims at providing the community with an agreed-upon factual resource. Interpretive work is instead mainly focused on creating novel perspectives. A second axe refers to two modes of exploration of new ideas: in-breadth, where novelty relates to adding new, previously unknown pieces to the mosaic, or in-depth, if novelty then happens by improving on previous results. All combinations possible, historians tend to focus on in-breadth interpretations, with the immediate consequence that growth accentuates intellectual fragmentation in the absence of further consolidating factors such as theory or technologies. Research on evidence might have a different impact by potentially scaling-up in the digital space, and in so doing influence the modes of interpretation in turn. This process is not dissimilar to the gradual rise in importance of research technologies and collaborative competition in the mathematical and natural sciences. This is perhaps the promise of the digital humanities.

Lausanne: EPFL

2018

p. 185.

DOI : 10.5075/epfl-thesis-8537

Comparing human and machine performances in transcribing 18th century handwritten Venetian script

S. Ares Oliveira 

Automatic transcription of handwritten texts has made important progress in the recent years. This increase in performance, essentially due to new architectures combining convolutional neural networks with recurrent neutral networks, opens new avenues for searching in large databases of archival and library records. This paper reports on our recent progress in making million digitized Venetian documents searchable, focusing on a first subset of 18th century fiscal documents from the Venetian State Archives. For this study, about 23’000 image segments containing 55’000 Venetian names of persons and places were manually transcribed by archivists, trained to read such kind of handwritten script. This annotated dataset was used to train and test a deep learning architecture with a performance level (about 10% character error rate) that is satisfactory for search use cases. This paper compares this level of reading performance with the reading capabilities of Italian-speaking transcribers. More than 8500 new human transcriptions were produced, confirming that the amateur transcribers were not as good as the expert. However, on average, the machine outperforms the amateur transcribers in this transcription tasks.

Digital Humanities Conference, Mexico City, Mexico, June 25-29.

Comparing human and machine performances in transcribing 18th century handwritten Venetian script

S. Ares Oliveira; F. Kaplan 

Automatic transcription of handwritten texts has made important progress in the recent years. This increase in performance, essentially due to new architectures combining convolutional neural networks with recurrent neutral networks, opens new avenues for searching in large databases of archival and library records. This paper reports on our recent progress in making million digitized Venetian documents searchable, focusing on a first subset of 18th century fiscal documents from the Venetian State Archives. For this study, about 23’000 image segments containing 55’000 Venetian names of persons and places were manually transcribed by archivists, trained to read such kind of handwritten script. This annotated dataset was used to train and test a deep learning architecture with a performance level (about 10% character error rate) that is satisfactory for search use cases. This paper compares this level of reading performance with the reading capabilities of Italian-speaking transcribers. More than 8500 new human transcriptions were produced, confirming that the amateur transcribers were not as good as the expert. However, on average, the machine outperforms the amateur transcribers in this transcription tasks.

2018-07-26

Digital Humanities Conference, Mexico City, Mexico, June 24-29, 2018.

The Hermenutic Circle of Data Visualization

D. Rodighiero 

According to Don Ihde (1990, 80–97), hermeneutic relations are a specific kind of technologically mediated I-world relations in which the technology must be “read” and “interpreted” in order to access the world. More recently, Peter-Paul Verbeek (2010, 145) introduced the notion of composite relations, featuring a double intentionality performed by human and non-human actors. The aim of this article is to expand these two concepts by reflecting on data visualization. In particular, we will deal with a visualization called Affinity Map, which displays the scholars of EPFL (École Polytechnique Fédérale de Lausanne) arranged according to a metric based on the collaboration. Two specificities characterize, for us, such configuration. 1) First, the subjectivization of what hermeneutics have called the “world of the text” because scholars are both readers and contributors. In other terms, hermeneutic relations are here technologically-mediated self-relations in a strong sense. 2) Second, the collectivization of the hermeneutic circle in each of its steps: subjects, data, designers, actualizations, visualizations, and readers. In this respect, the Affinity Map might be seen as a concrete encounter between postphenomenology, whose main focus is on the types of I-technology-world relations, and actor-network theory, which has always-already been concerned with collectives.

Human-Technology Relations, University of Twente, 11-13 July 2018.

Detecting Text Reuse in Newspapers Data with Passim

M. Romanello 

Hacking the News, co-located with DHNordic 2018, Helsinki, Finland, 5-6 March 2018.

The Scholar Index: Towards a Collaborative Citation Index for the Arts and Humanities

G. Colavizza; M. Romanello; M. Babetto; V. Barbay; L. Bolli et al. 

Mexico City, 26-29 June 2018.

Using Networks to Visualize Publications

D. Rodighiero 

Retrieval systems are often shaped as lists organized in pages. However, the majority of users look at the first page ignoring the other ones. This presentation concerns an alterna- tive way to present the results of a query using network visualizations.
 The presentation includes a case study that concerns a school of management. Its whole publications are arranged in a network visualization according to their lexical proximity, based on a technique called Term Frequency – Inverse Document Frequency (TF-IDF). These terms are further used to fill the space between the network nodes, creating a sort of semantic background. The case study shows pros and cons of such visual representa- tion through practical examples of term extraction and visualization interaction.

EUROLIB General Assembly, Joint Research Centre of European Commission – Ispra (VA), Italy, 30 May – 1 June 2018.

Mapping Affinities in Academic Organizations

D. Rodighiero; F. Kaplan; B. Beaude 

Scholarly affinities are one of the most fundamental hidden dynamics that drive scientific development. Some affinities are actual, and consequently can be measured through classical academic metrics such as co-authoring. Other affinities are potential, and therefore do not leave visible traces in information systems; for instance, some peers may share interests without actually knowing it. This article illustrates the development of a map of affinities for academic collectives, designed to be relevant to three audiences: the management, the scholars themselves, and the external public. Our case study involves the School of Architecture, Civil and Environmental Engineering of EPFL, hereinafter ENAC. The school consists of around 1,000 scholars, 70 laboratories, and 3 institutes. The actual affinities are modeled using the data available from the information systems reporting publications, teaching, and advising scholars, whereas the potential affinities are addressed through text mining of the publications. The major challenge for designing such a map is to represent the multi-dimensionality and multi-scale nature of the information. The affinities are not limited to the computation of heterogeneous sources of information; they also apply at different scales. The map, thus, shows local affinities inside a given laboratory, as well as global affinities among laboratories. This article presents a graphical grammar to represent affinities. Its effectiveness is illustrated by two actualizations of the design proposal: an interactive online system in which the map can be parameterized, and a large-scale carpet of 250 square meters. In both cases, we discuss how the materiality influences the representation of data, in particular the way key questions could be appropriately addressed considering the three target audiences: the insights gained by the management and their consequences in terms of governance, the understanding of the scholars’ own positioning in the academic group in order to foster opportunities for new collaborations and, eventually, the interpretation of the structure from a general public to evaluate the relevance of the tool for external communication.

Frontiers in Research Metrics and Analytics

2018-02-19

Vol. 3 , p. 4.

DOI : 10.3389/frma.2018.00004

Printing Walkable Visualizations

D. Rodighiero 

This article concerns a specific actor in the actualization process, the media. The conventional media for visualizations is the computer screen, a visual device that supports the practices of design and reading. However, visualizations also appear in other ways, for example as posters, articles, books, or projections. This article focuses, in particular, on a pretty unusual medium called floor or walkable visualization.

Transimage 2018

2018

5th Biennial Research Transdisciplinary Imaging Conference, Edunburgh, UK, April 18-20, 2018.

p. 58-73

DOI : 10.6084/m9.figshare.6104693.v2

Machine Vision Algorithms on Cadaster Plans

S. Ares Oliveira; I. di Lenardo; F. Kaplan 

Cadaster plans are cornerstones for reconstructing dense representations of the history of the city. They provide information about the city urban shape, enabling to reconstruct footprints of most important urban components as well as information about the urban population and city functions. However, as some of these handwritten documents are more than 200 years old, the establishment of processing pipeline for interpreting them remains extremely challenging. We present the first implementation of a fully automated process capable of segmenting and interpreting Napoleonic Cadaster Maps of the Veneto Region dating from the beginning of the 19th century. Our system extracts the geometry of each of the drawn parcels, classifies, reads and interprets the handwritten labels.

2017

Premiere Annual Conference of the International Alliance of Digital Humanities Organizations (DH 2017), Montreal, Canada, August 8-11, 2017.

Automatic information extraction from historical collections: the case of the 1808 venetian cadaster

S. Ares Oliveira 

The presentation reports on the on-going work to automatically process heterogeneous historical documents. After a quick overview of the general processing pipeline, a few examples are more comprehensively described. The recent progress in making large collections of digitised documents searchable is also be presented through the results of the automatic transcription of named entities in 18th century venetian fiscal documents. Finally, the case of the 1808 venetian cadaster is used to illustrate the general approach and the results of the processing of the whole 1808 cadaster are presented.

CREATE Salon ‘Heterogeneous archives’, Amsterdam, Netherlands, 05.04.2018.

Machine Vision algorithms on cadaster plans

S. Ares Oliveira 

Cadaster plans are cornerstones for reconstructing dense representations of the history of the city. However, as some of these handwritten documents are more than 200 years old, the establishment of processing pipeline for interpreting them remains extremely challenging. The talk will present the implementation of an automated process capable of segmenting and interpreting Napoleonic Cadaster Maps of the Veneto Region dating from the beginning of the 19th century. Using a deep learning approach, the system extracts the geometry of each of the drawn parcels, reads and interprets the handwritten labels. This efficient and automated process opens new perspectives to reconstitute the past.

PlatformDH, Antwerp, Belgium, 04.12.2017.

From Documents to Structured Data: First Milestones of the Garzoni Project

M. Ehrmann; G. Colavizza; O. Topalov; R. Cella; D. Drago et al. 

Led by an interdisciplinary consortium, the Garzoni project undertakes the study of apprenticeship, work and society in early modern Venice by focusing on a specific archival source, namely the Accordi dei Garzoni from the Venetian State Archives. The project revolves around two main phases with, in the first instance, the design and the development of tools to extract and render information contained in the documents (according to Semantic Web standards) and, as a second step, the examination of such information. This paper outlines the main progress and achievements during the first year of the project.

DHCommons

2016

num. 2.

Mapping affinities: visualizing academic practice through collaboration

D. Rodighiero / Director(s) : F. Kaplan; B. Beaude 

Academic affinities are one of the most fundamental hidden dynamics that drive scientific development. Some affinities are actual, and consequently can be measured through classical academic metrics such as co-authoring. Other affinities are potential, and therefore do not have visible traces in information systems; for instance, some peers may share scientific interests without actually knowing it. This thesis illustrates the development of a map of affinities for scientific collectives, which is intended to be relevant to three audiences: the management, the scholars themselves, and the external public. Our case study involves the School of Architecture, Civil and Environmental Engineering of EPFL, which consists of three institutes, seventy laboratories, and around one thousand employees. The actual affinities are modeled using the data available from the academic systems reporting publications, teaching, and advising, whereas the potential affinities are addressed through text mining of the documents registered in the information system. The major challenge for designing such a map is to represent the multi-dimensionality and multi-scale nature of the information. The affinities are not limited to the computation of heterogeneous sources of information, they also apply at different scales. Therefore, the map shows local affinities inside a given laboratory, as well as global affinities among laboratories. The thesis presents a graphical grammar to represent affinities. This graphical system is actualized in several embodiments, among which a large-scale carpet of 250 square meters and an interactive online system in which the map can be parameterized. In both cases, we discuss how the actualization influences the representation of data, in particular the way key questions could be appropriately addressed considering the three target audiences: the insights gained by the management and the relative decisions, the understanding of the researchers’ own positioning in the academic collective that might reveal opportunities for new synergies, and eventually the interpretation of the structure from an external standpoint that suggesting the relevance of the tool for communication.

EPFL

2018

p. 179.

DOI : 10.5075/epfl-thesis-8242

Littérature Potentielle 2.0

C. B. Daniel de Roulet 

Le Persil

2017

p. 24-25.

Machine Vision Algorithms on Cadaster Plans

S. Ares Oliveira; I. di Lenardo 

Cadaster plans are cornerstones for reconstructing dense representations of the history of the city. They provide information about the city urban shape, enabling to reconstruct footprints of most important urban components as well as information about the urban population and city functions. However, as some of these handwritten documents are more than 200 years old, the establishment of processing pipeline for interpreting them remains extremely challenging. We present the first implementation of a fully automated process capable of segmenting and interpreting Napoleonic Cadaster Maps of the Veneto Region dating from the beginning of the 19th century. Our system extracts the geometry of each of the drawn parcels, classifies, reads and interprets the handwritten labels.

Premiere Annual Conference of the International Alliance of Digital Humanities Organizations (DH 2017), Montreal, Canada, August 8-11, 2017.

Printing materials and technologies in the 15th−17th century book production: an undervalued research field

F. Albertin; E. Balliana; G. Pizzol; G. Colavizza; E. Zendri et al. 

We present a systematic non-invasive investigation of a large corpus of early printed books, exploiting multiple techniques. This work is part of a broader project — Argeia — aiming to study early printing technologies, their evolution and, potentially, the identification of physical/chemical fingerprints of different manufactures and/or printing dates. We analyzed sixty volumes, part of the important collection of the Ateneo Veneto in Venice (Italy), printed between the 15th and the 17th centuries in the main European manufacturing centers. We present here the results of the imaging analysis of the entire corpus and the X-Ray Fluorescence (XRF) investigation performed, focusing on the XRF data and their statistical treatment using a combination of Principal Component Analysis (PCA) and Logistic Regression. Thanks to the broad XRF investigation — more than 200 data points — and to the multidisciplinary approach, we were able to discriminate the provenances of the paper — in particular for the German and Venetian volumes — and we potentially identified a chemical fingerprint of Venetian papers.

Microchemical Journal

2018

Vol. 138 , p. 147-153.

DOI : 10.1016/j.microc.2017.12.010

Mapping large organizations

D. Rodighiero 

Today organizations are more than ever complex systems. They are large, ramified, distributed, and intertwined so that their organic structure seems like a tangle of activities. Day by day individuals contribute to keep these structures alive with their work, thoughts, and personalities, and as a result organizations rely on these daily practices. Contemporary sociology aims to untangle the network of practices through the analysis of digital traces that are distributed in desktop computers, smart phones, Wi-Fi and GPS signals, payment systems, badges, information systems, etc. Digital traces are all the information that individuals leave behind them during daily activi-ties. The challenge is therefore to recompose a faithful image of an organization us-ing the data that its members left behind in various forms.

IMD Business School, Lausanne, Switzerland, December 13, 2017.

Using Linked Open Data to Bootstrap a Knowledge Base of Classical Texts

M. Romanello; M. Pasin 

We describe a domain-specific knowledge base aimed at supporting the extraction of bibliographic references in the domain of Classics. In particular, we deal with references to canonical works of the Greek and Latin literature by providing a model that represents key aspects of this domain such as names and abbreviations of authors, the canonical structure of classical works, and links to related web resources. Finally, we show how the availability of linked data in the emerging Graph of Ancient World Data has helped bootstrapping the creation of our knowledge base.

Proceedings of the Second Workshop on Humanities in the Semantic Web (WHiSe II) co-located with 16th International Semantic Web Conference (ISWC 2017)

2017

Second Workshop on Humanities in the Semantic Web (WHiSe II) co-located with 16th International Semantic Web Conference (ISWC 2017), Vienna, Austria, October 22, 2017.

p. 3-14

Mapping the Analogous City

D. Rodighiero 

Re-descriptions: Aldo Rossi’s Architectural Composition. Chair of Architecture and the Public Building, Friday 1 December 2017, Room U 10:45 – 12:30. On the event of the publication in the new issue of OverHolland of the English translation of Ezio Bonfanti’s seminal essay ‘Elementi e costruzione: Note sull’architettura di Aldo Rossi’ (1970), the Chair of Public Building organises a lecture event to examine Rossi‘s compositional procedures, their relationship with his architectural theory and, more in general, their significance for architectural design research. In addition to the introduction of Bonfanti’s reading of Rossi’s work by Henk Engel and Stefano Milani, invited speaker Dario Rodighero (École polytechnique fédérale de Lausanne) will present the research ‘The Analogous City. The Map’ which examines ‘piece by piece’ Aldo Rossi‘s collage executed in 1976 in cooperation with Eraldo Consolascio, Bruno Reichlin and Fabio Reinhart. Programme: 10:45 – 11:00 Henk Engel (TUD), Introduction to OverHolland nos. 18-19; 11:00 – 11:30 Stefano Milani (TUD), Re-descriptions: On Ezio Bonfanti’s essay ‘Elements and constructions. Notes on Aldo Rossi’s architecture’; 11:30 – 12:15 Dario Rodighero (EPFL), Mapping the Analogous City; 12:15 – 12:45 Discussion.

Re-descriptions: Aldo Rossi’s Architectural Composition, Delft University of Technology, Netherlands, December 1, 2017.

Analyse multi-échelle de n-grammes sur 200 années d’archives de presse

V. Buntinx / Director(s) : F. Kaplan; A. Xanthos 

The recent availability of large corpora of digitized texts over several centuries opens the way to new forms of studies on the evolution of languages. In this thesis, we study a corpus of 4 million press articles covering a period of 200 years. The thesis tries to measure the evolution of written French on this period at the level of words and expressions, but also in a more global way by attempting to define integrated measures of linguistic evolution. The methodological choice is to introduce a minimum of linguistic hypotheses in this study by developing new measures around the simple notion of n-gram, a sequence of n consecutive words. The thesis explores on this basis the potential of already known concepts as temporal frequency profiles and their diachronic correlations, but also introduces new abstractions such as the notion of resilient linguistic kernel or the decomposition of profiles into solidified expressions according to simple statistical models. Through the use of distributed computational techniques, it develops methods to test the relevance of these concepts on a large amount of textual data and thus allows to propose a virtual observatory of the diachronic evolutions associated with a given corpus. On this basis, the thesis explores more precisely the multi-scale dimension of linguistic phenomena by considering how standardized measures evolve when applied to increasingly long n-grams. The discrete and continuous scale from the isolated entities (n=1) to the increasingly complex and structured expressions (1 < n < 10) offers a transversal axis of study to the classical differentiations that ordinarily structure linguistics: syntax, semantics, pragmatics, and so on. The thesis explores the quantitative and qualitative diversity of phenomena at these different scales of language and develops a novel approach by proposing multi-scale measurements and formalizations, with the aim of characterizing more fundamental structural aspects of the studied phenomena.

Lausanne: EPFL

2017

p. 362.

DOI : 10.5075/epfl-thesis-8180

Annotated References in the Historiography on Venice: 19th–21st centuries

G. Colavizza; M. Romanello 

We publish a dataset containing more than 40’000 manually annotated references from a broad corpus of books and journal articles on the history of Venice. References were considered from both reference lists and footnotes, include primary and secondary sources, in full or abbreviated form. The dataset comprises references from publications from the 19th to the 21st century. References were collected from a newly digitized corpus and manually annotated in all their constituent parts. The dataset is stored on a GitHub repository, persisted in Zenodo, and it is accompanied with code to train parsers in order to extract references from other publications. Two trained Conditional Random Fields models are provided along with their evaluation, in order to act as a baseline for a parsing shared task. No comparable public dataset exists to support the task of reference parsing in the humanities. The dataset is of interest to all working on the domain of reference parsing and citation extraction in the humanities.

Journal of Open Humanities Data

2017

Vol. 3 , p. 2.

DOI : 10.5334/johd.9

TimeRank: A dynamic approach to rate scholars using citations

M. Franceschet; G. Colavizza 

Rating has become a common practice of modern science. No rating system can be considered as final, but instead several approaches can be taken, which magnify different aspects of the fabric of science. We introduce an approach for rating scholars which uses citations in a dynamic fashion, allocating ratings by considering the relative position of two authors at the time of the citation among them. Our main goal is to introduce the notion of citation timing as a complement to the usual suspects of popularity and prestige. We aim to produce a rating able to account for a variety of interesting phenomena, such as positioning raising stars on a more even footing with established researchers. We apply our method on the bibliometrics community using data from the Web of Science from 2000 to 2016, showing how the dynamic method is more effective than alternatives in this respect.

Journal of Informetrics

2017

Vol. 11 , num. 4, p. 1128-1141.

DOI : 10.1016/j.joi.2017.09.003

Characterizing in-text citations in scientific articles: A large-scale analysis

K. W. Boyack; N. J. van Eck; G. Colavizza; L. Waltman 

We report characteristics of in-text citations in over five million full text articles from two large databases – the PubMed Central Open Access subset and Elsevier journals – as functions of time, textual progression, and scientific field. The purpose of this study is to understand the characteristics of in-text citations in a detailed way prior to pursuing other studies focused on answering more substantive research questions. As such, we have analyzed in-text citations in several ways and report many findings here. Perhaps most significantly, we find that there are large field-level differences that are reflected in position within the text, citation interval (or reference age), and citation counts of references. In general, the fields of Biomedical and Health Sciences, Life and Earth Sciences, and Physical Sciences and Engineering have similar reference distributions, although they vary in their specifics. The two remaining fields, Mathematics and Computer Science and Social Science and Humanities, have different reference distributions from the other three fields and between themselves. We also show that in all fields the numbers of sentences, references, and in-text mentions per article have increased over time, and that there are field-level and temporal differences in the numbers of in-text mentions per reference. A final finding is that references mentioned only once tend to be much more highly cited than those mentioned multiple times.

Journal of Informetrics

2018

Vol. 12 , num. 1, p. 59-73.

DOI : 10.1016/j.joi.2017.11.005

Méduse, vers des visualisations plus complexes que le réseau

A. Rigal; D. Rodighiero 

Dans le cadre de la « Conférence internationale des Humanités Numériques » qui s’est tenue en 2014 à Lausanne, nous avons réalisé une représentation. Celle-ci est une cartographie en réseau des auteurs et des mots-clés de la conférence. Les cartes résultantes sont reproduites sur divers objets : bâche, tapis, ouvrages, posters, tasses. Ces derniers avaient pour fonction de susciter l’intérêt des auteurs et leur identi cation au champ des humanités numériques. La qualité de la car- tographie en réseau est qu’elle exclut peu d’acteurs et dans notre cas peu de participants. De ce fait un grand nombre de participants à la conférence a pu se trouver sur la représentation et par là prendre part au collectif suggéré par les liens de la cartographie. Par ces reproductions, qui ne sont jamais vraiment mécaniques, la représentation a circulé en alimentant des interprétations qui tracent les contours d’un collectif propre à la conférence. Les traces fabriquées par les participants – commentaires de la cartographie, photos, souvenirs, tweets, etc. -, permettent de suivre la trajectoire de la représentation. Par conséquent, savoir si la représentation a réussi revient à enquêter sur l’étendue et la qualité de sa trajectoire entre les épreuves. L’enjeu de cet article est donc d’enquêter sur le design cartographique en tant qu’art du rassemblement, grâce aux outils du design cartographique.

Chronotopies, lecture et écriture des mondes en mouvement; Grenoble: Elya Éditions, 2017.

Linked Lexical Knowledge Bases Foundations and Applications

M. Ehrmann 

Computational Linguistics

2017

Vol. 43 , num. 2, p. 461-463.

DOI : 10.1162/COLI_r_00289

The Closer the Better: Similarity of Publication Pairs at Different Co-Citation Levels

G. Colavizza; K. W. Boyack; N. J. van Eck; L. Waltman 

We investigate the similarities of pairs of articles which are co-cited at the different co- citation levels of the journal, article, section, paragraph, sentence and bracket. Our results indicate that textual similarity, intellectual overlap (shared references), author overlap (shared authors), proximity in publication time all rise monotonically as the co-citation level gets lower (from journal to bracket). While the main gain in similarity happens when moving from journal to article co-citation, all level changes entail an increase in similarity, especially section to paragraph and paragraph to sentence/bracket levels. We compare results from four journals over the years 2010-2015: Cell, the European Journal of Operational Research, Physics Letters B and Research Policy, with consistent general outcomes and some interesting differences. Our findings motivate the use of granular co-citation information as defined by meaningful units of text, with implications for, among others, the elaboration of maps of science and the retrieval of scholarly literature.

Journal of the Association for Information Science and Technology

2018

Vol. 69 , num. 4, p. 600-609.

DOI : 10.1002/asi.23981

The Core Literature of the Historians of Venice

G. Colavizza 

Over the past decades, the humanities have been accumulating a growing body of literature at an increasing pace. How does this impact their traditional organization into disciplines and fields of research therein? This article considers history, by examining a citation network among recent monographs on the history of Venice. The resulting network is almost connected, clusters of monographs are identifiable according to specific disciplinary areas (history, history of architecture, and history of arts) or periods of time (middle ages, early modern, and modern history), and a map of the recent trends in the field is sketched. Most notably a set of highly cited works emerges as the core literature of the historians of Venice. This core literature comprises a mix of primary sources, works of reference, and scholarly monographs and is important in keeping the field connected: monographs usually cite a combination of few core and a variety of less well-cited works. Core primary sources and works of reference never age, while core scholarly monographs are replaced at a very slow rate by new ones. The reliance of new publications on the core literature is slowly rising over time, as the field gets increasingly more varied.

Frontiers in Digital Humanities

2017

Vol. 4 , p. 14.

DOI : 10.3389/fdigh.2017.00014

The structural role of the core literature in history

G. Colavizza 

The intellectual landscapes of the humanities are mostly uncharted territory. Little is known on the ways published research of humanist scholars defines areas of intellectual activity. An open question relates to the structural role of core literature: highly cited sources, naturally playing a disproportionate role in the definition of intellectual landscapes. We introduce four indicators in order to map the structural role played by core sources into connecting different areas of the intellectual landscape of citing publications (i.e. communities in the bibliographic coupling network). All indicators factor out the influence of degree distributions by internalizing a null configuration model. By considering several datasets focused on history, we show that two distinct structural actions are performed by the core literature: a global one, by connecting otherwise separated communities in the landscape, or a local one, by rising connectivity within communities. In our study, the global action is mainly performed by small sets of scholarly monographs, reference works and primary sources, while the rest of the core, and especially most journal articles, acts mostly locally.

Scientometrics

2017

Vol. 113 , num. 3, p. 1787-1809.

DOI : 10.1007/s11192-017-2550-4

Apprenticeship in Early Modern Venice

A. Bellavitis; R. Cella; G. Colavizza 

The desire of the Republican state to regulate the production and sale of food led to the establishment, during the twelfth century, of the Giustizia Vecchia, a magistracy which later developed an authority over the majority of the city’s guilds. The further decision to set a public register of contracts of apprenticeship reflects the ambition of Venetian authorities to regulate and control both vocational training and access to the urban job market, acting as a guarantor between masters and young apprentices. This chapter presents an historical overview of apprenticeship in early modern Venice, examining the development of the city’s legislation on the matter, and analysing a new sample of contracts recorded in the city’s apprenticeship registers during the sixteenth and seventeenth centuries. In particular, we discuss the complex relationship between the general legal framework established by Venetian public authorities and the particular set of norms detailed in guilds statutes. Our analysis reveals that apprenticeship contracts were used to accommodate a variety of situations, including paying for intense training to masked working contracts, while following the general framework provided by state and guild regulations. We then present an in-depth study of apprenticeship contracts from three crafts (goldsmiths, carpenters and printers), chosen for their economic importance, and because they possibly represented different realities in terms of technological specialization, capital (or labour) intensiveness and typology of market. This highlights yet another aspect of apprenticeship in Venice: the influence of guilds. Some guilds such as the Goldsmiths, were more closed to foreigners, favouring Venetians instead. Apprenticeship in early modern Venice is an institution which, despite appearing as highly regulated and formalized, accommodated a variety of realities with remarkable flexibility.

2017

A Simple Set of Rules for Characters and Place Recognition in French Novels

C. Bornet; F. Kaplan 

Frontiers in Digital Humanities

2017

Vol. 4 , num. 6.

DOI : 10.3389/fdigh.2017.00006

Man-machine interface method executed by an interactive device

F. Kaplan; J. Besuchet 

The aim of the present invention is to provide a cost effective solution for a man-machine interface without physical contact with the device to which a command should be given. This aim is achieved by a Man-machine interface method executed by an interactive device, comprising a display, an infra-red camera, an infra-red illumination system, this method executing the following steps : – capturing a first image by the camera with infra-red illumination – capturing a second image by the camera without infra-red illumination – Subtracting the two images to create a difference image – Creating a movement map using a high-pass filter to filter-out static part on a plurality of difference images – computing a barycentre of at least one region of the movement map, – Assigning at least one cursor based on the position of the barycentre movement, – Modifying information on the display based on the position of this cursor..

Patent number(s) :

  • EP2219097 (A1)

2010

Big Data of the Past

F. Kaplan; I. di Lenardo 

Big Data is not a new phenomenon. History is punctuated by regimes of data acceleration, characterized by feelings of information overload accompanied by periods of social transformation and the invention of new technologies. During these moments, private organizations, administrative powers, and sometimes isolated individuals have produced important datasets, organized following a logic that is often subsequently superseded but was at the time, nevertheless, coherent. To be translated into relevant sources of information about our past, these document series need to be redocumented using contemporary paradigms. The intellectual, methodological, and technological challenges linked to this translation process are the central subject of this article.

Frontiers in Digital Humanities

2017

Vol. 4 , num. 12, p. 1-12.

DOI : 10.3389/fdigh.2017.00012

Ancient administrative handwritten documents: virtual x-ray reading

F. Albertin; G. Margaritondo; F. Kaplan 

A method for detecting ink writings in a specimen comprising stacked pages, allowing a page-by-page reading without turning pages The method compris- es steps of taking a set of projection x-ray images for different positions of the specimen with respect to an x-ray source and a detector from an apparatus for taking projection x-ray images; storing the set of projection x-ray images in a suitable computer system; and processing the set of projection x-ray images to tomographically reconstruct the shape of the specimen.

Patent number(s) :

  • WO2015189817 (A3)
  • WO2015189817 (A2)

2016

Narrative Recomposition in the Context of Digital Reading

C. A. M. Bornet / Director(s) : F. Kaplan 

In any creative process, the tools one uses have an immediate influence on the shape of the final artwork. However, while the digital revolution has redefined core values in most creative domains over the last few decades, its impact on literature remains limited. This thesis explores the relevance of digital tools for several aspects of novels writing by focusing on two research questions: Is it possible for an author to edit better novels out of already published ones, given the access to adapted tools? And, will authors change their way of writing when they know how they are being read? This thesis is a multidisciplinary participatory study, actively involving the Swiss novelist Daniel de Roulet, to construct measures, visualizations, and digital tools aimed at leveraging the process of dynamic reordering of narrative material, similar to how one edits a video footage. We developed and tested various text analysis and visualization tools, the results of which were interpreted and used by the author to recompose a family saga out of material he has been writing for twenty-four years. Based on this research, we released Saga+, an online editing, publishing, and reading tool. The platform was handed out to third parties to improve existing writings, making new novels available to the public as a result. While many researchers have studied the structuration of texts either through global statistical features or micro-syntactic analyses, we demonstrate that by allowing visualization and interaction at an intermediary level of organisation, authors can manipulate their own texts in agile ways. By integrating readers’ traces into this newly revealed structure, authors can start to approach the question of optimizing their writing processes in ways that are similar to what is being practiced in other media industries. The introduction of tools for optimal composition opens new avenues for authors, as well as a controversial debate regarding the future of literature.

Lausanne: EPFL

2017

p. 206.

DOI : 10.5075/epfl-thesis-7592

Optimized scripting in Massive Open Online Courses

F. Kaplan; I. di Lenardo 

The Time Machine MOOC, currently under preparation, is designed to provide the necessary knowledge for students to use the editing tool of the Time Machine platform. The first test case of the platform in centered on our current work on the City of Venice and its archives. Small Teaching modules focus on specific skills of increasing difficulty: segmenting a word on a page, transcribing a word from a document series, georeferencing ancient maps using homologous points, disambiguating named entities, redrawing urban structures, finding matching details between paintings and writing scripts that perform automatically some of these tasks. Other skills include actions in the physical world, like scanning pages, books, maps or performing a photogrammetric reconstruction of a sculpture taking a large number of pictures. Eventually, some other modules are dedicated to general historic, linguistic, technical or archival knowledge that constitute prerequisites for mastering specific tasks. A general dependency graph has been designed, specifying in which order the skills can be acquired. The performance of most tasks can be tested using some pre-defined exercises and evaluation metrics, which allows for a precise evaluation of the level of mastery of each student. When the student successfully passes the test related to a skill, he or she gets the credentials to use that specific tool in the platform and starts contributing. However, the teaching options can vary greatly for each skill. Building upon the script concept developed by Dillenbourg and colleagues, we designed each tutorial as a parameterized sequence. A simple gradient descent method is used to progressively optimize the parameters in order to maximize the success rate of the students at the skill tests and therefore seek a form of optimality among the various design choices for the teaching methods. Thus, the more students use the platform, the more efficient teaching scripts become.

Dariah Teach, Université de Lausanne, Switzerland, March 23-24, 2017.

The references of references: a method to enrich humanities library catalogs with citation data

G. Colavizza; M. Romanello; F. Kaplan 

The advent of large-scale citation indexes has greatly impacted the retrieval of scientific information in several domains of research. The humanities have largely remained outside of this shift, despite their increasing reliance on digital means for information seeking. Given that publications in the humanities have a longer than average life-span, mainly due to the importance of monographs for the field, this article proposes to use domain-specific reference monographs to bootstrap the enrichment of library catalogs with citation data. Reference monographs are works considered to be of particular importance in a research library setting, and likely to possess characteristic citation patterns. The article shows how to select a corpus of reference monographs, and proposes a pipeline to extract the network of publications they refer to. Results using a set of reference monographs in the domain of the history of Venice show that only 7% of extracted citations are made to publications already within the initial seed. Furthermore, the resulting citation network suggests the presence of a core set of works in the domain, cited more frequently than average.

International Journal on Digital Libraries

2017

p. 1-11.

DOI : 10.1007/s00799-017-0210-1

S’affranchir des automatismes

B. Stiegler; F. Kaplan; D. Podalydès 

Fabuleuses mutations, Cité des Sciences, December 8, 2015.

Studying Linguistic Changes over 200 Years of Newspapers through Resilient Words Analysis

V. Buntinx; C. Bornet; F. Kaplan 

This paper presents a methodology to analyze linguistic changes in a given textual corpus allowing to overcome two common problems related to corpus linguistics studies. One of these issues is the monotonic increase of the corpus size with time, and the other one is the presence of noise in the textual data. In addition, our method allows to better target the linguistic evolution of the corpus, instead of other aspects like noise fluctuation or topics evolution. A corpus formed by two newspapers “La Gazette de Lausanne” and “Le Journal de Genève” is used, providing 4 million articles from 200 years of archives. We first perform some classical measurements on this corpus in order to provide indicators and visualizations of linguistic evolution. We then define the concept of a lexical kernel and word resilience, to face the two challenges of noises and corpus size fluctuations. This paper ends with a discussion based on the comparison of results from linguistic change analysis and concludes with possible future works continuing in that direction.

Frontiers in Digital Humanities

2017

Vol. 4 , p. 2.

DOI : 10.3389/fdigh.2017.00002

Rendre le passé présent

F. Kaplan 

La conception d’un espace à quatre dimensions, dont la navigation agile, permet de réintroduire une continuité fluide entre le présent et le passé, s’inscrit dans l’ancien rêve philosophico-technologique de la machine à remonter le temps. Le moment historique auquel nous sommes convié s’inscrit comme la continuité d’un long processus ou fiction, technologie, science et culture se mêlent. La machine à remonter le temps est cet horizon toujours discuté, progressivement approché, et, aujourd’hui peut-être pour la première fois atteignable.

Forum des 100, Université de Lausanne, Switzerland, Mai, 2016.

How to build an information time machine

F. Kaplan 

The Venice Time Machine project aims at building a multidimensional model of Venice and its evolution covering a period of more than 1000 years. Kilometers of archives are currently being digitized, transcribed and indexed setting the base of the largest database ever created on Venetian documents. Millions of photos are processed using machine vision algorithms and stored in a format adapted to high performance computing approaches. In addition to these primary sources, the content of thousands of monographs are indexed and made searchable. The information extracted from these diverse sources is organized in a semantic graph of linked data and unfolded in space and time as part of an historical geographical information system, based on high-resolution scanning of the city itself.

TEDxCaFoscariU, Venice, Italy, June, 2013.

La modélisation du temps dans les Digital Humanities

F. Kaplan 

Les interfaces numériques sont chaque jour optimisées pour proposer des navigations sans frictions dans les multiples dimensions du présent. C’est cette fluidité, caractéristique de ce nouveau rapport à l’enregistrement documentaire, que les Digital Humanities pourraient réussir à reintroduire dans l’exploration du passé. Un simple bouton devrait nous permettre de glisser d’une représentation du présent à la représentation du même référent, il y a 10, 100 ou 1000 ans. Idéalement, les interfaces permettant la navigation dans le temps devraient pouvoir offrir la même agilité d’action que celle nous permettent de zoomer et des zoomer sur des objets aussi larges et denses que le globe terrestre. La recherche textuelle, nouvelle porte d’entrée de la connaissance depuis le le XXIe siècle devrait pouvoir s’étendre avec la même simplicité aux contenus des documents du passé. La recherche visuelle, second grand moment de l’indexation du monde et dont les premiers résultats commencent à s’inviter sur la quotidienneté de nos pratiques numériques, pourrait être la clé de voute de l’accès aux milliards de documents qu’il nous faut maintenant rendre accessible sous format numérique. Pour rendre le passé présent, il faudrait le restructurer selon les logiques des structures de la société numérique. Que deviendrait le temps dans cette transformation ? Une simple nouvelle dimension de l’espace ? La réponse est peut-être plus subtile.

Regimes temporels et sciences historiques, Bern, October, 14, 2016.

L’Europe doit construire la première Time Machine

F. Kaplan 

Le projet Time Machine, en compétition dans la course pour les nouveaux FET Flagships, propose une infrastructure d’archivage et de calcul unique pour structurer, analyser et modéliser les données du passé, les réaligner sur le présent et permettre de se projeter vers l’avenir. Il est soutenu par 70 institutions provenant de 20 pays et par 13 programmes internationaux.

2016

Visual Link Retrieval in a Database of Paintings

B. L. A. Seguin; C. Striolo; I. di Lenardo; F. Kaplan 

This paper examines how far state-of-the-art machine vision algorithms can be used to retrieve common visual patterns shared by series of paintings. The research of such visual patterns, central to Art History Research, is challenging because of the diversity of similarity criteria that could relevantly demonstrate genealogical links. We design a methodology and a tool to annotate efficiently clusters of similar paintings and test various algorithms in a retrieval task. We show that pretrained convolutional neural network can perform better for this task than other machine vision methods aimed at photograph analysis. We also show that retrieval performance can be significantly improved by fine-tuning a network specifically for this task.

2016

VISART Workshop, ECCV, Amsterdam, September, 2016.

DOI : 10.1007/978-3-319-46604-0_52

Epidemics in Venice: On the Small or Large Nature of the Pre-modern World

G. Colavizza 

Marvel et al. [12] recently argued that the pre-modern contact world was physically and, by set inclusion, socially not small-world. Since the Black Death and similar plagues used to spread in well-defined waves, the argument goes, the underlying contact network could not have been small-world. I counter here that small-world contact networks were likely to exist in pre-modern times in a setting of the greatest importance for the outbreak of epidemics: urban environments. I show this by running epidemic diffusion simulations on the transportation network of Venice, verifying how such network becomes small-world when we account for naval transportation. Large epidemic outbreaks might not have been even possible without the catalyst of urban small-worlds.

Computational History and Data-Driven Humanities

2016

International Workshop on Computational History and Data-Driven Humanities, Dublin, Ireland, May 25, 2016.

p. 33-40

DOI : 10.1007/978-3-319-46224-0_4

Exploring Citation Networks to Study Intertextuality in Classics

M. Romanello 

Referring constitutes such an essential scholarly activity across disciplines that it has been regarded by [Unsworth 2000] as one of the scholarly primitives. In Classics, in particular, the references to passages of the ancient texts – the so-called canonical citations (or references) – play a prominent role. The potential of these citations, however, has not been fully exploited to date, despite the attention that they have recently received in the field of Digital Humanities. In this paper I discuss two aspects of making such citations computable. Firstly, I illustrate how they can be extracted from text by using Natural Language Processing techniques, especially Named Entity Recognition. Secondly, I discuss the creation of a three-level citation network to formalise the web of relations between texts that canonical references implicitly constitute. As I outline in the conclusive section of this paper, the possible uses of the extracted citation network include the development of search applications and recommender systems for bibliography; the enhancement of digital environments to read primary sources with links to related secondary literature; and the application of these network to the study of intertextuality and text reception.

Digital Humanities Quarterly

2016

Vol. 10 , num. 2, p. 22.

Clustering citation histories in the Physical Review

G. Colavizza; M. Franceschet 

We investigate publications through their citation histories – the history events are the citations given to the article by younger publications and the time of the event is the date of publication of the citing article. We propose a methodology, based on spectral clustering, to group citation histories, and the corresponding publications, into communities and apply multinomial logistic regression to provide the revealed communities with semantics in terms of publication features. We study the case of publications from the full Physical Review archive, covering 120 years of physics in all its domains. We discover two clear archetypes of publications – marathoners and sprinters – that deviate from the average middle-of-the-roads behaviour, and discuss some publication features, like age of references and type of publication, that are correlated with the membership of a publication into a certain community.

Journal of Informetrics

2016

Vol. 10 , num. 4, p. 1037-1051.

DOI : 10.1016/j.joi.2016.07.009

Diachronic Evaluation of NER Systems on Old Newspapers

M. Ehrmann; G. Colavizza; Y. Rochat; F. Kaplan 

In recent years, many cultural institutions have engaged in large-scale newspaper digitization projects and large amounts of historical texts are being acquired (via transcription or OCRization). Beyond document preservation, the next step consists in providing an enhanced access to the content of these digital resources. In this regard, the processing of units which act as referential anchors, namely named entities (NE), is of particular importance. Yet, the application of standard NE tools to historical texts faces several challenges and performances are often not as good as on contemporary documents. This paper investigates the performances of different NE recognition tools applied on old newspapers by conducting a diachronic evaluation over 7 time-series taken from the archives of Swiss newspaper Le Temps.

Proceedings of the 13th Conference on Natural Language Processing (KONVENS 2016)

2016

13th Conference on Natural Language Processing (KONVENS 2016)Conference on Natural Language Processing, Bochum, GermanyBochum, Germany, September 19-21, 2016September 19–21, 2016.

p. 97-107

Les entités nommées pour le traitement automatique des langues

D. Nouvel; M. Ehrmann; S. Rosset 

Le monde numérisé et connecté produit de grandes quantités de données. Analyser automatiquement le langage naturel est un enjeu majeur pour les applications de recherches sur le Web, de suivi d’actualités, de fouille, de veille, d’opinion, etc. Les recherches menées en extraction d’information ont montré l’importance de certaines unités, telles que les noms de personnes, de lieux et d’organisations, les dates ou les montants. Le traitement de ces éléments, les « entités nommées », a donné lieu au développement d’algorithmes et de ressources utilisées par les systèmes informatiques. Théorique et pratique, cet ouvrage propose des outils pour définir ces entités, les identifier, les lier à des bases de connaissance ou pour procéder à l’évaluation des systèmes.

ISTE editions, 2015.

ISBN : 978-1-78405-104-4

Wikipedia’s Miracle

F. Kaplan; N. Nova 

Wikipedia has become the principle gateway to knowledge on the web. The doubts about information quality and the rigor of its collective negotiation process during its first couple of years have proved unfounded. Whether this delights or horrifies us, Wikipedia has become part of our lives. Both flexible in its form and content, the online encyclopedia will continue to constitute one of the pillars of digital culture for decades to come. It is time to go beyond prejudices and to study its true nature and better understand the emergence of this “miracle.”

Lausanne: EPFL PRESS, 2016.

ISBN : 978-2889143931

Le miracle Wikipédia

F. Kaplan; N. Nova 

Wikipédia s’est imposée comme la porte d’entrée principale de la connaissance sur le web. Les débats de ses premières années concernant la qualité des informations produites ou le bien-fondé du processus de négociation collective sont aujourd’hui dépassés. Que l’on s’en réjouisse ou qu’on le déplore, Wikipédia fait maintenant partie de notre vie. Flexible à la fois dans sa forme et dans ses contenus, l’encyclopédie en ligne continuera sans doute de constituer un des piliers de la culture numérique lors des prochaines décennies. Au-delà des préjugés, il s’agit maintenant d’étudier sa véritable nature et de comprendre à rebours comment un tel « miracle » a pu se produire.

Lausanne: Presses Polytechniques et Universitaires Romandes, 2016.

ISBN : 978-2-88915-143-1

La culture internet des mèmes

F. Kaplan; N. Nova 

Nous sommes à un moment de transition dans l’histoire des médias. Sur Internet, des millions de personnes produisent, altèrent et relaient des « mèmes », contenus numériques aux motifs stéréotypés. Cette « culture » offre un paysage nouveau, riche et complexe à étudier. Pour la première fois, un phénomène à la fois global et local, populaire et, d’une certaine manière, élitiste, construit, « médié » et structuré par la technique, peut être observé avec précision. Étudier les mèmes, c’est non seulement comprendre ce qu’est et sera peut-être la culture numérique, mais aussi inventer une nouvelle approche permettant de saisir la complexité des circulations de motifs à l’échelle mondiale.

Lausanne: Presses Polytechniques et Universitaires Romandes, 2016.

ISBN : 978-2-88915-142-4

Circulation of Opinions in Visualization Reading

D. Rodighiero 

Visualizations need interpretation as a way to grasp the meaning of visual representation. They are complex, and often the process of creation is hidden to the public. Because of this, the following text illustrates a way to read the visual representation of data by analysing the reading in three intervals: detail, visualization, and context. These three different moments make explicit the structure of reading, which will end with the interpretation—the moment in which the observer gets insights and becomes conscious about a personal kind of knowledge. Interpretation, which is composed of personal opinions, is a very important medium to keep the information circulating and to permit an open dialogue with other observers who are reading the same visualization, even in the medical field. In this paper the photography of Luigi Ghirri will illustrate the schematic approach; successively, the three intervals will be exemplified using a medical example, where my parents will be involved in the reading of a blood test. The simple idea is that, through the circulation of opinions and the dialogue, visualization interpretation will foster the creation of a common knowledge and improve the capacity of reading by each single observer.

Proceedings of the Workshop on Valuable Visualization of Healthcare Information: from the quantified self data to conversations

2016

International Working Conference on Advanced Visual Interfaces (AVI 2016), Bari, Italy, June 7-10, 2016.

p. 13-19

Visual Patterns Discovery in Large Databases of Paintings

I. di Lenardo; B. L. A. Seguin; F. Kaplan 

The digitization of large databases of works of arts photographs opens new avenue for research in art history. For instance, collecting and analyzing painting representations beyond the relatively small number of commonly accessible works was previously extremely challenging. In the coming years,researchers are likely to have an easier access not only to representations of paintings from museums archives but also from private collections, fine arts auction houses, art historian However, the access to large online database is in itself not sufficient. There is a need for efficient search engines, capable of searching painting representations not only on the basis of textual metadata but also directly through visual queries. In this paper we explore how convolutional neural network descriptors can be used in combination with algebraic queries to express powerful search queries in the context of art history research.

2016

Digital Humanities 2016, Krakow, Polland, July 11-16, 2016.

Visualizing Complex Organizations with Data

D. Rodighiero 

The Affinity Map is a project founded by the ENAC whose aim is to provide an instrument to understand organizations. The photograph shows the disclosure of the first map for the ENAC Research Day. The visualization was presented to scholars who are displayed in the representation itself.

IC Research Day, Lausanne, Switzerland, June 30, 2016.

Navigating through 200 years of historical newspapers

Y. Rochat; M. Ehrmann; V. Buntinx; C. Bornet; F. Kaplan 

This paper aims to describe and explain the processes behind the creation of a digital library composed of two Swiss newspapers, namely Gazette de Lausanne (1798-1998) and Journal de Genève (1826-1998), covering an almost two-century period. We developed a general purpose application giving access to this cultural heritage asset; a large variety of users (e.g. historians, journalists, linguists and the general public) can search through the content of around 4 million articles via an innovative interface. Moreover, users are offered different strategies to navigate through the collection: lexical and temporal lookup, n-gram viewer and named entities.

2016

International Conference on Digital Preservation (IPRES), Bern, Switzerland, October 3-6, 2016.

Cross-lingual Linking of Multi-word Entities and their corresponding Acronyms

G. Jacquet; M. Ehrmann; R. Steinberger; J. Väyrynen 

This paper reports on an approach and experiments to automatically build a cross-lingual multi-word entity resource. Starting from a collection of millions of acronym/expansion pairs for 22 languages where expansion variants were grouped into monolingual clusters, we experiment with several aggregation strategies to link these clusters across languages. Aggregation strategies make use of string similarity distances and translation probabilities and they are based on vector space and graph representations. The accuracy of the approach is evaluated against Wikipedia’s redirection and cross-lingual linking tables. The resulting multi-word entity resource contains 64,000 multi-word entities with unique identifiers and their 600,000 multilingual lexical variants. We intend to make this new resource publicly available.

Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016)

2016

10th International Conference on Language Resources and Evaluation, Portorož, Slovenia, May 2016.

Named Entity Resources – Overview and Outlook

M. Ehrmann; D. Nouvel; S. Rosset 

Recognition of real-world entities is crucial for most NLP applications. Since its introduction some twenty years ago, named entity processing has undergone a significant evolution with, among others, the definition of new tasks (e.g. entity linking) and the emergence of new types of data (e.g. speech transcriptions, micro-blogging). These pose certainly new challenges which affect not only methods and algorithms but especially linguistic resources. Where do we stand with respect to named entity resources? This paper aims at providing a systematic overview of named entity resources, accounting for qualities such as multilingualism, dynamicity and interoperability, and to identify shortfalls in order to guide future developments.

Proceedings of the 9th International Conference on Language Resources and Evaluation

2016

10th International Conference on Language Resources and Evaluation, Portorož, Slovenia, May 2016.

JRC-Names: Multilingual Entity Name variants and titles as Linked Data

M. Ehrmann; G. Jacquet; R. Steinberger 

Since 2004 the European Commission’s Joint Research Centre (JRC) has been analysing the online version of printed media in over twenty languages and has automatically recognised and compiled large amounts of named entities (persons and organisations) and their many name variants. The collected variants not only include standard spellings in various countries, languages and scripts, but also frequently found spelling mistakes or lesser used name forms, all occurring in real-life text (e.g. Benjamin/Binyamin/Bibi/Benyam’in/Biniamin/Беньямин/بنيامين Netanyahu/Netanjahu/N\'{e}tanyahou/Netahny/Нетаньяху/\نتنياهو). This entity name variant data, known as JRC-Names, has been available for public download since 2011. In this article, we report on our efforts to render JRC-Names as Linked Data (LD), using the lexicon model for ontologies lemon. Besides adhering to Semantic Web standards, this new release goes beyond the initial one in that it includes titles found next to the names, as well as date ranges when the titles and the name variants were found. It also establishes links towards existing datasets, such as DBpedia and Talk-Of-Europe. As multilingual linguistic linked dataset, JRC-Names can help bridge the gap between structured data and natural languages, thus supporting large-scale data integration, e.g. cross-lingual mapping, and web-based content processing, e.g. entity linking. JRC-Names is publicly available through the dataset catalogue of the European Union’s Open Data Portal.

Semantic Web

2017-01-02

Vol. 8 , num. 2, p. 283-295.

DOI : 10.3233/SW-160228

Self-Recognition in Data Visualization: how people see themselves in social visualizations

D. Rodighiero; L. Cellard 

Self-recognition is an intimate act performed by people. Inspired by Paul Ricoeur, we reflect upon the action of self-recognition, especially when data visualization represents the observer itself. Along the article, the reader is invited to think about this specific relationship through concepts like the personal identity stored in information systems, the truthfulness at the core of self-recognition, and the mutual-recognition among community members. In the context of highly interdisciplinary research, we unveil two protagonists in data visualization: the designer and the observer – the designer as the creator and the observer as the viewer of a visualization. This article deals with some theoretical aspects behind data visualization, a discipline more complex than normally expected. We believe that data visualization deserves a conceptual framework, and this investigation pursues this intention. For this reason, we look at the designer as not just a technician in the visualization production, but as a contemporary ethnologist – the designer as a professional working in a social environment to comprehend the context and formulate a specific inquiry with the help of appropriate visual languages.

PubPub

2016

Reading Data Together

D. Rodighiero 

Network visualizations are the most complex visualizations possible, but sometimes they are not capable of describing system-complexity. Even if they are the most widely employed visualization techniques, they still have limitations. Indeed a) their relations are not sufficient to analyse complexity and b) networks do not distinguish between qualitative differences of represented entities. Starting from the actual network model, how could one manipulate this visualization to improve complexity comprehension? In this paper, we propose a solution called trajectory. The trajectory has two major points of difference compared to the network: the trajectory a) represents not only distances, but also durations, and b) it displays kinetic entities according to their evolution with time. The discourse is articulated around these four points. Considering that networks are tools widely used by digital humanists, we propose a new language to improve the quality of represented data: a new network based on a vertical timeline. Complexification of the network visualization is not just a new language, but also a tool that would give the field of Digital Humanities the most complex of all possible visualizations.

2016

VVH 2016 – 1st International Workshop on “Valuable visualization of healthcare information”: from the quantified self data to conversations., Bari, Italy, June 7-10, 2016.

Studying Linguistic Changes on 200 Years of Newspapers

V. Buntinx; C. Bornet; F. Kaplan 

Large databases of scanned newspapers open new avenues for studying linguistic evolution. By studying a two-billion-word corpus corresponding to 200 years of newspapers, we compare several methods in order to assess how fast language is changing. After critically evaluating an initial set of methods for assessing textual distance between subsets corresponding to consecutive years, we introduce the notion of a lexical kernel, the set of unique words that maintain themselves over long periods of time. Focusing on linguistic stability instead of linguistic change allows building more robust measures to assess long term phenomena such as word resilience. By systematically comparing the results obtained on two subsets of the corpus corresponding to two independent newspapers, we argue that the results obtained are independent of the specificity of the chosen corpus, and are likely to be the results of more general linguistic phenomena.

2016

Digital Humanities 2016, Kraków, Poland, July 11-16, 2016.

L’historien et l’algorithme

F. Kaplan; M. Fournier; M-A. Nuessli 

Les relations houleuses qu’histoire et informatique entretiennent ne sont pas nouvelles et la révolution des sciences historiques annoncée depuis plusieurs décennies continue de se faire attendre. Dans ce chapitre, nous aimerions néanmoins tenter de montrer qu’une évolution inédite est aujourd’hui à l’oeuvre dans les sciences historiques et que cette transformation est différente de celle qui a caractérisé, il y a quelques décennies l’arrivée de la « cliométrie » et des méthodes quantitatives. Notre hypothèse est que nous assistons par les effets de deux processus complémentaires à une généralisation des algorithmes comme objets médiateurs de la connaissance historique.

Le Temps des Humanités Digitales; FYP Editions, 2014. p. 49–63.

ISBN : 978-2-36405-122-5

A Method for Record Linkage with Sparse Historical Data

G. Colavizza; M. Ehrmann; Y. Rochat 

Massive digitization of archival material, coupled with automatic document processing techniques and data visualisation tools offers great opportunities for reconstructing and exploring the past. Unprecedented wealth of historical data (e.g. names of persons, places, transaction records) can indeed be gathered through the transcription and annotation of digitized documents and thereby foster large-scale studies of past societies. Yet, the transformation of hand-written documents into well-represented, structured and connected data is not straightforward and requires several processing steps. In this regard, a key issue is entity record linkage, a process aiming at linking different mentions in texts which refer to the same entity. Also known as entity disambiguation, record linkage is essential in that it allows to identify genuine individuals, to aggregate multi-source information about single entities, and to reconstruct networks across documents and document series. In this paper we present an approach to automatically identify coreferential entity mentions of type Person in a data set derived from Venetian apprenticeship contracts from the early modern period (16th-18th c.). Taking advantage of a manually annotated sub-part of the document series, we compute distances between pairs of mentions, combining various similarity measures based on (sparse) context information and person attributes.

2016

Digital Humanities Conference 2016, Krakow, Poland, July 11-16, 2016.

A View on Venetian Apprenticeship through the Garzoni Database

G. Colavizza 

A sample of contracts of apprenticeship from three periods in the history of early modern Venice is analysed, as recorded in the archive of the Giustizia Vecchia, a venetian magistracy. The periods are the end of the 16th century, the 1620s and the 1650s. A set of findings is discussed. First, the variety of professions represented in the dataset reduces over time, as the proportion of venetian apprentices increases, in accordance with previous literature highlighting the decline of the venetian economy during the 17th century. Secondly, apprenticeships are found to be divided into two broad groups: those who stipulated a payment to be given by the master to the apprentice (circa 80%), and those who did not. The first group is suggested to represent contracts used in part, sometimes exclusively, to hire cheap workforce as well as to provide training. Lastly, professional profiles are introduced, as a combination of statistics which provide evidence of three typologies of professions with respect to apprenticeship market dynamics.

Garzoni Apprendistato e formazione tra Venezia e l’Europa in età moderna

2017

Garzoni Apprendistato, Lavoro e Società a Venezia e in Europa, XVI-XVIII secolo, Venice, Italy, October 10-11, 2014.

p. 235-260

The References of References: Enriching Library Catalogs via Domain-Specific Reference Mining

G. Colavizza; M. Romanello; F. Kaplan 

The advent of large-scale citation services has greatly impacted the retrieval of scientific information for several domains of research. The Humanities have largely remained outside of this shift despite their increasing reliance on digital means for information seeking. Given that publications in the Humanities probably have a longer than average life-span, mainly due to the importance of monographs in the field, we propose to use domain-specific reference monographs to bootstrap the enrichment of library catalogs with citation data. We exemplify our approach using a corpus of reference monographs on the history of Venice and extracting the network of publications they refer to. Preliminary results show that on average only 7% of extracted references are made to publications already within such corpus, therefore suggesting that reference monographs are effective hubs for the retrieval of further resources within the domain.

Proceedings of the Third Workshop on Bibliometric-enhanced Information Retrieval co-located with the 38th European Conference on Information Retrieval (ECIR 2016)

2016

3rd International Workshop on Bibliometric-enhanced Information Retrieval (BIR2016), Padua, Italy, March 20-23, 2016.

p. 32-43

Two lectures about representing scientific communities by data visualization

D. Rodighiero 

These lectures present a research that investigates the representation of communities, and the way to foster their understanding by different audiences. Communities are complex multidimensional entities intrinsically difficult to represent synthetically. The way to represent them is likely to differ depending on the audience considered: governing entities trying to make decision for the future of the community, general public trying to understand the nature of the community and the members of the community themselves. This work considers two types of communities as example: a scientific organization and an arising domain: the EPFL institutional community composed of faculty members and researchers and, at a world wide level, the emerging community of Digital Humanities researchers. For both cases, the research is organised as a process going from graphical research to actual materialization as physical artefacts (posters, maps, etc.), possibly extended using digital devices (augmented reality applications). Through iterative cycles of design and experimentation, the research explores theoretically (representation theory, studies about networks, cartography, etc.) and experimentally (development of methods to assess the relevance of each representation depending on the target audiences) how to create effective community mapping. Its global ambition is to inform a theory of design helping to understand how certain community representations can lead to actual cognitive shifts in the way a community is understood.

First Day – Design Creation

The lecture proposes a new way to look at scientific communities. Dealing with a very complex situation, where literacy production is enormous and decisions are made using metrics that are judged obsolete by all, we propose a visual way to understand the community organization. How do scholars work together? What is the intermediary object which makes scientific researchers work together? This first session transforms the current situation into a visual object, a design artefact that embodies the elemental in the creation of maps to understand and evaluate scientific communities.

Second Day – Use of the Maps

The lecture proposes the use of maps to understand and evaluate scientific communities. As continuation of yesterday’s lecture, the topic of the day is how to present elementary objects—which represents publications, teaching, grants and subjects of matters—in a map. Several maps will be shown, representing a precise scientific community inside the EPFL, but with the perspective to make them adaptable to other communities. Moreover, much attention will be dedicated to the reading and interpretation of these maps. Finally a web-based software will be introduced, to illustrate to members and managers of any given community the benefit of a visual representation of a scientific organisation.

Academic Training Lecture Regular Programme, CERN, Geneva, Switzerland, March 14-15, 2016.

La Ville Analogue, la Carte

D. Rodighiero 

This new publication of The Analogous City, an artwork produced by Aldo Rossi, Eraldo Consolascio, Bruno Reichlin and Fabio Reinhart for the Venice Biennale of Architecture in 1976, is part of a museographic installation for the exhibition Aldo Rossi – The Window of the Poet at the Bonnefanten Museum in Maastricht. To gauge and explore this seminal work, Archizoom relied on Dario Rodighiero, candidate on the Doctoral Programme for Architecture and Sciences of the Cities, and designer at the Digital Humanities Lab (DHLAB) at EPFL. Conceived as a genuine urban project, The Analogous City displays an aggregation of architectures drawn from collective and personal memories. What happens if we isolate the forms that Aldo Rossi and his friends so consciously placed in relation to each other? Rodighiero simply decomposed it into the original references and then returned the pieces to the artwork, thus allowing us to simultaneously see the work and its visual vocabulary. An application based on augmented reality has been created to work in tandem with this publication by displaying the complete references belonging to the collage on different layers suspended over the artwork. By downloading the free application and installing it on your tablet or mobile phone, you can recreate the interaction of the museum installation whenever and wherever you are.

Aldo Rossi, La Finestra del Poeta, EPFL, Lausanne, Switzerland, February 29, 2016.

Métaphores machinales

F. Kaplan 

Au fil des siècles, l’homme se voit comme une machine successivement hydropneumatique, mécanique, électrique et aujourd’hui numérique. Chaque nouvelle invention offre une nouvelle perspective sur le vivant sans jamais toutefois être complètement satisfaisante. Il reste toujours “quelque chose” qui semble difficilement réductible à un mécanisme et pour beaucoup ce quelque chose que nous ne voyons que par différence, fait le propre de l’homme.

L’Homme-machine et ses avatars. Entre science, philosophie et littérature – XVIIe-XXIe siècles; Vrin, 2011. p. 237-240.

ISBN : 978-2-7116-2349-5

The Trajectories Tool: Amplifying Network Visualization Complexity

A. Rigal; D. Rodighiero; L. Cellard 

Network visualizations are the most complex visualizations possible, but sometimes they are not capable of describing system-complexity. Even if they are the most widely employed visualization techniques, they still have limitations. Indeed a) their relations are not sufficient to analyse complexity and b) networks do not distinguish between qualitative differences of represented entities. Starting from the actual network model, how could one manipulate this visualization to improve complexity comprehension? In this paper, we propose a solution called t rajectory. The trajectory has two major points of difference compared to the network: the trajectory a) represents not only distances, but also durations, and b) it displays kinetic entities according to their evolution with time. The discourse is articulated around these four points. Considering that networks are tools widely used by digital humanists, we propose a new language to improve the quality of represented data: a new network based on a vertical timeline. Complexification of the network visualization is not just a new language, but also a tool that would give the field of Digital Humanities the most complex of all possible visualizations.

2016

Digital Humanities 2016, Krakóv, Poland, 12-16 July 2016.

Venezia e l’invenzione del paesaggio urbano tra laguna e città

I. di Lenardo 

Acqua e Cibo. Storie di Laguna e Città; Marsilio, 2015. p. 35-39.

ISBN : 978-8-83172-195-0

The Venice Time Machine

F. Kaplan 

The Venice Time Machine is an international scientific programme launched by the EPFL and the University Ca’Foscari of Venice with the generous support of the Fondation Lombard Odier. It aims at building a multidimensional model of Venice and its evolution covering a period of more than 1000 years. The project ambitions to reconstruct a large open access database that could be used for research and education. Thanks to a parternship with the Archivio di Stato in Venice, kilometers of archives are currently digitized, transcribed and indexed setting the base of the largest database ever created on Venetian documents. The State Archives of Venice contain a massive amount of hand-written documentation in languages evolving from medieval times to the 20th century. An estimated 80 km of shelves are filled with over a thousand years of administrative documents, from birth registrations, death certificates and tax statements, all the way to maps and urban planning designs. These documents are often very delicate and are occasionally in a fragile state of conservation. In complementary to these primary sources, the content of thousands of monographies have been indexed and made searchable.

Proceedings of the 2015 ACM Symposium on Document Engineering

2015

ACM Symposium on Document Engineering, Lausanne, Switzerland, September 08 – 11, 2015.

DOI : 10.1145/2682571.2797071

Integration, incorporation, interface: L’evolution des systèmes techniques

F. Kaplan 

Cahiers de l’Institut de la Methode

2009

Vol. 31 , p. 8–16.

Venice Time Machine : Recreating the density of the past

I. di Lenardo; F. Kaplan 

This article discusses the methodology used in the Venice Time Machine project (http://vtm.epfl.ch) to reconstruct a historical geographical information system covering the social and urban evolution of Venice over a period of 1,000 years. Given the time span considered, the project used a combination of sources and a specific approach to align heterogeneous historical evidence into a single geographic database. The project is based on a mass digitization project of one of the largest archives in Venice, the Archivio di Stato. One goal of the project is to build a kind of ‘Google map’ of the past, presenting a hypothetical reconstruction of Venice in 2D and 3D for any year starting from the origins of the city to present-day Venice.

2015

Digital Humanities 2015, Sydney, June 29 – July 3, 2015.

Trajectoire d’une représentation cartographique en réseau

A. Rigal; D. Rodighiero 

Dans le cadre de la « Conférence internationale des Humanités Numériques » qui s’est tenue en 2014 à Lausanne, nous avons réalisé une représentation. Celle-ci est une cartographie en réseau des auteurs et des mots-clés de la conférence. Les cartes résultantes sont reproduites sur divers objets : bâche, tapis, ouvrages, posters, tasses. Ces derniers avaient pour fonction de susciter l’intérêt des auteurs et leur identi cation au champ des humanités numériques. La qualité de la car- tographie en réseau est qu’elle exclut peu d’acteurs et dans notre cas peu de participants. De ce fait un grand nombre de participants à la conférence a pu se trouver sur la représentation et par là prendre part au collectif suggéré par les liens de la cartographie. Par ces reproductions, qui ne sont jamais vraiment mécaniques, la représentation a circulé en alimentant des interprétations qui tracent les contours d’un collectif propre à la conférence. Les traces fabriquées par les participants – commentaires de la cartographie, photos, souvenirs, tweets, etc. -, permettent de suivre la trajectoire de la représentation. Par conséquent, savoir si la représentation a réussi revient à enquêter sur l’étendue et la qualité de sa trajectoire entre les épreuves. L’enjeu de cet article est donc d’enquêter sur le design cartographique en tant qu’art du rassemblement, grâce aux outils du design cartographique.

Cartes & Géomatique: Temps, Art & Cartographie

2015

Vol. 1 , num. 225, p. 33-41.

On Mining Citations to Primary and Secondary Sources in Historiography

G. Colavizza; F. Kaplan 

We present preliminary results from the Linked Books project, which aims at analysing citations from the histo- riography on Venice. A preliminary goal is to extract and parse citations from any location in the text, especially footnotes, both to primary and secondary sources. We detail a pipeline for these tasks based on a set of classifiers, and test it on the Archivio Veneto, a journal in the domain.

2015

Clic-IT 2015, Trento, Italy, December 3-4, 2015.

Interactive device and method for transmitting commands from a user

F. Kaplan 

According to the present invention, it is provided an interactive device comprising a display, a camera, an image analyzing means, said interactive device comprising means to acquire an image with the camera, the analyzing means detecting at least a human face on the acquired image and displaying on the display at least a pattern where the human face was detected wherein the interactive device further comprises means to determine a halo region extending at least around the pattern and means to add into the halo region at least one interactive zone related to a command, means to detect movement onto the interactive zone and means to execute the command by said device.

Patent number(s) :

  • US8126221 (B2)
  • US2009208052 (A1)
  • EP2090961 (A1)

2012

Character network analysis of Émile Zola’s Les Rougon-Macquart

Y. Rochat 

In this work, we use network analysis methods to sketch a typology of fiction novels based on characters and their proximity in the narration. We construct character networks modelling the twenty novels composing Les Rougon-Macquart, written by Émile Zola. To categorise them, we rely on methods that track down major and minor characters relative to the character-systems. For that matter, we use centrality measures such as degree and eigenvector centrality. Eventually, with this analysis of a small corpus, we open the stage for a large-scale analysis of novels through their character networks.

2015

Digital Humanities 2015, Sydney, June 29 – July 3, 2015.

Text Line Detection and Transcription Alignment: A Case Study on the Statuti del Doge Tiepolo

F. Slimane; A. Mazzei; L. Tomasin; F. Kaplan 

In this paper, we propose a fully automatic system for the transcription alignment of historical documents. We introduce the ‘Statuti del Doge Tiepolo’ data that include images as well as transcription from the 14th century written in Gothic script. Our transcription alignment system is based on forced alignment technique and character Hidden Markov Models and is able to efficiently align complete document pages.

2015

Digital humanities, Sydney, Australia, June 29 – July 3, 2015.

Anatomy of a Drop-Off Reading Curve

C. Bornet; F. Kaplan 

Not all readers finish the book they start to read. Electronic media allow to us to measure more precisely how this “drop-off” effect unfolds as readers are reading a book. A curve showing how many people have read each chapter of a book is likely to be progressively going down as part of them interrupt their reading “journey”. This article is an initial study about the shape of these “drop­off” reading curves.

2015

DH2015, Sydney, Australia, June 29 – July 3, 2015.

Inversed N-gram viewer: Searching the space of word temporal profiles

V. Buntinx; F. Kaplan 

2015

Digital Humanities 2015, Sydney, Australia, 29 June–3 July 2015.

The DHLAB Trajectory

D. Rodighiero; A. Rigal; L. Cellard 

This visualisation represents the research activity of the Digital Humanities Lab through publications and co-authorship. Vertical disposition is ordered by time: each layer is a different year of publications, from the lab’s foundation to nowadays. The layers displays the collaboration networks: two researchers are linked if they published together. The vertical trajectories represent the activity of a researcher through the time. The authors position is fix in the space. As consequence, the trajectories become a linear representation of collaborations continuity. The laboratory is here transformed in a geometrical structure which evolves in time despite the members instability.

IC Research Day 2015, EPFL, SwissTech Convention Center, EPFL, Lausanne, Switzerland, 30-6, 2015.

The Analogous City, The Map

D. Rodighiero 

This new publication of The Analogous City, an artwork produced by Aldo Rossi, Eraldo Consolascio, Bruno Reichlin and Fabio Reinhart for the Venice Biennale of Architecture in 1976, is part of a museographic installation for the exhibition Aldo Rossi – The Window of the Poet at the Bonnefanten Museum in Maastricht. To gauge and explore this seminal work, Archizoom relied on Dario Rodighiero, candidate on the Doctoral Programme for Architecture and Sciences of the Cities, and designer at the Digital Humanities Lab (DHLAB) at EPFL. Conceived as a genuine urban project, The Analogous City displays an aggregation of architectures drawn from collective and personal memories. What happens if we isolate the forms that Aldo Rossi and his friends so consciously placed in relation to each other? Rodighiero simply decomposed it into the original references and then returned the pieces to the artwork, thus allowing us to simultaneously see the work and its visual vocabulary. An application based on augmented reality has been created to work in tandem with this publication by displaying the complete references belonging to the collage on different layers suspended over the artwork. By downloading the free application and installing it on your tablet or mobile phone, you can recreate the interaction of the museum installation whenever and wherever you are.

Lausanne: Archizoom, 2015.

ISBN : 978-2-8399-1667-7

Research Notes: Information demand about the US presidential candidates in Switzerland. Investigating information retrieval via the Google AdWords platform

A. Jobin 

2012

Just-in-time Sociology, Lausanne, Switzerland, December 4.

Representing the Digital Humanities Community: Unveiling The Social Network Visualization of an International Conference

D. Rodighiero 

This paper deals with the sense of represent- ing both a new domain as Digital Humanities and its community. Based on a case study, where a set of visualizations was used to represent the community attending the international Digital Humanities conference of 2014 in Lausanne, Switzerland, the meaning of representing a community is investigated in the light of the theories of three acknowledged authors, namely Charles Sanders Peirce for his notion of the interpretant, Ludwig Wittgenstein for his insights on the use of language, and finally Bruno Latour for his ideas of representing politics. There results a proposal to designing and interpreting social network visualizations in a more thoughtful way, while remaining aware of the relation between objects in the real world and their visualizations. As this type of work pertains to a wider scope, we propose bringing a theoretical framework to a young domain such as data visualization.

Parsons Journal of Information Mapping

2015

Vol. VII , num. 2.

Quelques réflexions préliminaires sur la Venice Time Machine

F. Kaplan 

Encore aujourd’hui la plupart des historiens ont l’habitude de travailler en toutes petites équipes, se focalisant sur des problématiques très spécifiques. Ils n’échangent que très rarement leurs notes ou leurs données, percevant à tort ou à raison que leurs travaux de recherche préparatoire sont à la base de l’originalité de leurs travaux futurs. Prendre conscience de la dimension et la densité informationnelle des archives comme celle de Venise doit nous faire réaliser de l’impossibilité pour quelques historiens, travaillant de manière non coordonnée de couvrir avec une quelconque systématicité un objet aussi vaste. Si nous voulons tenter de transformer une archive de 80 kilomètres couvrant mille ans d’histoire en un système d’information structuré il nous faut développer un programme scientifique collaboratif, coordonné et massif. Nous sommes devant une entité informationnelle trop grande. Seule une collaboration scientifique internationale peut tenter d’en venir à bout.

L’archive dans quinze ans; Louvain-la-Neuve: Academia, 2015. p. 161–179.

ISBN : 978-2-8061-0225-6

A Map for Big Data Research in Digital Humanities

F. Kaplan 

This article is an attempt to represent Big Data research in digital humanities as a structured research field. A division in three concentric areas of study is presented. Challenges in the first circle – focusing on the processing and interpretations of large cultural datasets – can be organized linearly following the data processing pipeline. Challenges in the second circle – concerning digital culture at large – can be structured around the different relations linking massive datasets, large communities, collective discourses, global actors, and the software medium. Challenges in the third circle – dealing with the experience of big data – can be described within a continuous space of possible interfaces organized around three poles: immersion, abstraction, and language. By identifying research challenges in all these domains, the article illustrates how this initial cartography could be helpful to organize the exploration of the various dimensions of Big Data Digital Humanities research.

Frontiers in Digital Humanities

2015

Vol. 2 , num. 1, p. 1–7.

DOI : 10.3389/fdigh.2015.00001

Mapping the Early Modern News Flow: An Enquiry by Robust Text Reuse Detection

G. Colavizza; M. Infelise; F. Kaplan 

Early modern printed gazettes relied on a system of news exchange and text reuse largely based on handwritten sources. The reconstruction of this information exchange system is possible by detecting reused texts. We present a method to individuate text borrowings within noisy OCRed texts from printed gazettes based on string kernels and local text alignment. We apply our methods on a corpus of Italian gazettes for the year 1648. Beside unveiling substantial overlaps in news sources, we are able to assess the editorial policy of different gazettes and account for a multi-faceted system of text reuse.

Social Informatics

2015

HistoInformatics 2014.

p. 244-253

DOI : 10.1007/978-3-319-15168-7_31

The Labeling System: A New Approach to Overcome the Vocabulary Bottleneck

M. Piotrowski; G. Colavizza; F. Thiery; K-C. Bruhn 

Shared controlled vocabularies are a prerequisite for collaborative annotation and semantic interchange. The creation and maintenance of such vocabularies is, however, time-consuming and expensive. The diversity of research questions in the humanities makes it virtually impossible to create shared controlled vocabularies that cover a wide range of potential applications and satisfy the needs of diverse stakeholders. In this paper we present a novel conceptual approach for mitigating these problems. We propose that projects define their own vocabularies as needed and link the vocabulary terms to one or more concepts in a reference thesaurus, so that the project-specific term effectively serves as a “label” for a set of shared concepts. We also describe the implementation of this approach in the Labeling System. The Labeling System is a Web application that allows users to easily import concepts or create SKOS vocabularies and link the vocabulary terms to concepts from one or more reference thesauri.

Proceedings of DH-CASE ’14

2014

DH CASE 2014.

DOI : 10.1145/2657480.2657482

X-ray Spectrometry and imaging for ancient handwritten document

F. Albertin; A. Astolfo; M. Stampanoni; E. Peccenini; Y. Hwu et al. 

We detected handwritten characters in ancient documents from several centuries with different synchrotron x-ray imaging techniques. The results were correlated to those of x-ray fluorescence analysis. In most cases, heavy elements produced high image quality suitable for tomography reconstruction leading to virtual page-by-page “reading”. When absorption is too low, differential phase contrast (DPC) imaging can reveal the characters from the substrate morphology. This paves the way to new strategies for information harvesting during mass digitization programs. This study is part of the Venice Time Machine project, an international research program aiming at transforming the immense venetian archival records into an open access digital information system. The Archivio di Stato in Venice holds about 80 kms of archival records documenting every aspects of a 1000 years of Venetian history. A large part of these records take the form of ancient bounded registers that can only be digitize through cautious manual operations. Each page must be turned manually in order to be photographed. Our project explore new ways to virtually “read” manuscripts, without opening them,. We specifically plan to use x-ray tomography to computer-extract page-by-page information from sets of projection images. The raw data can be obtained without opening or manipulating the manuscripts, reducing the risk of damage and speeding up the process. The present tests demonstrate that the approach is feasible. Furthermore, they show that over a very long period of time the common recipes used in Europe for inks in “normal” handwritings – ship records, notary papers, commercial transactions, demographic accounts, etc. – very often produced a high concentration of heavy or medium-heavy elements such as Fe, Hg and Ca. This opens the way in general to x-ray analysis and imaging. Furthermore, it could lead to a better understanding of the deterioration mechanisms in the search for remedies. The most important among the results that we will present is tomographic reconstruction. We simulated books with stacks of manuscript fragments and obtained from sets of projection images individual views — that correspond indeed to a virtual page-by-page “reading” without opening the volume.

European Conference on X-Ray Spectrometry

2014

European Conference on X-Ray Spectrometry, EXRS2014, Bologna,

Virtual X-ray Reading (VXR) of Ancient Administrative Handwritten Documents

F. Albertin; A. Astolfo; M. Stampanoni; E. Peccenini; Y. Hwu et al. 

The study of ancient documents is too often confined to specimens of high artistic value or to official writings. Yet, a wealth of information is often stored in administrative records such as ship records, notary papers, work contract, tax declaration, commercial transactions or demographic accounts. One of the best examples is the Venice Time Machine project that targets a massive digitization and information extraction program of Venetian archives. The Archivio di Stato in Venice holds about 80 kms of archival documents spanning over ten centuries and documenting every aspect of Venetian Mediterranean Empire. If unlocked and transformed in a digital information system, this information could change significantly our understanding of European history. We are exploring new ways to facilitate and speed up this broad task, exploiting x-ray techniques, notably those based on synchrotron light. . Specifically, we plan to use x-ray tomography to computer-extract page-by-page information from sets of projection images. The raw data can be obtained without opening or manipulating the bounded administrative registers, reducing the risk of damage and accelerating the process. We present here positive tests of this approach. First, we systematically analyzed the ink composition of a sample of Italian handwritings spanning over several centuries. Then, we performed x-ray imaging with different contrast mechanisms (absorption, scattering and refraction) using the differential phase contrast (DPC) mode of the TOMCAT beamline of the Swiss Light Source (SLS). Finally, we selected cases of high contrast to perform tomographic reconstruction and demonstrate page-by-page handwriting recognition. The experiments concerned both black inks from different centuries and red ink from the 15th century. For the majority of the specimens, we found in the ink areas heavy or medium-heavy elements such as Fe, Ca, Hg, Cu and Zn. This eliminates a major question about our approach, since the documentation on the nature of inks for ancient administrative records is quite scarce. As a byproduct, the approach can produce valuable information on the ink-substrate interaction with the objective to understand and prevent corrosion and deterioration.

Synchrotron Radiation in Art and Archaeology

2014

Synchrotron Radiation in Art and Archaeology, SR2A 14.

X-ray spectrometry and imaging for ancient administrative handwritten documents

F. Albertin; M. Stampanoni; E. Peccenini; Y. Hwu; F. Kaplan et al. 

‘Venice Time Machine’ is an international program whose objective is transforming the ‘Archivio di Stato’ – 80 km of archival records documenting every aspect of 1000 years of Venetian history – into an open-access digital information bank. Our study is part of this project: We are exploring new, faster, and safer ways to digitalize manuscripts, without opening them, using X-ray tomography. A fundamental issue is the chemistry of the inks used for administrative documents: Contrary to pieces of high artistic or historical value, for such items, the composition is scarcely documented. We used X-ray fluorescence to investigate the inks of four Italian ordinary handwritten documents from the 15th to the 17th century. The results were correlated to X-ray images acquired with different techniques. In most cases, iron detected in the ‘iron gall’ inks produces image absorption contrast suitable for tomography reconstruction, allowing computer extraction of handwriting information from sets of projections. When absorption is too low, differential phase contrast imaging can reveal the characters from the substrate morphology

X-Ray Spectrometry

2015

Vol. 44 , num. 3, p. 93-98.

DOI : 10.1002/xrs.2581

Ancient administrative handwritten documents: X-ray analysis and imaging

F. Albertin; A. Astoflo; E. Peccenini; Y. Hwu; F. Kaplan et al. 

Handwritten characters in administrative antique documents from three centuries have been detected using different synchrotron X-ray imaging techniques. Heavy elements in ancient inks, present even for everyday administrative manuscripts as shown by X-ray fluorescence spectra, produce attenuation contrast. In most cases the image quality is good enough for tomography reconstruction in view of future applications to virtual page-by-page `reading’. When attenuation is too low, differential phase contrast imaging can reveal the characters from refractive index effects. The results are potentially important for new information harvesting strategies, for example from the huge Archivio di Stato collection, objective of the Venice Time Machine project

Journal of Synchrotron Radiation

2015

Vol. 22 , num. 3, p. 446-451.

DOI : 10.1107/S1600577515000314

La simulation humaine : le roman-fleuve comme terrain d’expérimentation narrative

C. Bornet; D. de Roulet; F. Kaplan 

Dans cet article nous présentons la démarche et les premiers résultats d’une recherche participative menée conjointement par le laboratoire d’humanités digitales de l’EPFL (DHLAB) et l’écrivain suisse Daniel de Roulet. Dans cette étude, nous explorons les façons dont la lecture numérique est susceptible d’influencer la façon d’écrire et de réorganiser des récits complexes, de type roman-fleuve ou saga. Nous exposons également nos premières conclusions ainsi que les possibles travaux futures, dans ce domaine très vaste et peu étudié à ce jour.

Cahiers de Narratologie

2014

num. 27.

DOI : 10.4000/narratologie.7042

Il pleut des chats et des chiens: Google et l’impérialisme linguistique

F. Kaplan; D. Kianfar 

Au début du mois de décembre dernier, quiconque demandait à Google Traduction l’équivalent italien de l’expression « Cette fille est jolie » obtenait une proposition étrange : Questa ragazza è abbastanza, littéralement « Cette fille est assez ». La beauté s’était lost in translation — perdue en cours de traduction. Comment un des traducteurs automatiques les plus performants du monde, fort d’un capital linguistique unique constitué de milliards de phrases, peut-il commettre une erreur aussi grossière ? La réponse est simple : il passe par l’anglais. « Jolie » peut se traduire par pretty, qui signifie à la fois « joli » et « assez ». Le second sens correspond à l’italien abbastanza.

2015

The oltramontani Network in Venice: Hans von Aachen in Context

I. di Lenardo 

Thanks to recent archival and historical researches it is now possible to specify the identity of some personalities told in the Lives of Van Mander relating and in close contact with Hans von Aachen. The reconstruction of Venice and Treviso context, in which the artist moves, shows a thick network of relationships woven by Flemish and German communities. The presence of a portrait by Hans von Aachen in the collection of paintings of Francesco Vrients is information very valuable: firstly outlines the painter as an intimate friend of the family Vrients, and in the same time the discovery of the inscription on the drawings of Cephalus and Procri (presented for this exhbition) it is an important pointer for profiling the Vrients-circle and its relationships with the flemish jewellers lobby. Indeed is him the collector of Maastricht mentioned by Van Mander and one of the most eminent flemish personality in the lagoon, around whom, probably, gravitated intellectuals and artists: is a fact that in his house, in Campo Santa Maria Formosa, found hospitality the literate Pieter Cornelisz de Hooft on the occasion of his trip in Italy in 1599. Additional documents shall also specify the role of Gaspar Rem in a venetian and international context: his strong tie to the circle of the “Sadelers” who, especially with a shrewd art dealer like Giusto, play a crucial role promoting artists “Oltramontani” weaving friendship with Dirck de Vries, Rottenhammer, Joannes Koenig to name a few.

Hans von Aachen in Context, Proceedings of the International Conference

2010

Hans von Aachen in Context, Proceedings of the International Conference, Prague, September 22–25, 2010.

p. 28-37

Carlo Helman : merchant, patron and collector in the Antwerp – Venice migrant network

I. di Lenardo 

This contribution is part of the monographic number of the Nederlands Yearbook for History of Art dedicated to a large overview on the “Art and Migration. Nethelandish Artists on the Move, 1400-1750”. In the dynamics of migration, circulation, establishing trough Europe in the Modern Era, the network’s analysis play a fundamental role. The essay explores the prominent role played by Antwerp merchants in Venice in forging contacts between artists, patrons and agent of art in promoting the exchange of goods and ideas within their adopted home. In the course of the 16th century, and more particularly towards the end of that period, the complex network of Netherlandish merchant families, operating on a European level, played a crucial role in the circulation of artists, paintings and other artworks in Italy and beyond. The article proposed here deals with Carlo Helman, a Venetian resident of Antwerp origins, a major figure whose importance in this context has been insufficiently studied. Helman’s family firm traded in practically every kind of commodity, ranging from wool and spices to pearls and diamonds, and, indeed, artworks, “in omnibus mundis regnis”, as we read in the commemorative inscription on his monumental tomb in the Venetian church of Santa Maria Formosa. A high-class international trader in Venice, Helman was consul of the “Nattione Fiamenga”. Helman had a conspicuous collection of art, including classics of the “Venetian maniera” like Titian, Veronese and Bassano, but also important pictures by Northern masters. Moreover, his collection contained a remarkable cartographic section. In Venice, Helman had contacts with the Bassano dynasty, Paolo Fiammingo, Dirck de Vries, Lodewijck Toeput (Pozzoserrato) and the Sadeler brothers, artists who, in one way or another, introduced novel themes and typologies on the Italian, and, indeed, European market. The dedication to Helman on a print by Raphael Sadeler, reproducing Bassano’s Parable of the Sower, photographs the merchant’s role in the diffusion of Bassanesque themes in the North. Helman’s connections with the Zanfort brothers, dealers in tapestries and commercial agents of Hieronymus Cock are further indications of the merchant’s exemplary role of collector, merchant and agent of artists in a European network of “art” commerce.

Art and Migration. Netherlandish Artists on the Move, 1400-1750; Leiden: Brill, 2014. p. 325-347.

ISBN : 9789004270534

“Cities of Fire”. Iconography, Fortune and the Circulation of Fire Paintings in Flanders and Italy in the XVI Century.

I. di Lenardo 

The Wounded City. The representation of Urban Disasters in European Art (XV-XX Centuries); Leiden: Brill, 2014. p. 100–115.

ISBN : 9789004284913

DOI : 10.1163/9789004300682_006

Digital Humanities 2014: representing a controverted definition

D. Rodighiero 

The network portrays all keywords used in the Digital Humanities 2014 conference, which will take place in Lausanne, Switzerland. The keywords‚ represented by nodes‚have been freely chosen by each author attending the conference, contributed via their papers and posters. Edges represent keywords appearing together in a contribution. The weight of the edges measures the occurrence of keywords pairs, multiplied by the number of authors creating them. The visualization is meant as a talking point, to foster a debate about the controversial definition of the Digital Humanities domain.

IC Research Day 2014, EPFL, Lausanne, Switzerland, June 12, 2014.

Character Networks and Centrality

Y. Rochat / Director(s) : H. Volken; F. Kaplan 

A character network represents relations between characters from a text; the relations are based on text proximity, shared scenes/events, quoted speech, etc. Our project sketches a theoretical framework for character network analysis, bringing together narratology, both close and distant reading approaches, and social network analysis. It is in line with recent attempts to automatise the extraction of literary social networks (Elson, 2012; Sack, 2013) and other studies stressing the importance of character- systems (Woloch, 2003; Moretti, 2011). The method we use to build the network is direct and simple. First, we extract co-occurrences from a book index, without the need for text analysis. We then describe the narrative roles of the characters, which we deduce from their respective positions in the network, i.e. the discourse. As a case study, we use the autobiographical novel Les Confessions by Jean-Jacques Rousseau. We start by identifying co-occurrences of characters in the book index of our edition (Slatkine, 2012). Subsequently, we compute four types of centrality: degree, closeness, betweenness, eigenvector. We then use these measures to propose a typology of narrative roles for the characters. We show that the two parts of Les Confessions, written years apart, are structured around mirroring central figures that bear similar centrality scores. The first part revolves around the mentor of Rousseau; a figure of openness. The second part centres on a group of schemers, depicting a period of deep paranoia. We also highlight characters with intermediary roles: they provide narrative links between the societies in the life of the author. The method we detail in this complete case study of character network analysis can be applied to any work documented by an index.

University of Lausanne

2014

Trajectoire d’une représentation cartographique en réseau

A. Rigal; D. Rodighiero 

Dans le cadre de la « Conférence internationale des Humanités Numériques » qui s’est tenue en 2014 à Lausanne, nous avons réalisé une cartographie en réseau des auteurs et des mots-clés de la conférence. Les cartes en résultant sont reproduites sur divers objets : bâche, tapis, ouvrages, posters, tasses. Ces derniers avaient pour fonction de susciter l’intérêt des auteurs et leur identification au champ des humanités numériques. La qualité de la cartographie en réseau est qu’elle exclut peu d’acteurs et dans notre cas peu de participants. De ce fait un grand nombre de participants à la conférence a pu se trouver sur la représentation et par là prendre part au collectif suggéré par les liens de la cartographie. Par ces reproductions, qui ne sont jamais vraiment mécaniques, la représentation a circulé en alimentant des interprétations qui tracent les contours d’un collectif propre à la conférence. Les traces fabriquées par les participants – commentaires de la cartographie, photos, souvenirs, tweets, etc. -, rendent possible de suivre la trajectoire de la représentation. Par conséquent, savoir si la représentation a réussi revient à enquêter sur l’étendue et la qualité de sa trajectoire entre les épreuves. L’enjeu de cette communication est donc d’enquêter sur le design cartographique en tant qu’art du rassemblement, grâce aux outils du design cartographique.

Colloque International Art, Temps, Cartographie, Milan, Italie, décembre 4-5, 2014.

Encoding metaknowledge for historical databases

M-A. Nuessli; F. Kaplan 

Historical knowledge is fundamentally uncertain. A given account of an historical event is typically based on a series of sources and on sequences of interpretation and reasoning based on these sources. Generally, the product of this historical research takes the form of a synthesis, like a narrative or a map, but does not give a precise account of the intellectual process that led to this result. Our project consists of developing a methodology, based on semantic web technologies, to encode historical knowledge, while documenting, in detail, the intellectual sequences linking the historical sources with a given encoding, also know as paradata. More generally, the aim of this methodology is to build systems capable of representing multiple historical realities, as they are used to document the underlying processes in the construction of possible knowledge spaces.

Digital Humanities 2014

2014

Digital Humanities 2014, Lausanne, Switzerland, July 7-12, 2014.

p. 288-289

A New Text-Independent GMM Writer Identification System Applied to Arabic Handwriting

F. Slimane; V. Margner 

This paper proposes a system for text-independent writer identification based on Arabic handwriting using only 21 features. Gaussian Mixture Models (GMMs) are used as the core of the system. GMMs provide a powerful representation of the distribution of features extracted using a fixed-length sliding window from the text lines and words of a writer. For each writer a GMM is built and trained using words and text lines images of that writer. At the recognition phase, the system returns log-likelihood scores. The GMM model(s) with the highest score(s) is (are) selected depending if the score is computed in Top-1 or Top-n level. Experiments using only word and text line images from the freely available Arabic Handwritten Text Images Database written by Multiple Writers (AHTID/MW) demonstrate a good performance for the Top-1, Top-2, Top-5 and Top-10 results.

Proceedings of 14th International Conference on Frontiers in Handwriting Recognition

2014

14th International Conference on Frontiers in Handwriting Recognition (ICFHR 2014), Crete Island, Greece, September 1-4, 2014.

p. 708-713

ICFHR2014 Competition on Arabic Writer Identification Using AHTID/MW and KHATT Databases

F. Slimane; S. Awaida; A. Mezghani; M. T. Parvez; S. Kanoun et al. 

This paper describes the first edition of the Arabic writer identification competition using AHTID/MW and KHATT databases held in the context of the 14th International Conference on Frontiers in Handwriting Recognition (ICFHR2014). This competition has used the new freely available Arabic Handwritten Text Images Database written by Multiple Writers (AHTID/MW) and the Arabic handwritten text database called KHATT presented in ICFHR2012. We propose three tasks in this Arabic writer identification competition: the first and second are based respectively on word and text line level using the AHTID/MW database and the third one is paragraph based using the KHATT database. We received one system for the second task, three systems for the third task and none for the first task. All systems are tested in a blind manner using a set of images kept internal. A short description of the participating groups, their systems, the experimental setup, and the observed results are presented.

Proceedings of 14th International Conference on Frontiers in Handwriting Recognition

2014

14th International Conference on Frontiers in Handwriting Recognition (ICFHR 2014), Crete Island, Greece, September 1-4, 2014.

p. 797-802

La question de la langue à l’époque de Google

F. Kaplan 

En 2012, Google a réalisé un chiffre d’affaires de 50 milliards de dollars un résultat financier impressionnant pour une entreprise créée il y a seulement une quinzaine d’années. 50 milliards de dollars représentent 140 millions de dollars par jour, 5 millions de dollars par heure. Si vous lisez ce chapitre en une dizaine de minutes, Google aura, entre temps, réalisé presque un million de dollars de revenu. Que vend Google pour réaliser des performances financières si impressionnantes ? Google vend des mots, des millions de mots.

Digital Studies Organologie des savoirs et technologies de la connaissance; Limoge: Fyp, 2014. p. 143-156.

ISBN : 978-2-364051089

Fantasmagories au musée

F. Kaplan 

L’utilisation de plus en plus prégnante des nouvelles technologies dans les musées et bibliothèques (tablettes tactiles, audioguides, écrans interactifs, etc.) diviserait les publics entre ceux qui recherchent la compréhension et ceux pour qui prime l’émotion. Comment alors concilier expérience collective partagée et dispositifs techniques ? Comment des cartels virtuels flottant dans les airs peuvent devenir des “fantasmagories didactiques” ? Retour d’expérience muséographique de réalité mixte autour de l’utilisation de vitrines virtuelles “holographiques”.

Alliage

2014

Vol. 73 , p. 132-140.

A Preparatory Analysis of Peer-Grading for a Digital Humanities MOOC

F. Kaplan; C. Bornet 

Over the last two years, Massive Open Online Classes (MOOCs) have been unexpectedly successful in convincing large number of students to pursue online courses in a variety of domains. Contrary to the “learn anytime anywhere” moto, this new generation of courses are based on regular assignments that must be completed and corrected on a fixed schedule. Successful courses attracted about 50 000 students in the first week but typically stabilised around 10 000 in the following weeks, as most courses demand significant involvement. With 10 000 students, grading is obviously an issue, and the first successful courses tended to be technical, typically in computer science, where various options for automatic grading system could be envisioned. However, this posed a challenge for humanities courses. The solution that has been investigated for dealing with this issue is peer-grading: having students grade the work of one another. The intuition that this would work was based on some older results showing high correlation between professor grading, peer-grading and self-grading. The generality of this correlation can reasonably be questioned. There is a high chance that peer-grading works for certain domains, or for certain assignment, but not for others. Ideally this should be tested experimentally before launching any large-scale courses. EPFL is one of the first European schools to experiment with MOOCs in various domains. Since the launch of these first courses, preparing an introductory MOOC on Digital Humanities was one of our top priorities. However, we felt it was important to first validate the kind of peer-grading strategy we were planning to implement on a smaller set of students, to determine if it would actually work for the assignments we envisioned. This motivated the present study which was conducted during the first semester of our masters level introductory course on Digital Humanities at EPFL.

Digital Humanities 2014 : Book of Abstracts

2014

Digital Humanities 2014, Lausanne, 7-12 July.

p. 227-229

GMM-based Handwriting Style Identification System for Historical Documents

F. Slimane; T. Schaßan; V. Märgner 

In this paper, we describe a novel method for handwriting style identification. A handwriting style can be common to one or several writer. It can represent also a handwriting style used in a period of the history or for specific document. Our method is based on Gaussian Mixture Models (GMMs) using different kind of features computed using a combined fixed-length horizontal and vertical sliding window moving over a document page. For each writing style a GMM is built and trained using page images. At the recognition phase, the system returns log-likelihood scores. The GMM model with the highest score is selected. Experiments using page images from historical German document collection demonstrate good performance results. The identification rate of the GMM-based system developed with six historical handwriting style is 100%.

Proceedings of the 6th International Conference of Soft Computing and Pattern Recognition

2014

6th International Conference of Soft Computing and Pattern Recognition, Tunis, Tunisia, August 11-14, 2014.

p. 387-392

Linguistic Capitalism and Algorithmic Mediation

F. Kaplan 

Google’s highly successful business model is based on selling words that appear in search queries. Organizing several million auctions per minute, the company has created the first global linguistic market and demonstrated that linguistic capitalism is a lucrative business domain, one in which billions of dollars can be realized per year. Google’s services need to be interpreted from this perspective. This article argues that linguistic capitalism implies not an economy of attention but an economy of expression. As several million users worldwide daily express themselves through one of Google’s interfaces, the texts they produce are systematically mediated by algorithms. In this new context, natural languages could progressively evolve to seamlessly integrate the linguistic biases of algorithms and the economical constraints of the global linguistic economy.

Representations

2014

Vol. 127 , num. 1, p. 57-63.

Analyse des réseaux de personnages dans Les Confessions de Jean-Jacques Rousseau

Y. Rochat; F. Kaplan 

Cet article étudie le concept de centralité dans les réseaux de personnages apparaissant dans Les Confessions de Jean-Jacques Rousseau. Notre objectif est ainsi de caractériser certains aspects des rôles des personnages du récit sur la base de leurs cooccurrences dans le texte. We sketch a theoretical framework for literary network analysis, bringing together narratology, distant reading and social network analysis. We extract co-occurrences from a book index without the need for text analysis and describe the narrative roles of the characters. As a case study, we use the autobiographical novel Les Confessions from Jean-Jacques Rousseau. Eventually, we compute four types of centrality — degree, closeness, betweenness, eigenvector — and use these measures to propose a typology of narrative roles for the characters.

Les Cahiers du Numérique

2014

Vol. 10 , num. 3, p. 109-133.

DOI : 10.3166/LCN.10.3.109‐133

A Network Analysis Approach of the Venetian Incanto System

Y. Rochat; M. Fournier; A. Mazzei; F. Kaplan 

The objective of this paper was to perform new analyses about the structure and evolution of the Incanto system. The hypothesis was to go beyond the textual narrative or even cartographic representation thanks to network analysis, which could potentially offer a new perspective to understand this maritime system.

DH 2014 book of abstracts

2014

Digital Humanities 2014, Lausanne, July 7-12, 2014.

A social network analysis of Rousseau’s autobiography “Les Confessions”

Y. Rochat; F. Kaplan; C. Bornet 

We propose an analysis of the social network composed of the characters appearing in Jean-Jacques Rousseau’s autobiographic Les Confessions, with existence of edges based on co-occurrences. This work consists of twelve volumes, that span over fifty years of his life. Having a unique author allows us to consider the book as a coherent work, unlike some of the historical texts from which networks often get extracted, and to compare the evolution of patterns of characters through the books on a common basis. Les Confessions, considered as one of the first modern autobiographies, has the originality to let us compose a social network close to the reality, only with a bias introduced by the author, that has to be taken into account during the analysis. Hence, with this paper, we discuss the interpretation of networks based on the content of a book as social networks. We also, in a digital humanities approach, discuss the relevance of this object as an historical source and a narrative tool.

DH 2013 book of abstracts

2013

Digital Humanities 2013, Lincoln, Nebraska, USA, July 15-19, 2013.

Questions méthodologiques sur les usages de Twitter

Y. Rochat 

Journée d’étude : Les usages académiques de Twitter, Université Toulouse II – Le Mirail, France, Mai 16, 2013.

Modeling Venice’s maritime network – End 13th to Mid. 15th centuries

M. Fournier; Y. Rochat 

2014

International Workshop ERC World Seastems – Maritime Networks in Space and Time, Paris, June 16-18, 2014.

Analyse de réseaux sur les Confessions de Rousseau

Y. Rochat; F. Kaplan 

2013

Humanités délivrées, Lausanne, Switzerland, October 1-2, 2013.

Character networks in Les Confessions from Jean-Jacques Rousseau

Y. Rochat; F. Kaplan 

2014

Texas Digital Humanities Conference, Houston, Texas, USA, April 10-12, 2014.

Semi-Automatic Transcription Tool for Ancient Manuscripts

M. M. J-A. Simeoni 

In this work, we investigate various techniques from the fields of shape analysis and image processing in order to construct a semi-automatic transcription tool for ancient manuscripts. First, we design a shape matching procedure using shape contexts, introduced in [1], and exploit this procedure to compute different distances between two arbitrary shapes/words. Then, we use Fischer discrimination to combine these distances in a single similarity measure and use it to naturally represent the words on a similarity graph. Finally, we investigate an unsupervised clustering analysis on this graph to create groups of semantically similar words and propose an uncertainty measure associated with the attribution of one word to a group. The clusters together with the uncertainty measure form the core of the semi-automatic transcription tool, that we test on a dataset of 42 words. The average classification accuracy achieved with this technique on this dataset is of 86%, which is quiet satisfying. This tool allows to reduce the actual number of words we need to type to transcript a document of 70%.

IC Research Day 2014: Challenges in Big Data, SwissTech Convention Center, Lausanne, Switzerland, June 12, 2014.

Attentional Processes in Natural Reading: the Effect of Margin Annotations on Reading Behaviour and Comprehension

A. Mazzei; T. Koll; F. Kaplan; P. Dillenbourg 

We present an eye tracking study to investigate how natural reading behavior and reading comprehension are influenced by in-context annotations. In a lab experiment, three groups of participants were asked to read a text and answer comprehension questions: a control group without taking annotations, a second group reading and taking annotations, and a third group reading a peer-annotated version of the same text. A self-made head-mounted eye tracking system was specifically designed for this experiment, in order to study how learners read and quickly re-read annotated paper texts, in low constrained experimental conditions. In the analysis, we measured the phenomenon of annotation-induced overt attention shifts in reading, and found that: (1) the reader’s attention shifts toward a margin annotation more often when the annotation lies in the early peripheral vision, and (2) the number of attention shifts, between two different types of information units, is positively related to comprehension performance in quick re-reading. These results can be translated into potential criteria for knowledge assessment systems.

ETRA ’14: Proceedings of the Symposium on Eye Tracking Research and Applications

2014

ACM Symposium on Eye Tracking Research and Applications, Safety Harbor, USA, March 26-28, 2014.

p. 235–238

DOI : 10.1145/2578153.2578195

Les “Big data” du passé

F. Kaplan 

Les sciences humaines sont sur le point de vivre un bouleversement comparable à celui qui a frappé la biologie dans les trente dernières années. Cette révolution consiste essentiellement en un changement d’échelle dans l’ambition et la taille des projets de recherche. Nous devons former une nouvelle génération de jeunes chercheurs préparés pour cette transformation.

Bulletin SAGW

2013

num. 4, p. 54-55.

3D Model-Based Gaze Estimation in Natural Reading: a Systematic Error Correction Procedure based on Annotated Texts

A. Mazzei; S. Eivazi; Y. Marko; F. Kaplan; P. Dillenbourg 

Studying natural reading and its underlying attention processes requires devices that are able to provide precise measurements of gaze without rendering the reading activity unnatural. In this paper we propose an eye tracking system that can be used to conduct analyses of reading behavior in low constrained experimental settings. The system is designed for dual-camera-based head-mounted eye trackers and allows free head movements and note taking. The system is composed of three different modules. First, a 3D model-based gaze estimation method computes the reader’s gaze trajectory. Second, a document image retrieval algorithm is used to recognize document pages and extract annotations. Third, a systematic error correction procedure is used to post-calibrate the system parameters and compensate for spatial drifts. The validation results show that the proposed method is capable of extracting reliable gaze data when reading in low constrained experimental conditions.

ETRA ’14: Proceedings of the Symposium on Eye Tracking Research and Applications

2014

ACM Symposium on Eye Tracking Research and Applications, Safety Harbor, USA, March 26-28, 2014.

p. 87–90

DOI : 10.1145/2578153.2578164

Expanding Eye-Tracking Methods to Explain the Socio-Cognitive Effects of Shared Annotations

A. Mazzei / Director(s) : P. Dillenbourg; F. Kaplan 

Social technologies are leading to transformations in education, by empowering the way learners connect to each other, by introducing new means for teaching and learning and by reshaping the way knowledge is delivered. Annotating texts is a learning strategy that is spontaneously adopted by learners during lectures or while reading text books. Students still individually take annotations on paper, that tend to be thrown away when the learning goal is achieved. Students also engage occasionally in the spontaneous practice of note sharing. Our work explores experimentally the benefits of note-taking and note-sharing behaviour. First, we study how sharing student-annotated instructional texts can improve learning, enriching an experimental approach with a new eye tracking method. Second, we conduct experiments on computer-mediated note sharing in the classroom. Our results demonstrate the virtuous circle of note-taking: both annotating while reading and reading annotated documents lead to a better learning achievement. In the first experimental study, we measure if the presence of annotations, on an instruc- tional text, can influence the reading pattern, and how visual features of annotations can elicit the reader’s attention. To complement the results concerning learning and reading comprehension, we look into the readers’ gaze patterns, to explain how the students’ learning outcome relates to reading annotated texts. For this purpose, we design a novel eye tracker, that can be used to study reading and note taking in unconstrained experimental settings. This eye tracking system is based on a systematic error correction procedure that exploits the appearance similarity between the annotated texts and the spatial distribution of the fixation points. We show that this method can extract accurate gaze measures without introducing experimental constraints, that could disturb the note taking process and affect the readers’ comprehension. In the second study, we move from a controlled experimental setting to the classroom. We discuss how the use of technology can facilitate a spontaneous transition from personal to shared annotations and support students in the learning process. We complement the analysis by reporting a friendship bias in browsing the shared annotated material. We further speculate on the potential of shared annotations in triggering adaptations of the instructional material and teaching workflow. These two studies provided an insightful understanding of the effects of student-generated annotations on reading comprehension, and the underlying impact on a community of learners. The obtained findings should inspire further experimentation on social learning environments, meant to facilitate knowledge sharing through shared annotations and their diffusion within educational institutions.

Lausanne: EPFL

2013

DOI : 10.5075/epfl-thesis-5917

The practical confrontation of engineers with a new design endeavour: The case of digital humanties

F. Kaplan; D. Vinck 

This chapter shows some of the practices of engineers use when they are confronted to completely new situations, when they enter into an emerging field where methods and paradigms are not yet stabilized. Following the engineers here would help to shed light on their practices when they are confronted to new fields and new interlocutors. This is the case for engineers and computer scientists who engage themselves with human and social sciences to imagine, design, develop and implement digital humanities (DH) with specific hardware, software and infrastructure.

Engineering Practice in a Global Context; London, UK: CRC Press, 2013. p. 61-78.

ISBN : 978-0-415-63696-4

Simple models of distributed co-ordination

F. Kaplan 

Distributed co-ordination is the result of dynamical processes enabling independent agents to coordinate their actions without the need of a central co-ordinator. In the past few years, several computational models have illustrated the role played by such dynamics for self-organizing communication systems. In particular, it has been shown that agents could bootstrap shared convention systems based on simple local adaptation rules. Such models have played a pivotal role for our understanding of emergent language processes. However, only few formal or theoretical results have been published about such systems. Deliberately simple computational models are discussed in this paper in order to make progress in understanding the underlying dynamics responsible for distributed coordination and the scaling laws of such systems. In particular, the paper focuses on explaining the convergence speed of those models, a largely under-investigated issue. Conjectures obtained through empirical and qualitative studies of these simple models are compared with results of more complex simulations and discussed in relation to theoretical models formalized using Markov chains, game theory and Polya processes.

Connection Science

2005

Vol. 17 , num. 3-4, p. 249-270.

DOI : 10.1080/09540090500177596

Le cercle vertueux de l’annotation

F. Kaplan 

Annoter est bon pour la compréhension immédiate du lecteur. Lire des textes annotés permet de mieux les comprendre. Cette double pertinence de l’annotation, confirmée par l’expérience, peut expliquer son succès séculaire.

Le lecteur à l’oeuvre; Gollion, Suisse: Infolio, 2013. p. 57-68.

Dyadic pulsations as a signature of sustainability in correspondence networks

M. Aeschbach; P-Y. Brandt; F. Kaplan 

Proceedings of Digital Humanities 2013

2013

Digital Humanities 2013, Lincoln, Nebraska, USA, July 15-19, 2013.

Are Google’s linguistic prosthesis biased towards commercially more interesting expressions? A preliminary study on the linguistic effects of autocompletion algorithms.

A. Jobin; F. Kaplan 

Google’s linguistic prosthesis have become common mediators between our intended queries and their actual expressions. By correcting a mistyped word or extending a small string of letters into a statistically plausible continuation, Google offers a valuable service to users. However, Google might also be transforming a keyword with no or little value into a keyword for which bids are more likely. Since Google’s word bidding algorithm accounts for most of the company’s revenues, it is reasonable to ask whether linguistic prosthesis are biased towards commercially more interesting expressions. This study describes a method allowing for progressing in this understanding. Based on an optimal experiment design algorithm, we are reconstructing a model of Google’s autocompletion and value assignment functions. We can then explore and question the various possible correlations between the two functions. This is a first step towards the larger goal of understanding how Google’s linguistic economy impacts natural language.

Conference Abstracts

2013

Digital Humanities 2013, Lincoln, Nebraska, USA, July 15-19, 2013.

p. 245-248

L’ordinateur du XXIe siècle sera un robot

F. Kaplan 

Et l’Homme créa le robot; Paris: Musée des Arts et Métiers / Somogy éditions d’Art, 2012.

ISBN : 9782757205907

Living With a Vacuum Cleaning Robot – A 6-month Ethnographic Study

J. Fink; V. Bauwens; F. Kaplan; P. Dillenbourg 

Little is known about the usage, adoption process and long-term effects of domestic service robots in people’s homes. We investigated the usage, acceptance and process of adoption of a vacuum cleaning robot in nine households by means of a six month ethnographic study. Our major goals were to explore how the robot was used and integrated into daily practices, whether it was adopted in a durable way, and how it impacted its environment. We studied people’s perception of the robot and how it evolved over time, kept track of daily routines, the usage patterns of cleaning tools, and social activities related to the robot. We integrated our results in an existing framework for domestic robot adoption and outlined similarities and differences to it. Finally, we identified several factors that promote or hinder the process of adopting a domestic service robot and make suggestions to further improve human-robot interactions and the design of functional home robots toward long-term acceptance.

International Journal of Social Robotics

2013

Vol. 5 , num. 3, p. 389-408.

DOI : 10.1007/s12369-013-0190-2

Paper Interfaces: an HCI Approach to Geometry Education

Q. Bonnard / Director(s) : P. Dillenbourg; F. Kaplan 

Paper interfaces are an alternative for controlling a computer. Typically, users interact with pieces of paper which are detected by a camera and augmented with relevant information by a projector. The development of paper interfaces, historically, aimed at merging digital and physical versions of documents, enabling to transparently work on one or the other. Furthermore, in the new era of natural interaction techniques, the special affordances of paper can be of great value as a basis for tangible interaction and Augmented Reality: digital objects, linked to paper artefacts, can be manipulated by folding, cutting, orienting, etc. In the context of classroom technologies, paper interfaces are especially appropriated, because paper is integrated and ubiquitous in the school environment and learning processes. Students and teachers are familiar with its properties and know how to interact with it. The goal of this dissertation is to explore the possibilities of using paper as a support for the learning content and more importantly as a tangible body. We focus on geometry education at primary school, because it is a domain where these two aspects of paper can be extensively exploited: pupils can write formulas or draw figures, and they can also move cardboard shapes, or fold along symmetry axes. We designed five series of pedagogical activities: classifying quadrilaterals, mastering the protractor, communicating angles, exploring symmetries, and describing physical transformations with geometrical concepts. These activities are experimented in increasingly valid settings, such that the last series took place in regular classrooms. We also developed methods to analyse the learning activity happening during these experimentations. Our studies revealed important insights on paper interfaces and their application in classroom education. The variety of collaborative scripts that could be created shows the flexibility of paper as a material for building user interfaces that support pedagogical designs. Such flexibility can be further exploited to enable teachers to create their own pedagogical Augmented Reality applications. We also observed many examples of uses that were not intended in the original design, which we refer to as “creative appropriation”. As a result, our paper interfaces were integrated in the everyday conditions of the classroom, used intuitively by the pupils, and managed autonomously by the teachers.

Lausanne: EPFL

2012

DOI : 10.5075/epfl-thesis-5579

La bibliothèque comme interface physique de découverte et lieu de curation collective

F. Kaplan 

Une bibliothèque est toujours un volume organisé en deux sous-espaces : une partie publique (front-end) avec laquelle les usages peuvent interagir, une partie cachée (back-end) utilisée pour la logistique et le stockage. À la Bibliothèque Nationale de France, c’est un système robotisé qui fait la jonction entre les espaces immenses et sous-terrains ouverts au public et les quatre tours qui stockent les livres. L’architecte Dominique Perrault a imaginé une vertigineuse bibliothèque-machine où la circulation des hommes a été pensée symétriquement à la circulation des livres …

Documentaliste – Sciences de l’information

2012

Vol. 49 , num. 3, p. 12-13.

Can a Table Regulate Participation in Top Level Managers’ Meetings?

F. Roman; S. Mastrogiacomo; D. Mlotkowski; F. Kaplan; P. Dillenbourg 

We present a longitudinal study on the participation regulation effects in the presence of a speech aware interactive table. This study focuses on training meetings of groups of top level managers, whose compositions do not change, in a corporate organization. We show that an effect of balancing participation develops over time. We also report other emerging group-specific features such as interaction patterns and signatures, leadership effects, and behavioral changes between meetings. Finally we collect feedback from the participants and analyze qualitatively the human and social aspects of the participants interaction mediated by the technology.

Proceedings of the International Conference on Supporting Group Work GROUP’12

2012

International Conference on Supporting Group Work GROUP’12, Sanibel, Florida, USA, October 27-31, 2012.

Supporting opportunistic search in meetings with tangible tabletop

N. Li; F. Kaplan; O. Mubin; P. Dillenbourg 

Web searches are often needed in collocated meetings. Many research projects have been conducted for supporting collaborative search in information-seeking meetings, where searches are executed both intentionally and intensively. However, for most common meetings, Web searches may happen randomly with low-intensity. They neither serve as main tasks nor major activities. This kind of search can be referred to as opportunistic search. The area of opportunistic search in meetings has not yet been studied. Our research is based upon this motivation. We propose an augmented tangible tabletop system with a semi-ambient conversation-context-aware surface as well as foldable paper browsers for supporting opportunistic search in collocated meetings. In this paper, we present our design of the system and initial findings.

2012

the 2012 ACM annual conference extended abstracts, Austin, Texas, USA, 05-10 05 2012.

DOI : 10.1145/2212776.2223837

How books will become machines

F. Kaplan 

This article is an attempt to reframe the evolution of books into a larger evolutionary theory. A central concept of this theory is the notion of regulated representation. A regulated representation is governed by a set of production and usage rules.ur core hypothesis is that regulated representations get more regular over time. The general process of this regulating tendency is the transformation of a convention into a mechanism. The regulation usually proceeds in two consecutive steps, firstly mechanizing the representation production rules and secondly its conventional usages. Ultimately, through this process, regulated representations tend to become machines.

Lire demain : Des manuscrits antiques à l’ère digitale; Lausanne: PPUR, 2012. p. 27-44.

ISBN : 978-2880749583

Hands-on Symmetry with Augmented Reality on Paper

Q. Bonnard; A. Legge; F. Kaplan; P. Dillenbourg 

Computers have been trying to make their way into education, because they can allow learners to manipulate abstract notions and explore problem spaces easily. However, even with the tremendous potential of computers in education, their integration into formal learning has had limited success. This may be due to the fact that computer interfaces completely rupture the existing tools and curricula. We propose paper interfaces as a solution. Paper interfaces can be manipulated and annotated yet still maintain the processing power and dynamic displays of computers. We focus on geometry, which allows us to fully harness these two interaction modalities: for example, cutting a complex paper shape into simpler forms shows how to compute an area. We use a camera-projector system to project information on pieces of paper detected with a 2D barcode. We developed and experimented with several activities based on this system for geometry learning, however, we focus our presentation on one activity addressing symmetry. This activity is based on a sheet where a part of its content is scanned, and then reprojected according to one or more symmetry axes. Such a sheet is used to illustrate, in real time, how a symmetric drawing is constructed. Anything in the input area can be reflected: ink paper shapes, or physical objects. We show how the augmented sheets provide an easy solution for teachers to develop their own augmented reality activities by reporting on the collaboration with three teachers. These teachers successfully used the activities in their classes, integrating them in the normal course of their teaching. We also relate how paper interfaces let pupils express their creativity while working on geometry.

2012

9th International Conference on Hands-on Science, Antalya, Turkey, October 16 – 21, 2012.

Paper Interfaces to Support Pupils and Teachers in Geometry

Q. Bonnard; F. Kaplan; P. Dillenbourg 

Digital Ecosystems for Collaborative Learning: Embedding Personal and Collaborative Devices to Support Classrooms of the Future (DECL). Workshop in the International Conference of the Learning Sciences (ICLS), Sydney, Australia, July 2.

Tangible Paper Interfaces: Interpreting Pupils’ Manipulations

Q. Bonnard; P. Jermann; A. Legge; F. Kaplan; P. Dillenbourg 

Paper interfaces merge the advantages of the digital and physical world. They can be created using normal paper augmented by a camera+projector system. They are particularly promising for applications in education, because paper is already fully integrated in the classroom, and computers can augment them with a dynamic display. However, people mostly use paper as a document, and rarely for its characteristics as a physical body. In this article, we show how the tangible nature of paper can be used to extract information about the learning activity. We present an augmented reality activity for pupils in primary schools to explore the classification of quadrilaterals based on sheets, cards, and cardboard shapes. We present a preliminary study and an in-situ, controlled study, making use of this activity. From the detected positions of the various interface elements, we show how to extract indicators about problem solving, hesitation, difficulty levels of the exercises, and the division of labor among the groups of pupils. Finally, we discuss how such indicators can be used, and how other interfaces can be designed to extract different indicators.

2012

Interactive Tabletops and Surfaces 2012 Conference, Cambridge, Massachusetts, USA, November 11-14 2012.

Paper Interfaces for Learning Geometry

Q. Bonnard; H. Verma; F. Kaplan; P. Dillenbourg 

Paper interfaces offer tremendous possibilities for geometry education in primary schools. Existing computer interfaces designed to learn geometry do not consider the integration of conventional school tools, which form the part of the curriculum. Moreover, most of computer tools are designed specifically for individual learning, some propose group activities, but most disregard classroom-level learning, thus impeding their adoption. We present an augmented reality based tabletop system with interface elements made of paper that addresses these issues. It integrates conventional geometry tools seamlessly into the activity and it enables group and classroom-level learning. In order to evaluate our system, we conducted an exploratory user study based on three learning activities: classifying quadrilaterals, discovering the protractor and describing angles. We observed how paper interfaces can be easily adopted into the traditional classroom practices.

EC-TEL 2012: 21st Century Learning for 21st Century Skills

2012

7th European Conference on Technology Enhanced Learning, Saarbrücken, Germany, September 18-21, 2012.

p. 37-50

DOI : 10.1007/978-3-642-33263-0_4

From hardware and software to kernels and envelopes: a concept shift for robotics, developmental psychology, and brain sciences

F. Kaplan; P-Y. Oudeyer 

Neuromorphic and Brain-Based robots; Cambridge: Cambridge University Press, 2011. p. 217-250.

ISBN : 978-0-521-76978-8

L’homme, l’animal et la machine : Perpétuelles redéfinitions

G. Chapouthier; F. Kaplan 

Les animaux ont-ils une conscience ? Les machines peuvent-elles se montrer intelligentes ? Chaque nouvelle découverte des biologistes, chaque progrès technologique nous invite à reconsidérer le propre de l’homme. Ce livre, fruit de la collaboration entre Georges Chapouthier, biologiste et philosophe de la biologie, et Frédéric Kaplan, ingénieur spécialiste de l’intelligence artificielle et des interfaces homme-machine, fait le point sur les multiples manières dont les animaux et les machines peuvent être comparés aux êtres humains. Après un panorama synthétique des capacités des animaux et des machines à apprendre, développer une conscience, ressentir douleur ou émotion, construire une culture ou une morale, les auteurs détaillent ce qui nous lie à nos alter-egos biologiques ou artificiels : attachement, sexualité, droit, hybridation. Au-delà, ils explorent des traits qui semblent spécifiquement humains – l’imaginaire, l’âme ou le sens du temps – mais pour combien de temps encore… Une exploration stimulante au coeur des mystères de la nature humaine, qui propose une redéfinition de l’homme dans son rapport à l’animal et à la machine.

CNRS Editions, Paris, 2011.

ISBN : 9782271070722

Anthropomorphic Language in Online Forums about Roomba, AIBO and the iPad

J. Fink; O. Mubin; F. Kaplan; P. Dillenbourg 

What encourages people to refer to a robot as if it was a living being? Is it because of the robot’s humanoid or animal-like shape, its movements or rather the kind of inter- action it enables? We aim to investigate robots’ characteristics that lead people to anthropomorphize it by comparing different kinds of robotic devices and contrasting it to an interactive technology. We addressed this question by comparing anthro- pomorphic language in online forums about the Roomba robotic vacuum cleaner, the AIBO robotic dog, and the iPad tablet computer. A content analysis of 750 postings was carried out. We expected to find the highest amount of anthropomorphism in the AIBO forum but were not sure about how far people referred to Roomba or the iPad as a lifelike artifact. Findings suggest that people anthropomorphize their robotic dog signifi- cantly more than their Roomba or iPad, across different topics of forum posts. Further, the topic of the post had a significant impact on anthropomorphic language.

Proceedings of the IEEE International Workshop on Advanced Robotics and its Social Impacts (ARSO 2012)

2012

The IEEE International Workshop on Advanced Robotics and its Social Impacts (ARSO 2012), Technische Universität München, Munich, Germany, May 21-23, 2012.

p. 54-59

DOI : 10.1109/ARSO.2012.6213399

HRI in the home: A Longitudinal Ethnographic Study with Roomba

J. Fink; V. Bauwens; O. Mubin; F. Kaplan; P. Dillenbourg 

Personal service robots, such as the iRobot Roomba vacuum cleaner provide a promising opportunity to study human-robot interaction (HRI) in domestic environments. Still rather little is known about long-term impacts of robotic home appliances on people’s daily routines and attitudes and how they evolve over time. We investigate these aspects through a longitudinal ethnographic study with nine households, to which we gave a Roomba cleaning robot. During six months, data is gathered through a combination of qualitative and quantitative methods.

1st Symposium of the NCCR robotics, Zürich, Switzerland, June 16, 2011.

Roomba is not a Robot; AIBO is still Alive! Anthropomorphic Language in Online Forums

J. Fink; O. Mubin; F. Kaplan; P. Dillenbourg 

Anthropomorphism describes people’s tendency to ascribe humanlike qualities to non-human artifacts, such as robots. We investigated anthropomorphic language in 750 posts of online forums about the Roomba robotic vacuum cleaner, the AIBO robotic dog and the iPad tablet computer. Results of this content analysis suggest a significant difference for anthropomorphic language usage among the three technologies. In contrast to Roomba and iPad, the specific characteristics of the robotic dog enhanced a more social interaction and lead people to use considerably more anthropomorphic language.

3rd International Conference on Social Robotics, ICSR 2011, Amsterdam, The Netherlands, November 24-25, 2011.

People’s Perception of Domestic Service Robots: Same Household, Same Opinion?

J. Fink; V. Bauwens; O. Mubin; F. Kaplan; P. Dillenbourg 

The study presented in this paper examined people’s perception of domestic service robots by means of an ethnographic study. We investigated initial reactions of nine households who lived with a Roomba vacuum cleaner robot over a two week period. To explore people’s attitude and how it changed over time, we used a recurring questionnaire that was filled at three different times, integrated in 18 semi-structured qualitative interviews. Our findings suggest that being part of a specific household has an impact how each individual household member perceives the robot. We interpret that, even though individual experiences with the robot might differ from one other, a household shares a specific opinion about the robot. Moreover our findings also indicate that how people perceived Roomba did not change drastically over the two week period.

Proceedings of the 3rd International Conference on Social Robotics, ICSR 2011

2011

3rd International Conference on Social Robotics, Amsterdam, The Netherlands, November 24-25, 2011.

p. 204-213

DOI : 10.1007/978-3-642-25504-5

Classroom orchestration : The third circle of usability

P. Dillenbourg; G. Zufferey; H. S. Alavi; P. Jermann; L. H. S. Do et al. 

We analyze classroom orchestration as a question of usability in which the classroom is the user. Our experiments revealed design features that reduce the global orchestration load. According to our studies in vocational schools, paper-based interfaces have the potential of making educational workflows tangible, i.e. both visible and manipulable. Our studies in university classes converge on minimalism: they reveal the effectiveness o tools that make visible what is invisible but do not analyze, predict or decide for teachers. These studies revealed a third circle of usability. The first circle concerns individual usability (HCI). The second circle is about design for teams (CSCL/CSCW). The third circle raises design choices that impart visibility, reification and minimalism on classroom orchestration. The fact that a CSCL environment allows or not students to look at what the next team is doing (e.g. tabletops versus desktops) illustrates the third circle issues that are important for orchestration.

Connecting Computer-Supported Collaborative Learning to Policy and Practice: CSCL2011 Conference Proceedings. Volume I — Long Papers

2011

9th International Conference on Computer Supported Collaborative Learning, Hong Kong, China, July 4-8, 2011.

p. 510-517

Qu’est-ce que la curiosité?

D. SCHUEPACH; M. WINKLER 

2010

Advisor(s): J. DEONNA; F. KAPLAN; F. CLEMENT

C’est quoi ce morceau?

V. EMERY; M. PROENCA; P. HANHART 

2010

Advisor(s): J. DEONNA; F. KAPLAN

Machine Mélomane

P-F. LAQUERRE 

2010

Advisor(s): J. DEONNA; F. KAPLAN

A 99 Dollar Head-Mounted Eye Tracker

Y. Marko; A. Mazzei; F. Kaplan; P. Dillenbourg 

Head-mounted eye-trackers are powerful research tools to study attention processes in various contexts. Most existing commercial solutions are still very expensive, limiting the current use of this technology. We present a hardware design to build, at low cost, a camera-based head-mounted eye tracker using two cameras and one infrared LED. A Playstation Eye camera (PEye) is fixed on an eyeglasses frame and positioned under one eye to track its movements. The filter of the PEye is replaced by another one (Optolite 750nm) that blocks the visible light spectrum. The focal length of the PEye needs to be re-adjusted in order to obtain a sharp image of the eye. This is done by increasing the distance between the charge coupled device (CCD) and the lens by a few millimeters. One IR-LED (Osram SFH485P) is installed near the PEye lens to impose an artificial infrared lighting which produces the so-called “dark pupil effect”. This is done while respecting the Minimum Safe Working Distance. We positioned a second camera on the front side of the eyeglasses frame. Preliminary applicative tests indicate an accuracy of approximately one degree of visual angle, which makes this tool relevant for many eye-tracking projects.

In F. Vitu, E. Castet, & L. Goffart (Eds.), Abstracts of the 16th European Conference on Eye Movements. Presented at the ECEM, Marseille., Marseille, August 21-25, 2011.

Producing and Reading Annotations on Paper Documents: a geometrical framework for eye-tracking studies

A. Mazzei; F. Kaplan; P. Dillenbourg 

The printed textbook remains the primary medium for studying in educational systems. Learners use personal annotation strategies while reading. These practices play an important role in supporting working memory, enhancing recall and influencing attentional processes. To be able to study these cognitive mechanisms we have designed and built a lightweight head mounted eye-tracker. Contrary to many eye trackers that require the readers head to stay still, our system permits complete freedom of movement and thus enables to study reading behaviors as if they were performed in everyday life. To accomplish this task we developed a geometrical framework to determine the localization of the gaze on a flattened document page. The eye tracker embeds a dual camera system which synchronously records the reader’s eye movements and the paper document. The framework post-processes these two video streams. Firstly it performs a monocular 3D-tracking of the human eyeball to infer a plausible 3d gaze trajectory. Secondly it applies a feature point based method to recognize the document page and estimate its planar pose robustly. Finally it disambiguates their relative position optimizing the system parameters. Preliminary tests show that the proposed method is accurate enough to obtain reliable fixations on textual elements.

Symposium N°13: Interacting with electronic and mobile media: Oculomotor and cognitive effects. In F. Vitu, E. Castet, & L. Goffart (Eds.), Abstracts of the 16th European Conference on Eye Movements. Presented at the ECEM, Marseille., Marseille, August 21-25, 2011.

Paper Interface Design for Classroom Orchestration

S. Cuendet; Q. Bonnard; F. Kaplan; P. Dillenbourg 

Designing computer systems for educational purpose is a difficult task. While many of them have been developed in the past, their use in classrooms is still scarce. We make the hypothesis that this is because those systems take into account the needs of individuals and groups, but ignore the requirements inherent in their use in a classroom. In this work, we present a computer system based on a paper and tangible interface that can be used at all three levels of interaction: individual, group, and classroom. We describe the current state of the interface design and why it is appropriate for classroom orchestration, both theoretically and through two examples for teaching geometry.

CHI, Vancouver, BC, Canada, May 7-12, 2011.

Surfaces émotionelles

C. EMERY; H. ARZE 

2008

Advisor(s): F. KAPLAN

Masques émotionnels

T-P. NGUYEN; L. HASSEN; E. NOVIKOVA 

2008

Advisor(s): F. KAPLAN

Le micro-théâtre des émotions

N. PAREDES; C. SANCHEZ; M. PERROD 

2008

Advisor(s): F. KAPLAN

Surfer avec les émotions

C. WEBER; P. VUISTINER; Y. PAQUIER; S. BIANCO 

2008

Advisor(s): F. KAPLAN

Détection automatique du rire

C. MARX 

2008

Advisor(s): F. KAPLAN

Reconnaissance visuelle de la surprise

C. MUMENTHALER 

2008

Advisor(s): F. KAPLAN

Vocal Sticky Notes: Adding Audio Comments on Paper Documents

Q. Bonnard; A. Mazzei; F. Kaplan; P. Dillenbourg 

In this paper we present a tool to annotate paper documents with vocal comments. This tool does not require specially processed documents, and allows natural and simple interactions: sticking a note to add a comment, and place an object on it to listen to the record. A pilot experiment in which teachers used this tool to annotate reports revealed that vocal comments require an extra effort compared to writing. We discuss future work that could either fix or take advantage of this extra effort.

28th ACM Conference on Human Factors in Computing Systems, Atlanta, Georgia, USA, April 10–15, 2010.

A Paper Interface for Code Exploration

Q. Bonnard; F. Kaplan; P. Dillenbourg 

We describe Paper Code Explorer, a paper based interface for code exploration. This augmented reality system is designed to offer active exploration tools for programmers confronted with the problem of getting familiar with a large codebase. We first present an initial qualitative study that proved to be useful for informing the design of this system and then describe its main characteristics. As discussed in the conclusion, paper has many intrinsic advantages for our application.

12th ACM International Conference on Ubiquitous Computing, Copenhagen, Denmark, September 26-29, 2010.

Cognitive and social effects of handwritten annotations

A. Mazzei; F. Kaplan; P. Dillenbourg 

This article first describes a method for extracting and classifying handwritten annotations on printed documents using a simple camera integrated in a lamp. The ambition of such a research is to offer a seamless integration of notes taken on printed paper in our daily interactions with digital documents. Existing studies propose a classification of annotations based on their form and function. We demonstrate a method for automating such a classification and report experimental results showing the classification accuracy. In the second part of the article we provide a road map for conducting user-centered studies using eye-tracking systems aiming to investigate the cognitive roles and social effects of annotations. Based on our understanding of some research questions arising from this experiment, in the last part of the article we describe a social learning environment that facilitates knowledge sharing across a class of students or a group of colleagues through shared annotations.

Red-conference, rethinking education in the knowledge society, Monte Verità, Switzerland, March 7-10, 2011.

Extraction and Classification of Handwritten Annotations

A. Mazzei; F. Kaplan; P. Dillenbourg 

This article describes a method for extracting and classifying handwritten annotations on printed documents using a simple camera integrated in a lamp or a mobile phone. The ambition of such a research is to offer a seamless integration of notes taken on printed paper in our daily interactions with digital documents. Existing studies propose a classification of annotations based on their form and function. We demonstrate a method for automating such a classification and report experimental results showing the classification accuracy.

1st International Workshop on Paper Computing, a Ubicomp 2010 workshop (PaperComp 2010), Copenhagen, Denmark, September 26, 2010.

Augmenting Face-to-Face Collaboration with Low-Resolution Semi-Ambient Feedback

K. Bachour / Director(s) : P. Dillenbourg; F. Kaplan 

Conversations are a daily human activity, and a crucial part of face-to-face collaboration in virtually any context. It is also a highly engaging activity that requires the full attention of participants involved in it. This is why computers have generally been perceived as intrusive in the world of human conversation, for they take some of their user’s attentive focus, reducing their capacity to engage with the other. However, computers today are no longer limited to pieces of technology that we place in front of us or hold in our hands while we interact directly with them via keyboards, touch screens or other input devices. Some computers now hide in our environment, avoiding our attention, achieving whatever function is required of them without us even knowing they are there, and leaving us to focus on the tasks that are important to us. We present a system to explore the role computers can take in face-to-face conversations within the context of this new computing paradigm. Our interactive table, which we call Reflect, monitors the conversation taking place around it via embedded microphones and displays relevant information about member participation on its surface in a discreet and unobtrusive manner. We raise several questions about how such a device can be used to improve the quality of face-to-face collaboration. In particular, we explore whether or not this system is capable altering user behavior and under what conditions this is possible. We also examine whether or it is possible to achieve a change in user behavior while remaining unobtrusive. In addition, we look at the use of such a device outside the scope of face-to-face collaboration by examining its role in the world of communication training. Finally we study the transition process and the design changes needed to bring such a device out of the laboratory and into the real world. To answer these questions, we describe two user studies conducted on the Reflect table. In the first study, we show how the table can be used to promote balanced participation and we examine the conditions under which this is possible. In the second study, we test such a system’s ability to change the way people speak during a conversation, and show some of the difficulties in achieving that, as well as some differences in how male and female users respond to such a device. We then take the Reflect table outside of the laboratory and explore its use in the real world. We explore the changes to the system design that are needed for such a transition to take place. We also show how the table is perceived by users outside the scope of a laboratory study.

Lausanne: EPFL

2010

p. 116.

DOI : 10.5075/epfl-thesis-4895

An Interactive Table for Supporting Participation Balance in Face-to-Face Collaborative Learning

K. Bachour; F. Kaplan; P. Dillenbourg 

We describe an interactive table designed for supporting face-to-face collaborative learning. The table, Reflect, addresses the issue of unbalanced participation during group discussions. By displaying on its surface a shared visualization of member participation, Reflect is meant to encourage participants to avoid the extremes of over- and under-participation. We report on a user study that validates some of our hypotheses on the effect the table would have on its users. Namely we show that Reflect leads to more balanced collaboration, but only under certain conditions. We also show different effects the table has on over- and under-participators.

IEEE Transactions on Learning Technologies

2010

Vol. 3 , num. 3, p. 203-213.

DOI : 10.1109/TLT.2010.18

Low-Resolution Ambient Awareness Tools for Educational Support

K. Bachour; H. Seiied Alavi; F. Kaplan; P. Dillenbourg 

We examine an approach in technology-enhanced learning that avoids deviation from existing pedagogical practices as these are often reluctant to change. This is accomplished by designing technology to augment learning activities that are already in common practice. We implemented two ambient awareness tools, Lantern and Reflect, in line with this approach. The former is tailored for recitation sections and improves student productivity while the latter promotes participation balance in face-to-face collaboration. Both devices allow very limited interaction and provide low-resolution feedback, keeping the actual learning tasks at the center of the student’s focus. We show that the approach we examine coupled with this simple design makes these tools effective and easy to adopt.

CHI 2010 Workshop: The Future of HCI And Education, Atlanta, Georgia, USA, April 11, 2010.

Distributed Awareness for Class Orchestration

H. Alavi; P. Dillenbourg; F. Kaplan 

The orchestration process consists of managing classroom interactions at multiple levels: individual activities, teamwork and class-wide sessions. We study the process of orchestration in recitation sections, i.e. when students work on their assignments individually or in small groups with the presence of teaching assistants who give help on demand. Our empirical study revealed that recitation sections suffer from inefficient orchestration. Too much attention is devoted to the management of the relationship between students and teaching assistants, which prevent both sides from concentrating on their main task. We present a model of students’ activities during recitation sections that emphasize the issue of mutual awareness, i.e. monitoring help needs and TA’s availability. To tackle these difficulties, we developed two awareness tools. Both tools convey the same information: which exercise each group is working on, whether it has asked for help and for how long. In the centralized version, named Shelf, students provide information with a personal response system and the status of each team is juxtaposed on a central display. In the distributed version, named Lantern, each team provides information by interacting with a lamp placed on its table. The display is distributed over the classroom, the information being spatially associated to each group. We are now comparing these two versions in an empirical study with two first year undergraduate classes in Physics. Preliminary results show that both versions increase the efficiency of interaction between students and teaching assistants. This contribution focused on the distributed version.

Learning In The Synergy Of Multiple Disciplines, Proceedings

2009

EC-TEL 2009, Synergy of Disciplines., Nice, September 29th – October 2nd, 2009.

p. 211-225

DOI : 10.1007/978-3-642-04636-0_21

On the Importance of Spatial Configuration of Information

H. Alavi; P. Dillenbourg; F. Kaplan 

The spatial layout of information influences collaborative interactions. We compared two awareness tools which give information on the status of participants in a collaborative work, one displaying it on a single screen and the other distributing it in the room.

2009

11th International Conferenece on Ubiquitous Computing, Florida, September 30-October 3,2009.

Paper-based Concept Map: the Effects of Tabletop on an Expressive Collaborative Learning Task

S. Do-Lenh; F. Kaplan; P. Dillenbourg 

Augmented tabletops have recently attracted considerable attention in the literature. However, little has been known about the effects that these interfaces have on learning tasks. In this paper, we report on the results of an empirical study that explores the usage of tabletop systems in an expressive collaborative learning task. In particular, we focus on measuring the difference in learning outcomes at individual and group levels between students using two interfaces: traditional computer and augmented tabletop with tangible input. No significant effects of the interface on individual learning gain were found. However, groups using traditional computer learned significantly more from their partners than those using tabletop interface. Further analysis showed an interaction effect of the condition and the group heterogeneity on learning outcomes. We also present our qualitative findings in terms of how group interactions and strategy differ in the two conditions.

The 23rd BCS conference on Human Computer Interaction (HCI 2009)

2009

September 1-5, 2009.

p. 149-158

Le corps comme variable expérimentale

F. Kaplan; P. Y. Oudeyer 

L’évolution des concepts de corps et de processus d’animation dans le domaine de la robotique conduit aujourd’hui à définir le concept d’un noyau, ensemble d’algorithmes stables, indépendant des espaces corporels auxquels ils s’appliquent. Il devient alors possible d’étudier la manière dont certaines inscriptions corporelles, considérées comme des variables, structurent le comportement et, à plus long terme, le développement d’un robot. Cette démarche méthodologique peut mener à une approche originale du développement soulignant l’importance d’un corps variable aux frontières en continuelle redéfinition.

Revue philosophique de la France et de l’etranger

2008

Vol. 133 , num. 3, p. 287-298.

DOI : 10.3917/rphi.083.0287

Multi-Finger Interactions with Papers on Augmented Tabletops

S. Do-Lenh; F. Kaplan; A. Sharma; P. Dillenbourg 

Although many augmented tabletop systems have shown the potential and usability of finger-based interactions and paper-based interfaces, they have mainly dealt with each of them separately. In this paper, we introduce a novel method aimed to improve human natural interactions on augmented tabletop systems, which enables multiple users to use both fingertips and physical papers as mediums for interaction. This method uses computer vision techniques to detect multi-fingertips both over and touching the surface in real-time regardless of their orientations. Fingertip and touch positions would then be used in combination with paper tracking to provide a richer set of interaction gestures that the users can perform in collaborative scenarios.

TEI ’09: Proceedings of the 3rd International Conference on Tangible and Embedded Interaction

2009

3rd International Conference on Tangible and Embedded Interaction (TEI 2009), Cambridge, UK, February 16-18, 2009.

p. 267–274

DOI : 10.1145/1517664.1517720

Interpersonal Computers for Higher Education

F. Kaplan; S. Do-Lenh; K. Bachour; G. Y. Kao; C. Gault et al. 

Interactive Artifacts and Furniture Supporting Collaborative Work and Learning; Springer US, 2009. p. 129-145.

DOI : 10.1007/978-0-387-77234-9_8

Reflect : An Interactive Table for Regulating Face-to-Face Collaborative Learning

K. Bachour; F. Kaplan; P. Dillenbourg 

In face-to-face collaborative learning, unbalanced participation often leads to the undersirable result of some participants experiencing lower learning outcomes than others. Providing feedback to the participants on the level of their participation could have a positive effect on their ability to self-regulate, leading to a more balanced collaboration. We propose a new approach for providing this feedback that takes the shape of a meeting table with a reactive visualization displayed on its surface. The meeting table monitors the collaborative interaction taking place around it using embedded microphones and displays a real- time feedback to the participants on an array of LEDs, inviting them to balance their collaboration. We report on an ongoing study that currently shows a positive effect our table has on group regulation.

Times of Convergence: Technologies Across Learning Contexts; Berlin / Heidelberg: Springer, 2008. p. 39-48.

ISBN : 978-3-540-87604-5

DOI : 10.1007/978-3-540-87605-2_5

Intrinsically motivated machines

F. Kaplan; P-Y. Oudeyer 

Children seem intrinsically motivated to manipulate, to explore, to test, to learn and they look for activities and situations that provide such learning opportunities. Inspired by research in developmental psychology and neuroscience, some researchers have started to address the problem of designing intrinsic motivation systems. A robot controlled by such systems is able to autonomously explore its environment not to fulfill predefined tasks but driven by an incentive to search for situations where learning happens efficiently. In this paper, we present the origins of these intrinsically motivated machines, our own research in this novel field and we argue that intrinsic motivation might be a crucial step towards machines capable of life-long learning and open-ended development.

50 Years of AI, Festschrift; Springer Verlag, 2007. p. 304-315.

ISBN : 978-3-540-77295-8

DOI : 10.1007/978-3-540-77296-5_27

Classification of dog barks: a machine learning approach

C. Molnár; F. Kaplan; P. Roy; F. Pachet; P. Pongracz et al. 

In this study we analyzed the possible context-specific and individual-specific features of dog barks using a new machine-learning algorithm. A pool containing more than 6,000 barks, which were recorded in six different communicative situations was used as the sound sample. The algorithm’s task was to learn which acoustic features of the barks, which were recorded in different contexts and from different individuals, could be distinguished from another. The program conducted this task by analyzing barks emitted in previously identified contexts by identified dogs. After the best feature set had been obtained (with which the highest identification rate was achieved), the efficiency of the algorithm was tested in a classification task in which unknown barks were analyzed. The recognition rates we found were highly above chance level: the algorithm could categorize the barks according to their recorded situation with an efficiency of 43% and with an efficiency of 52% of the barking individuals. These findings suggest that dog barks have context-specific and individual-specific acoustic features. In our opinion, this machine learning method may provide an efficient tool for analyzing acoustic data in various behavioral studies.

Animal Cognition

2008

Vol. 11 , num. 3, p. 389-400.

DOI : 10.1007/s10071-007-0129-9

A tabletop display for self-regulation in face-to-face Meetings

K. Bachour; F. Kaplan; P. Dillenbourg 

IEEE TABLETOP 2007, Portland, RHODE ISLAND, USA, October 2007.

Docklamp: a portable projector-camera system

F. Kaplan; S. Do-Lenh; P. Dillenbourg 

2nd IEEE TableTop workshop, Newport, RHODE ISLAND, USA, October 10-12 2007.

The science of laughter

F. Kaplan 

The rediscovery of intelligence – 20 years of AI – in Zurich and world-wide; Zurich: AI-Lab Zurich, 2007.

The progress drive hypothesis: an interpretation of early imitation

F. Kaplan; P-Y. Oudeyer 

Models and Mechanims of Imitation and Social Learning: Behavioural, Social and Communication Dimensions; Cambridge: Cambridge University Press., 2007. p. 361-377.

Futur 2.0: Comprendre les 20 prochaines années

M. Layet; P. Bultez-Adams; F. Kaplan 

Comment vivrons-nous dans 20 ans ? Unique en son genre, ce beau livre illustré, accessible à tous, est le compagnon idéal des rêveurs de futur, de ceux qui souhaitent le comprendre pour mieux le construire. Sous forme d’une pédagogie du futur et grâce à un contenu riche de sens, Futur 2.0 sensibilise le lecteur à tout ce qui pourrait changer son quotidien dans les deux prochaines décennies. Comment les mutations technologiques amorcées vont-elles influer sur notre façon de se déplacer, travailler, se soigner, se cultiver, jouer, se nourrir, agir dans notre environnement, seul et avec les autres ? Chercheurs, philosophes, sociologues ou encore artistes exposent de façon claire et précise les enjeux socioculturels, économiques et écologiques de notre futur. Ils imaginent et racontent leur vision du monde de demain en soulevant un coin du rideau sur les défis et la créativité qui permettront à chacun de nous de construire notre avenir. Depuis 20 ans, le Futuroscope prône la pédagogie douce en faisant des désirs d’avenir et des envies de renouvellement de soi son territoire. Mêlant fiction, sciences et technologies avancées, cet ouvrage collectif et optimiste est une invitation à inventer, à ouvrir notre esprit, à diversifier nos connaissances, pour s’interroger ensemble sur les 20 prochaines années, tenter d’en dessiner les contours et réenchanter le futur.

Fyp editions, 2007.

ISBN : 2916571043

Intrinsic Motivation Systems for Autonomous Mental Development

P-Y. Oudeyer; F. Kaplan; V. Hafner 

Exploratory activities seem to be intrinsicallyrewarding for children and crucial for their cognitive development. Can a machine be endowed with such an intrinsic motivation system? This is the question we study in this paper, presenting a number of computational systems that try to capture this drive towards novel or curious situations. After discussing related research coming from developmental psychology, neuroscience, developmental robotics and active learning, this article presents the mechanism of Intelligent Adaptive Curiosity, an intrinsic motivation system which pushes a robot towards situations in which it maximizes its learning progress. This drive makes the robot focus on situations which are neither too predictable nor too unpredictable thus permitting autonomous mental development. The complexity of the robot’s activities autonomously increases and complex developmental sequences self-organize without being constructed in a supervised manner. Two experiments are presented illustrating the stage-like organization emerging with this mechanism. In one of them, a physical robot is placed on a baby playmat with objects that it can learn to manipulate. Experimental results show that the robot first spends time in situations which are easy to learn, then shifts its attention progressively to situations of increasing difficulty, avoiding situations in which nothing can be learnt. Finally, these various results are discussed in relation to more complex forms of behavioural organization and data coming from developmental psychology.

IEEE Transactions on Evolutionary Computation

2007

Vol. 11 , num. 1, p. 265–286.

DOI : 10.1109/TEVC.2006.890271

Un robot motivé pour apprendre : le role des motivations intrinseques dans le developpement sensorimoteur

F. Kaplan; P-Y. Oudeyer 

Cet article présente des travaux récents qui illustrent comment un robot doté d’un système de motivation intrinsèque peut explorer son environnement et apprendre une succession de tâches qui n’ont pas été spécifiées par son programmeur. Un programme générique contrôle le robot et le pousse à rechercher des situations où ses progrès en prédiction sont maximaux. Ces situations, que l’on appelle ” niches de progrès “, dépendent des opportunités présentes dans l’environnement mais aussi de la morphologie, des contraintes cognitives spécifiques, et de l’expérience passée du robot. Des premiers résultats ont été obtenus dans le domaine de la locomotion, de la découverte des ” affordances “, et des échanges prélinguistiques. Dans chacune de ces expériences, le robot explore les situations ” intéressantes ” de son point de vue par rapport à ses capacités d’apprentissage et les contraintes de son espace sensorimoteur. L’article discute les résultats de ces premières expériences et conclut sur la possibilité de fournir en retour aux neurosciences et à la psychologie, inspiratrices de ces travaux en robotique, de nouvelles pistes de réflexions et de nouveaux concepts pour penser les processus de développement chez l’enfant.

Enfance

2007

Vol. 59 , num. 1, p. 46–58.

Language Evolution as a Darwinian Process: Computational Studies

P-Y. Oudeyer; F. Kaplan 

This paper presents computational experiments that illustrate how one can precisely conceptualize language evolution as a Darwinian process. We show that there is potentially a wide diversity of replicating units and replication mechanisms involved in language evolution. Computational experiments allow us to study systemic properties coming out of populations of linguistic replicators: linguistic replicators can adapt to specific external environments; they evolve under the pressure of the cognitive constraints of their hosts, as well as under the functional pressure of communication for which they are used; one can observe neutral drift; coalitions of replicators may appear, forming higher level groups which can themselves become subject to competion and selection.

Cognitive Processing

2007

Vol. 8 , p. 21-35.

DOI : 10.1007/s10339-006-0158-3

In search of the neural circuits of intrinsic motivation

F. Kaplan; P-Y. Oudeyer 

Children seem to acquire new know-how in a continuous and open-ended manner. In this paper, we hypothesize that an intrinsic motivation to progress in learning is at the origins of the remarkable structure of children’s developmental trajectories. In this view, children engage in exploratory and playful activities for their own sake, not as steps toward other extrinsic goals. The central hypothesis of this paper is that intrinsically motivating activities correspond to expected decrease in prediction error. This motivation system pushes the infant to avoid both predictable and unpredictable situations in order to focus on the ones that are expected to maximize progress in learning. Based on a computational model and a series of robotic experiments, we show how this principle can lead to organized sequences of behavior of increasing complexity characteristic of several behavioral and developmental patterns observed in humans.We then discuss the putative circuitry underlying such an intrinsic motivation system in the brain and formulate two novel hypotheses. The first one is that tonic dopamine acts as a learning progress signal. The second is that this progress signal is directly computed through a hierarchy of microcortical circuits that act both as prediction and metaprediction systems.

Frontiers in Neuroscience

2007

Vol. 1 , num. 1, p. 225-236.

DOI : 10.3389/neuro.01/1.1.017.2007

Computational models in the debate over language learnability

F. Kaplan; P-Y. Oudeyer; B. Bergen 

Computational models have played a central role in the debate over language learnability. This article discusses how they have been used in different stances, from generative views to more recently introduced explanatory frameworks based on embodiment, cognitive development and cultural evolution. By digging into the details of certain specific models, we show how they organize, transform and rephrase defining questions about what makes language learning possible for children. Finally, we present a tentative synthesis to recast the debate using the notion of learning bias.

Infant and Child Development

2008

Vol. 17 , num. 1, p. 55–80.

DOI : 10.1002/icd.544

Information-theoretic framework for unsupervised activity classification

F. Kaplan; V. Hafner 

This article presents a mathematical framework based on information theory to compare multivariate sensory streams. Central to this approach is the notion of configuration: a set of distances between information sources, statistically evaluated for a given time span. As information distances capture simultaneously effects of physical closeness, intermodality, functional relationship and external couplings, a configuration can be interpreted as a signature for specific patterns of activity. This provides ways for comparing activity sequences by viewing them as points in an activity space. Results of experiments with an autonomous robot illustrate how this framework can be used to perform unsupervised activity classification.

Advanced Robotics

2006

Vol. 20 , num. 10, p. 1087-1103.

DOI : 10.1163/156855306778522514