Deep (Hi)storytelling

Amsterdam, July 2nd, 2026

Dr. Leakey had long been hoping for this family trip to Amsterdam, ever since a DNA test had shown him that his family origins were partly Dutch. While organizing the trip, the doctor from Colorado, ever meticulous, had read extensively about the Dutch Golden Age and its famous painters. However, nothing could have prepared him for the emotions that would follow his entry with his wife and daughter into Rembrandt’s Night Watch gallery at the Rijksmuseum in Amsterdam.

As he moves towards the colossal 12X14 feet canvas famously displaying a group of harquebusiers led by Captain Frans Banninck Cocq, he takes out his smartphone which has just vibrated in his pocket. A notification from the museum's smart app invites him to listen to a message: “Hello Dr. Leakey. Welcome to the Night Watch Gallery. May we draw your attention to information that connects you personally to this masterpiece?”

Before he can answer, he hears an excited "yes" from his daughter beside him, like a pre-echo. The voice from the museum app continues: "Look closely at the character with the black hat on the right, Dr. Leakey. The one who looks to the right while his arm is stretched out to the left.” A picture on his smartphone highlights a character with a light beard. “This is Rombout Kemp. He was a sergeant, a deacon, and a clothing merchant. We have good reason to believe that he is your ancestor. Do you want to know more?”

Guided by their daughter's enthusiasm, the Leakeys spend the rest of the day not only discovering Kemp and their Dutch origins but literally walking in the footsteps of their family history. Linked to a large interconnected database called the Amsterdam Time Machine, the museum's application creates a fully personalized tour of the city, starting with the discovery of a second portrait of Rombout Kemp, posing among the governors of a charity organization, in another room of the Rijksmuseum.

Following the 3D map created by the application, they go to the church of Nieiuwe Kerk where Kemp's funeral took place in 1653. In front of Nieuwendjik's house, where Kemp lived with his wife Elsje, they are given a virtual tour of the family interior reconstructed from the inventory of the inheritance. Geolocation software guides them to the bookstores where Kemp, a Calvinist, liked to go, and on to the barracks where his eldest son Artus served as a civil guard. They learn how one of Kemp's descendants joined the West India Company and was sent to America, where he entered into the fur trade with the Algonquins...

If you have never heard of the Time Machines that historians and computer scientists are building in Europe today, this fiction may seem too good to be true. This is not the case. On the contrary, it is a glimpse into the phenomenal potential that digital preservation of heritage offers to the cultural industries and tourism of tomorrow.

Digital technology, biotechnology, and other innovations are creating a field at the crossroads of artistic and cultural disciplines and technologies. Called "ArtTech", it is already offering fertile soil; music has just been revolutionized by Spotify and cinema by Netflix. This is only the beginning. Museums, libraries, and all cultural institutions that have functioned more or less in the same way for centuries, will soon be transformed into digital content providers, especially in Europe.

Indeed, thanks to the depth and diversity of Europe’s history and the vitality of its creative industries - which directly employ twice as many people (7.8 million) as automobile production (4.4 million) - the Old Continent has all the assets to build new digital platforms to take advantage of this opportunity. But Europe has to move decidedly. Because Silicon Valley has also identified its potential.

San Francisco, May 21st, 2019

"Your DNA offers much more than information about your origins. It can help you plan your next travel." The teaser of the press release announcing the alliance of the genomics company 23andMe with shared housing giant AirBnB highlights the potential of disruptive technologies in cultural consumption.

23andMe's Director of Communication Christine Pai explains: “If a 23andMe customer has origins in southern Italy, our new service will find a trullo (a kind of traditional housing) in Apulia that will serve as a base for exploring his origins.” The company developed a genetic testing service for the general public to discover medical predispositions, but it can reveal genealogical information as well. This service, Ancestry Composition, determines where a client's DNA comes from across more than a thousand regions of the world already.

With the inclusion of Ancestry and MyHeritage's competing services, 30 million people have performed these types of tests since they became popular in 2017. At the same time, travel in search of family histories is a rising trend, according to AirBnB, which claims “a 500% increase since 2014 in the number of travelers using the platform for this purpose.” Marrying the two databases to create a service capable of hyper-personalizing a journey thus seems infallible business logic in the age of selfies.

However, AirBnB and 23andMe still lack an essential piece to give their users the kind of enhanced experience of the fictional Leakeys in Amsterdam: databases containing the millions of individual stories of our ancestors. It is in Venice one must look for them, starting back a few years ago.

Venice, November 10th, 2015

Former Swiss Technology Institute of Lausanne (EPFL) President Patrick Aebischer (see his interview about the launch of the Arttech venture fund in our bonus) warns: "You feel something physical when you discover the archives of the Republic of Venice.” By visiting the 300 rooms where private and public documents have been collected since 1815 in the convent of the Basilica of Santa Maria Gloriosa dei Frari, one can actually “physically” understand the scale of the challenge that is the digitization project of the 80 kilometers long archives, a project known as the Venice Time Machine of the EPFL. A little later, in a room of the Ca' Foscari University overlooking the Grand Canal and Palazzo Grassi, it is its potential that we discover.

EPFL professor Frédéric Kaplan, who is leading this digitization of the history of the Serenissima, plays with images: “Big data from the past, Google Earth for time and space… Middle Ages ‘Facebook’.” He explains: “Thanks to the thousands of data points we scan in the archives, we can zoom in on an individual at a specific date. We follow him from a birth register to a notarial deed to find his house, business, and reconstitute his life.”

On this day it is the third autumn session of the project, which brings together the Venetian University, the Venice Archives, and EPFL. The students are presenting their works focused on the year 1740, and in doing so embody the prediction of historian Emmanuel Leroy-Ladurie, who stated in 1968: “Historians will become programmers.”

A first team has reconstituted the functions of the buildings in the Rialto district by crossing the land census (castatici) with the first cadastre (sommarione) of the city, imposed by Napoleon in 1808. Their ‘Google Map’ reveals, among other things, the explosion of luxury boutiques in the area. The next group has extracted transactions from tax records (savi alle decime) from a sample of senior officials to build a ‘LinkedIn’ of their relationships. A third group has used the death register during carnival week and the Lent weeks that followed to establish links between causes of mortality and social behavior.

“The Venetians documented everything,” explains Frédéric Kaplan. In the 17th century, at the height of the city's prosperity, with a population of about 160,000 people (three times that of today), half of the jobs were in administration. There were magistrates for everything. To navigate this sprawling paperwork, the Venitians had developed information systems, combining words and numbers from one register to another."

Frédéric Kaplan and his team have transformed these paper links into Internet hyperlinks. They have developed artificial intelligence algorithms that read handwritten scripts to, for example, follow a proper name on a property deed to a tax register. And their search engines are augmented with time sliders to choose a date to visit this incredibly dense past.

But the Venice Time Machine does not just connect the data to bring up the tingle of individual stories. On her PC, art historian Isabella di Leonardo also shows her urban simulation model, which combines cadastral plans with paintings, architectural sketches, and photos to recreate in 3D the very construction of Venice over the centuries.

For Frédéric Kaplan, it is clear that these technologies, like those of circular scanners that digitize books or the X-ray technology developed at the University of Erlangen to read fragile works without opening them, have a universal use. “Any city can use the same techniques.” This is the whole idea of the European Time Machine and the community of some 400 researchers and companies who came together to coordinate the project last May in Holland.

Amsterdam, May 4th, 2019

Only accessible from the covered alleyway of an old books market, the Minerva courtyard of the Ouidemanhuispoort, now part of the University of Amsterdam, resonates with excitement. This is the first meeting of the European Time Machine consortium since the project was selected two months earlier as one of the six finalists of the €1 billion European FET Flagship project funding over ten years. The optimism is tangible.

The choice of Amsterdam for this conference is no coincidence. The city already has its own Amsterdam Time Machine. “In 2016, we invited Frédéric Kaplan to present his work in Venice at a conference at the University of Amsterdam,” explains Julia Noordegraaf, Professor of Digital Heritage in the Department of Media Studies at the University of Amsterdam. “We were blown away. And inspired, two years after launching our first LinkedIn project for the art market of the Dutch Golden Age.”

With Amsterdam's prosperity having been historically based on its commercial success, the city's 50 kilometers of archives contain millions of extremely detailed notarial deeds. “Everyone had to deal with notaries at some point in their lives,” says Julia Noordegraaf. “Property inventories contain accurate information, including information on paintings. Starting in 1550, our archives are a little less deep than those of Venice. But by adding other sources such as baptisms, weddings, etc., we had enough data to launch the Amsterdam Time Machine.”

The result is impressive. The meta-database, named ALiDA, brings together the different sources in a unified format of interconnected data, containing 630,000 scans of notarial records with more than 1.5 million proper names linked to seven other indexes in the city's archives (e.g. 500,000 marriage records, 1.2 million baptisms, etc.). When linked to the commercial acts, or the Ecartico database, with its 47,422 people connected to cultural industries between 1475 and 1740, these data bring to life the city's social interactions between the 15th and 18th centuries.

A tool called AdamLink even allows you to track the evolution of 6500 streets year after year. “That's how I realized that I live in an old cinema, which is ironic because I'm a film historian,” explains Julia Noordegraaf. Together with colleagues at the Huygens Institute for Dutch History and the Municipality of Amsterdam, she is now developing an interface to reconstruct the history of interiors in virtual images based on notarial deeds.

For Julia Noordegraaf and the other representatives of the consortium of the European Time Machine (400 organizations in 34 countries), the Venice experience is a tremendous time saver. This makes them confident about the chances of their own projects from Lisbon to Oslo and from Istanbul to Dublin.

And this despite a major setback.

A few days after the Amsterdam meeting, the European Union decided to simplify its research funding mechanisms by eliminating the FET Flagship programs and their symbolic billion. However, this did not dampen the enthusiasm of the researchers involved with the European project. “It's so exciting, we'll do it anyway,” Julia Noordegraf said soon after.

In good Cartesian style, Frédéric Kaplan highlights the research in artificial intelligence that will be advanced by analyzing these huge unstructured databases. “It's the same kind of social big data that Facebook uses to train its machine learning programs,” he compares. His confidence is strengthened by the launch of Time Machines in more than twenty cities who already support them financially, such as Jerusalem, Nuremberg, Antwerp, Utrecht, Naples, Paris, Dresden....

As we cross a bridge in Amsterdam, Frédéric Kaplan stops suddenly when I mention the idea of future tourism based on Time Machines and DNA analysis. “It's funny you say that because we've initiated contacts with a Mormon organization that manages genealogical indexes in the United States.” Frédéric Kaplan knows that while Time Machines are important for researchers in the Humanities and social sciences, politicians who fund this research also expect economic results.

It is in front of a beer at the end of the day that Harry Verwayen, the director of Europeana - a database of 60 million digitized cultural artifacts that is a partner of the European Time Machine - opens one of the many promising avenues on this subject. “These Time Machines will serve as support and inspiration for the mirror worlds that virtual, mixed, and augmented reality technologies are beginning to create.” He is right. And this idea will be the subject of the next episode of our "ArtTech" exploration.

Editing: Katherine Lingenfelter