Designing a Data Lifeboat

About the project

You have probably been affected by web services that go dark or disappear, often with little or no warning. We think that’s not good enough, especially for an archive as precious as Flickr (and your photos), so we want to design a better way.

Today, the Flickr holds “tens of billions of images” documenting our planet from the first days of photography to just a moment ago. What if—should the ship go down—we had an archival copy of your Flickr presence ready? Simply admitting this might happen and preparing for it is a form of preservation. We call it a data lifeboat.

It’s all at risk—though not in imminent danger—and that’s why the foundation has been set up. SmugMug has acknowledged the risk and set us the task of imagining and determining how to make sure this huge piece of human history doesn’t sink.

This year (2024)

We will work initially with the smaller and openly-licensed subset of imagery held within the Flickr Commons. Using this collection as our baseline, we will explore the edges of what’s required to create a data lifeboat that’s transportable, buoyant, and robust.

Here’s what we envision:

A Data Lifeboat is an archival piece of Flickr, not all of the 50 billion images and their metadata. For example, a Lifeboat might contain all the photos tagged with “sunflower” or all the Recipes to Share group submissions. Whatever facet of the data you can think of, you could generate a Data Lifeboat for it. We envision an archival sliver richer than a mere folder of JPGs: one where you can navigate the content to explore and understand its networked context. Even better, an archival sliver that is updated if things change at flickr.com.

This year our goals are to create several rough prototypes of the software, develop a reasonably detailed understanding of the main technical challenges, prepare a survey of critical ongoing legal issues, and establish a robust design direction for further product development. You can follow our progress in our blog posts:

• Data Lifeboat: NEH Grant Update 1
• Data Lifeboat Update 2: More questions than answers
• Data Lifeboat: Deeper research into the challenge of archiving social media objects
• Data Lifeboat Update 3

We’re proud to have been awarded a Digital Humanities Advancement Grant by the National Endowment for the Humanities to support this exploration, too. Thank you, NEH!