Designing a Data Lifeboat

Flickr.com was born in 2004. It’s one of the oldest social web platforms we have. Today, there are tens of billions of images, uploaded by millions of people around the world. There’s a picture of every city and most landmarks, as well as vernacular, intimate representations of our daily lives. It’s one of the biggest collections humanity has ever assembled.

Within this vast collection are countless images truly worth preserving for generations to come, so how do we secure its enduring legacy and accessibility? The entirety of Flickr is simply too big for any one archive to hold and take care of. We know we can’t save everything, nor should we. Which is why we created Data Lifeboat.

The idea came about to address the question ‘what if Flickr goes down?’ How can we prepare for the worst-case scenario? We use the metaphor of the Lifeboat because even the most seaworthy vessels carry emergency equipment, whether or not they expect to sink.

What is a Data Lifeboat?

Data Lifeboat is a tool to archive ‘slivers’ of the huge Flickr collection. Essentially, it is a piece of software that creates a downloadable, compressed file containing images and their metadata – that means comments, tags, galleries, all the stuff that gives Flickr its uniquely social character. We love pondering how a Data Lifeboat might be discovered and viewed in 100 years, and what technologies and social practices might make that possible.

Data Lifeboats are designed to be self-contained, long lasting, versatile, and decipherable. Perhaps our favorite feature in a Data Lifeboat is its built-in mechanism for the creator to leave a “note to the future”.

Features of a Data Lifeboat:

1. Self Contained: Everything needed to access, understand, and maintain the contents exists within the Data Lifeboat itself. All you need is a web browser (and we think they’re going to last a long time).
2. Long lasting: A Data Lifeboat embodies low-fi resilience, built in HTML with no external dependencies. We want the Data Lifeboat to remain operable and legible for decades to come.
3. Versatile: A Data Lifeboat is designed to be a flexible container to fit the preservation needs of both individuals and organizations. Whether it’s your family archive, your local Harley Davidson motorcycle club, or every photo of the Manchester Vimto statue on Flickr, a Data Lifeboat can accommodate this.
4. Legible: Each Data Lifeboat comes with two features that enhance future legibility: The viewer provides a clear overview including permissions and licensing and the README is a free-text input — a sort of ‘note to the future’ — where creators can provide essential context about why this collection was made, who made it, and what they would like to happen to it.

We have designed Data Lifeboat with archival principles at its core. As we’ve learned from conversations with archivists, effective preservation requires conscientious curation:

• Meaningful: What goes into a Data Lifeboat is determined by Flickr members themselves, meaning the preservation energies go towards the photos that matter most to people
• Start small: By capturing a carefully selected ‘sliver’ of Flickr, rather than attempting to save it all, we increase the likelihood of successful preservation
• Well-described: Data Lifeboats keep metadata intact, preserving not just photos, but also their Flickr-specific context – like comments, galleries, and tags. The stuff that gives Flickr its uniquely social character.

How does it work?

• Log into your Flickr.com account
• Assemble the photos (this can be a gallery, an album, single photos, or an entire photostream)
• Check the contents
• Complete your README (a message for future viewers)
• Preview
• We bake it
• Then you can download your Data Lifeboat!

Where would a Data Lifeboat land?

We are working with two scenarios:

1. A Beach: Initially, your Data Lifeboat will be downloaded by you and put wherever you want to store it. It could be your Dropbox account, or that hard drive by your computer. From our perspective, this is a scenario you control and manage entirely as you want.
2. In the Safe Harbor Network: (in development.) We are planning to create a distributed network of managed “docks” or servers where Data Lifeboats can be kept by trusted members of the network for long-term safekeeping. If you’ve heard of LOCKSS, it’s a bit like that.

There are already some great examples of Data Lifeboats we’ve made so far:

• The Flickr Commons 1K Collection Data Lifeboat
  • • There’s a downloadable version on the Internet Archive or
    • A more “glacial” version on the decentralized InterPlanetary File System (IPFS)

The Timeline of Data Lifeboat

2021: Data Lifeboat first mentioned as a response to the question ‘what if Flickr.com goes down’?”

2022: Flickr Foundation is established

2023: National Endowment for the Humanities (NEH) Digital Humanities Advancement Grant Level I funding is secured for the R&D phase of Data Lifeboat

2024: First prototype of Data Lifeboat is delivered; Funding from the Mellon Foundation Public Knowledge Grant secured to complete deep user research and further product planning

2025: Alpha release is on the way! Ongoing research with Flickr members

2026: Beta release…

Want to keep up-to-date? You can follow our progress in real time on the blog: Data Lifeboat Archives

Our Research Approach

The design of the Data Lifeboat is informed by research and expert consultation, generously supported by the National Endowment for the Humanities and the Mellon Foundation.

Responsive to existing needs. ensuring we design against redundancy, we mapped the landscape of existing digital preservation systems. Participatory research with actual people allowed us pinpoint what the Data Lifeboat should be responding to and communing with.

Prioritizes ethics and care. we wanted to develop a tool built on modern archival best practices, particularly the C.A.R.E. principles for data stewardship (Collective benefit, Authority to control, Responsibility, and Ethics). This framework places care at the centre of preservation, ensuring ethical considerations aren’t overshadowed by technological needs.

Collaboratively conceived. Many people and organizations have played a vital role in shaping Data Lifeboat, offering their diverse expertise to guide our thinking and design. We are grateful for the contributions of staff from the following organizations and look forward to fostering future collaborations:

Aalborg City Archives/Aalborg Stadsarkiv
Aalto University
Bodleian Libraries
Boston Public Library
The British Library
Cambridge University Library
Capital Area District Libraries
CLOCKSS
Community Archives of Belleville and Hastings County
Congregation of the Sisters of St. Joseph in Canada Archives
Creative Commons
Digital Preservation Coalition
GLAM-E Lab
Filecoin Foundation for the Decentralized Web
The Internet Society
The Library of Congress
The National Archives (UK)
National Archives (US)
National Library of Wales/Llyfrgell Genedlaethol Cymru
Permanent.org
Port Morien Digital Archives
Provincial Archives of Alberta
Royal Danish Library/Det Kongelige Bibliotek
Schlesinger Library at Harvard Radcliffe Institute
San Diego Air & Space Museum
University of Bristol
University of Edinburgh
University of Washington

Documentation

• NEH/DHAG 2024 Data Lifeboat Whitepaper
• Mellon Foundation Public Knowledge Grant, Report (forthcoming)