Working together to archive the Web

As part of the cooperation between the BnF and its associated centres in the French regions, printer legal deposit libraries participate in the selection of websites as part of collaborative thematic crawls (electoral web collections since 2004, COVID-19 collection in 2020, Environmental Issues and Artificial Intelligence collections in 2021) and regional collections (Alsace, Lorraine, Montpellier Languedoc-Roussillon, Provence-Alpes-Côte d’Azur, Réunion). Guided tours in “Archives de l’internet”, themed editorial selections of archived sites, have been created on the basis of these collections: the Nancy library, for example, has created a guided tour of the Lorraine Web.

In addition, since a 2014 decree, these libraries have been able to provide access to the BnF’s web legal deposit collections via the “Archives de l’internet” application. This access has been deployed in 21 establishments, enabling users to consult more than 40 titles of the regional daily press in PDF format.

The aim of the ResPaDon project, Réseau de Partenaires pour l’exploration de Données numériques (Network of Partners for the Exploration of Digital Data), run by the BnF and the University of Lille in partnership with the Campus Condorcet and Sciences Po from 2021 to 2023, was to develop and diversify the research uses of the web archive collections held at the BnF. It brought together a community of web archivists, documentation professionals and researchers to promote and facilitate the academic use of web archives. The project gave rise to a study day and an international conference, and was punctuated by a series of thematic workshops (collaborative crawls, typology of research projects, educational uses, etc.).

In 2003, the BnF co-founded the International Internet Preservation Consortium (IIPC) alongside the Internet Archive Foundation and 10 other national libraries.
This Consortium was created in response to the growing demand for international cooperation in web archiving for the benefit of future generations. It now has around 50 members on five continents.

The Consortium’s mission is to:

• collect, preserve and provide access to the richest possible body of internet content throughout the world;
• encourage the development and collaborative use of free software, techniques and standards for the creation of international and interoperable archives;
• promote initiatives and legislation in favour of the collection, preservation and access to internet content;
• encourage and support action by libraries, archives, museums and cultural heritage institutions to preserve web archives.

Every year, the General Assembly and the associated conference on web archiving provide an opportunity for workshops and meetings.

The BnF participates in the governance of the Consortium and in the activities of its working groups. In particular, it is helping to build collaborative collections on transnational themes, such as the Olympic Games, and events with a global reach, such as the Arab Spring, the COVID-19 pandemic and the war in Ukraine.

Most of the software used by the BnF to archive the Web has been developed by members of the IIPC. It was also within this consortium that the WARC format (ISO 28500), used for storing and preserving web archives, was designed.

The IIPC maintains a list of web archiving resources: techniques, formats, archiving and access tools, and offers a number of online training modules.

A community has also formed around NetArchiveSuite, a crawl management tool developed by the Royal Library of Denmark and adopted by the BnF in 2008. Developments in this open-source software are now supported by the community, which has also been joined by the national libraries of Austria, Spain and Estonia.

The BnF has recently undertaken to promote “BnF Collecte du web”, a selection tool that it has developed and which is backed up by NetArchiveSuite, to this community. In addition to these tools, members share a common legal framework and similar working methods, which means that the community is conducive to sharing experiences and exchanging practices. The Web Legal Deposit team is also involved in research programmes, conferences and meetings, particularly for the BnF’s European and French-speaking partners.