This operation is performed in accordance with the legal deposit of the Internet as established by the French Heritage Code (art. L131-1 to L133-1), following the Law on Copyright passed on August 1st, 2006. Legal deposit is one of the main methods available to the BnF to ensure the growth and development of its collections.
Harvesting settings
BnF uses a spider called Heritrix (http://crawler.archive.org) to harvest websites. The robot’s identification field is "User-Agent : Mozilla/5.0 (compatible; bnf.fr_bot; ...)". It always applies high politeness rules (delays between two requests) in order not to stress the producers' servers.
Inquiries
If despite this the performance of your website is affected by this operation, please report it by email to robot@bnf.fr. We will propose a solution as soon as possible.
Tuesday, September 13, 2011