[Netarchivesuite-curator] BnF NAS update for January
geraldine.camile at bnf.fr
geraldine.camile at bnf.fr
Wed Jan 8 09:18:38 CET 2020
Dear All,
A Happy New Year and best wishes to all for 2020 from the BnF web
archiving team!
Our broad crawl, which started on October 12th, finished on December 12th.
It represents 2.2 billion URLs and 118.17 TB of compressed data. Despite
technical problems related to our infrastructure (25% of the jobs were
killed by their HarvestController because Heritrix needed too much time to
initialise), it took less time than last year (11 weeks in 2019). Its size
exceeds our initial budget of 110 TB due to an average weight per URL
which is higher than our estimates (from 55421 bytes to 57044 bytes).
We'll analyse the reports to understand this increase: it probably comes
from an evolution of the websites.
We welcome a newcomer in our team: Alexandre Faye. He will be in charge of
cooperation with researchers and international cooperation.
Best regards,
The BnF digital legal deposit team
Exposition Tolkien, voyage en Terre du Milieu - du 22 octobre 2019 au 16 février 2020 - BnF - François-Mitterrand Avant d'imprimer, pensez à l'environnement.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://ml.sbforge.org/pipermail/netarchivesuite-curator/attachments/20200108/c8d5bc1a/attachment.html>
More information about the Netarchivesuite-curator
mailing list