[Netarchivesuite-curator] BnF NAS update for June

[auriane.quoix at bnf.fr]

Dear all,

The preparations for the broad crawl have begun. We started working on 
several projects:
- the problem of false URLs 404 error
- blacklists and DNS problems
We are currently carrying out an inventory of these identified problems 
and we are considering several options at this stage to try to resolve the 
problem of URLs with 404 error, such as blocking links from javascript 
libraries which pose a problem, rethinking the use of javascript parsers 
and opening the discussion within IIPC about these 404 error URLs.
Regarding blacklists, we plan to identify and to contact platforms that 
list aggressive IPs and we will see if it would be possible to modify the 
frequency of Heritrix requests every 6 hours.

The Official Publications and Auction House harvests will be launched this 
month. The first one contains both websites of ministries, public 
establishments, independent administrative authorities and local 
authorities and the second one concerns French auction houses websites. 
More than 1300 and 200 sites have respectively been selected for a budget 
of approximately 6.5 and 1 TB.

Best regards,

The BnF digital legal deposit team

Expositions  L’invention de la Renaissance – L’humaniste, le prince et l’artiste  jusqu'au 16 juin 2024 | Richelieu –  La France sous leurs yeux – 200 regards de photographes sur les années 2020  du 19 mars au 23 juin 2024 | François-Mitterrand Avant d'imprimer, pensez à l'environnement. 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://ml.sbforge.org/pipermail/netarchivesuite-curator/attachments/20240604/d9b06a7d/attachment.html>