[Netarchivesuite-curator] BnF NAS update for August

[peter.stirling at bnf.fr]

Hello all,

As in previous years, in July we started to work on our annual broad 
crawl. We have asked for seed lists from our different partners 
(registrars, registers and BnF databases). In 2016, we have managed to 
expand the number of domains from TLDs for overseas French departments 
(.gf, .gp, .mq, .pf) and from regional TLDs (.alsace, .bzh, .paris): at 
this point, we have more than 36,000 domain names from these TLDs.

In July and August, we are using our tool nas-preload to deduplicate URLs 
and domains from the seven different identified sources and to check the 
DNS response, with the aim of transferring only active domains into NAS.

We have just started to work on the migration from Heritrix 1 to Heritrix 
3. We plan to achieve the first stage of non-regression at the end of 
February 2017. It is a big challenge as we also have to adjust other tools 
connected to Heritrix 3.

Best regards,
The BnF Digital Legal Deposit team
Expositions : 
Miquel Barceló. Sol y sombra  - du 22 mars 2016 au 28 août 2016 - BnF - François-Mitterrand Avant d'imprimer, pensez à l'environnement. 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ml.sbforge.org/pipermail/netarchivesuite-curator/attachments/20160803/4f9d1192/attachment.html>