[Netarchivesuite-curator] Netarchive NAS update for October

[Sabine Schostag]

Dear NAS colleagues

Better late than never: here is an update on October/November activities from Netarchivet

Collection policy and strategy
are now published on  http://netarkivet.dk/om-netarkivet/

Broad crawl
we finished our 3. broad crawl
[cid:image001.jpg at 01D11709.D5FDA750]


Cultural heritage Cluster:
On October, 19 the Danish e-Infrastructure Cooperation officially opened the Cultural Heritage Cluster. Now Netarchivet’s users have the possibility to use a super computer on the data in our web archive (big data). Subsequently Netarchivet is invited to participate in a research project with the working title: Humanities - Through the Lenses of Dig Data//The Voices of the Humanities in a Digital world/universe (together with internet researchers)


Heritrix 3:
We are working on the migration to H3, at the same time we want to reduce our number of harvest templates.

Wayback full text search:
Is officially open now. We published the manuals on netarchivet.dk and informed all actual users

RESAW:
We participate in an application (fund raising) from fra Horizon2020, especially the wor packages Research, Access and interoperability, Legal issues.

Event harvests:
We still run an event crawl on the refugee crisis
We are starting an event crawl on the Danish opt-out on Justice and Home affairs
Furthermore we will have a one week event crawl next week: that week is called ”media week”, just a “normal” week, where Statsbiblioteket collects the non-commercial radio and tv-station’s programs. Netarchivet collects their web pages.



PS.

Inspiration

Maybe you already have seen this: the Portuguese web archive has produced a “commercial” Arquivo.pt (vídeo oficial)<https://www.youtube.com/watch?v=2HEudlXPV4o&feature=youtu.be&list=PLKfzD5UuSdETtSCX_TM02nSP7JDmGFGIE>



Best,

SAbine

Sabine Schostag
Web curator
THE NETARCHIVE
[cid:image001.png at 01CFE4A6.C9C92360]STATE AND UNIVERSITY LIBRARY
Victor Albecks Vej 1
8000 AARHUS C
DENMARK
VAT NO. 1010 0682
___________________________________________
http://netarkivet.dk/in-english/

From: Netarchivesuite-curator [mailto:netarchivesuite-curator-bounces at ml.sbforge.org] On Behalf Of peter.stirling at bnf.fr
Sent: Friday, October 02, 2015 2:30 PM
To: netarchivesuite-curator at ml.sbforge.org
Subject: [Netarchivesuite-curator] BnF NAS update for October

Hello all,

That's it! We started our 2015 broad crawl on September 22nd with 4.5 million seed domains.

We need to be careful with the bandwidth: only ten threads are possible by crawler during the day, so we do not go over the 400 Mbits/s allocated for web archiving at the BnF. During the night, each crawler has 250 threads.

For the same attention to bandwidth, we also started the crawl with only five crawlers, and have added by groups of five to go up to 40 crawlers. In case of any problem, we can decrease the number of crawlers in the same way.

Each domain has a maximum budget of 2,300 URL, and each job has a maximum time of three days.

We hope to finish this crawl in two months with a total budget of 60 TB.

Best regards,
The BnF digital legal deposit team
________________________________

Entrez dans l'Histoire des rois de France en participant ? l'acquisition du br?viaire royal de Saint-Louis de Poissy<http://www.bnf.fr/fr/acces_dedies/mecenat_partenariat/s.mecenat_saint_louis.html>

Avant d'imprimer, pensez ? l'environnement.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ml.sbforge.org/pipermail/netarchivesuite-curator/attachments/20151104/3191f5f7/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 20867 bytes
Desc: image001.jpg
URL: <http://ml.sbforge.org/pipermail/netarchivesuite-curator/attachments/20151104/3191f5f7/attachment-0001.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 584 bytes
Desc: image002.png
URL: <http://ml.sbforge.org/pipermail/netarchivesuite-curator/attachments/20151104/3191f5f7/attachment-0001.png>