[Netarchivesuite-users] DNS preparation, domain state active?
clara.wiatrowski at bnf.fr
clara.wiatrowski at bnf.fr
Tue Nov 2 18:20:26 CET 2021
Hi Peter,
It's me. According to BnF an active domain is a domain with a valid
config.
When there is no DNS response we set the default config associated to the
domain like this one :
Max number of objects / bytes (depended on your crawl settings) is set to
zero (and we had a comment).
So NAS won't take the domain.
Do you need more infos ?
Best,
Clara Wiatrowski
De : "Peter Svanberg" <Peter.Svanberg at kb.se>
A : "'netarchivesuite-users at ml.sbforge.org'"
<netarchivesuite-users at ml.sbforge.org>
Date : 02/11/2021 17:11
Objet : [Netarchivesuite-users] DNS preparation, domain state active?
Envoyé par : "NetarchiveSuite-users"
<netarchivesuite-users-bounces at ml.sbforge.org>
Someone (Tue?) talked about a domain having some status, “active” and
others. I can’t find that, please explain.
/Peter
Från: Peter Svanberg
Skickat: den 2 november 2021 09:26
Till: 'netarchivesuite-users at ml.sbforge.org'
<netarchivesuite-users at ml.sbforge.org>
Ämne: DNS in NAS/Heritrik
Hello!
In preparing for a new broad crawl we gather info on new domain names.
Some issues around that.
1) I have a memory of reading about some of you had done some scripts
concerning domains and seeds. What was that about? What around this does
NAS/Heritrix not handle on its own?
2) Well, one thing we’ve found is that NAS assumes that a domain X answers
on URL http://www.X, but that is not true. We found hundreds of domains
which have no www.X host but answers on http://X. Maybe this should be
changed in some way in NAS?
Med vänlig hälsning
Peter Svanberg
Teknisk handläggare
Insamling och metadata
Insamling 1
Kungliga biblioteket
Box 5039, 102 41 Stockholm
Besöksadress: Karlavägen 96, Stockholm
+46 10 709 32 78
Peter.Svanberg at kb.se
www.kb.se
_______________________________________________
NetarchiveSuite-users mailing list
NetarchiveSuite-users at ml.sbforge.org
https://ml.sbforge.org/mailman/listinfo/netarchivesuite-users
Découvrez toute la programmation culturelle de la rentrée à la BnF
Pass BnF lecture/culture : bibliothèques, expositions, conférences, concerts en illimité pour 15 € / an – Acheter en ligne Avant d'imprimer, pensez à l'environnement.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://ml.sbforge.org/pipermail/netarchivesuite-users/attachments/20211102/6fbc4e60/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 12822 bytes
Desc: not available
URL: <https://ml.sbforge.org/pipermail/netarchivesuite-users/attachments/20211102/6fbc4e60/attachment-0001.gif>
More information about the NetarchiveSuite-users
mailing list