[Netarchivesuite-users] Number of domains/job recommendation?
Peter.Svanberg at kb.se
Fri Jan 24 09:14:40 CET 2020
Another short question: what do you think is a good level (*) on the number of domains per job on a broad crawl?
We started a crawl being unaware of that we had a strange value on maxTotalSize. That lead to allmost 13000 jobs with on average 37 domains in each – not so good. (But it made us learn how to stop a broad crawl.)
What should we choose?
National library of Sweden
(*) I would have liked to use the very Swedish word "lagom" here ...
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the NetarchiveSuite-users