[Netarchivesuite-users] Number of domains/job recommendation?

[Peter Svanberg]

Another short question: what do you think is a good level (*) on the number of domains per job on a broad crawl?

We started a crawl being unaware of that we had a strange value on maxTotalSize. That lead to allmost 13000 jobs with on average 37 domains in each – not so good. (But it made us learn how to stop a broad crawl.)

What should we choose?

Regards,

Peter Svanberg
National library of Sweden

(*) I would have liked to use the very Swedish word "lagom" here ...
https://en.wikipedia.org/wiki/Lagom

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://ml.sbforge.org/pipermail/netarchivesuite-users/attachments/20200124/04b86917/attachment.html>