[Netarchivesuite-users] Lots of -50 status codes

[Peter Svanberg]

Hello all!

I get quite a lot of -50 status lines in my crawl.log (test snapshot runs, NetarchiveSuite 5.5). I haven't seen any pattern in which kind of URL:s that gets this - sometimes it is URL:s which redirects to entrance/top level page (should be banned!), sometimes it's quite ordinary URL:s, often to images, I think. And manual fetching later always works.

One pattern is that it seems as if it either

*         all requests to a certain host are -50, or

*         first a series of 200 and then ditto -50
so they are not intermixed. That could imply some problem - or automatic blocking after a while? - on the host.

What are your experiences?

-----

Peter Svanberg

National Library of Sweden
Phone: +46 10 709 32 78

E-mail: peter.svanberg at kb.se
Web: www.kb.se



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://ml.sbforge.org/pipermail/netarchivesuite-users/attachments/20190614/2c0dbdc3/attachment.html>