[Netarchivesuite-users] NAS handling of "no space left" – just hangs?

[Peter Svanberg]

When a harvester process fills the disk there is only warnings in the Heritrix logfile:

2021-12-11 10:55:56.179 WARNING thread-65 com.google.common.cache.LocalCache.processPendingNotifications() Exception thrown by removal listener
com.sleepycat.je<http://com.sleepycat.je/>.DiskLimitException: (JE 7.5.11) Disk usage is not within je.maxDisk or je.freeDisk limits and write operations are prohibited: maxDiskLimit=0 freeDiskLimit=5,368,709,120 adj
ustedMaxDiskLimit=0 maxDiskOverage=0 freeDiskShortage=1,183,277,056 diskFreeSpace=4,185,432,064 availableLogSize=-1,183,277,056 totalLogSize=1,266 activeLogSize=1,266 reservedLogSize=0 prote
ctedLogSize=0 protectedLogSizeMap={}

NAS doesn't seem to notice that anything has happened. But all Heritrix processes hangs.

Is that expected or are there some parameter, configuration or other error on our part?

What is the best way to reset afterwards? Restart the NAS system on the harvester? Manually tear down hanging jobs? Linux kill?

Also, the origin for filling the disk was a > 150 Gbyte ris file  (RecordingInputStream). Any experience of that? This file seems to just contain nulls. Could a non-zero value on fetchHttp.maxLengthBytes avoid that?

Regards,

[KB Logo]<https://www.kb.se/>

Peter Svanberg
Technical officer

Kungliga biblioteket
Stockholm
+46 10 709 32 78
Peter.Svanberg at kb.se<mailto:Peter.Svanberg at kb.se>
www.kb.se<https://www.kb.se/>




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://ml.sbforge.org/pipermail/netarchivesuite-users/attachments/20211213/d3ca2af2/attachment.html>