[Netarchivesuite-devel] Multiple jobs submitted simultaneously under 5.3.1

[sara.aubry at bnf.fr]

Hi Andreas,

Are your problems coming up because you just launched a broad crawl?

At BnF, we are still running 5.3.0 with default settings on these 
parameters:
settings.harvester.harvesting.sendReadyInterval on 30s 
settings.harvester.harvesting.sendReadyDelay on 1000ms

We are currently testing 5.3.1 on very small crawls (working well)
and we will start bigger crawls next week. I'll let you know
how it goes.

Sara 




De :    <aponb at gmx.at>
A :     <netarchivesuite-devel at ml.sbforge.org>
Date :  28/06/2017 11:43
Objet : [Netarchivesuite-devel] Multiple jobs submitted simultaneously 
under 5.3.1
Envoyé par :    Netarchivesuite-devel 
<netarchivesuite-devel-bounces at ml.sbforge.org>



If was running Nas Version on 5.3.1 in production and did get a huge 
number of jobs with the same Configurations submitted. This must be the 
behavior of https://sbforge.org/jira/browse/NAS-2614 which was fixed for 
Version 5.3.1 - the strange thing is, that I had not any problems in 
Version 5.3.0.

Is anyone experiencing the same issue?
As suggested I set settings.harvester.harvesting.sendReadyInterval to 
300 and I am using settings.harvester.harvesting.sendReadyDelay with 
value 300

Also the HarvestJobManagerApplication dies with OutOfMemory Exception, 
even when started with parameter -Xmx4096m

20:28:11.823 ERROR d.n.c.lifecycle.PeriodicTaskExecutor - Task threw 
exception: java.lang.OutOfMemoryError: Java heap space
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: 
Java heap space
         at java.util.concurrent.FutureTask.report(FutureTask.java:122) 
~[na:1.8.0_77]
         at java.util.concurrent.FutureTask.get(FutureTask.java:192) 
~[na:1.8.0_77]
         at 
dk.netarkivet.common.lifecycle.PeriodicTaskExecutor.checkExecution(PeriodicTaskExecutor.java:171) 

[common-core-5.3.1.jar:UNKNOWN_REVISION]
         at 
dk.netarkivet.common.lifecycle.PeriodicTaskExecutor.access$500(PeriodicTaskExecutor.java:47) 

[common-core-5.3.1.jar:UNKNOWN_REVISION]
         at 
dk.netarkivet.common.lifecycle.PeriodicTaskExecutor$1.run(PeriodicTaskExecutor.java:152) 

[common-core-5.3.1.jar:UNKNOWN_REVISION]

Do you have any thoughts on this?
Regards
a.

_______________________________________________
Netarchivesuite-devel mailing list
Netarchivesuite-devel at ml.sbforge.org
https://ml.sbforge.org/mailman/listinfo/netarchivesuite-devel


Expositions : 
Le monde selon Topor  - jusqu'au 16 juillet 2017 - BnF - François-Mitterrand 
La bibliothèque, la nuit – Bibliothèques mythiques en réalité virtuelle  - jusqu'au 13 août 2017 - BnF - François-Mitterrand Avant d'imprimer, pensez à l'environnement. 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://ml.sbforge.org/pipermail/netarchivesuite-devel/attachments/20170629/18dbb9a7/attachment.html>