<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body>
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif;" dir="ltr">
<p>I've created an issue <a href="https://sbforge.org/jira/browse/NAS-2682" class="OWAAutoLink" id="LPlnk879065" previewremoved="true">https://sbforge.org/jira/browse/NAS-2682</a> for this and I have some suspicions about whose code might be responsible for
the problem, although right now I can't see anything obviously wrong.</p>
<p><br>
</p>
<div id="Signature">
<div name="divtagdefaultwrapper" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:; margin:0">
<div>--</div>
<div>Colin Rosenthal PhD</div>
<div>Senior IT Consultant</div>
<div>Royal Danish Library (Aarhus)</div>
</div>
</div>
</div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Netarchivesuite-devel <netarchivesuite-devel-bounces@ml.sbforge.org> on behalf of sara.aubry@bnf.fr <sara.aubry@bnf.fr><br>
<b>Sent:</b> Tuesday, July 11, 2017 11:18:10 AM<br>
<b>To:</b> netarchivesuite-devel@ml.sbforge.org<br>
<b>Subject:</b> Re: [Netarchivesuite-devel] Multiple jobs submitted simultaneously under 5.3.1</font>
<div> </div>
</div>
<div><font size="2" face="sans-serif">Hi everyone,</font><br>
<br>
<font size="2" face="sans-serif">Just a quick note to let you know that we have launched a broad crawl test with 5.3.1 at the end of last week.</font><br>
<font size="2" face="sans-serif">And everything went smooth: we generated 872 jobs, ran 20 of them using 10 crawlers, job status are consistent</font><br>
<font size="2" face="sans-serif">and there is nothing wrong with the broker.</font><br>
<br>
<font size="2" face="sans-serif">We have the following configuration:</font><br>
<font size="2" face="sans-serif">- CentOS 7.3 (which seems to be similar to Red Hat 4.8)</font><br>
<font size="2" face="sans-serif">- Java(TM) SE Runtime Environment (build 1.8.0_40-b25) 64-Bit</font><br>
<font size="2" face="sans-serif">- OpenMQ (MessageQueue5.1)</font><br>
<br>
<br>
<font size="2" face="sans-serif">Maybe more important, we are using this configuration on the scheduler.</font><br>
<font size="2" face="sans-serif"> <scheduler></font><br>
<font size="2" face="sans-serif"> <jobtimeouttime>31536000</jobtimeouttime></font><br>
<font size="2" face="sans-serif"> <jobgenerationperiode>60</jobgenerationperiode></font><br>
<font size="2" face="sans-serif"> <jobGen></font><br>
<font size="2" face="sans-serif"> <class>dk.netarkivet.harvester.scheduler.jobgen.FixedDomainConfigurationCountJobGenerator</class></font><br>
<font size="2" face="sans-serif"> <objectLimitIsSetByQuotaEnforcer>false</objectLimitIsSetByQuotaEnforcer></font><br>
<font size="2" face="sans-serif"> <domainConfigSubsetSize>5000</domainConfigSubsetSize></font><br>
<font size="2" face="sans-serif"> <config></font><br>
<font size="2" face="sans-serif"> <fixedDomainCountSnapshot>5000</fixedDomainCountSnapshot></font><br>
<font size="2" face="sans-serif"> <fixedDomainCountFocused>500</fixedDomainCountFocused></font><br>
<font size="2" face="sans-serif"> <excludeDomainsWithZeroBudget>true</excludeDomainsWithZeroBudget></font><br>
<font size="2" face="sans-serif"> <postponeUnregisteredChannel>false</postponeUnregisteredChannel></font><br>
<font size="2" face="sans-serif"> </config></font><br>
<font size="2" face="sans-serif"> </jobGen></font><br>
<font size="2" face="sans-serif"> </scheduler></font><br>
<br>
<font size="2" face="sans-serif">If I remember well, at KB and ONB, you are using a different job generator that tries to make homogenous jobs sizes based</font><br>
<font size="2" face="sans-serif">on the previous harvest. The one we are using is making jobs taking the domains in alphabetical order.</font><br>
<br>
<font size="2" face="sans-serif">Hope this help,</font><br>
<br>
<font size="2" face="sans-serif">Sara</font><br>
<br>
<br>
<br>
<font size="1" color="#5f5f5f" face="sans-serif">De : </font><font size="1" face="sans-serif"><aponb@gmx.at></font><br>
<font size="1" color="#5f5f5f" face="sans-serif">A : </font><font size="1" face="sans-serif"><netarchivesuite-devel@ml.sbforge.org></font><br>
<font size="1" color="#5f5f5f" face="sans-serif">Date : </font><font size="1" face="sans-serif">29/06/2017 11:13</font><br>
<font size="1" color="#5f5f5f" face="sans-serif">Objet : </font><font size="1" face="sans-serif">Re: [Netarchivesuite-devel] Multiple jobs submitted simultaneously under 5.3.1</font><br>
<font size="1" color="#5f5f5f" face="sans-serif">Envoyé par : </font><font size="1" face="sans-serif">Netarchivesuite-devel <netarchivesuite-devel-bounces@ml.sbforge.org></font><br>
<hr noshade="">
<br>
<br>
<br>
<font size="3">Hi Sara,<br>
<br>
I forgot to mention that the problems were coming up with our daily crawls. The intention was to deploy 5.3.1, waiting for some daily crawls, before starting the broad crawl.<br>
<br>
Thanks for your settings and for telling how your broad crawl will work!<br>
</font><br>
<font size="2" face="Arial">Hi Andreas,</font><font size="3"><br>
</font><font size="2" face="Arial"><br>
Are your problems coming up because you just launched a broad crawl?</font><font size="3"><br>
</font><font size="2" face="Arial"><br>
At BnF, we are still running 5.3.0 with default settings on these parameters:<br>
settings.harvester.harvesting.sendReadyInterval on 30s <br>
settings.harvester.harvesting.sendReadyDelay on 1000ms</font><font size="3"><br>
</font><font size="2" face="Arial"><br>
We are currently testing 5.3.1 on very small crawls (working well)<br>
and we will start bigger crawls next week. I'll let you know<br>
how it goes.</font><font size="3"><br>
</font><font size="2" face="Arial"><br>
Sara </font><font size="3"><br>
<br>
<br>
<br>
</font><font size="1" color="#5f5f5f" face="sans-serif"><br>
De : </font><a href="mailto:aponb@gmx.at"><font size="1" color="blue" face="sans-serif"><u><aponb@gmx.at></u></font></a><font size="1" color="#5f5f5f" face="sans-serif"><br>
A : </font><a href="mailto:netarchivesuite-devel@ml.sbforge.org"><font size="1" color="blue" face="sans-serif"><u><netarchivesuite-devel@ml.sbforge.org></u></font></a><font size="1" color="#5f5f5f" face="sans-serif"><br>
Date : </font><font size="1" face="sans-serif">28/06/2017 11:43</font><font size="1" color="#5f5f5f" face="sans-serif"><br>
Objet : </font><font size="1" face="sans-serif">[Netarchivesuite-devel] Multiple jobs submitted simultaneously under 5.3.1</font><font size="1" color="#5f5f5f" face="sans-serif"><br>
Envoyé par : </font><font size="1" face="sans-serif">Netarchivesuite-devel
</font><a href="mailto:netarchivesuite-devel-bounces@ml.sbforge.org"><font size="1" color="blue" face="sans-serif"><u><netarchivesuite-devel-bounces@ml.sbforge.org></u></font></a><font size="3"><br>
</font>
<hr noshade="">
<font size="3"><br>
<br>
</font><tt><font size="2"><br>
If was running Nas Version on 5.3.1 in production and did get a huge <br>
number of jobs with the same Configurations submitted. This must be the <br>
behavior of </font></tt><a href="https://sbforge.org/jira/browse/NAS-2614"><tt><font size="2" color="blue"><u>https://sbforge.org/jira/browse/NAS-2614</u></font></tt></a><tt><font size="2">which was fixed for
<br>
Version 5.3.1 - the strange thing is, that I had not any problems in <br>
Version 5.3.0.<br>
<br>
Is anyone experiencing the same issue?<br>
As suggested I set settings.harvester.harvesting.sendReadyInterval to <br>
300 and I am using settings.harvester.harvesting.sendReadyDelay with <br>
value 300<br>
<br>
Also the HarvestJobManagerApplication dies with OutOfMemory Exception, <br>
even when started with parameter -Xmx4096m<br>
<br>
20:28:11.823 ERROR d.n.c.lifecycle.PeriodicTaskExecutor - Task threw <br>
exception: java.lang.OutOfMemoryError: Java heap space<br>
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: <br>
Java heap space<br>
at java.util.concurrent.FutureTask.report(FutureTask.java:122) <br>
~[na:1.8.0_77]<br>
at java.util.concurrent.FutureTask.get(FutureTask.java:192) <br>
~[na:1.8.0_77]<br>
at <br>
dk.netarkivet.common.lifecycle.PeriodicTaskExecutor.checkExecution(PeriodicTaskExecutor.java:171)
<br>
[common-core-5.3.1.jar:UNKNOWN_REVISION]<br>
at <br>
dk.netarkivet.common.lifecycle.PeriodicTaskExecutor.access$500(PeriodicTaskExecutor.java:47)
<br>
[common-core-5.3.1.jar:UNKNOWN_REVISION]<br>
at <br>
dk.netarkivet.common.lifecycle.PeriodicTaskExecutor$1.run(PeriodicTaskExecutor.java:152)
<br>
[common-core-5.3.1.jar:UNKNOWN_REVISION]<br>
<br>
Do you have any thoughts on this?<br>
Regards<br>
a.<br>
<br>
_______________________________________________<br>
Netarchivesuite-devel mailing list</font></tt><tt><font size="2" color="blue"><u><br>
</u></font></tt><a href="mailto:Netarchivesuite-devel@ml.sbforge.org"><tt><font size="2" color="blue"><u>Netarchivesuite-devel@ml.sbforge.org</u></font></tt></a><font size="3" color="blue"><u><br>
</u></font><a href="https://ml.sbforge.org/mailman/listinfo/netarchivesuite-devel"><tt><font size="2" color="blue"><u>https://ml.sbforge.org/mailman/listinfo/netarchivesuite-devel</u></font></tt></a><font size="3"><br>
</font><font size="3" face="sans-serif"><br>
</font>
<hr>
<p><font size="3" face="sans-serif">Expositions :</font><font size="3" color="blue" face="sans-serif"><b><i><u><br>
</u></i></b></font><a href="http://www.bnf.fr/fr/evenements_et_culture/anx_expositions/f.monde_topor.html"><font size="3" color="blue" face="sans-serif"><b><i><u>Le monde selon Topor</u></i></b></font></a><font size="3" face="sans-serif">- jusqu'au 16 juillet
2017 - BnF - François-Mitterrand</font><font size="3" color="blue" face="sans-serif"><b><i><u><br>
</u></i></b></font><a href="http://www.bnf.fr/fr/evenements_et_culture/anx_expositions/f.bibliotheque_la_nuit.html"><font size="3" color="blue" face="sans-serif"><b><i><u>La bibliothèque, la nuit – Bibliothèques mythiques en réalité virtuelle
</u></i></b></font></a><font size="3" face="sans-serif">- jusqu'au 13 août 2017 - BnF - François-Mitterrand</font></p>
<p><font size="3" color="#008000" face="sans-serif"><b>Avant d'imprimer, pensez à l'environnement.</b></font></p>
<p><font size="3"><br>
</font><br>
<tt><font size="3">_______________________________________________<br>
Netarchivesuite-devel mailing list<br>
</font></tt><a href="mailto:Netarchivesuite-devel@ml.sbforge.org"><tt><font size="3" color="blue"><u>Netarchivesuite-devel@ml.sbforge.org</u></font></tt></a><tt><font size="3"><br>
</font></tt><a href="https://ml.sbforge.org/mailman/listinfo/netarchivesuite-devel"><tt><font size="3" color="blue"><u>https://ml.sbforge.org/mailman/listinfo/netarchivesuite-devel</u></font></tt></a><tt><font size="3"><br>
</font></tt></p>
<p><tt><font size="2">_______________________________________________<br>
Netarchivesuite-devel mailing list<br>
Netarchivesuite-devel@ml.sbforge.org<br>
</font></tt><a href="https://ml.sbforge.org/mailman/listinfo/netarchivesuite-devel"><tt><font size="2">https://ml.sbforge.org/mailman/listinfo/netarchivesuite-devel</font></tt></a><tt><font size="2"><br>
</font></tt></p>
<p><font face="sans-serif"></p>
<hr>
<p>Expositions :<br>
<strong><em><a href="http://www.bnf.fr/fr/evenements_et_culture/anx_expositions/f.monde_topor.html">Le monde selon Topor</a></em></strong> - jusqu'au 16 juillet 2017 - BnF - François-Mitterrand<br>
<strong><em><a href="http://www.bnf.fr/fr/evenements_et_culture/anx_expositions/f.bibliotheque_la_nuit.html">La bibliothèque, la nuit – Bibliothèques mythiques en réalité virtuelle
</a></em></strong>- jusqu'au 13 août 2017 - BnF - François-Mitterrand</p>
<p style="color:#008000"><strong>Avant d'imprimer, pensez à l'environnement.</strong></p>
</font></div>
</body>
</html>