<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
tt
{mso-style-priority:99;
font-family:"Courier New";}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
span.EstiloCorreo21
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 3.0cm 70.85pt 3.0cm;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="ES" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Hi,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Thanks so much for your answer.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">We are using the versión 5.1… So, We’ll try to install the one you told me and see if It works.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">I don’t know the reason for so much memory in our JVM parameters. The person who was on the Project before me, configured It, and he
didn’t leave any notes or documentation. Anyways, We could try with the parameters you said in the new version and I will write again if there is any problem.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Thank you for your help!
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">(:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Miguel.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">De:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> bert.wendland@bnf.fr <bert.wendland@bnf.fr>
<br>
<b>Enviado el:</b> jueves, 6 de mayo de 2021 17:41<br>
<b>Para:</b> netarchivesuite-users@ml.sbforge.org<br>
<b>CC:</b> Monzón, Fernando <f.monzon@bne.es>; PE-BDH-002 <pebdh002@bne.es><br>
<b>Asunto:</b> RE: [Netarchivesuite-users] NAS - Problems with Broker<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">Hello Miguel,</span><br>
<br>
<span style="font-size:10.0pt;font-family:"Arial",sans-serif">What version of the broker do you use?
</span><br>
<br>
<span style="font-size:10.0pt;font-family:"Arial",sans-serif">I cannot explain what happened to your broker, but we once had similar problems which occurred during our snapshot crawl. It turned out that the version we used then, Message Queue 5.1, was unstable
in a NAS environment. We changed to Open Message Queue 4.5.2 and we never had problems again. So maybe this will help you, too.</span><br>
<br>
<span style="font-size:10.0pt;font-family:"Arial",sans-serif">Although OpenMQ 4.5.2 is rather old now, it perfectly does its job and there are no issues of incompatibility with NAS at all.
</span><br>
<br>
<span style="font-size:10.0pt;font-family:"Arial",sans-serif">We use this configuration:</span><br>
<span style="font-size:10.0pt;font-family:"Arial",sans-serif">imq.jms.min_threads=10</span><br>
<span style="font-size:10.0pt;font-family:"Arial",sans-serif">imq.instanceconfig.version=300</span><br>
<span style="font-size:10.0pt;font-family:"Arial",sans-serif">imq.autocreate.destination.maxNumProducers=-1</span><br>
<span style="font-size:10.0pt;font-family:"Arial",sans-serif">imq.jms.tcp.port=33700</span><br>
<span style="font-size:10.0pt;font-family:"Arial",sans-serif">imq.autocreate.queue.maxNumActiveConsumers=100</span><br>
<span style="font-size:10.0pt;font-family:"Arial",sans-serif">imq.jms.max_threads=1000</span><br>
<br>
<span style="font-size:10.0pt;font-family:"Arial",sans-serif">and JVM parameters: -Xms512m -Xmx2048m -Xss512k</span><br>
<br>
<span style="font-size:10.0pt;font-family:"Arial",sans-serif">So, -Xms24g in your config is really heavy! Are you sure you need that much?</span><br>
<br>
<span style="font-size:10.0pt;font-family:"Arial",sans-serif">Regards,</span><br>
<span style="font-size:10.0pt;font-family:"Arial",sans-serif"> Bert</span><br>
<span style="font-size:10.0pt;font-family:"Arial",sans-serif">-- <br>
Ingénieur de production pour l'archivage de l'internet<br>
Département des systèmes d'information<br>
Bibliothèque nationale de France<br>
Quai François-Mauriac<br>
75706 Paris Cedex 13<br>
Tél. : 01 53 79 45 58</span><br>
<br>
<br>
<br>
<br>
<span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#5F5F5F">De : </span><span style="font-size:7.5pt;font-family:"Arial",sans-serif">"PE-BDH-002" <<a href="mailto:pebdh002@bne.es">pebdh002@bne.es</a>></span><br>
<span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#5F5F5F">A : </span><span style="font-size:7.5pt;font-family:"Arial",sans-serif">"'netarchivesuite-users@ml.sbforge.org'" <<a href="mailto:netarchivesuite-users@ml.sbforge.org">netarchivesuite-users@ml.sbforge.org</a>></span><br>
<span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#5F5F5F">Cc : </span><span style="font-size:7.5pt;font-family:"Arial",sans-serif">"Monzón, Fernando" <<a href="mailto:f.monzon@bne.es">f.monzon@bne.es</a>></span><br>
<span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#5F5F5F">Date : </span><span style="font-size:7.5pt;font-family:"Arial",sans-serif">06/05/2021 12:26</span><br>
<span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#5F5F5F">Objet : </span><span style="font-size:7.5pt;font-family:"Arial",sans-serif">[Netarchivesuite-users] NAS - Problems with Broker</span><br>
<span style="font-size:7.5pt;font-family:"Arial",sans-serif;color:#5F5F5F">Envoyé par : </span><span style="font-size:7.5pt;font-family:"Arial",sans-serif">"NetarchiveSuite-users" <<a href="mailto:netarchivesuite-users-bounces@ml.sbforge.org">netarchivesuite-users-bounces@ml.sbforge.org</a>></span><o:p></o:p></p>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="100%" noshade="" style="color:#A0A0A0" align="center">
</div>
<p class="MsoNormal"><br>
<br>
<br>
<span style="font-family:"Calibri",sans-serif">Hi!</span><br>
<span style="font-family:"Calibri",sans-serif">I’m Miguel Soleto, from the National Library of Spain. I will try to explain the problems We are having with the Broker.</span><br>
<span style="font-family:"Calibri",sans-serif">First, this is the architecture of our installation (each module on a different machine). NAS Version: 5.4.2:</span><br>
<span style="font-family:Symbol">· </span> <span style="font-family:Symbol"> </span>
<span style="font-family:Symbol"> </span> <span style="font-family:Symbol"> </span>
<span style="font-family:Symbol"> </span><span style="font-family:"Calibri",sans-serif">4 BitArchives.</span><br>
<span style="font-family:Symbol">· </span> <span style="font-family:Symbol"> </span>
<span style="font-family:Symbol"> </span> <span style="font-family:Symbol"> </span>
<span style="font-family:Symbol"> </span><span style="font-family:"Calibri",sans-serif">1 Aggregator.</span><br>
<span style="font-family:Symbol">· </span> <span style="font-family:Symbol"> </span>
<span style="font-family:Symbol"> </span> <span style="font-family:Symbol"> </span>
<span style="font-family:Symbol"> </span><span style="font-family:"Calibri",sans-serif">2 Indexers.</span><br>
<span style="font-family:Symbol">· </span> <span style="font-family:Symbol"> </span>
<span style="font-family:Symbol"> </span> <span style="font-family:Symbol"> </span>
<span style="font-family:Symbol"> </span><span style="font-family:"Calibri",sans-serif">3 OpenWayback.</span><br>
<span style="font-family:Symbol">· </span> <span style="font-family:Symbol"> </span>
<span style="font-family:Symbol"> </span> <span style="font-family:Symbol"> </span>
<span style="font-family:Symbol"> </span><span style="font-family:"Calibri",sans-serif">1 Broker.</span><br>
<span style="font-family:Symbol">· </span> <span style="font-family:Symbol"> </span>
<span style="font-family:Symbol"> </span> <span style="font-family:Symbol"> </span>
<span style="font-family:Symbol"> </span><span style="font-family:"Calibri",sans-serif">1 Postgresql database (1 Master and 1 Slave).</span><br>
<span style="font-family:Symbol">· </span> <span style="font-family:Symbol"> </span>
<span style="font-family:Symbol"> </span> <span style="font-family:Symbol"> </span>
<span style="font-family:Symbol"> </span><span style="font-family:"Calibri",sans-serif">1 IndexServerApplication and 1 ViewerProxyApplication.</span><br>
<span style="font-family:Symbol">· </span> <span style="font-family:Symbol"> </span>
<span style="font-family:Symbol"> </span> <span style="font-family:Symbol"> </span>
<span style="font-family:Symbol"> </span><span style="font-family:"Calibri",sans-serif">1 GUIApplication, 1 ArcRepositoryApplication, 1 BitarchiveMonitorApplication and 1 HarvestJobManagerApplication.</span><br>
<span style="font-family:Symbol">· </span> <span style="font-family:Symbol"> </span>
<span style="font-family:Symbol"> </span> <span style="font-family:Symbol"> </span>
<span style="font-family:Symbol"> </span><span style="font-family:"Calibri",sans-serif">110 Spiders, Working 80, the other 30 doing nothing.</span><br>
<span style="font-family:"Calibri",sans-serif"> </span><br>
<span style="font-family:"Calibri",sans-serif">Here are the issues We are experiencing:
</span><br>
<span style="font-family:"Calibri",sans-serif">This Sunday (2nd May), the Broker seemed to work, but no Jobs were running. With a ps command on the machine, We saw the Java process of the Broker, but We had lots of “Connection refused” on the logs of the GUIApplication.
After that, We killed the Java process (Broker machine) and a few minutes later, it seemed to be OK again.</span><br>
<span style="font-family:"Calibri",sans-serif">This Tuesday, the Java process of the Broker died with no apparent reason. In that moment, We realized that the file “config.properties” were overwritten, and every line on that file was commented, except for the
last one: </span><br>
<span style="font-family:"Calibri",sans-serif">#Last Update:</span><br>
<span style="font-family:"Calibri",sans-serif">#Tue May 04 09:22:52 CEST 2021 </span>
<br>
<b><span style="font-family:"Calibri",sans-serif">imq.instanceconfig.version=300</span></b><br>
<span style="font-family:"Calibri",sans-serif"> </span><br>
<span style="font-family:"Calibri",sans-serif">After that, We re-launched the Java process again, and everything were OK.</span><br>
<span style="font-family:"Calibri",sans-serif"> </span><br>
<span style="font-family:"Calibri",sans-serif">Yesterday, We had the same problem: the Broker died with apparently no reason.</span><br>
<span style="font-family:"Calibri",sans-serif"> </span><br>
<span style="font-family:"Calibri",sans-serif">On the GUIApplication logs, We have the error code “C4056”. I have searched for that code, and this is what Oracle says:
<b>Cause A Message Queue client received a GOOD_BYE message from broker. </b></span><br>
<b><span style="font-family:"Calibri",sans-serif"> </span></b><br>
<span style="font-family:"Calibri",sans-serif">So, We are worried about the performance of the Broker, beacuse when It dies, all the platform stops working. We had a BASH script on the Broker machine that looks for the process: if It isn’t running, It send
us an e-mail. </span><br>
<span style="font-family:"Calibri",sans-serif"> </span><br>
<span style="font-family:"Calibri",sans-serif">Has anyone had a problem like this? Any idea about what is happening in our NAS installation? May be a memory problem, or just too much conections for the Broker? Should I send our “default.properties” file to
you? Any guide or documentation about the config.properties or default.properties parameters?</span><br>
<span style="font-family:"Calibri",sans-serif">The Broker process is configured with this parameters: -Xms24g -Xmx24g -Xss192m -XX:MaxGCPauseMillis=5000.</span><br>
<span style="font-family:"Calibri",sans-serif"> </span><br>
<span style="font-family:"Calibri",sans-serif">Thank you all!</span><br>
<span style="font-family:"Calibri",sans-serif">Best regards,</span><br>
<span style="font-family:"Calibri",sans-serif">Miguel.</span><o:p></o:p></p>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="100%" align="center">
</div>
<p class="MsoNormal"><span style="font-size:7.5pt">Este mensaje y cualquier fichero adjunto están dirigidos únicamente a sus destinatarios y contiene información confidencial. Si usted ha recibido este correo electrónico por error, le informamos que no puede
realizar ninguna revisión, alteración, impresión, copia, transmisión, difusión ni utilización alguna de este mensaje ni de cualquier fichero adjunto que pudiese contener. La realización de cualquiera de los actos indicados está expresamente prohibida por las
Normas que regulan estas materias. Por todo ello se solicita que, en caso de existir error en la recepción de este mensaje, se lo notifique al remitente respondiendo a este e-mail y elimine el mensaje y su contenido inmediatamente. La Biblioteca Nacional de
España se reserva las acciones legales que le correspondan en el caso de que se infrinja lo indicado anteriormente.</span><o:p></o:p></p>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="100%" align="center">
</div>
<p class="MsoNormal"><span style="font-size:7.5pt">The information in this e-mail and any attachments is confidential and it is intended for the addressee only. If you have received this e-mail in error, you are notified that any revision, amendment, print,
copy, disclosure, distribution or use of the contents is unauthorized. Carrying out any of the above actions, is expressly banned by rules governing this matter. Hence we request that if you are not the intended recipient, please notify the sender answering
this e-mail, and delete the message and any attachments. The National Library of Spain reserves itself the right to take the appropriate legal actions in the event of the above mentioned matter is being infringed.</span>
<o:p></o:p></p>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="100%" align="center">
</div>
<p class="MsoNormal"><tt><span style="font-size:10.0pt">_______________________________________________</span></tt><span style="font-size:10.0pt;font-family:"Courier New""><br>
<tt>NetarchiveSuite-users mailing list</tt><br>
<tt><a href="mailto:NetarchiveSuite-users@ml.sbforge.org">NetarchiveSuite-users@ml.sbforge.org</a></tt><br>
</span><a href="https://ml.sbforge.org/mailman/listinfo/netarchivesuite-users"><tt><span style="font-size:10.0pt">https://ml.sbforge.org/mailman/listinfo/netarchivesuite-users</span></tt></a><o:p></o:p></p>
<div class="MsoNormal" align="center" style="text-align:center"><span style="font-family:"Arial",sans-serif">
<hr size="2" width="100%" align="center">
</span></div>
<p><span style="font-family:"Arial",sans-serif">En raison des directives gouvernementales liées à la situation sanitaire, les expositions restent fermées jusqu'à nouvelle consigne. Les manifestations culturelles ne peuvent pas accueillir de public mais sont
en grande partie <a href="https://www.bnf.fr/fr/actualites/retrouvez-les-conferences-de-janvier-2021-en-ligne">
diffusées en ligne</a>.<o:p></o:p></span></p>
<p><span style="font-family:"Arial",sans-serif">La bibliothèque tous publics est ouverte du mardi au vendredi de 10 h à 17 h.<br>
Les bibliothèques de recherche sont ouvertes, sur le site François-Mitterrand, le lundi de 14 h à 17 h et du mardi au vendredi de 10 h à 17 h, et, sur les sites Richelieu, Arsenal et Opéra, de 10 h à 17 h du lundi au vendredi.
<a href="https://www.bnf.fr/fr/reouverture-des-salles-de-lecture-de-la-bnf-modalites-dacces">
Consulter les modalités d'accès</a><o:p></o:p></span></p>
<p><strong><span style="font-family:"Arial",sans-serif;color:green">Avant d'imprimer, pensez à l'environnement.</span></strong><span style="font-family:"Arial",sans-serif;color:green"><o:p></o:p></span></p>
</div>
<hr width="100%">
<font size="1">Este mensaje y cualquier fichero adjunto están dirigidos únicamente a sus destinatarios y contiene información confidencial. Si usted ha recibido este correo electrónico por error, le informamos que no puede realizar ninguna revisión, alteración,
impresión, copia, transmisión, difusión ni utilización alguna de este mensaje ni de cualquier fichero adjunto que pudiese contener. La realización de cualquiera de los actos indicados está expresamente prohibida por las Normas que regulan estas materias. Por
todo ello se solicita que, en caso de existir error en la recepción de este mensaje, se lo notifique al remitente respondiendo a este e-mail y elimine el mensaje y su contenido inmediatamente. La Biblioteca Nacional de España se reserva las acciones legales
que le correspondan en el caso de que se infrinja lo indicado anteriormente.</font>
<hr width="100%">
<font size="1">The information in this e-mail and any attachments is confidential and it is intended for the addressee only. If you have received this e-mail in error, you are notified that any revision, amendment, print, copy, disclosure, distribution or use
of the contents is unauthorized. Carrying out any of the above actions, is expressly banned by rules governing this matter. Hence we request that if you are not the intended recipient, please notify the sender answering this e-mail, and delete the message
and any attachments. The National Library of Spain reserves itself the right to take the appropriate legal actions in the event of the above mentioned matter is being infringed.</font>
<hr width="100%">
</body>
</html>