[Netarchivesuite-users] NAS - Problems with Broker

Thu May 6 17:40:49 CEST 2021

Hello Miguel,

What version of the broker do you use? 

I cannot explain what happened to your broker, but we once had similar 
problems which occurred during our snapshot crawl. It turned out that the 
version we used then, Message Queue 5.1, was unstable in a NAS 
environment. We changed to Open Message Queue 4.5.2 and we never had 
problems again. So maybe this will help you, too.

Although OpenMQ 4.5.2 is rather old now, it perfectly does its job and 
there are no issues of incompatibility with NAS at all. 

We use this configuration:
imq.jms.min_threads=10
imq.instanceconfig.version=300
imq.autocreate.destination.maxNumProducers=-1
imq.jms.tcp.port=33700
imq.autocreate.queue.maxNumActiveConsumers=100
imq.jms.max_threads=1000

and JVM parameters: -Xms512m -Xmx2048m -Xss512k

So, -Xms24g in your config is really heavy! Are you sure you need that 
much?

Regards,
  Bert
-- 
Ingénieur de production pour l'archivage de l'internet
Département des systèmes d'information
Bibliothèque nationale de France
Quai François-Mauriac
75706 Paris Cedex 13
Tél. : 01 53 79 45 58

De :    "PE-BDH-002" <pebdh002 at bne.es>
A :     "'netarchivesuite-users at ml.sbforge.org'" 
<netarchivesuite-users at ml.sbforge.org>
Cc :    "Monzón, Fernando" <f.monzon at bne.es>
Date :  06/05/2021 12:26
Objet : [Netarchivesuite-users] NAS - Problems with Broker
Envoyé par :    "NetarchiveSuite-users" 
<netarchivesuite-users-bounces at ml.sbforge.org>

Hi!
I’m Miguel Soleto, from the National Library of Spain. I will try to 
explain the problems We are having with the Broker.
First, this is the architecture of our installation (each module on a 
different machine). NAS Version: 5.4.2:
·         4 BitArchives.
·         1 Aggregator.
·         2 Indexers.
·         3 OpenWayback.
·         1 Broker.
·         1 Postgresql database (1 Master and 1 Slave).
·         1 IndexServerApplication and 1 ViewerProxyApplication.
·         1 GUIApplication, 1 ArcRepositoryApplication, 1 
BitarchiveMonitorApplication and 1 HarvestJobManagerApplication.
·         110 Spiders, Working 80, the other 30 doing nothing.

Here are the issues We are experiencing: 
This Sunday (2nd May), the Broker seemed to work, but no Jobs were 
running. With a ps command on the machine, We saw the Java process of the 
Broker, but We had lots of “Connection refused” on the logs of the 
GUIApplication. After that, We killed the Java process (Broker machine) 
and a few minutes later, it seemed to be OK again.
This Tuesday, the Java process of the Broker died with no apparent reason. 
In that moment, We realized that the file “config.properties” were 
overwritten, and every line on that file was commented, except for the 
last one: 
#Last Update:
#Tue May 04 09:22:52 CEST 2021 
imq.instanceconfig.version=300

After that, We re-launched the Java process again, and everything were OK.

Yesterday, We had the same problem: the Broker died with apparently no 
reason.

On the GUIApplication logs, We have the error code “C4056”. I have 
searched for that code, and this is what Oracle says: Cause A Message 
Queue client received a GOOD_BYE message from broker. 

So, We are worried about the performance of the Broker, beacuse when It 
dies, all the platform stops working. We had a BASH script on the Broker 
machine that looks for the process: if It isn’t running, It send us an 
e-mail. 

Has anyone had a problem like this? Any idea about what is happening in 
our NAS installation? May be a memory problem, or just too much conections 
for the Broker? Should I send our “default.properties” file to you? Any 
guide or documentation about the config.properties or default.properties 
parameters?
The Broker process is configured with this parameters: -Xms24g -Xmx24g 
-Xss192m -XX:MaxGCPauseMillis=5000.

Thank you all!
Best regards,
Miguel.
Este mensaje y cualquier fichero adjunto están dirigidos únicamente a sus 
destinatarios y contiene información confidencial. Si usted ha recibido 
este correo electrónico por error, le informamos que no puede realizar 
ninguna revisión, alteración, impresión, copia, transmisión, difusión ni 
utilización alguna de este mensaje ni de cualquier fichero adjunto que 
pudiese contener. La realización de cualquiera de los actos indicados está 
expresamente prohibida por las Normas que regulan estas materias. Por todo 
ello se solicita que, en caso de existir error en la recepción de este 
mensaje, se lo notifique al remitente respondiendo a este e-mail y elimine 
el mensaje y su contenido inmediatamente. La Biblioteca Nacional de España 
se reserva las acciones legales que le correspondan en el caso de que se 
infrinja lo indicado anteriormente. The information in this e-mail and any 
attachments is confidential and it is intended for the addressee only. If 
you have received this e-mail in error, you are notified that any 
revision, amendment, print, copy, disclosure, distribution or use of the 
contents is unauthorized. Carrying out any of the above actions, is 
expressly banned by rules governing this matter. Hence we request that if 
you are not the intended recipient, please notify the sender answering 
this e-mail, and delete the message and any attachments. The National 
Library of Spain reserves itself the right to take the appropriate legal 
actions in the event of the above mentioned matter is being infringed. 
_______________________________________________
NetarchiveSuite-users mailing list
NetarchiveSuite-users at ml.sbforge.org
https://ml.sbforge.org/mailman/listinfo/netarchivesuite-users

En raison des directives gouvernementales liées à la situation sanitaire, les expositions restent fermées jusqu'à nouvelle consigne. Les manifestations culturelles ne peuvent pas accueillir de public mais sont en grande partie  diffusées en ligne . La bibliothèque tous publics est ouverte du mardi au vendredi de 10 h à 17 h. 
Les bibliothèques de recherche sont ouvertes, sur le site François-Mitterrand, le lundi de 14 h à 17 h et du mardi au vendredi de 10 h à 17 h, et, sur les sites Richelieu, Arsenal et Opéra, de 10 h à 17 h du lundi au vendredi.  Consulter les modalités d'accès Avant d'imprimer, pensez à l'environnement. 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://ml.sbforge.org/pipermail/netarchivesuite-users/attachments/20210506/828743d7/attachment-0001.html>