[Netarchivesuite-users] NAS - Problems with Broker

Fri May 7 09:24:16 CEST 2021

Hi,
Thanks so much for your answer.
We are using the versión 5.1… So, We’ll try to install the one you told me and see if It works.
I don’t know the reason for so much memory in our JVM parameters. The person who was on the Project before me, configured It, and he didn’t leave any notes or documentation. Anyways, We could try with the parameters you said in the new version and I will write again if there is any problem.

Thank you for your help!
(:

Regards,
Miguel.

De: bert.wendland at bnf.fr <bert.wendland at bnf.fr>
Enviado el: jueves, 6 de mayo de 2021 17:41
Para: netarchivesuite-users at ml.sbforge.org
CC: Monzón, Fernando <f.monzon at bne.es>; PE-BDH-002 <pebdh002 at bne.es>
Asunto: RE: [Netarchivesuite-users] NAS - Problems with Broker

Hello Miguel,

What version of the broker do you use?

I cannot explain what happened to your broker, but we once had similar problems which occurred during our snapshot crawl. It turned out that the version we used then, Message Queue 5.1, was unstable in a NAS environment. We changed to Open Message Queue 4.5.2 and we never had problems again. So maybe this will help you, too.

Although OpenMQ 4.5.2 is rather old now, it perfectly does its job and there are no issues of incompatibility with NAS at all.

We use this configuration:
imq.jms.min_threads=10
imq.instanceconfig.version=300
imq.autocreate.destination.maxNumProducers=-1
imq.jms.tcp.port=33700
imq.autocreate.queue.maxNumActiveConsumers=100
imq.jms.max_threads=1000

and JVM parameters: -Xms512m -Xmx2048m -Xss512k

So, -Xms24g in your config is really heavy! Are you sure you need that much?

Regards,
  Bert
--
Ingénieur de production pour l'archivage de l'internet
Département des systèmes d'information
Bibliothèque nationale de France
Quai François-Mauriac
75706 Paris Cedex 13
Tél. : 01 53 79 45 58

De :        "PE-BDH-002" <pebdh002 at bne.es<mailto:pebdh002 at bne.es>>
A :        "'netarchivesuite-users at ml.sbforge.org'" <netarchivesuite-users at ml.sbforge.org<mailto:netarchivesuite-users at ml.sbforge.org>>
Cc :        "Monzón, Fernando" <f.monzon at bne.es<mailto:f.monzon at bne.es>>
Date :        06/05/2021 12:26
Objet :        [Netarchivesuite-users] NAS - Problems with Broker
Envoyé par :        "NetarchiveSuite-users" <netarchivesuite-users-bounces at ml.sbforge.org<mailto:netarchivesuite-users-bounces at ml.sbforge.org>>
________________________________

Hi!
I’m Miguel Soleto, from the National Library of Spain. I will try to explain the problems We are having with the Broker.
First, this is the architecture of our installation (each module on a different machine). NAS Version: 5.4.2:
•         4 BitArchives.
•         1 Aggregator.
•         2 Indexers.
•         3 OpenWayback.
•         1 Broker.
•         1 Postgresql database (1 Master and 1 Slave).
•         1 IndexServerApplication and 1 ViewerProxyApplication.
•         1 GUIApplication, 1 ArcRepositoryApplication, 1 BitarchiveMonitorApplication and 1 HarvestJobManagerApplication.
•         110 Spiders, Working 80, the other 30 doing nothing.

Here are the issues We are experiencing:
This Sunday (2nd May), the Broker seemed to work, but no Jobs were running. With a ps command on the machine, We saw the Java process of the Broker, but We had lots of “Connection refused” on the logs of the GUIApplication. After that, We killed the Java process (Broker machine) and a few minutes later, it seemed to be OK again.
This Tuesday, the Java process of the Broker died with no apparent reason. In that moment, We realized that the file “config.properties” were overwritten, and every line on that file was commented, except for the last one:
#Last Update:
#Tue May 04 09:22:52 CEST 2021
imq.instanceconfig.version=300

After that, We re-launched the Java process again, and everything were OK.

Yesterday, We had the same problem: the Broker died with apparently no reason.

On the GUIApplication logs, We have the error code “C4056”. I have searched for that code, and this is what Oracle says: Cause A Message Queue client received a GOOD_BYE message from broker.

So, We are worried about the performance of the Broker, beacuse when It dies, all the platform stops working. We had a BASH script on the Broker machine that looks for the process: if It isn’t running, It send us an e-mail.

Has anyone had a problem like this? Any idea about what is happening in our NAS installation? May be a memory problem, or just too much conections for the Broker? Should I send our “default.properties” file to you? Any guide or documentation about the config.properties or default.properties parameters?
The Broker process is configured with this parameters: -Xms24g -Xmx24g -Xss192m -XX:MaxGCPauseMillis=5000.

Thank you all!
Best regards,
Miguel.
________________________________
Este mensaje y cualquier fichero adjunto están dirigidos únicamente a sus destinatarios y contiene información confidencial. Si usted ha recibido este correo electrónico por error, le informamos que no puede realizar ninguna revisión, alteración, impresión, copia, transmisión, difusión ni utilización alguna de este mensaje ni de cualquier fichero adjunto que pudiese contener. La realización de cualquiera de los actos indicados está expresamente prohibida por las Normas que regulan estas materias. Por todo ello se solicita que, en caso de existir error en la recepción de este mensaje, se lo notifique al remitente respondiendo a este e-mail y elimine el mensaje y su contenido inmediatamente. La Biblioteca Nacional de España se reserva las acciones legales que le correspondan en el caso de que se infrinja lo indicado anteriormente.
________________________________
The information in this e-mail and any attachments is confidential and it is intended for the addressee only. If you have received this e-mail in error, you are notified that any revision, amendment, print, copy, disclosure, distribution or use of the contents is unauthorized. Carrying out any of the above actions, is expressly banned by rules governing this matter. Hence we request that if you are not the intended recipient, please notify the sender answering this e-mail, and delete the message and any attachments. The National Library of Spain reserves itself the right to take the appropriate legal actions in the event of the above mentioned matter is being infringed.
________________________________
_______________________________________________
NetarchiveSuite-users mailing list
NetarchiveSuite-users at ml.sbforge.org<mailto:NetarchiveSuite-users at ml.sbforge.org>
https://ml.sbforge.org/mailman/listinfo/netarchivesuite-users
________________________________

En raison des directives gouvernementales liées à la situation sanitaire, les expositions restent fermées jusqu'à nouvelle consigne. Les manifestations culturelles ne peuvent pas accueillir de public mais sont en grande partie diffusées en ligne<https://www.bnf.fr/fr/actualites/retrouvez-les-conferences-de-janvier-2021-en-ligne>.

La bibliothèque tous publics est ouverte du mardi au vendredi de 10 h à 17 h.
Les bibliothèques de recherche sont ouvertes, sur le site François-Mitterrand, le lundi de 14 h à 17 h et du mardi au vendredi de 10 h à 17 h, et, sur les sites Richelieu, Arsenal et Opéra, de 10 h à 17 h du lundi au vendredi. Consulter les modalités d'accès<https://www.bnf.fr/fr/reouverture-des-salles-de-lecture-de-la-bnf-modalites-dacces>

Avant d'imprimer, pensez à l'environnement.

________________________________
Este mensaje y cualquier fichero adjunto están dirigidos únicamente a sus destinatarios y contiene información confidencial. Si usted ha recibido este correo electrónico por error, le informamos que no puede realizar ninguna revisión, alteración, impresión, copia, transmisión, difusión ni utilización alguna de este mensaje ni de cualquier fichero adjunto que pudiese contener. La realización de cualquiera de los actos indicados está expresamente prohibida por las Normas que regulan estas materias. Por todo ello se solicita que, en caso de existir error en la recepción de este mensaje, se lo notifique al remitente respondiendo a este e-mail y elimine el mensaje y su contenido inmediatamente. La Biblioteca Nacional de España se reserva las acciones legales que le correspondan en el caso de que se infrinja lo indicado anteriormente.
________________________________
The information in this e-mail and any attachments is confidential and it is intended for the addressee only. If you have received this e-mail in error, you are notified that any revision, amendment, print, copy, disclosure, distribution or use of the contents is unauthorized. Carrying out any of the above actions, is expressly banned by rules governing this matter. Hence we request that if you are not the intended recipient, please notify the sender answering this e-mail, and delete the message and any attachments. The National Library of Spain reserves itself the right to take the appropriate legal actions in the event of the above mentioned matter is being infringed.
________________________________
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://ml.sbforge.org/pipermail/netarchivesuite-users/attachments/20210507/2260560e/attachment-0001.html>