[Netarchivesuite-devel] # messages on HARVEST_MON is growing all the time

Søren Vejrup Carlsen svc at kb.dk
Mon Feb 7 11:36:15 CET 2011


Hi all.
We have just rolled out 3.14.0 in the Netarkivet PROD, and immediately stumbled upon two major problems.
The consumers to the MONITOR queues cannot keep up with the producers of these queues.

After just a few days, we have the following broker status:



PROD_COMMON_ANY_HIGHPRIORITY_HACO                              Queue  RUNNING  1          8          0      0      0.0

PROD_COMMON_ANY_LOWPRIORITY_HACO                               Queue  RUNNING  0          16         0      0      0.0

PROD_COMMON_FRONTIERMON                                        Queue  RUNNING  29         2          1      1      1739.0

PROD_COMMON_HARVESTMON                                         Queue  RUNNING  30         2          28538  2      2376.3276

PROD_COMMON_INDEX_SERVER                                       Queue  RUNNING  30         1          0      0      0.0

PROD_COMMON_MONITOR                                            Queue  RUNNING  158        1          91543  1      937.5843

PROD_COMMON_THE_REPOS                                          Queue  RUNNING  47         1          0      0      0.0

PROD_COMMON_THE_SCHED                                          Queue  RUNNING  29         1          0      0      0.0


Notice, that there is 28538 waiting messages (from 30 producers) in the COMMON_HARVESTMON queue, and 91543 (from 158 producers) waiting message in the COMMON_MONITOR queue.

We have increased the crawlLoppWaitTime from the default 20 secs to 60 seconds, and the reregisterDelay from 1 minute to 5 minutes.
These increases were hoped to fix the problem with congestion in the MONITOR message-queues.

Have anyone have similar problems in their production environment?

Best Regards
Søren V. Carlsen, QA of netarchiveSuite

---------------------------------------------------------------------------
Søren Vejrup Carlsen, Department of Digital Preservation, Royal Library, Copenhagen, Denmark
tlf: (+45) 33 47 48 41
email: svc at kb.dk
----------------------------------------------------------------------------
Non omnia possumus omnes
--- Macrobius, Saturnalia, VI, 1, 35 -------

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ml.sbforge.org/pipermail/netarchivesuite-devel/attachments/20110207/b5cbff9d/attachment-0002.html>


More information about the Netarchivesuite-devel mailing list