[Netarchivesuite-devel] # messages on HARVEST_MON is growing all the time
Søren Vejrup Carlsen
svc at kb.dk
Tue Feb 8 10:48:13 CET 2011
It turned out, that we had misspelled the reregisterdelay setting, so the default setting of 1 minute was still in effect instead of the 5 minutes, the wanted!
We have now corrected this, and now we don't any problems with the COMMON_MONITOR anymore.
However, the HARVEST_MONITOR is still quite full, but as full as the last time:
Name Type State Producers Consumers Msgs
Total Count UnAck Avg Size
-------------------------------------------------------------------------------------------------------------------------
PROD_COMMON_ANY_HIGHPRIORITY_HACO Queue RUNNING 1 14 0 0 0.0
PROD_COMMON_ANY_LOWPRIORITY_HACO Queue RUNNING 0 16 0 0 0.0
PROD_COMMON_FRONTIERMON Queue RUNNING 29 1 0 0 0.0
PROD_COMMON_HARVESTMON Queue RUNNING 30 1 4645 1 2371.5286
PROD_COMMON_INDEX_SERVER Queue RUNNING 28 1 0 0 0.0
PROD_COMMON_MONITOR Queue RUNNING 158 1 0 0 0.0
PROD_COMMON_THE_REPOS Queue RUNNING 45 1 0 0 0.0
PROD_COMMON_THE_SCHED Queue RUNNING 29 1 0 0 0.0
Best Regards
Søren V. Carlsen, Netarkivet
Fra: netarchivesuite-devel-bounces at lists.gforge.statsbiblioteket.dk [mailto:netarchivesuite-devel-bounces at lists.gforge.statsbiblioteket.dk] På vegne af aponb at gmx.at
Sendt: 7. februar 2011 13:43
Til: netarchivesuite-devel at lists.gforge.statsbiblioteket.dk
Emne: Re: [Netarchivesuite-devel] # messages on HARVEST_MON is growing all the time
I had this Problem in Version 3.8. Then I used reregisterDelay setting with 10 minutes and I had no problems anymore in Version 3.10 and 3.12. Can't tell about Version 3.14 as I am starting testing it now!
We have just rolled out 3.14.0 in the Netarkivet PROD, and immediately stumbled upon two major problems.
The consumers to the MONITOR queues cannot keep up with the producers of these queues.
After just a few days, we have the following broker status:
PROD_COMMON_ANY_HIGHPRIORITY_HACO Queue RUNNING 1 8 0 0 0.0
PROD_COMMON_ANY_LOWPRIORITY_HACO Queue RUNNING 0 16 0 0 0.0
PROD_COMMON_FRONTIERMON Queue RUNNING 29 2 1 1 1739.0
PROD_COMMON_HARVESTMON Queue RUNNING 30 2 28538 2 2376.3276
PROD_COMMON_INDEX_SERVER Queue RUNNING 30 1 0 0 0.0
PROD_COMMON_MONITOR Queue RUNNING 158 1 91543 1 937.5843
PROD_COMMON_THE_REPOS Queue RUNNING 47 1 0 0 0.0
PROD_COMMON_THE_SCHED Queue RUNNING 29 1 0 0 0.0
Notice, that there is 28538 waiting messages (from 30 producers) in the COMMON_HARVESTMON queue, and 91543 (from 158 producers) waiting message in the COMMON_MONITOR queue.
We have increased the crawlLoppWaitTime from the default 20 secs to 60 seconds, and the reregisterDelay from 1 minute to 5 minutes.
These increases were hoped to fix the problem with congestion in the MONITOR message-queues.
Have anyone have similar problems in their production environment?
Best Regards
Søren V. Carlsen, QA of netarchiveSuite
---------------------------------------------------------------------------
Søren Vejrup Carlsen, Department of Digital Preservation, Royal Library, Copenhagen, Denmark
tlf: (+45) 33 47 48 41
email: svc at kb.dk<mailto:svc at kb.dk>
----------------------------------------------------------------------------
Non omnia possumus omnes
--- Macrobius, Saturnalia, VI, 1, 35 -------
_______________________________________________
Netarchivesuite-devel mailing list
Netarchivesuite-devel at lists.gforge.statsbiblioteket.dk<mailto:Netarchivesuite-devel at lists.gforge.statsbiblioteket.dk>
https://lists.gforge.statsbiblioteket.dk/mailman/listinfo/netarchivesuite-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ml.sbforge.org/pipermail/netarchivesuite-devel/attachments/20110208/d9002a45/attachment-0002.html>
More information about the Netarchivesuite-devel
mailing list