[Netarchivesuite-devel] # messages on HARVEST_MON is growing all the time
aponb at gmx.at
aponb at gmx.at
Mon Feb 7 13:42:46 CET 2011
I had this Problem in Version 3.8. Then I used reregisterDelay setting
with 10 minutes and I had no problems anymore in Version 3.10 and 3.12.
Can't tell about Version 3.14 as I am starting testing it now!
>
> We have just rolled out 3.14.0 in the Netarkivet PROD, and immediately
> stumbled upon two major problems.
>
> The consumers to the MONITOR queues cannot keep up with the producers
> of these queues.
>
> After just a few days, we have the following broker status:
>
> PROD_COMMON_ANY_HIGHPRIORITY_HACO Queue
> RUNNING 1 8 0 0 0.0
>
> PROD_COMMON_ANY_LOWPRIORITY_HACO Queue
> RUNNING 0 16 0 0 0.0
>
> PROD_COMMON_FRONTIERMON Queue
> RUNNING 29 2 1 1 1739.0
>
> PROD_COMMON_HARVESTMON Queue
> RUNNING 30 2 28538 2 2376.3276
>
> PROD_COMMON_INDEX_SERVER Queue
> RUNNING 30 1 0 0 0.0
>
> PROD_COMMON_MONITOR Queue
> RUNNING 158 1 91543 1 937.5843
>
> PROD_COMMON_THE_REPOS Queue
> RUNNING 47 1 0 0 0.0
>
> PROD_COMMON_THE_SCHED Queue
> RUNNING 29 1 0 0 0.0
>
> Notice, that there is 28538 waiting messages (from 30 producers) in
> the COMMON_HARVESTMON queue, and 91543 (from 158 producers) waiting
> message in the COMMON_MONITOR queue.
>
> We have increased the crawlLoppWaitTime from the default 20 secs to 60
> seconds, and the reregisterDelay from 1 minute to 5 minutes.
>
> These increases were hoped to fix the problem with congestion in the
> MONITOR message-queues.
>
> Have anyone have similar problems in their production environment?
>
> Best Regards
>
> Søren V. Carlsen, QA of netarchiveSuite
>
> ---------------------------------------------------------------------------
>
> Søren Vejrup Carlsen, Department of Digital Preservation, Royal
> Library, Copenhagen, Denmark
>
> tlf: (+45) 33 47 48 41
>
> email: svc at kb.dk
>
> ----------------------------------------------------------------------------
>
> Non omnia possumus omnes
>
> --- Macrobius, Saturnalia, VI, 1, 35 -------
>
>
> _______________________________________________
> Netarchivesuite-devel mailing list
> Netarchivesuite-devel at lists.gforge.statsbiblioteket.dk
> https://lists.gforge.statsbiblioteket.dk/mailman/listinfo/netarchivesuite-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ml.sbforge.org/pipermail/netarchivesuite-devel/attachments/20110207/8416291f/attachment-0002.html>
More information about the Netarchivesuite-devel
mailing list