[Netarchivesuite-devel] # messages on HARVEST_MON is growing all the time

Søren Vejrup Carlsen svc at kb.dk
Tue Feb 8 10:48:13 CET 2011


It turned out, that we had misspelled the reregisterdelay setting, so the default setting of 1 minute was still in effect instead of the 5 minutes, the wanted!
We have now corrected this, and now we don't any problems with the COMMON_MONITOR anymore.
However, the HARVEST_MONITOR is still quite full, but as full as the last time:

Name                                                                           Type    State   Producers  Consumers           Msgs
                                                                                         Total    Count  UnAck  Avg Size
-------------------------------------------------------------------------------------------------------------------------
PROD_COMMON_ANY_HIGHPRIORITY_HACO                           Queue  RUNNING  1          14         0      0      0.0
PROD_COMMON_ANY_LOWPRIORITY_HACO                            Queue  RUNNING  0          16         0      0      0.0
PROD_COMMON_FRONTIERMON                                     Queue  RUNNING  29         1          0      0      0.0
PROD_COMMON_HARVESTMON                                      Queue  RUNNING  30         1          4645   1      2371.5286
PROD_COMMON_INDEX_SERVER                                    Queue  RUNNING  28         1          0      0      0.0
PROD_COMMON_MONITOR                                         Queue  RUNNING  158        1          0      0      0.0
PROD_COMMON_THE_REPOS                                       Queue  RUNNING  45         1          0      0      0.0
PROD_COMMON_THE_SCHED                                       Queue  RUNNING  29         1          0      0      0.0


Best Regards
Søren V. Carlsen, Netarkivet

Fra: netarchivesuite-devel-bounces at lists.gforge.statsbiblioteket.dk [mailto:netarchivesuite-devel-bounces at lists.gforge.statsbiblioteket.dk] På vegne af aponb at gmx.at
Sendt: 7. februar 2011 13:43
Til: netarchivesuite-devel at lists.gforge.statsbiblioteket.dk
Emne: Re: [Netarchivesuite-devel] # messages on HARVEST_MON is growing all the time

I had this Problem in Version 3.8. Then I used reregisterDelay setting with 10 minutes and I had no problems anymore in Version 3.10 and 3.12. Can't tell about Version 3.14 as I am starting testing it now!

We have just rolled out 3.14.0 in the Netarkivet PROD, and immediately stumbled upon two major problems.
The consumers to the MONITOR queues cannot keep up with the producers of these queues.

After just a few days, we have the following broker status:



PROD_COMMON_ANY_HIGHPRIORITY_HACO                              Queue  RUNNING  1          8          0      0      0.0

PROD_COMMON_ANY_LOWPRIORITY_HACO                               Queue  RUNNING  0          16         0      0      0.0

PROD_COMMON_FRONTIERMON                                        Queue  RUNNING  29         2          1      1      1739.0

PROD_COMMON_HARVESTMON                                         Queue  RUNNING  30         2          28538  2      2376.3276

PROD_COMMON_INDEX_SERVER                                       Queue  RUNNING  30         1          0      0      0.0

PROD_COMMON_MONITOR                                            Queue  RUNNING  158        1          91543  1      937.5843

PROD_COMMON_THE_REPOS                                          Queue  RUNNING  47         1          0      0      0.0

PROD_COMMON_THE_SCHED                                          Queue  RUNNING  29         1          0      0      0.0


Notice, that there is 28538 waiting messages (from 30 producers) in the COMMON_HARVESTMON queue, and 91543 (from 158 producers) waiting message in the COMMON_MONITOR queue.

We have increased the crawlLoppWaitTime from the default 20 secs to 60 seconds, and the reregisterDelay from 1 minute to 5 minutes.
These increases were hoped to fix the problem with congestion in the MONITOR message-queues.

Have anyone have similar problems in their production environment?

Best Regards
Søren V. Carlsen, QA of netarchiveSuite

---------------------------------------------------------------------------
Søren Vejrup Carlsen, Department of Digital Preservation, Royal Library, Copenhagen, Denmark
tlf: (+45) 33 47 48 41
email: svc at kb.dk<mailto:svc at kb.dk>
----------------------------------------------------------------------------
Non omnia possumus omnes
--- Macrobius, Saturnalia, VI, 1, 35 -------






_______________________________________________

Netarchivesuite-devel mailing list

Netarchivesuite-devel at lists.gforge.statsbiblioteket.dk<mailto:Netarchivesuite-devel at lists.gforge.statsbiblioteket.dk>

https://lists.gforge.statsbiblioteket.dk/mailman/listinfo/netarchivesuite-devel



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ml.sbforge.org/pipermail/netarchivesuite-devel/attachments/20110208/d9002a45/attachment-0002.html>


More information about the Netarchivesuite-devel mailing list