[Netarchivesuite-devel] Priority 5 bugs detected

[Søren Vejrup Carlsen]

I have now located the problem with the current deduplicator.
It turns out that the use of the sparse-range-filter is included in the deduplicator, but disabled by default.
To enable it, you need to set "use-sparse-range-filter" to true in the configuration of the DeDuplicator in all the harvest templates:

Just add "<boolean name="use-sparse-range-filter">true</boolean>" 
below the line: <boolean name="stats-per-host">true</boolean>

Regards Søren

-----Oprindelig meddelelse-----
Fra: netarchivesuite-devel-bounces at lists.gforge.statsbiblioteket.dk på vegne af Kåre Fiedler Christiansen
Sendt: fr 26-06-2009 12:10
Til: netarchivesuite-devel at lists.gforge.statsbiblioteket.dk
Emne: [Netarchivesuite-devel] Priority 5 bugs detected
 
Hi all,

We need to make a fix in the stable release, due to new problems found.
Or rather, old problems, for some of them.

Durign upgrade of deduplicator, our fix to dramatically decrease the
memory usage of Lucene was somehow rolled back. This means the bugs 1078
and 1079 are reopened and have priority 5.

Bjarne is in the process of submitting another bug, that may stop
harvesting entirely, due to a half-dead process blocking new harvester
processes. The practical outcome is that jobs are accepted, but die
immediately because a new harvester cannot be started. The bug has
number 1711. Making JMX more resilient might have meant that this
problem had never shown up, i.e. more work as described in the original
evaluation of 1336. This bug has priority 5.

This half-dead harvester is detectable, although not fixable. What we
can do is die in JMXHeritrixController line 530, and send an error
notification. Right now we continue, although it is almost certain that
this will lead to greater trouble.

Best,
  Kåre

_______________________________________________
Netarchivesuite-devel mailing list
Netarchivesuite-devel at lists.gforge.statsbiblioteket.dk
https://lists.gforge.statsbiblioteket.dk/mailman/listinfo/netarchivesuite-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ml.sbforge.org/pipermail/netarchivesuite-devel/attachments/20090626/367dd012/attachment-0001.html>