[Netarchivesuite-devel] Failed jobs and Jobtimeouttime parameter
sara.aubry at bnf.fr
sara.aubry at bnf.fr
Tue Jun 8 09:18:04 CEST 2010
Hi Søren,
Thanks for your answer.
What if we manually change the status in the DB from failed back to
started again?
Is the communication going to be ok again?
Or is the broker storing some information about failed marked jobs?
Sara
Message de : Søren Vejrup Carlsen <svc at kb.dk>
07/06/2010 10:55
Envoyé par :
<netarchivesuite-devel-bounces at lists.gforge.statsbiblioteket.dk>
Veuillez répondre à
<netarchivesuite-devel at lists.gforge.statsbiblioteket.dk>
Pour
"netarchivesuite-devel at lists.gforge.statsbiblioteket.dk"
<netarchivesuite-devel at lists.gforge.statsbiblioteket.dk>
Copie
"sara.aubry at info.unicaen.fr" <sara.aubry at info.unicaen.fr>
Objet
Re: [Netarchivesuite-devel] Failed jobs and Jobtimeouttime parameter
Hi Sara.
The purpose of this parameter is to move the jobs from status STARTED to
status FAILED
When we have waited for a week, after which we no longer expect the job to
terminate. You might want to raise this number to 4*10080 (i.e. a month).
This was implemented in FR 1014 No good way to mark a non-reported-stopped
job as FAILED or DONE.
The method
dk.netarkivet.harvester.scheduler.HarvestScheduler.stopTimeoutJobs called
from the scheduleJobs method in the same class handles this task.
The data from the harvesters will always be processed, but the status of
the job will still be failed!
The class responsible for the processing of the feedback from the
harvesters is the
dk.netarkivet.harvester.scheduler.HarvestSchedulerMonitorServer and
specifically the method: processCrawlStatusMessage(CrawlStatusMessage
cmsg)
Best regards
Søren
-----Oprindelig meddelelse-----
Fra: netarchivesuite-devel-bounces at lists.gforge.statsbiblioteket.dk
[mailto:netarchivesuite-devel-bounces at lists.gforge.statsbiblioteket.dk] På
vegne af sara.aubry at bnf.fr
Sendt: 4. juni 2010 19:00
Til: netarchivesuite-devel at lists.gforge.statsbiblioteket.dk
Cc: sara.aubry at info.unicaen.fr
Emne: [Netarchivesuite-devel] Failed jobs and Jobtimeouttime parameter
Hello everyone,
A week after the start of our second stage, our first 35 jobs were marked
as Failed in the database although they are still up and running on the
crawlers.
Nicolas went through the code and found a Jobtimeouttime parameter which
is set on 10080 minutes.
What is the purpose of this parameter ?
He also read that it should not keep the HarvestController from updating
the job status to Finished and the job details (start and end time, bytes
and documents harvested for each domain...) as soon as the job is
finished.
Can you confirmed this? We don't want to have inconsistent information in
the job details.
Thanks!
Sara
Avant d'imprimer, pensez ? l'environnement.
_______________________________________________
Netarchivesuite-devel mailing list
Netarchivesuite-devel at lists.gforge.statsbiblioteket.dk
https://lists.gforge.statsbiblioteket.dk/mailman/listinfo/netarchivesuite-devel
Avant d'imprimer, pensez à l'environnement.
More information about the Netarchivesuite-devel
mailing list