[Netarchivesuite-users] Running out of disk space

[sara.aubry at bnf.fr]

Hi Andreas,

We have tested at the scale of our broad crawl and it does work!

Sara









Message de : Colin Samuel Rosenthal <csr at statsbiblioteket.dk> 
                      06/09/2010 14:48

Envoyé par : 
<netarchivesuite-users-bounces at lists.gforge.statsbiblioteket.dk>

Veuillez répondre à 
<netarchivesuite-users at lists.gforge.statsbiblioteket.dk>



Pour
"netarchivesuite-users at lists.gforge.statsbiblioteket.dk" 
<netarchivesuite-users at lists.gforge.statsbiblioteket.dk>
Copie

Objet
Re: [Netarchivesuite-users] Running out of disk space



Just a wild suggestion ...

You could mount another drive, move some of the harvested files over to 
the mounted drive, and then create links to them in
the harvest directory. It might work but we've certainly never tested it!
cheers,
Colin
________________________________________
From: netarchivesuite-users-bounces at lists.gforge.statsbiblioteket.dk 
[netarchivesuite-users-bounces at lists.gforge.statsbiblioteket.dk] On Behalf 
Of aponb at gmx.at [aponb at gmx.at]
Sent: Monday, September 06, 2010 2:41 PM
To: NetarchiveSuite-users
Subject: [Netarchivesuite-users] Running out of disk space

Currently I am running a job which was defined with too high
configuration limits for the maximum number of bytes for each included
domain (or too many domains for that job). Now the disk space on that
crawler machine is getting close. Does anybody have a clue how I could
prevent this job from failing?
I could move the oldest arc files out of the job crawling directory (in
my case ./8802harvester/3666_1283259960121/arcs) to an other location,
so that I can get more free space. But than I will get a problem later,
when all collected arc-Files get processed after completion of that job
and some are missing.
What can I do or what would you suggest to do in that case?
Thanks for reading
a.


_______________________________________________
NetarchiveSuite-users mailing list
NetarchiveSuite-users at lists.gforge.statsbiblioteket.dk
https://lists.gforge.statsbiblioteket.dk/mailman/listinfo/netarchivesuite-users


_______________________________________________
NetarchiveSuite-users mailing list
NetarchiveSuite-users at lists.gforge.statsbiblioteket.dk
https://lists.gforge.statsbiblioteket.dk/mailman/listinfo/netarchivesuite-users







Avant d'imprimer, pensez à l'environnement.