[Netarchivesuite-users] Upload errors

[Kåre Fiedler Christiansen]

Hi.

Although it's difficult to say it for certain, due to an inefficiency in
the logging, the most probable cause for your failure is that the port
allocated for a http remote file is already in use.

You can probably find a more detailed error message in the log for the
HarvestControllerApplication. In the next version the full stack trace
will be included in the log.

You can test my theory by inspecting the settingsfile, and seeing what
port is allocated for RemoteFile port, and then checking if that port is
already in use, using netstat.

$ netstat -nlp | grep :$PORTNO

should give you the pid of the java process listening on the port. You
can then 

$ ps -fp

to see more information aboput the process.

I hope this helps.

Hilsen
  Kåre

On Mon, 2008-06-02 at 11:46 +0200, Søren Vejrup Carlsen wrote:
> Hi there.
> It is difficult to answer your question without looking at the settings.xml, the application is using.
> This will tell us what kind of RemoteFile it is trying to create a singleton for (and so we can better figure out what the problem is)
> In the next stable release, this information will be in the log as well.
> So please send us your settings.xml for the harvester application
> 
> Best, 
> Søren
> > -----Original Message-----
> > From: netarchivesuite-users-bounces at lists.gforge.statsbiblioteket.dk
> > [mailto:netarchivesuite-users-bounces at lists.gforge.statsbiblio
> > teket.dk]O
> > n Behalf Of aponb at gmx.at
> > Sent: Monday, June 02, 2008 9:22 AM
> > To: netarchivesuite-users at lists.gforge.statsbiblioteket.dk
> > Subject: [Netarchivesuite-users] Upload errors
> > 
> > 
> > I checked the jobs which run over the weekend and realized 
> > that some of 
> > them failed due to an upload error.
> > I've got following exception in my logs:
> > 
> > Error uploading arcfile 
> > '/home/onbpre/netarchive/scripts/simple_harvest/server2/49_121
> > 2195698720/arcs/49-1-20080531020618-00008-webcrawler01.arc.gz'
> >  Will be moved to 
> > '/home/onbpre/netarchive/scripts/simple_harvest/oldjobs2'
> > dk.netarkivet.common.exceptions.IOFailure: Could not store 
> > 'server2/49_1212195698720/arcs/49-1-20080531020618-00008-webcr
> > awler01.arc.gz' after 3 attempts. Giving up.
> > Client-side exception occurred while storing 
> > 'server2/49_1212195698720/arcs/49-1-20080531020618-00008-webcr
> > awler01.arc.gz' on attempt number 1 of 3.
> > Error creating singleton
> >     
> > dk.netarkivet.common.utils.SettingsFactory.getInstance(Setting
> > sFactory.java:102)
> >     
> > dk.netarkivet.common.distribute.RemoteFileFactory.getInstance(
> > RemoteFileFactory.java:51)
> >     
> > dk.netarkivet.common.distribute.RemoteFileFactory.getDistribut
> > efileInstance(RemoteFileFactory.java:74)
> >     
> > dk.netarkivet.archive.arcrepository.distribute.StoreMessage.<i
> > nit>(StoreMessage.java:55)
> >     
> > dk.netarkivet.archive.arcrepository.distribute.JMSArcRepositor
> > yClient.store(JMSArcRepositoryClient.java:230)
> >     
> > dk.netarkivet.harvester.harvesting.HarvestController.uploadFil
> > es(HarvestController.java:299)
> >     
> > dk.netarkivet.harvester.harvesting.HarvestController.storeFile
> > s(HarvestController.java:244)
> >     
> > dk.netarkivet.harvester.harvesting.distribute.HarvestControlle
> > rServer.processHarvestInfoFile(HarvestControllerServer.java:550)
> >     
> > dk.netarkivet.harvester.harvesting.distribute.HarvestControlle
> > rServer.access$300(HarvestControllerServer.java:83)
> >     
> > dk.netarkivet.harvester.harvesting.distribute.HarvestControlle
> > rServer$HarvesterThread.run(HarvestControllerServer.java:647)
> > Client-side exception occurred while storing 
> > 'server2/49_1212195698720/arcs/49-1-20080531020618-00008-webcr
> > awler01.arc.gz' on attempt number 2 of 3.
> > Error creating singleton
> >     
> > dk.netarkivet.common.utils.SettingsFactory.getInstance(Setting
> > sFactory.java:102)
> >     
> > dk.netarkivet.common.distribute.RemoteFileFactory.getInstance(
> > RemoteFileFactory.java:51)
> >     
> > dk.netarkivet.common.distribute.RemoteFileFactory.getDistribut
> > efileInstance(RemoteFileFactory.java:74)
> >     
> > dk.netarkivet.archive.arcrepository.distribute.StoreMessage.<i
> > nit>(StoreMessage.java:55)
> >     
> > dk.netarkivet.archive.arcrepository.distribute.JMSArcRepositor
> > yClient.store(JMSArcRepositoryClient.java:230)
> >     
> > dk.netarkivet.harvester.harvesting.HarvestController.uploadFil
> > es(HarvestController.java:299)
> >     
> > dk.netarkivet.harvester.harvesting.HarvestController.storeFile
> > s(HarvestController.java:244)
> >     
> > dk.netarkivet.harvester.harvesting.distribute.HarvestControlle
> > rServer.processHarvestInfoFile(HarvestControllerServer.java:550)
> >     
> > dk.netarkivet.harvester.harvesting.distribute.HarvestControlle
> > rServer.access$300(HarvestControllerServer.java:83)
> >     
> > dk.netarkivet.harvester.harvesting.distribute.HarvestControlle
> > rServer$HarvesterThread.run(HarvestControllerServer.java:647)
> > Client-side exception occurred while storing 
> > 'server2/49_1212195698720/arcs/49-1-20080531020618-00008-webcr
> > awler01.arc.gz' on attempt number 3 of 3.
> > Error creating singleton
> >     
> > dk.netarkivet.common.utils.SettingsFactory.getInstance(Setting
> > sFactory.java:102)
> >     
> > dk.netarkivet.common.distribute.RemoteFileFactory.getInstance(
> > RemoteFileFactory.java:51)
> >     
> > dk.netarkivet.common.distribute.RemoteFileFactory.getDistribut
> > efileInstance(RemoteFileFactory.java:74)
> >     
> > dk.netarkivet.archive.arcrepository.distribute.StoreMessage.<i
> > nit>(StoreMessage.java:55)
> >     
> > dk.netarkivet.archive.arcrepository.distribute.JMSArcRepositor
> > yClient.store(JMSArcRepositoryClient.java:230)
> >     
> > dk.netarkivet.harvester.harvesting.HarvestController.uploadFil
> > es(HarvestController.java:299)
> >     
> > dk.netarkivet.harvester.harvesting.HarvestController.storeFile
> > s(HarvestController.java:244)
> >     
> > dk.netarkivet.harvester.harvesting.distribute.HarvestControlle
> > rServer.processHarvestInfoFile(HarvestControllerServer.java:550)
> >     
> > dk.netarkivet.harvester.harvesting.distribute.HarvestControlle
> > rServer.access$300(HarvestControllerServer.java:83)
> >     
> > dk.netarkivet.harvester.harvesting.distribute.HarvestControlle
> > rServer$HarvesterThread.run(HarvestControllerServer.java:647)
> > 
> > 
> > 
> > My questions are now. What went wrong and what can i do that 
> > it will not 
> > happen again. And how can I transfer the files from the oldjobs 
> > directory in a offical way? Or should I just copy each gz 
> > file from the 
> > oldjobs2/jobid/arc directory to my bitarchive?
> > 
> > Thanks in advance for giving me a hint!
> > 
> > _______________________________________________
> > NetarchiveSuite-users mailing list
> > NetarchiveSuite-users at lists.gforge.statsbiblioteket.dk
> > https://lists.gforge.statsbiblioteket.dk/mailman/listinfo/neta
> rchivesuite-users
> 
> _______________________________________________
> NetarchiveSuite-users mailing list
> NetarchiveSuite-users at lists.gforge.statsbiblioteket.dk
> https://lists.gforge.statsbiblioteket.dk/mailman/listinfo/netarchivesuite-users