[Netarchivesuite-users] Upload errors

aponb at gmx.at aponb at gmx.at
Mon Jun 2 09:22:18 CEST 2008


I checked the jobs which run over the weekend and realized that some of 
them failed due to an upload error.
I've got following exception in my logs:

Error uploading arcfile '/home/onbpre/netarchive/scripts/simple_harvest/server2/49_1212195698720/arcs/49-1-20080531020618-00008-webcrawler01.arc.gz' Will be moved to '/home/onbpre/netarchive/scripts/simple_harvest/oldjobs2'
dk.netarkivet.common.exceptions.IOFailure: Could not store 'server2/49_1212195698720/arcs/49-1-20080531020618-00008-webcrawler01.arc.gz' after 3 attempts. Giving up.
Client-side exception occurred while storing 'server2/49_1212195698720/arcs/49-1-20080531020618-00008-webcrawler01.arc.gz' on attempt number 1 of 3.
Error creating singleton
    dk.netarkivet.common.utils.SettingsFactory.getInstance(SettingsFactory.java:102)
    dk.netarkivet.common.distribute.RemoteFileFactory.getInstance(RemoteFileFactory.java:51)
    dk.netarkivet.common.distribute.RemoteFileFactory.getDistributefileInstance(RemoteFileFactory.java:74)
    dk.netarkivet.archive.arcrepository.distribute.StoreMessage.<init>(StoreMessage.java:55)
    dk.netarkivet.archive.arcrepository.distribute.JMSArcRepositoryClient.store(JMSArcRepositoryClient.java:230)
    dk.netarkivet.harvester.harvesting.HarvestController.uploadFiles(HarvestController.java:299)
    dk.netarkivet.harvester.harvesting.HarvestController.storeFiles(HarvestController.java:244)
    dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer.processHarvestInfoFile(HarvestControllerServer.java:550)
    dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer.access$300(HarvestControllerServer.java:83)
    dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer$HarvesterThread.run(HarvestControllerServer.java:647)
Client-side exception occurred while storing 'server2/49_1212195698720/arcs/49-1-20080531020618-00008-webcrawler01.arc.gz' on attempt number 2 of 3.
Error creating singleton
    dk.netarkivet.common.utils.SettingsFactory.getInstance(SettingsFactory.java:102)
    dk.netarkivet.common.distribute.RemoteFileFactory.getInstance(RemoteFileFactory.java:51)
    dk.netarkivet.common.distribute.RemoteFileFactory.getDistributefileInstance(RemoteFileFactory.java:74)
    dk.netarkivet.archive.arcrepository.distribute.StoreMessage.<init>(StoreMessage.java:55)
    dk.netarkivet.archive.arcrepository.distribute.JMSArcRepositoryClient.store(JMSArcRepositoryClient.java:230)
    dk.netarkivet.harvester.harvesting.HarvestController.uploadFiles(HarvestController.java:299)
    dk.netarkivet.harvester.harvesting.HarvestController.storeFiles(HarvestController.java:244)
    dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer.processHarvestInfoFile(HarvestControllerServer.java:550)
    dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer.access$300(HarvestControllerServer.java:83)
    dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer$HarvesterThread.run(HarvestControllerServer.java:647)
Client-side exception occurred while storing 'server2/49_1212195698720/arcs/49-1-20080531020618-00008-webcrawler01.arc.gz' on attempt number 3 of 3.
Error creating singleton
    dk.netarkivet.common.utils.SettingsFactory.getInstance(SettingsFactory.java:102)
    dk.netarkivet.common.distribute.RemoteFileFactory.getInstance(RemoteFileFactory.java:51)
    dk.netarkivet.common.distribute.RemoteFileFactory.getDistributefileInstance(RemoteFileFactory.java:74)
    dk.netarkivet.archive.arcrepository.distribute.StoreMessage.<init>(StoreMessage.java:55)
    dk.netarkivet.archive.arcrepository.distribute.JMSArcRepositoryClient.store(JMSArcRepositoryClient.java:230)
    dk.netarkivet.harvester.harvesting.HarvestController.uploadFiles(HarvestController.java:299)
    dk.netarkivet.harvester.harvesting.HarvestController.storeFiles(HarvestController.java:244)
    dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer.processHarvestInfoFile(HarvestControllerServer.java:550)
    dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer.access$300(HarvestControllerServer.java:83)
    dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer$HarvesterThread.run(HarvestControllerServer.java:647)



My questions are now. What went wrong and what can i do that it will not 
happen again. And how can I transfer the files from the oldjobs 
directory in a offical way? Or should I just copy each gz file from the 
oldjobs2/jobid/arc directory to my bitarchive?

Thanks in advance for giving me a hint!




More information about the NetarchiveSuite-users mailing list