[Netarchivesuite-users] Upload error

aponb at gmx.at aponb at gmx.at
Fri Mar 20 14:25:35 CET 2009


Just got an upload error from just one file (out of some 100) with the 
following Message:

Error uploading arcfile '/home/netarchive/apps/netarchivesuite/ONB/harvester_7051/5_1237407101891/arcs/5-3-20090319002508-00023-webcrawler06.onb.ac.at.arc.gz' Will be moved to '/home/netarchive/apps/netarchivesuite/ONB/oldjobs'
dk.netarkivet.common.exceptions.IOFailure: Could not store '/home/netarchive/apps/netarchivesuite/ONB/harvester_7051/5_1237407101891/arcs/5-3-20090319002508-00023-webcrawler06.onb.ac.at.arc.gz' after 3 attempts. Giving up.
Client-side exception occurred while storing '/home/netarchive/apps/netarchivesuite/ONB/harvester_7051/5_1237407101891/arcs/5-3-20090319002508-00023-webcrawler06.onb.ac.at.arc.gz' on attempt number 1 of 3.
dk.netarkivet.common.exceptions.ArgumentNotValid: Error creating singleton of class 'dk.netarkivet.common.distribute.HTTPRemoteFile': 
	at dk.netarkivet.common.utils.SettingsFactory.getInstance(SettingsFactory.java:102)
	at dk.netarkivet.common.distribute.RemoteFileFactory.getInstance(RemoteFileFactory.java:51)
	at dk.netarkivet.common.distribute.RemoteFileFactory.getDistributefileInstance(RemoteFileFactory.java:74)
	at dk.netarkivet.archive.arcrepository.distribute.StoreMessage.<init>(StoreMessage.java:55)
	at dk.netarkivet.archive.arcrepository.distribute.JMSArcRepositoryClient.store(JMSArcRepositoryClient.java:240)
	at dk.netarkivet.harvester.harvesting.HarvestController.uploadFiles(HarvestController.java:320)
	at dk.netarkivet.harvester.harvesting.HarvestController.storeFiles(HarvestController.java:263)
	at dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer.processHarvestInfoFile(HarvestControllerServer.java:550)
	at dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer.access$300(HarvestControllerServer.java:83)
	at dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer$HarvesterThread.run(HarvestControllerServer.java:647)
Caused by: dk.netarkivet.common.exceptions.IOFailure: Unable to checksum file '/home/netarchive/apps/netarchivesuite/ONB/harvester_7051/5_1237407101891/arcs/5-3-20090319002508-00023-webcrawler06.onb.ac.at.arc.gz'
	at dk.netarkivet.common.distribute.HTTPRemoteFile.<init>(HTTPRemoteFile.java:88)
	at dk.netarkivet.common.distribute.HTTPRemoteFile.getInstance(HTTPRemoteFile.java:114)
	at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at dk.netarkivet.common.utils.SettingsFactory.getInstance(SettingsFactory.java:100)
	... 9 more
Caused by: java.io.IOException: Input/output error
	at java.io.FileInputStream.readBytes(Native Method)
	at java.io.FileInputStream.read(FileInputStream.java:199)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
	at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
	at java.io.DataInputStream.read(DataInputStream.java:83)
	at dk.netarkivet.common.utils.MD5.generateMD5onFile(MD5.java:91)
	at dk.netarkivet.common.distribute.HTTPRemoteFile.<init>(HTTPRemoteFile.java:86)
	... 14 more


I also tried it with the Upload Tool, but it failed with the same error.
The message obviously means that there was an error in writing the file 
on the local harvest machine (maybe a harddisk problem) - do you agree 
in that?
Is there anyway to bring that file in the bitarchive? But probably not, 
as I am not able to unzip that file.

And another question. This happened with a job which included 153 
configurations in a full harvest. What does this failed status now mean 
for the next step in the full harvest? Will the whole job repeated now? 
Will the uploaded files belonging to this failed job used for 
deduplication (I assume yes)?

Thanks again in advance for your answers
Regards
a.



More information about the NetarchiveSuite-users mailing list