[Netarchivesuite-users] Upload errors

aponb at gmx.at aponb at gmx.at
Mon Jun 2 14:58:15 CEST 2008


>
> It's an artifact of running everything on one machine with the same
> settingsfile :-/
>
> The reply for the store message is sent to a JMS queue, and the queue
> name is generated in a way that is expected to be uniq by using the
> hostname and a portnumber, plus a string identifying the queue.
>
> In this case, it will probably share the queuename with a harvester.
> That gives a 50/50 chance that you will actually receive a reply.
>
> The solution is to use a settings-file made for this tool, with a
> specialised port, and specifying the setting file using 
>
> "-Ddk.netarkivet.settings.file=/path/to/settings-file-for-tool.xml"
>
> or overriding the portnumber on the command line, using
>
> "-Dsettings.common.http.port=9999" 
>
> (where 9999 is a free port number)
>
> I have updated feature request 291 with a link to this conversation.
> https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=291 <https://gforge.statsbiblioteket.dk/tracker/index.php?func=detail&aid=291>
>   

Thanks for the help. Changing of the port changed the behaviour, 
although it doesn't loop over the "find result". But it came back with 
the message that the file was processed, but then it stands still. Here 
is my log:

Connecting to ArcRepository
02.06.2008 14:48:17 
dk.netarkivet.archive.arcrepository.distribute.JMSArcRepositoryClient <init>
INFO: JMSArcRepositoryClient will retry a store 3 times and timeout on 
each try after 3600000 milliseconds, and timeout on each getrequest 
after 60000 milliseconds.
02.06.2008 14:48:17 dk.netarkivet.common.distribute.JMSConnectionSunMQ 
<init>
INFO: Creating instance of 
dk.netarkivet.common.distribute.JMSConnectionSunMQ
02.06.2008 14:48:17 dk.netarkivet.common.distribute.JMSConnection 
initConnection
INFO: JMS connection. Broker:172.16.14.141;
02.06.2008 14:48:17 dk.netarkivet.common.distribute.JMSConnection 
initConnection
INFO: JMS connection. Port:7676;
02.06.2008 14:48:17 
dk.netarkivet.archive.arcrepository.distribute.JMSArcRepositoryClient <init>
INFO: JMSArcRepository listens for replies on channel '[Queue 
'PROD_COMMON_THIS_HACO_127_0_0_1_8076']'
Uploading file 
'/home/onbpre/netarchive/scripts/simple_harvest/oldjobs2/67_1212368498739/arcs/67-1-20080602010158-00000-webcrawler01.arc.gz'...
02.06.2008 14:48:17 org.slf4j.impl.JCLLoggerAdapter info
INFO: Logging to org.slf4j.impl.JCLLoggerAdapter(org.mortbay.log) via 
org.mortbay.log.Slf4jLog
02.06.2008 14:48:17 org.slf4j.impl.JCLLoggerAdapter info
INFO: jetty-6.1.1
02.06.2008 14:48:17 org.slf4j.impl.JCLLoggerAdapter info
INFO: Started SocketConnector @ 0.0.0.0:9000
02.06.2008 14:48:18 dk.netarkivet.archive.arcrepository.ArcRepository store
INFO: Store started: '67-1-20080602010158-00000-webcrawler01.arc.gz'
02.06.2008 14:48:18 dk.netarkivet.archive.arcrepository.ArcRepository 
replyOK
INFO: Store OK: '67-1-20080602010158-00000-webcrawler01.arc.gz'
Uploading file 
'/home/onbpre/netarchive/scripts/simple_harvest/oldjobs2/67_1212368498739/arcs/67-1-20080602010158-00000-webcrawler01.arc.gz' 
succeeded
Uploading file 
'/home/onbpre/netarchive/scripts/simple_harvest/oldjobs2/67_1212368498739/arcs/67-1-20080602010847-00006-webcrawler01.arc.gz'...
02.06.2008 14:48:20 dk.netarkivet.archive.arcrepository.ArcRepository store
INFO: Store started: '67-1-20080602010847-00006-webcrawler01.arc.gz'
02.06.2008 14:48:20 dk.netarkivet.archive.bitarchive.Bitarchive upload
INFO: Upload: dk.netarkivet.common.distribute.HTTPRemoteFile at 7ed081
02.06.2008 14:48:22 dk.netarkivet.archive.bitarchive.Bitarchive upload
INFO: Upload: completed uploading 
67-1-20080602010847-00006-webcrawler01.arc.gz
02.06.2008 14:48:22 
dk.netarkivet.archive.bitarchive.distribute.BitarchiveServer visit
INFO: Sending reply: 
ID:298-127.0.0.1(a0:39:84:eb:6d:13)-34420-1212410900397: To 
PROD_ONB_ANY_BA ReplyTo PROD_COMMON_THE_ARCREPOS OK Arcfile: 
67-1-20080602010847-00006-webcrawler01.arc.gz
02.06.2008 14:48:22 
dk.netarkivet.archive.bitarchive.distribute.BitarchiveMonitorServer visit
INFO: Received BatchMessage
ID:301-127.0.0.1(a0:39:84:eb:6d:13)-34420-1212410902476: To 
PROD_ONB_THE_BAMON ReplyTo PROD_COMMON_THE_ARCREPOS OK Job: Checksum job 
dk.netarkivet.archive.arcrepository.bitpreservation.ChecksumJob: [Files 
Processed = 0; Files  failed = 0]
02.06.2008 14:48:22 dk.netarkivet.archive.bitarchive.Bitarchive batch
INFO: Starting batch job: 
dk.netarkivet.archive.arcrepository.bitpreservation.ChecksumJob
02.06.2008 14:48:23 dk.netarkivet.archive.bitarchive.Bitarchive batch
INFO: Finished batch job 
dk.netarkivet.archive.arcrepository.bitpreservation.ChecksumJob with 
result: 0 failures in processing 1 files at 127.0.0.1_8073
02.06.2008 14:48:23 
dk.netarkivet.archive.bitarchive.distribute.BitarchiveMonitorServer visit
INFO: Received batch ended from bitarchive '127.0.0.1_8073':
BatchEndedMessage for batch job 
ID:3038-127.0.0.1(b5:34:89:97:72:e6)-34418-1212410902479
 From Bitarchive 127.0.0.1_8073
FilesProcessed = 1
ID:1590-127.0.0.1(c9:d7:3:14:5e:75)-34406-1212410903178: To 
PROD_ONB_THE_BAMON ReplyTo PROD_COMMON_ERROR OK
02.06.2008 14:48:23 
dk.netarkivet.archive.bitarchive.distribute.BitarchiveMonitorServer 
doBatchReply
INFO: BatchReplyMessage: '
BatchReplyMessage for batch job 
ID:301-127.0.0.1(a0:39:84:eb:6d:13)-34420-1212410902476
FilesProcessed = 1
FilesFailed = 0
ID:3045-127.0.0.1(f2:a5:69:86:49:82)-34416-1212410903202: To 
PROD_COMMON_THE_ARCREPOS ReplyTo PROD_ONB_THE_BAMON OK' sent from BA 
monitor to queue: '[Queue 'PROD_COMMON_THE_ARCREPOS']'
02.06.2008 14:48:23 dk.netarkivet.archive.arcrepository.ArcRepository 
replyOK
INFO: Store OK: '67-1-20080602010847-00006-webcrawler01.arc.gz'




More information about the NetarchiveSuite-users mailing list