[Netarchivesuite-users] HravestController loses connection to Heritrix

nicolas.giraud at bnf.fr nicolas.giraud at bnf.fr
Mon May 4 10:51:17 CEST 2009


Hi,

I experience repeatedly my jobs failing by losing the JMX/RMI connection 
to Heritrix. This happens every time a job lasts more than a couple of 
hours, though sometimes it happens much sooner. I have attached the error 
notification that I get to this message. What can be the cause of such an 
error? The ports are properly open, I am clueless as what goes wrong.

The jobs are reported as failed, however the HarvestControllers are stuck 
when this happens, and a manual restart is needed for them to go on 
processing submitted jobs.

Thanks for your help,

Nicolas


----- Réacheminé par Nicolas GIRAUD/ETS/BnF le 04/05/2009 10:37 -----







Message de : robot at bnf.fr 
                      30/04/2009 18:53


Pour
nicolas.giraud at bnf.fr
Copie

Objet
Netarkivet error: Fatal error while operating job 'Job 8 (state = 
SUBMITTED, HD = 7, priority = LOWPRIORITY, forcemaxcount = -1, 
forcemaxbytes = 20000000, orderxml = default_obeyrobots, numconfigs = 
1000)'



acheron2.bnf.fr
dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer$HarvesterThread.run(HarvestControllerServer.java:670)
Fatal error while operating job 'Job 8 (state = SUBMITTED, HD = 7, 
priority = LOWPRIORITY, forcemaxcount = -1, forcemaxbytes = 20000000, 
orderxml = default_obeyrobots, numconfigs = 1000)'
dk.netarkivet.common.exceptions.IOFailure: Error during crawling. The 
crawl may have been only partially completed.
                 at 
dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer$HarvesterThread.run(HarvestControllerServer.java:657)
Caused by: dk.netarkivet.common.exceptions.IOFailure: Failed to connect to 
URL service:jmx:rmi:///jndi/rmi://localhost:8170/jmxrmi after 0 attempts
                 at 
dk.netarkivet.common.utils.JMXUtils.getJMXConnector(JMXUtils.java:383)
                 at 
dk.netarkivet.harvester.harvesting.JMXHeritrixController.getHeritrixJMXConnector(JMXHeritrixController.java:944)
                 at 
dk.netarkivet.harvester.harvesting.JMXHeritrixController.executeHeritrixCommand(JMXHeritrixController.java:868)
                 at 
dk.netarkivet.harvester.harvesting.JMXHeritrixController.crawlIsEnded(JMXHeritrixController.java:474)
                 at 
dk.netarkivet.harvester.harvesting.HeritrixLauncher.doCrawlLoop(HeritrixLauncher.java:214)
                 at 
dk.netarkivet.harvester.harvesting.HeritrixLauncher.doCrawl(HeritrixLauncher.java:196)
                 at 
dk.netarkivet.harvester.harvesting.HarvestController.runHarvest(HarvestController.java:221)
                 at 
dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer$HarvesterThread.run(HarvestControllerServer.java:650)
Caused by: java.io.IOException: Failed to retrieve RMIServer stub: 
javax.naming.CommunicationException [Root exception is 
java.rmi.ConnectIOException: error during JRMP connection establishment; 
nested exception is: 
                 java.net.SocketTimeoutException: Read timed out]
                 at 
javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:338)
                 at 
javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:248)
                 at 
dk.netarkivet.common.utils.JMXUtils.getJMXConnector(JMXUtils.java:369)
                 ... 7 more
Caused by: javax.naming.CommunicationException [Root exception is 
java.rmi.ConnectIOException: error during JRMP connection establishment; 
nested exception is: 
                 java.net.SocketTimeoutException: Read timed out]
                 at 
com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:101)
                 at 
com.sun.jndi.toolkit.url.GenericURLContext.lookup(GenericURLContext.java:185)
                 at 
javax.naming.InitialContext.lookup(InitialContext.java:392)
                 at 
javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(RMIConnector.java:1886)
                 at 
javax.management.remote.rmi.RMIConnector.findRMIServer(RMIConnector.java:1856)
                 at 
javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:257)
                 ... 9 more
Caused by: java.rmi.ConnectIOException: error during JRMP connection 
establishment; nested exception is: 
                 java.net.SocketTimeoutException: Read timed out
                 at 
sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:286)
                 at 
sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:184)
                 at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:322)
                 at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown 
Source)
                 at 
com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:97)
                 ... 14 more
Caused by: java.net.SocketTimeoutException: Read timed out
                 at java.net.SocketInputStream.socketRead0(Native Method)
                 at 
java.net.SocketInputStream.read(SocketInputStream.java:129)
                 at 
java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
                 at 
java.io.BufferedInputStream.read(BufferedInputStream.java:237)
                 at 
java.io.DataInputStream.readByte(DataInputStream.java:248)
                 at 
sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:228)
                 ... 18 more






Avant d'imprimer, pensez à l'environnement. 
Consider the environment before printing this mail.   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ml.sbforge.org/pipermail/netarchivesuite-users/attachments/20090504/d1af8da5/attachment-0002.html>


More information about the NetarchiveSuite-users mailing list