[Netarchivesuite-users] NetarchiveSuite Version: 4.4.0 does not run jobs
Mikis Seth Sørensen
mss at statsbiblioteket.dk
Thu Oct 30 11:12:28 CET 2014
Hi Meelis
Well, it seems that the HarvestControllerServers are receiving messages
(invalid) inconsistent with the messages (valid) the
HarvestJobManagerApplication are sending back as responses to the
HarvestControllerServers registration requests. This could be caused by
the messagebroker having a queue of persistent messages from earlier runs
of the NetarchiveSuite system.
So lets try starting a system purged of messages like this:
* Shut down the NAS system
* Shut down the broker
* Verify no java processes are running
* Start a clean broker using: bin/imqbrokerd -reset store
* Verify only the mq.sys dst exists: bin/imqcmd list dst
* Restart the NAS system
Good luck :-)
Mikis
On 10/29/14, 4:24 PM, "Meelis Mihhailov" <meelis at nlib.ee> wrote:
>Hi Mikis
>
>Cleared all the logs and started with a fresh files in order to get the
>newest data.
>While starting with startall script I can now see lines telling me that
>channel "FOCUSED" is invalid:
>
>---
>Oct 29, 2014 5:09:48 PM
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>visit
>SEVERE: Received message stating that channel 'FOCUSED' is invalid. Will
>stop.
>---
>
> From the log file I see no conflict lines. Seems that all is OK and
>everything seems to work as it should. As far as I understand :)
>I'll add result from the startup and logfile content below.
>
>Meelis
>--------------------------------------------------------------------------
>----------------------------------------------------
>
>
>crawler at veebiarhiiv:~/BASE$ ./startall_veebiarhiiv_digar_ee.sh
>--------------------------------------------
>STARTING MACHINE: crawler at localhost
>crawler at localhost's password:
>Starting all applications on: localhost
>Starting linux application: GUIApplication_testcrawler
>Starting linux application: ArcRepositoryApplication_testcrawler
>Starting linux application: BitarchiveMonitorApplication_testcrawler
>Starting linux application: BitarchiveApplication_testcrawler
>Starting linux application: HarvestControllerApplication_low
>Starting linux application: HarvestControllerApplication_high_1
>Starting linux application: HarvestControllerApplication_high_2
>Starting linux application: HarvestControllerApplication_high_3
>Starting linux application: HarvestControllerApplication_high_4
>Starting linux application: HarvestControllerApplication_high_5
>Starting linux application: HarvestControllerApplication_high_6
>Starting linux application: HarvestControllerApplication_high_7
>Starting linux application: HarvestControllerApplication_high_8
>Starting linux application: HarvestControllerApplication_high_9
>Starting linux application: HarvestControllerApplication_high_10
>Starting linux application: HarvestControllerApplication_high_11
>Starting linux application: HarvestControllerApplication_high_12
>Starting linux application: HarvestControllerApplication_high_13
>Starting linux application: HarvestControllerApplication_high_14
>Starting linux application: HarvestControllerApplication_high_15
>Starting linux application: HarvestControllerApplication_high_16
>Starting linux application: HarvestControllerApplication_high_17
>Starting linux application: HarvestControllerApplication_high_18
>Starting linux application: HarvestControllerApplication_high_19
>Starting linux application: HarvestControllerApplication_high_20
>Starting linux application: IndexServerApplication_testcrawler
>Starting linux application: ViewerProxyApplication_testcrawler
>Starting linux application: HarvestJobManagerApplication_testcrawler
>Starting dk.netarkivet.archive.arcrepository.ArcRepository
>Version: 4.4.0 status RELEASE
>dk.netarkivet.archive.arcrepository.ArcRepository Running
>log4j:WARN No appenders could be found for logger
>(com.mchange.v2.log.MLog).
>log4j:WARN Please initialize the log4j system properly.
>Starting dk.netarkivet.archive.bitarchive.distribute.BitarchiveServer
>Version: 4.4.0 status RELEASE
>dk.netarkivet.archive.bitarchive.distribute.BitarchiveServer Running
>Starting
>dk.netarkivet.archive.bitarchive.distribute.BitarchiveMonitorServer
>Version: 4.4.0 status RELEASE
>dk.netarkivet.archive.bitarchive.distribute.BitarchiveMonitorServer
>Running
>Starting dk.netarkivet.common.webinterface.GUIWebServer
>Version: 4.4.0 status RELEASE
>dk.netarkivet.common.webinterface.GUIWebServer Running
>log4j:WARN No appenders could be found for logger (org.mortbay.log).
>log4j:WARN Please initialize the log4j system properly.
>Starting
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Version: 4.4.0 status RELEASE
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Running
>Oct 29, 2014 5:09:48 PM
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>visit
>SEVERE: Received message stating that channel 'FOCUSED' is invalid. Will
>stop.
>Starting
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Version: 4.4.0 status RELEASE
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Running
>Starting
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Version: 4.4.0 status RELEASE
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Running
>Starting
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Version: 4.4.0 status RELEASE
>Starting
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Version: 4.4.0 status RELEASE
>Starting
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Version: 4.4.0 status RELEASE
>Starting
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Version: 4.4.0 status RELEASE
>Starting
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Version: 4.4.0 status RELEASE
>Starting
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Version: 4.4.0 status RELEASE
>Starting
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Version: 4.4.0 status RELEASE
>Starting
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Version: 4.4.0 status RELEASE
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Running
>Oct 29, 2014 5:09:47 PM
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>visit
>SEVERE: Received message stating that channel 'FOCUSED' is invalid. Will
>stop.
>Starting
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Version: 4.4.0 status RELEASE
>Starting
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Version: 4.4.0 status RELEASE
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Running
>Oct 29, 2014 5:09:47 PM
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>visit
>SEVERE: Received message stating that channel 'FOCUSED' is invalid. Will
>stop.
>Starting
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Version: 4.4.0 status RELEASE
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Running
>Oct 29, 2014 5:09:47 PM
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>visit
>SEVERE: Received message stating that channel 'FOCUSED' is invalid. Will
>stop.
>Starting
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Version: 4.4.0 status RELEASE
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Running
>Oct 29, 2014 5:09:47 PM
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>visit
>SEVERE: Received message stating that channel 'FOCUSED' is invalid. Will
>stop.
>Starting
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Version: 4.4.0 status RELEASE
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Running
>Oct 29, 2014 5:09:47 PM
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>visit
>SEVERE: Received message stating that channel 'FOCUSED' is invalid. Will
>stop.
>Starting
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Version: 4.4.0 status RELEASE
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Running
>Oct 29, 2014 5:09:47 PM
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>visit
>SEVERE: Received message stating that channel 'FOCUSED' is invalid. Will
>stop.
>Starting
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Version: 4.4.0 status RELEASE
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Running
>Oct 29, 2014 5:09:48 PM
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>visit
>SEVERE: Received message stating that channel 'FOCUSED' is invalid. Will
>stop.
>Starting
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Version: 4.4.0 status RELEASE
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Running
>Starting
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Version: 4.4.0 status RELEASE
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Running
>Starting
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Version: 4.4.0 status RELEASE
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>Running
>Oct 29, 2014 5:09:46 PM
>dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
>visit
>SEVERE: Received message stating that channel 'SNAPSHOT' is invalid.
>Will stop.
>Starting dk.netarkivet.harvester.indexserver.IndexServer
>Version: 4.4.0 status RELEASE
>Starting dk.netarkivet.viewerproxy.ViewerProxy
>Version: 4.4.0 status RELEASE
>--------------------------------------------------------------------------
>--------------------------------------------------------------------------
>------------
>
>* HarvestJobManagerApplication_testcrawler0.log.0
>--------------------------------------------------------------------------
>--------------------------------------------------------------------------
>------------
>Oct 29, 2014 5:09:46 PM dk.netarkivet.common.utils.Settings getAll
>FINE: Searching for a setting for key:
>settings.common.replicas.replica.replicaId
>Oct 29, 2014 5:09:47 PM dk.netarkivet.common.utils.Settings getAll
>FINE: Value found in loaded data: A
>Oct 29, 2014 5:09:46 PM
>dk.netarkivet.common.distribute.JMSConnectionSunMQ <init>
>INFO: Creating instance of
>dk.netarkivet.common.distribute.JMSConnectionSunMQ
>Oct 29, 2014 5:09:49 PM
>dk.netarkivet.common.distribute.JMSConnectionSunMQ getConnectionFactory
>INFO: Establishing SunMQ JMS Connection to 'localhost:7676'
>Oct 29, 2014 5:09:49 PM
>dk.netarkivet.harvester.datamodel.HarvestDBConnection initDataSource
>INFO: Connection pool initialized with the following values:
>- minPoolSize=5
>- maxPoolSize=10
>- acquireIncrement=5
>- maxStatements=0
>- maxStatementsPerConnection=0
>- idleConnTestPeriod=0
>- idleConnTestQuery='null'
>- idleConnTestOnCheckin=false
>Oct 29, 2014 5:09:50 PM dk.netarkivet.harvester.scheduler.JobDispatcher
><init>
>INFO: Creating JobDispatcher
>Oct 29, 2014 5:09:50 PM dk.netarkivet.common.utils.ApplicationUtils
>logAndPrint
>INFO: Starting dk.netarkivet.harvester.scheduler.HarvestJobManager
>Version: 4.4.0 status RELEASE
>Oct 29, 2014 5:09:50 PM dk.netarkivet.common.utils.ApplicationUtils
>startApp
>INFO: Using settings files
>'/arhiiv/testcrawler/TESTCRAWLER/conf/settings_HarvestJobManagerApplicatio
>n_testcrawler.xml'
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.monitor.distribute.JMSMonitorRegistryClient register
>INFO: Registering this client for monitoring every 1 minutes, using
>hostname 'veebiarhiiv.nlib.ee' and JMX/RMI ports 8118/8218
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.common.management.MBeanConnectorCreator exposeJMXMBeanServer
>INFO: Registered mbean server in registry on port 8118 communicating on
>port 8218 using password file 'conf/jmxremote.password'.
>Service URL is
>service:jmx:rmi://veebiarhiiv.nlib.ee:8218/jndi/rmi://veebiarhiiv.nlib.ee:
>8118/jmxrmi
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.common.lifecycle.LifeCycleComponent start
>FINE: Starting
>dk.netarkivet.harvester.scheduler.HarvestJobManager at 3ab99c20
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry register
>INFO: Harvester low registered on channel SNAPSHOT
>Oct 29, 2014 5:09:50 PM dk.netarkivet.common.utils.ApplicationUtils
>logAndPrint
>INFO: dk.netarkivet.harvester.scheduler.HarvestJobManager Running
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry logStatus
>INFO: HarvestChannelRegistry status:
> - SNAPSHOT { low }
>Oct 29, 2014 5:09:50 PM dk.netarkivet.harvester.scheduler.JobSupervisor
>rescheduleLeftOverJobs
>INFO: 0 jobs has been resubmitted.
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvesterStatusReceiver visit
>INFO: Sent a message to notify that harvest channel 'SNAPSHOT' is valid.
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry register
>INFO: Harvester high_1 registered on channel FOCUSED
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry logStatus
>INFO: HarvestChannelRegistry status:
> - SNAPSHOT { low }
> - FOCUSED { high_1 }
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvesterStatusReceiver visit
>INFO: Sent a message to notify that harvest channel 'FOCUSED' is valid.
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry register
>INFO: Harvester high_2 registered on channel FOCUSED
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry logStatus
>INFO: HarvestChannelRegistry status:
> - SNAPSHOT { low }
> - FOCUSED { high_1, high_2 }
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvesterStatusReceiver visit
>INFO: Sent a message to notify that harvest channel 'FOCUSED' is valid.
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry register
>INFO: Harvester high_4 registered on channel FOCUSED
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry logStatus
>INFO: HarvestChannelRegistry status:
> - SNAPSHOT { low }
> - FOCUSED { high_1, high_2, high_4 }
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvesterStatusReceiver visit
>INFO: Sent a message to notify that harvest channel 'FOCUSED' is valid.
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry register
>INFO: Harvester high_6 registered on channel FOCUSED
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry logStatus
>INFO: HarvestChannelRegistry status:
> - SNAPSHOT { low }
> - FOCUSED { high_1, high_2, high_4, high_6 }
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvesterStatusReceiver visit
>INFO: Sent a message to notify that harvest channel 'FOCUSED' is valid.
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry register
>INFO: Harvester high_3 registered on channel FOCUSED
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry logStatus
>INFO: HarvestChannelRegistry status:
> - SNAPSHOT { low }
> - FOCUSED { high_1, high_2, high_3, high_4, high_6 }
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvesterStatusReceiver visit
>INFO: Sent a message to notify that harvest channel 'FOCUSED' is valid.
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry register
>INFO: Harvester high_5 registered on channel FOCUSED
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry logStatus
>INFO: HarvestChannelRegistry status:
> - SNAPSHOT { low }
> - FOCUSED { high_1, high_2, high_3, high_4, high_5, high_6 }
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvesterStatusReceiver visit
>INFO: Sent a message to notify that harvest channel 'FOCUSED' is valid.
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry register
>INFO: Harvester high_7 registered on channel FOCUSED
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry logStatus
>INFO: HarvestChannelRegistry status:
> - SNAPSHOT { low }
> - FOCUSED { high_1, high_2, high_3, high_4, high_5, high_6,
>high_7 }
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvesterStatusReceiver visit
>INFO: Sent a message to notify that harvest channel 'FOCUSED' is valid.
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry register
>INFO: Harvester high_10 registered on channel FOCUSED
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry logStatus
>INFO: HarvestChannelRegistry status:
> - SNAPSHOT { low }
> - FOCUSED { high_1, high_10, high_2, high_3, high_4, high_5,
>high_6, high_7 }
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvesterStatusReceiver visit
>INFO: Sent a message to notify that harvest channel 'FOCUSED' is valid.
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry register
>INFO: Harvester high_9 registered on channel FOCUSED
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry logStatus
>INFO: HarvestChannelRegistry status:
> - SNAPSHOT { low }
> - FOCUSED { high_1, high_10, high_2, high_3, high_4, high_5,
>high_6, high_7, high_9 }
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvesterStatusReceiver visit
>INFO: Sent a message to notify that harvest channel 'FOCUSED' is valid.
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry register
>INFO: Harvester high_11 registered on channel FOCUSED
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry logStatus
>INFO: HarvestChannelRegistry status:
> - SNAPSHOT { low }
> - FOCUSED { high_1, high_10, high_11, high_2, high_3, high_4,
>high_5, high_6, high_7, high_9 }
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvesterStatusReceiver visit
>INFO: Sent a message to notify that harvest channel 'FOCUSED' is valid.
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry register
>INFO: Harvester high_8 registered on channel FOCUSED
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry logStatus
>INFO: HarvestChannelRegistry status:
> - SNAPSHOT { low }
> - FOCUSED { high_1, high_10, high_11, high_2, high_3, high_4,
>high_5, high_6, high_7, high_8, high_9 }
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvesterStatusReceiver visit
>INFO: Sent a message to notify that harvest channel 'FOCUSED' is valid.
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry register
>INFO: Harvester high_12 registered on channel FOCUSED
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry logStatus
>INFO: HarvestChannelRegistry status:
> - SNAPSHOT { low }
> - FOCUSED { high_1, high_10, high_11, high_12, high_2, high_3,
>high_4, high_5, high_6, high_7, high_8, high_9 }
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvesterStatusReceiver visit
>INFO: Sent a message to notify that harvest channel 'FOCUSED' is valid.
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry register
>INFO: Harvester high_13 registered on channel FOCUSED
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry logStatus
>INFO: HarvestChannelRegistry status:
> - SNAPSHOT { low }
> - FOCUSED { high_1, high_10, high_11, high_12, high_13, high_2,
>high_3, high_4, high_5, high_6, high_7, high_8, high_9 }
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvesterStatusReceiver visit
>INFO: Sent a message to notify that harvest channel 'FOCUSED' is valid.
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry register
>INFO: Harvester high_14 registered on channel FOCUSED
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry logStatus
>INFO: HarvestChannelRegistry status:
> - SNAPSHOT { low }
> - FOCUSED { high_1, high_10, high_11, high_12, high_13,
>high_14, high_2, high_3, high_4, high_5, high_6, high_7, high_8, high_9 }
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvesterStatusReceiver visit
>INFO: Sent a message to notify that harvest channel 'FOCUSED' is valid.
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry register
>INFO: Harvester high_16 registered on channel FOCUSED
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry logStatus
>INFO: HarvestChannelRegistry status:
> - SNAPSHOT { low }
> - FOCUSED { high_1, high_10, high_11, high_12, high_13,
>high_14, high_16, high_2, high_3, high_4, high_5, high_6, high_7,
>high_8, high_9 }
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvesterStatusReceiver visit
>INFO: Sent a message to notify that harvest channel 'FOCUSED' is valid.
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry register
>INFO: Harvester high_15 registered on channel FOCUSED
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry logStatus
>INFO: HarvestChannelRegistry status:
> - SNAPSHOT { low }
> - FOCUSED { high_1, high_10, high_11, high_12, high_13,
>high_14, high_15, high_16, high_2, high_3, high_4, high_5, high_6,
>high_7, high_8, high_9 }
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvesterStatusReceiver visit
>INFO: Sent a message to notify that harvest channel 'FOCUSED' is valid.
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry register
>INFO: Harvester high_18 registered on channel FOCUSED
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry logStatus
>INFO: HarvestChannelRegistry status:
> - SNAPSHOT { low }
> - FOCUSED { high_1, high_10, high_11, high_12, high_13,
>high_14, high_15, high_16, high_18, high_2, high_3, high_4, high_5,
>high_6, high_7, high_8, high_9 }
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvesterStatusReceiver visit
>INFO: Sent a message to notify that harvest channel 'FOCUSED' is valid.
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry register
>INFO: Harvester high_19 registered on channel FOCUSED
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry logStatus
>INFO: HarvestChannelRegistry status:
> - SNAPSHOT { low }
> - FOCUSED { high_1, high_10, high_11, high_12, high_13,
>high_14, high_15, high_16, high_18, high_19, high_2, high_3, high_4,
>high_5, high_6, high_7, high_8, high_9 }
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvesterStatusReceiver visit
>INFO: Sent a message to notify that harvest channel 'FOCUSED' is valid.
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry register
>INFO: Harvester high_17 registered on channel FOCUSED
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry logStatus
>INFO: HarvestChannelRegistry status:
> - SNAPSHOT { low }
> - FOCUSED { high_1, high_10, high_11, high_12, high_13,
>high_14, high_15, high_16, high_17, high_18, high_19, high_2, high_3,
>high_4, high_5, high_6, high_7, high_8, high_9 }
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvesterStatusReceiver visit
>INFO: Sent a message to notify that harvest channel 'FOCUSED' is valid.
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry register
>INFO: Harvester high_20 registered on channel FOCUSED
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvestChannelRegistry logStatus
>INFO: HarvestChannelRegistry status:
> - SNAPSHOT { low }
> - FOCUSED { high_1, high_10, high_11, high_12, high_13,
>high_14, high_15, high_16, high_17, high_18, high_19, high_2, high_20,
>high_3, high_4, high_5, high_6, high_7, high_8, high_9 }
>Oct 29, 2014 5:09:50 PM
>dk.netarkivet.harvester.scheduler.HarvesterStatusReceiver visit
>INFO: Sent a message to notify that harvest channel 'FOCUSED' is valid.
>Oct 29, 2014 5:20:20 PM
>dk.netarkivet.harvester.datamodel.HarvestDefinitionDBDAO read
>FINE: Reading harvestdefinition w/ id 1
>Oct 29, 2014 5:20:20 PM
>dk.netarkivet.harvester.datamodel.HarvestDefinitionDBDAO read
>FINE: Partialharvest found w/ id 1
>Oct 29, 2014 5:20:20 PM dk.netarkivet.harvester.datamodel.ScheduleDBDAO
>read
>FINE: Creating frequency for (timeunit,anytime,numtimeunits,hour,
>minute, dayofweek,dayofmonth) = (4, true,12,null,null,null,null,)
>Oct 29, 2014 5:20:20 PM dk.netarkivet.harvester.datamodel.Frequency
>getNewInstance
>FINE: Creating a MONTHLY frequency.
>Oct 29, 2014 5:20:20 PM dk.netarkivet.harvester.datamodel.ScheduleDBDAO
>read
>FINE: Creating frequency for (timeunit,anytime,numtimeunits,hour,
>minute, dayofweek,dayofmonth) = (4, true,12,null,null,null,null,)
>Oct 29, 2014 5:20:20 PM dk.netarkivet.harvester.datamodel.Frequency
>getNewInstance
>FINE: Creating a MONTHLY frequency.
>Oct 29, 2014 5:20:20 PM
>dk.netarkivet.harvester.scheduler.HarvestJobGenerator$JobGeneratorTask
>generateJobs
>INFO: Starting to create jobs for harvest definition #1(MEELIS)
>Oct 29, 2014 5:20:20 PM
>dk.netarkivet.harvester.scheduler.jobgen.AbstractJobGenerator generateJobs
>INFO: Generating jobs for harvestdefinition # 1
>Oct 29, 2014 5:20:20 PM
>dk.netarkivet.harvester.scheduler.jobgen.DefaultJobGenerator
>processDomainConfigurationSubset
>FINE: Adding domainconfigs with the same order.xml for harvest # 1
>Oct 29, 2014 5:20:20 PM
>dk.netarkivet.harvester.scheduler.jobgen.AbstractJobGenerator getNewJob
>INFO: No channel mapping registered for harvest id 1, will use default.
>Oct 29, 2014 5:20:20 PM
>dk.netarkivet.harvester.datamodel.HeritrixTemplate
>editOrderXML_ArchiveFormat
>FINE: WARC format selected to be used by Heritrix
>Oct 29, 2014 5:20:20 PM dk.netarkivet.harvester.datamodel.JobDBDAO create
>FINE: Creating Job 1 (state = NEW, HD = 1, channel = FOCUSED, snapshot =
>false, forcemaxcount = -1, forcemaxbytes = 1000000000,
>forcemaxrunningtime = 0, orderxml = default_orderxml, numconfigs = 1,
>created = Wed Oct 29 17:20:20 EET 2014)
>Oct 29, 2014 5:20:20 PM dk.netarkivet.harvester.datamodel.Job
>getHarvestFilenamePrefix
>WARNING: HarvestnamePrefix not yet set for job 1. Set it by using the
>naming scheme. This should only happen for old jobs being read
>Oct 29, 2014 5:20:20 PM dk.netarkivet.harvester.datamodel.Job
>setDefaultHarvestNamePrefix
>FINE: Applying the default ArchiveFileNaming class
>'dk.netarkivet.harvester.harvesting.LegacyNamingConvention'.
>Oct 29, 2014 5:20:20 PM dk.netarkivet.harvester.datamodel.Job
>setDefaultHarvestNamePrefix
>FINE: The harvestPrefix of this job is: 1-1
>Oct 29, 2014 5:20:20 PM
>dk.netarkivet.harvester.scheduler.jobgen.DefaultJobGenerator
>processDomainConfigurationSubset
>FINE: Created # 1 jobs for harvest # 1
>Oct 29, 2014 5:20:20 PM
>dk.netarkivet.harvester.scheduler.jobgen.AbstractJobGenerator generateJobs
>INFO: Finished generating 1 jobs for harvestdefinition # 1
>Oct 29, 2014 5:20:20 PM
>dk.netarkivet.harvester.scheduler.HarvestJobGenerator$JobGeneratorTask$1
>run
>INFO: Created 1 jobs for harvest definition (MEELIS)
>Oct 29, 2014 5:20:20 PM
>dk.netarkivet.harvester.datamodel.HarvestDefinitionDBDAO update
>FINE: 1 partialharvests records updated
>Oct 29, 2014 5:20:20 PM
>dk.netarkivet.harvester.scheduler.HarvestJobGenerator$JobGeneratorTask$1
>run
>FINE: Removed HD #1(MEELIS) from list of harvestdefinitions to be
>scheduled. Harvestdefinitions still to be scheduled: []
>
>--------------------------------------------------------------------------
>--------------------------------------------------------------------------
>------------
>
>
>On 29.10.2014 16:58, Mikis Seth Sørensen wrote:
>> Sounds weird, it looks like the channel registered by the
>> HarvestControlServer and the HarvestJobManagers harvestchannel table
>>agree
>> on the exists of a FOCUSED channel.
>>
>> Does the HarvestJobManager log contain any relevant information about
>>the
>> FOCUSED channel conflict?
>>
>> Best
>> Mikis
>>
>> On 10/29/14, 11:17 AM, "Meelis Mihhailov" <meelis at nlib.ee> wrote:
>>
>>> Hi Nicholas
>>>
>>> Checked it and there are two values present :)
>>>
>>>
>>> crawler=# SELECT * from harvestchannel;
>>> id | name | issnapshot | isdefault | comments
>>>
>>>----+----------+------------+-----------+-------------------------------
>>>-
>>> 2 | FOCUSED | f | t | Channel for selective
>>>harvests
>>> 1 | SNAPSHOT | t | t | Channel for snapshot
>>>harvests
>>> (2 rows)
>>>
>>>
>>>
>>> -----------------------------------------------------
>>> Meelis Mihhailov
>>> Süsteemiadministraator / Systemadministrator
>>> Eesti Rahvusraamatukogu / National Library Of Estonia
>>>
>>> Telefon: 630 7178 / Phone: +372 630 7178
>>> E-post: meelis at nlib.ee / E-mail: meelis at nlib.ee
>>>
>>> Tõnismägi 2, 15189 Tallinn, ESTONIA
>>>
>>> www.eestirahvusraamatukogu.ee
>>> -----------------------------------------------------
>>>
>>>
>
>_______________________________________________
>NetarchiveSuite-users mailing list
>NetarchiveSuite-users at ml.sbforge.org
>http://ml.sbforge.org/mailman/listinfo/netarchivesuite-users
More information about the NetarchiveSuite-users
mailing list