[Netarchivesuite-users] Problems With the HarvestController With 4.4.1

Charles Tassell ctassell at gmail.com
Tue Dec 2 15:41:45 CET 2014 

Hi Søren,

   Thanks, so this is a known problem?  Would downgrading to 4.4.0 help 
or would I have to go back to 4.3 or earlier?

   Here is my HarvestJobManagerApplication0.log.0 file after a restart 
if that helps.  It says that the HIGH/LOWPRIORITY and FOCUSED channels 
have not been created, so yeah it seems to be an issue with talking to 
OpenMQ...  I checked the OpenMQ logs and there are no real errors...  
The NAS components connect as the guest user fine.

2-Dec-2014 11:15:03 AM dk.netarkivet.common.utils.Settings getAll
FINE: Searching for a setting for key: 
settings.common.replicas.replica.replicaId
2-Dec-2014 11:15:03 AM dk.netarkivet.common.utils.Settings getAll
FINE: Value found in loaded data: A
2-Dec-2014 11:15:02 AM 
dk.netarkivet.common.distribute.JMSConnectionSunMQ <init>
INFO: Creating instance of 
dk.netarkivet.common.distribute.JMSConnectionSunMQ
2-Dec-2014 11:15:08 AM 
dk.netarkivet.common.distribute.JMSConnectionSunMQ getConnectionFactory
INFO: Establishing SunMQ JMS Connection to 'localhost:7676'
2-Dec-2014 11:15:12 AM 
dk.netarkivet.harvester.datamodel.HarvestDBConnection initDataSource
INFO: Connection pool initialized with the following values:
- minPoolSize=5
- maxPoolSize=10
- acquireIncrement=5
- maxStatements=0
- maxStatementsPerConnection=0
- idleConnTestPeriod=0
- idleConnTestQuery='null'
- idleConnTestOnCheckin=false
2-Dec-2014 11:15:14 AM dk.netarkivet.harvester.scheduler.JobDispatcher 
<init>
INFO: Creating JobDispatcher
netarchive at webarchive:~/ROBLIB/conf/log$ cat 
HarvestJobManagerApplication0.log.0
2-Dec-2014 11:15:03 AM dk.netarkivet.common.utils.Settings getAll
FINE: Searching for a setting for key: 
settings.common.replicas.replica.replicaId
2-Dec-2014 11:15:03 AM dk.netarkivet.common.utils.Settings getAll
FINE: Value found in loaded data: A
2-Dec-2014 11:15:02 AM 
dk.netarkivet.common.distribute.JMSConnectionSunMQ <init>
INFO: Creating instance of 
dk.netarkivet.common.distribute.JMSConnectionSunMQ
2-Dec-2014 11:15:08 AM 
dk.netarkivet.common.distribute.JMSConnectionSunMQ getConnectionFactory
INFO: Establishing SunMQ JMS Connection to 'localhost:7676'
2-Dec-2014 11:15:12 AM 
dk.netarkivet.harvester.datamodel.HarvestDBConnection initDataSource
INFO: Connection pool initialized with the following values:
- minPoolSize=5
- maxPoolSize=10
- acquireIncrement=5
- maxStatements=0
- maxStatementsPerConnection=0
- idleConnTestPeriod=0
- idleConnTestQuery='null'
- idleConnTestOnCheckin=false
2-Dec-2014 11:15:14 AM dk.netarkivet.harvester.scheduler.JobDispatcher 
<init>
INFO: Creating JobDispatcher
2-Dec-2014 11:15:14 AM dk.netarkivet.common.utils.ApplicationUtils 
logAndPrint
INFO: Starting dk.netarkivet.harvester.scheduler.HarvestJobManager
Version: 4.4.1 status RELEASE
2-Dec-2014 11:15:14 AM dk.netarkivet.common.utils.ApplicationUtils startApp
INFO: Using settings files 
'/opt/netarchive/ROBLIB/conf/settings_HarvestJobManagerApplication.xml'
2-Dec-2014 11:15:14 AM 
dk.netarkivet.monitor.distribute.JMSMonitorRegistryClient register
INFO: Registering this client for monitoring every 1 minutes, using 
hostname 'webarchive.upei.ca' and JMX/RMI ports 8118/8218
2-Dec-2014 11:15:14 AM 
dk.netarkivet.common.management.MBeanConnectorCreator exposeJMXMBeanServer
INFO: Registered mbean server in registry on port 8118 communicating on 
port 8218 using password file 'conf/jmxremote.password'.
Service URL is 
service:jmx:rmi://webarchive.upei.ca:8218/jndi/rmi://webarchive.upei.ca:8118/jmxrmi
2-Dec-2014 11:15:14 AM dk.netarkivet.common.lifecycle.LifeCycleComponent 
start
FINE: Starting dk.netarkivet.harvester.scheduler.HarvestJobManager at 25dd9891
2-Dec-2014 11:15:14 AM dk.netarkivet.common.utils.ApplicationUtils 
logAndPrint
INFO: dk.netarkivet.harvester.scheduler.HarvestJobManager Running
2-Dec-2014 11:15:14 AM dk.netarkivet.harvester.scheduler.JobSupervisor 
rescheduleLeftOverJobs
INFO: 0 jobs has been resubmitted.
2-Dec-2014 11:15:15 AM 
dk.netarkivet.harvester.scheduler.HarvesterStatusReceiver visit
INFO: Sent a message to notify that harvest channel 'LOWPRIORITY' is 
invalid.
2-Dec-2014 11:15:15 AM 
dk.netarkivet.harvester.scheduler.HarvesterStatusReceiver visit
INFO: Sent a message to notify that harvest channel 'HIGHPRIORITY' is 
invalid.
2-Dec-2014 11:15:15 AM 
dk.netarkivet.harvester.datamodel.HarvestDefinitionDBDAO read
FINE: Reading harvestdefinition w/ id 1
2-Dec-2014 11:15:15 AM 
dk.netarkivet.harvester.datamodel.HarvestDefinitionDBDAO read
FINE: Partialharvest found w/ id 1
2-Dec-2014 11:15:16 AM dk.netarkivet.harvester.datamodel.ScheduleDBDAO read
FINE: Creating frequency for (timeunit,anytime,numtimeunits,hour, 
minute, dayofweek,dayofmonth) = (2, true,1,null,null,null,null,)
2-Dec-2014 11:15:16 AM dk.netarkivet.harvester.datamodel.Frequency 
getNewInstance
FINE: Creating a DAILY frequency.
2-Dec-2014 11:15:16 AM dk.netarkivet.harvester.datamodel.ScheduleDBDAO read
FINE: Creating frequency for (timeunit,anytime,numtimeunits,hour, 
minute, dayofweek,dayofmonth) = (2, true,1,null,null,null,null,)
2-Dec-2014 11:15:16 AM dk.netarkivet.harvester.datamodel.Frequency 
getNewInstance
FINE: Creating a DAILY frequency.
2-Dec-2014 11:15:16 AM 
dk.netarkivet.harvester.scheduler.HarvestJobGenerator$JobGeneratorTask 
generateJobs
INFO: Harvest channel 'FOCUSED' has not yet been registered by any 
harvester, hence harvest definition 'UniversityIsland' (1) cannot be 
processed by the job generator for now.

On 14-12-02 10:09 AM, Søren Vejrup Carlsen wrote:
> Hi Charles.
> We are currently investigating the problem.
> So please be patient.
>
> The cause is at first sight, that the attempt by the harvester to register itself fails.
> Reason why is that the response message is invalid. Why this is so, I don't know.
>
> There could be some information in the HarvestJobManagerApplication logs as to why the registering fails?
>
> Best regards
>
> Søren V. Carlsen
>
> ________________________________________
> Fra: NetarchiveSuite-users [netarchivesuite-users-bounces at ml.sbforge.org] på vegne af Charles Tassell [charles at islandadmin.ca]
> Sendt: 2. december 2014 14:08
> Til: netarchivesuite-users at ml.sbforge.org
> Emne: [Netarchivesuite-users] Problems With the HarvestController With  4.4.1
>
> Hi Everyone,
>
>     Sorry if this is a dupe, I posted it to the list yesterday but I
> don't think it went through as I wasn't subscribed.
>
>     I've recently been trying to get NetArchiverSuite 4.4.1 running but
> I'm not having much luck, and it seems to be related to the interaction
> between the controllers and the message broker service. Whenever I start
> the high/low controller processes I get the following in the startup logs:
>
> Starting
> dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
> Version: 4.4.1 status RELEASE
> dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
> Running
> 1-Dec-2014 10:49:53 AM
> dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer visit
> SEVERE: Received message stating that channel 'HIGHPRIORITY' is invalid.
> Will stop.
>
> Starting
> dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
> Version: 4.4.1 status RELEASE
> dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer
> Running
> 1-Dec-2014 10:49:53 AM
> dk.netarkivet.harvester.harvesting.distribute.HarvestControllerServer visit
> SEVERE: Received message stating that channel 'LOWPRIORITY' is invalid.
> Will stop.
>
>     I saw an earlier post from October where Mikis recommended restarting
> the broker and making sure the queue was clear.  I did that but the
> problem persists.  Any ideas as to how to fix it?  When I look at the
> broker logs I see the system auto-creating queues like
> HOSTNAME_COMMON_THIS_REPOS_CLIENT_IP_HCS_HIGH but no HIGHPRIORITY or
> LOWPRIORITY so I'm wondering if I need to create some sort of user
> accounts on the broker for the system to be able to register the channels.
>
>     Oh, and on another note, in the 4.4.1 deployment script the log and
> conf directories are not created automatically, which causes the install
> to fail (the start/kill scripts and config settings aren't copied
> over.)  I manually fixed this on my system during the install process
> but I thought I should mention it.
>
> _______________________________________________
> NetarchiveSuite-users mailing list
> NetarchiveSuite-users at ml.sbforge.org
> http://ml.sbforge.org/mailman/listinfo/netarchivesuite-users
>
> _______________________________________________
> NetarchiveSuite-users mailing list
> NetarchiveSuite-users at ml.sbforge.org
> http://ml.sbforge.org/mailman/listinfo/netarchivesuite-users

More information about the NetarchiveSuite-users mailing list