[Netarchivesuite-devel] Database bug in NAS3.10

Colin Rosenthal csr at statsbiblioteket.dk
Tue Jan 19 13:17:23 CET 2010

Hi everybody,
Since installing NAS 3.10 in our production environment we have been 
plagued by derby database deadlocks between the Domain update
process and the Job select process. A typical deadlock report looks like:

Jan 14, 2010 3:07:30 PM dk.netarkivet.harvester.datamodel.JobDBDAO read
WARNING: SQL error reading job 69183 in database SQLException trace:
SQL State:40001
Error Code:30000
java.sql.SQLTransactionRollbackException: A lock could not be obtained due to a deadlock, cycle of locks and waiters
Lock : ROW, DOMAINS, (346,21)
  Waiting XID : {9992407455, S} , APP, SELECT domains.name, configurations.name FROM domains, configurations, job_configs
WHERE job_configs.job_id = ?  AND job_configs.config_id = configurations.config_id  AND domains.domain_id
= configurations.domain_id
  Granted XID : {9992407447, X}
Lock : ROW, CONFIGURATIONS, (4213,61)
  Waiting XID : {9992407447, X} , APP, UPDATE configurations SET comments = ?, template_id = ( SELECT template_id FROM
ordertemplates WHERE name = ? ), maxobjects = ?, maxrate = ?, maxbytes = ? WHERE name = ? AND domain_id = ?
  Granted XID : {9992407455, S}
. The selected victim is XID : 9992407455.
        at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
        at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
        at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
        at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
        at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
        at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source)
        at org.apache.derby.impl.jdbc.EmbedResultSet.closeOnTransactionError(Unknown Source)
        at org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(Unknown Source)
        at org.apache.derby.impl.jdbc.EmbedResultSet.next(Unknown Source)
        at dk.netarkivet.harvester.datamodel.JobDBDAO.read(JobDBDAO.java:437)
        at dk.netarkivet.harvester.scheduler.HarvestScheduler.stopTimeoutJobs(HarvestScheduler.java:198)
        at dk.netarkivet.harvester.scheduler.HarvestScheduler.scheduleJobs(HarvestScheduler.java:239)
        at dk.netarkivet.harvester.scheduler.HarvestScheduler.access$100(HarvestScheduler.java:65)
        at dk.netarkivet.harvester.scheduler.HarvestScheduler$1.run(HarvestScheduler.java:150)
        at java.util.TimerThread.mainLoop(Timer.java:512)
        at java.util.TimerThread.run(Timer.java:462)

Our current best thinking is that this is related to a known derby bug:
http://issues.apache.org/jira/browse/DERBY-2991 .

We are currently working on the issue and actively trying to 
a) reproduce it in our test environment, and
b) find a fix or workaround.

We would be interested to hear if any other users have experienced anything similar.

Colin Rosenthal
IT Developer

More information about the Netarchivesuite-devel mailing list