[Netarchivesuite-devel] About the DBConnect class, running out of connections

Nicolas Giraud nikokode at gmail.com
Fri Mar 25 16:37:53 CET 2011


Ok, thanks for the tip.

Actually I ran into this problem on my dev platform since 3.13 if I remember
correctly, but since it never showed up in production, I never investigated
it thoroughly like today.

One thing I tried, is replacing the WeakHashMap mapping connections by
threads by a default HashMap, and logging whether threads are alive or not.
And it shows that there are still valid DB connections for dead threads. So
that's really a waste and a unsafe code here. A simple workaround would be
to have a thread periodically do the cleanup ... but I'll rather look at
c3p0.

As soon as I am sure I fully understood the problem I'll file a Jira bug.

Regards,

Nicolas

2011/3/25 Søren Vejrup Carlsen <svc at kb.dk>

>  Hi Nicolas.
>
> I believe, that we are already using “c3p0”  in the wayback module, which
> uses hibernate to access a simple derby database.
>
> Look especially at the code in the package dk.netarkivet.wayback.indexer
>
>
>
> I’m curious that you didn’t get this problem until launching 3.15.1. Why
> can that be?
>
>
>
> Best Regards
>
> Søren
>
>
>
> *Fra:* netarchivesuite-devel-bounces at lists.gforge.statsbiblioteket.dk[mailto:
> netarchivesuite-devel-bounces at lists.gforge.statsbiblioteket.dk] *På vegne
> af *Nicolas Giraud
> *Sendt:* 25. marts 2011 16:14
>
> *Til:* netarchivesuite-devel at lists.gforge.statsbiblioteket.dk
> *Emne:* Re: [Netarchivesuite-devel] About the DBConnect class, running out
> of connections
>
>
>
> The problem occurs whether the DB server is on the same machine or a remote
> host.
>
> I am considering to set up a real connection pool. As we have different
> target systems, I'd go for c3p0 or dbcp. Do you guys have some experience
> with these APIs, would you recommend one? C3p0 seems really simple to put in
> place.
>
> Best,
>
> Nicolas
>
> 2011/3/25 Søren Vejrup Carlsen <svc at kb.dk>
>
> Hi Nicolas.
>
> You could be right about NAS never closing the database connections. But
> the connections to the database timeout automatically,
>
> don’t they?
>
>
>
> We’ve had similar problems with derby in our production environment running
> 3.14.1.
>
>
>
> But we didn’t see in our test-environment.
>
> Are you running the database on the same server as the processes running?
>
>
>
> Best Regards
>
> Søren
>
>
>
> *Fra:* netarchivesuite-devel-bounces at lists.gforge.statsbiblioteket.dk[mailto:
> netarchivesuite-devel-bounces at lists.gforge.statsbiblioteket.dk] *På vegne
> af *Nicolas Giraud
> *Sendt:* 25. marts 2011 14:57
> *Til:* netarchivesuite-devel at lists.gforge.statsbiblioteket.dk
> *Emne:* [Netarchivesuite-devel] About the DBConnect class, running out of
> connections
>
>
>
> Hi,
>
> On our new production environment, we have run into a error where the DB
> has too many open connections. I've started investigating on this, by
> looking at the DBConnect class.
>
> First the "connection pool" in it seems bugged, as if any Thread in the
> WeakHashMap is garbage collected, the associated connection might still be
> open and is not closed. Also I don't see anywhere in all of the code a call
> to Connection#close(), except in DomainDBDao#close() which by the way is
> never called.
>
> Where are DB connections closed ? It seems they are never closed, unless I
> have completely misunderstood the code.
>
> Any help will be greatly appreciated.
>
> Cheers,
>
> --
> Nicolas Giraud
>
> ---------------------------------------------------------------------------------------------
> Développeur Archives du Web - Bibliothèque Nationale de France
> Web Archiving Developper - National Library of France
>
> ---------------------------------------------------------------------------------------------
>
>
> _______________________________________________
> Netarchivesuite-devel mailing list
> Netarchivesuite-devel at lists.gforge.statsbiblioteket.dk
>
> https://lists.gforge.statsbiblioteket.dk/mailman/listinfo/netarchivesuite-devel
>
>
>
>
> --
> Nicolas Giraud
>
> ---------------------------------------------------------------------------------------------
> Développeur Archives du Web - Bibliothèque Nationale de France
> Web Archiving Developper - National Library of France
>
> ---------------------------------------------------------------------------------------------
>
> _______________________________________________
> Netarchivesuite-devel mailing list
> Netarchivesuite-devel at lists.gforge.statsbiblioteket.dk
>
> https://lists.gforge.statsbiblioteket.dk/mailman/listinfo/netarchivesuite-devel
>
>


-- 
Nicolas Giraud
---------------------------------------------------------------------------------------------
Développeur Archives du Web - Bibliothèque Nationale de France
Web Archiving Developper - National Library of France
---------------------------------------------------------------------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ml.sbforge.org/pipermail/netarchivesuite-devel/attachments/20110325/e05adb92/attachment-0002.html>


More information about the Netarchivesuite-devel mailing list