Re: JDBC Threads Locking up
- From: "joeNOSPAM@xxxxxxx" <joe.weinstein@xxxxxxxxx>
- Date: Wed, 20 Jun 2007 17:35:29 -0000
On Jun 20, 12:03 am, samee...@xxxxxxxxx wrote:
On Jun 19, 12:58 pm, "joeNOS...@xxxxxxx" <joe.weinst...@xxxxxxxxx>
wrote:
On Jun 19, 10:59 am, samee...@xxxxxxxxx wrote:
On Jun 15, 4:20 pm, "joeNOS...@xxxxxxx" <joe.weinst...@xxxxxxxxx>
wrote:
On Jun 15, 11:02 am, samee...@xxxxxxxxx wrote:
Hi,
We are using Apache,Tomcat 5,ojdbc14.jar,Oracle 10g based system and
observing that all 100 tomcat threads are getting stuck in following
situation.
87 threads are stuck in "waiting to lock <0x718fc270> (a
oracle.jdbc.pool.OracleConnectionEventListener)"
TP-Processor100 daemon prio=1 tid=0x5c430080 nid=0x7964 waiting for
monitor entry [0x546c2000..0x546c3868]
at
oracle.jdbc.pool.OracleConnectionEventListener.connectionClosed(OracleConnectionEventListener.java:
129)
- waiting to lock <0x718fc270> (a
oracle.jdbc.pool.OracleConnectionEventListener)
at
oracle.jdbc.pool.OraclePooledConnection.callListener(OraclePooledConnection.java:
482)
at
oracle.jdbc.pool.OraclePooledConnection.logicalClose(OraclePooledConnection.java:
445)
- locked <0x6254f030> (a oracle.jdbc.pool.OraclePooledConnection)
at
oracle.jdbc.driver.OracleConnection.logicalClose(OracleConnection.java:
2936)
- locked <0x6254f0f0> (a oracle.jdbc.driver.OracleConnection)
at oracle.jdbc.driver.OracleConnection.close(OracleConnection.java:
1454)
- locked <0x6254f0f0> (a oracle.jdbc.driver.OracleConnection)
One thread has locked "<0x718fc270> (a
oracle.jdbc.pool.OracleConnectionEventListener)" and it is waiting to
lock <0x718fc0e0> (a oracle.jdbc.pool.OracleConnectionCacheImpl).
TP-Processor45 daemon prio=1 tid=0x081cf9b8 nid=0x7d0 waiting for
monitor entry [0x562fa000..0x562fb868]
at
oracle.jdbc.pool.OracleConnectionCacheImpl.closeSingleConnection(OracleConnectionCacheImpl.java:
677)
- waiting to lock <0x718fc0e0> (a
oracle.jdbc.pool.OracleConnectionCacheImpl)
at
oracle.jdbc.pool.OracleConnectionCacheImpl.reusePooledConnection(OracleConnectionCacheImpl.java:
592)
at
oracle.jdbc.pool.OracleConnectionEventListener.connectionClosed(OracleConnectionEventListener.java:
130)
- locked <0x718fc270> (a
oracle.jdbc.pool.OracleConnectionEventListener)
at
oracle.jdbc.pool.OraclePooledConnection.callListener(OraclePooledConnection.java:
482)
at
oracle.jdbc.pool.OraclePooledConnection.logicalClose(OraclePooledConnection.java:
445)
- locked <0x62354878> (a oracle.jdbc.pool.OraclePooledConnection)
at
oracle.jdbc.driver.OracleConnection.logicalClose(OracleConnection.java:
2936)
- locked <0x62354938> (a oracle.jdbc.driver.OracleConnection)
at oracle.jdbc.driver.OracleConnection.close(OracleConnection.java:
1454)
- locked <0x62354938> (a oracle.jdbc.driver.OracleConnection)
One thread has locked <0x718fc0e0>, but it is waiting in
socketRead0(), so the lock can not released.
This stack trace is originating from while obtaining a new database
connection.
TP-Processor10 daemon prio=1 tid=0x084d4f50 nid=0x70d5 runnable
[0x5cd7c000..0x5cd7e868]
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at oracle.net.ns.Packet.receive(Unknown Source)
at oracle.net.ns.DataPacket.receive(Unknown Source)
at oracle.net.ns.NetInputStream.getNextPacket(Unknown Source)
t oracle.net.ns.NetInputStream.read(Unknown Source)
at oracle.net.ns.NetInputStream.read(Unknown Source)
at oracle.net.ns.NetInputStream.read(Unknown Source)
at oracle.jdbc.ttc7.MAREngine.unmarshalUB1(MAREngine.java:931)
at oracle.jdbc.ttc7.MAREngine.unmarshalSB1(MAREngine.java:893)
at oracle.jdbc.ttc7.Oall7.receive(Oall7.java:375)
at oracle.jdbc.ttc7.TTC7Protocol.doOall7(TTC7Protocol.java:1894)
at oracle.jdbc.ttc7.TTC7Protocol.parseExecuteFetch(TTC7Protocol.java:
1094)
- locked <0x62bb7a10> (a oracle.jdbc.ttc7.TTC7Protocol)
at
oracle.jdbc.driver.OracleStatement.executeNonQuery(OracleStatement.java:
2132)
at
oracle.jdbc.driver.OracleStatement.doExecuteOther(OracleStatement.java:
2015)
at
oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:
2877)
at oracle.jdbc.driver.OracleStatement.execute(OracleStatement.java:
925)
- locked <0x62bc4228> (a oracle.jdbc.driver.OracleStatement)
- locked <0x62bb7a80> (a oracle.jdbc.driver.OracleConnection)
at oracle.jdbc.dbaccess.DBAccess.setNlsParamsClient(DBAccess.java:
1680)
at oracle.jdbc.ttc7.TTC7Protocol.initNls(TTC7Protocol.java:1592)
- locked <0x62bb7a10> (a oracle.jdbc.ttc7.TTC7Protocol)
at oracle.jdbc.driver.OracleConnection.<init>(OracleConnection.java:
406)
at
oracle.jdbc.driver.OracleDriver.getConnectionInstance(OracleDriver.java:
521)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:325)
at java.sql.DriverManager.getConnection(DriverManager.java:512)
- locked <0xa1efd890> (a java.lang.Class)
at java.sql.DriverManager.getConnection(DriverManager.java:140)
- locked <0xa1efd890> (a java.lang.Class)
at
oracle.jdbc.pool.OracleDataSource.getConnection(OracleDataSource.java:
171)
- locked <0x718fc1e0> (a
oracle.jdbc.pool.OracleConnectionPoolDataSource)
at
oracle.jdbc.pool.OracleConnectionPoolDataSource.getPhysicalConnection(OracleConnectionPoolDataSource.java:
149)
at
oracle.jdbc.pool.OracleConnectionPoolDataSource.getPooledConnection(OracleConnectionPoolDataSource.java:
95)
- locked <0x718fc1e0> (a
oracle.jdbc.pool.OracleConnectionPoolDataSource)
at
oracle.jdbc.pool.OracleConnectionPoolDataSource.getPooledConnection(OracleConnectionPoolDataSource.java:
63)
- locked <0x718fc1e0> (a
oracle.jdbc.pool.OracleConnectionPoolDataSource)
at
oracle.jdbc.pool.OracleConnectionCacheImpl.getNewPoolOrXAConnection(OracleConnectionCacheImpl.java:
547)
at
oracle.jdbc.pool.OracleConnectionCacheImpl.getPooledConnection(OracleConnectionCacheImpl.java:
404)
at
oracle.jdbc.pool.OracleConnectionCacheImpl.getConnection(OracleConnectionCacheImpl.java:
298)
- locked <0x718fc0e0> (a oracle.jdbc.pool.OracleConnectionCacheImpl)
at
oracle.jdbc.pool.OracleConnectionCacheImpl.getConnection(OracleConnectionCacheImpl.java:
268)
- locked <0x718fc0e0> (a oracle.jdbc.pool.OracleConnectionCacheImpl)
Did somebody ran into a similar situation? Please Help.
Thanks,
Sameer
Show how your pool is defined. This points to a serious lack of
concurrency
consideration in the Oracle driver. One terrible thing we can see in
the
driver is that internally it is calling DriverManager.getConnection()!
This
is a huge gaff on the part of the driver because all DriverManager
calls
are class-synchronized, and many other of these calls are made by
drivers
and even SQLException constructors, so one long-running DriverManager
call
can block all other JDBC in the entire JVM.
I suggest you stop using the Oracle driver-based pooling if Tomcat
offers
it's own pooling.
Joe Weinstein at BEA Systems
Here are the connection pool parameters
CacheFixedWaitIdleTime = 30
CacheFixedWaitTimeout = 30
CacheFixedWaitTimeout = 30
CacheScheme = 2
CacheSize = 30
CacheTimeToLiveTimeout = 180
MaxLimit = 100
MaxStatements = 0
MinLimit = 30
NetworkProtocol = tcp
StmtCacheSize = 0
ThreadWakeUpInterval = 300
LoginTimeout = 0
ExplicitCachingEnabled = false
ImplicitCachingEnabled = false
(I posted this 2 days ago, but looks like it did not get posted)
Download the latest (10.2.0.3) version of the ojdbc14.jar,
and try it. Duplicate the problem and post the entire thread dump.
That's not the whole pool definition. You should be specifying
the oracle.Driver class or an Oracle XADataSource. I suspect you're
defining the pool to use the wrong driver class and/or with
the wrong properties.
Joe Weinstein at BEA Systems
This is the connection pool initialization code
ods = new OracleConnectionCacheImpl();
// Set the connection specific attributes
ods.setURL("jdbc:oracle:thin:@host_name:1521:instance_name");
ods.setUser(...);
ods.setPassword(...);
ods.setMaxLimit(100);
ods.setMinLimit(30);
ods.setCacheScheme(1); // Dynamic Scheme
I should mention here that the same drive and configuration has worked
for us for a long
time in the past. We did roll out new code and only after that this
issue started surfacing.
No code has been added/changed in connection pooler. Do you think some
application code can
cause this issue? like calling conn.close() (i.e. returning it to
pool)
but still continue using it or any other improper DB level locking?
Does tomcat provide a connection pooling implementation?
If so, use that. Don't use the Oracle driver-level pooling.
COnfigure the Tomcat (or other pooling implementation) to
simply use the plain driver class of the thin driver.
Joe
.
- References:
- JDBC Threads Locking up
- From: sameergn
- Re: JDBC Threads Locking up
- From: joeNOSPAM@xxxxxxx
- Re: JDBC Threads Locking up
- From: sameergn
- Re: JDBC Threads Locking up
- From: joeNOSPAM@xxxxxxx
- Re: JDBC Threads Locking up
- From: sameergn
- JDBC Threads Locking up
- Prev by Date: Re: JDBC Threads Locking up
- Next by Date: How to sort a table values
- Previous by thread: Re: JDBC Threads Locking up
- Next by thread: Re: JDBC Threads Locking up
- Index(es):
Relevant Pages
|
