While he have running this production environments for a long time, we thought we have tuned this very well; on database as on application server. As I spoke earlier on Java Garbage collection on http://orasoa.blogspot.com/2011/09/soa-11g-improved-memory-and-garbage.html, and on tuning the SOA Suite 11g; http://orasoa.blogspot.com/2011/08/tuning-soa-11g-in-nutshell.html.
We first thought, it had to do the the applications we made; Composites, Custom Java Classes, Services. Is something changed in the meantime? We also thought it had to do with the amount of data we are processing, is there more data processed than normal? Something changed in the database? The network? The physical storage? All of them were not changed, looking in the program code; we did not find anything special.
We tried to analyse the a heap dump, digged into log files, Oracle Support Bug Database and many-many-other-things. I stared a long time to this
#
# A fatal error has been detected by the Java Runtime Environment:
#
# java.lang.OutOfMemoryError: requested 32756 bytes for ChunkPool::allocate. Out of swap space?
#
The I was triggered by a bug in the Oracle/Sun Java; http://bugs.sun.com/view_bug.do?bug_id=6973402 . For some reason I downloaded the latest Java 1.6 SDK and applied this on one managed server on one the SOA 11g Suite.
The result was amazing!
- Less CPU
- Less Heap usage
- Normal garbage collections
We see this difference also on de Oracle Service Bus (!). My advise, roll out Java 1.6.0 build 30 (or later) on your existing production environments, SOA Suite, WebCenter, ServiceBus, etc!
Here are the results of our instance running without Java 1.6.0_b30 and at the bottom with Java 1.6.0_b30. And see the incredible difference. I did not show the CPU graphs with the garbage collections, but those are also different.
Java Heap Before applying Java 1.6.0 Build 30
Java Heap After applying Java 1.6.0 Build 30
3 comments:
That's interesting Marc. Your graphs suggest that before the update it was using about 1100-650=450MB heap in 20 mins, whereas after about 800-400=400MB. Of course that's nothing to be sniffed at, but am I missing something else?
Simon
Hi Simon, The test was executed on a production environment, the first graph is a managed server without the patch, the second with the patch. Overall time, both managed servers handled the same load. You do not miss something. We saw the same difference on OSB, SOA, ADF managed servers.
That's absorbing Marc. Your graphs advance that afore the amend it was application about 1100-650=450MB abundance in 20 mins, admitting afterwards about 800-400=400MB. Of advance that's annihilation to be sniffed at, but am I missing article else?
call center outsourcing
Post a Comment