Tuning BPEL in a nutshell

Ok, after doing some projects, I run into issues regarding performance and stability. The to tune the BPEL process manager take this into account:

Make sure you have to latest patches applied of BPEL. The latest patches can be found a metalink.

If you are using BPEL processes that are intensively using the DB-Adapter or AQ-Adapter. Verify that the max number of connections is high. Change the connection pooling in the data-source.xml file. For 10.1.3 this file can be found in $ORACLE_HOME/j2ee/oc4j_soa/config/data-sources.xml. Make sure your oc4j-ra.xml file (DbAdapter/AqAdapter) is using data sources in the JNDI lookup. In 10.1.0.2 verify that the max-read max-write connections are high.

If you are using many async processes that are using the hydration store for persistence, verify that the max number of connections is high. Change the connection pooling in the data-source.xml file. This file for 10.1.0.2 and 10.1.3 can be found in $ORACLE_HOME/j2ee/oc4j_soa/config/data-sources.xml.

While we see locks on the ydration store when we run a high load of process, increase the initrans of the tables "CUBE_INSTANCE" "CUBE_SCOPE".

• alter table CUBE_INSTANCE initrans 16;
• alter table CUBE_SCOPE initrans 16;

You can also apply this for the indexes; alter index initrans

Make sure that the following formula applies:

• SUM(WorkBean + InvokerBean) >= SUM(dspMaxThread for each domain)
• SUM(dspMaxThreads for each domain) <= BPEL Datasource

Do not increase the Java Heapsize to large. This could lead into swapping or too much overhead in JVM. In general I suggest to set the -Xms and -Xmx equal and make these 1 GB or 2GB (the half of psychical memory). Leave at least 30% memory for de rest; O/S, Apache...

Garbage collection is happening when the BPEL PM is overloaded with tasks. Full garbage collection will take place. A parameter that influence this behavior,a bit, is to reduce the Java stack size. By default the stacksize is 512KB. This is for each thread. If you have short running processes, they do not occupy a large stack. You can reduce the stack size. Use the value -Xss128k or even -Xss96k.

The determine the correct setting for WorkerBean and the InvokerBean is hard to tell. My experience is to set the WorkBean to a value around 100 and the InvokerBean 50. Run some performance test to determine your ideal values on which the system is performing well. If you have CPU left, increase these values and also the dspMaxThreads for the particular domain.

Another nice performance advantage is to increase the value of the "com.oracle.bpel.expirationAgent.threadCount". By default this value is 10. Run tests to determine your value. I experienced good results with 30 and 50. The value van be set in the file:

• 10.1.0.2: "$ORACLE_HOME/integration/orabpel/domains/default/config/quartz-config.properties"
• 10.1.3: "$ORACLE_HOME/bpel/domains/default/config/resources-quartz.properties"

When you have a lot of BPEL processes instances running, the BPEL PM must get the unique instance number to create this instance, and even to store it in the dehydration store, the key is retrieved from a internal hash table. If it runs out of this table, it will be refilled. The initial size of this table is 100. Increasing this value to large number reduces contention. The property can be found in the domain.xml file for the particular BPEL domain. The name of the property is "instanceKeyBlockSize". Set this value high, for example 100000.

Another bottleneck I saw, was in the Apache threads, the Apache client process. Tune the number of spare servers well. Set the value MaxRequestsPerChild to a non zero value. Otherwise the child process will run forever. If the process has handle 1024 requests, it will die and apache can stat a new process if needed

httpd.conf: MaxRequestsPerChild 1024

In the $OC4J_HOME/j2ee/home/config/transaction-manager.xml file you set the global transaction timeout with the transaction-timeout attribute of the <transaction-manager> element. For example, if you wanted to set the global transaction timeout to 1800 seconds, you would do as follows: <transaction-manager ... transaction-timeout="1800"
...
</transaction-manager>


Useful links:

• Performance Tuning Synchronous BPEL Processes
• Oracle BPEL Process Manager Performance Tuning
• Handle transactions timeouts
  

• bpel_admin_10.1.3.1.0.pdf
• bpel-performance-webinar.pdf
• Java garbage collection 1.4.2
• Java 1.4.2_whitepaper
• Java 1.5 Garbage collection