SOA 11g: Automatic Recovery

A cool feature in SOA 11g, is the functionality to recover faulted instances on a particular time. This feature is introduced in PS2. Instances that are faulted and marked to be recovered, can be retried on a particular time.

It is some times a disadvantage, while you do not want this feature. By default it is recovering faulted instance form 00:00 hours. If your instances are still failing, that soa-infra tablespaces grows.

The feature can be disables or changed on an other time, via Enterprise Manager by setting the BPEL property under soa-infra -> SOA Administartion -> BPEL Properties -> More Advanced BPEL Properties -> recoveryConfig.

 

Put the value maxMessageRaiseSize to 0, to disable auto recovery. I even set the begin and end time the same.

Scripting your Oracle SOA 11g Cluster

Did you ever tried to create your own WLST script to create a SOA 11g clustered domain? I did :-) By default the SOA 11g comes out with his own template. The bad thing about this template, it contains a predefined managed server and local machine and all the targetted objects (libraries, applications, JDBC, JMS, startup and shutdown classess.

It took me some time to find out what I need to create a clusted domain, based on my own specification. I wanna share you the information that is needed to create a SOA domain with the correct targets.

Before you begin you have to do some things. I expect you use the out-of-the-box SOA 11g template to create your domain. This will result in a domain, that contains all the applications, libraries, etc. But it also contains a managed server named 'soa_server1' and an machine 'LocalMachine'. These object will not have your settings regarding to logging, naming-convention, memory and many other settings. They are not usefull. 

If you remove these objects, you loose the SOA 11g configuration, the targetted libraries/application/classes/jdbc and JMS.

For JMS it is even worse, the default configuration is based on a non-clustered setup. Therefore you need to run a seperate script to migrate these JMS configuration to a clustered aware configuration.
If you do not use the GUI interface, but you only wanna use WLST scripting, this is my approach:

• Create a SOA domain based on the default templates.
• Start the Administration Server.
• Execute the next script.
• Perform the 'soa-createUDD.py' to create the JMS Cluster:

• $WLS_SERVER/common/bin/wlst.sh $ORACLE_HOME/bin/soa-createUDD.py --domain_home /data/user_projects/domains/MyDomain --soacluster My-Cluster --create_jms true

• Apply the Java Object Caching script, configure-joc.py
• Pack and Unpack the domain configuration to the other server
• Start the Managed Servers.

Happy SOA-ing :-)

targetClusters=['My-Cluster']

targetLibraries=[
  'adf.oracle.domain#1.0@11.1.1.2.0'
, 'adf.oracle.domain.webapp#1.0@11.1.1.2.0'
, 'jsf#1.2@1.2.9.0'
, 'jstl#1.2@1.2.0.1'
, 'ohw-rcf#5@5.0'
, 'ohw-uix#5@5.0'
, 'UIX#11@11.1.1.1.0'
, 'oracle.adf.dconfigbeans#1.0@11.1.1.2.0'
, 'oracle.adf.management#1.0@11.1.1.2.0'
, 'oracle.dconfig-infra#11@11.1.1.1.0'
, 'oracle.jrf.system.filter'
, 'oracle.jsp.next#11.1.1@11.1.1'
, 'oracle.pwdgen#11.1.1@11.1.1.2.0'
, 'oracle.rules#11.1.1@11.1.1'
, 'oracle.sdp.client#11.1.1@11.1.1'
, 'oracle.sdp.messaging#11.1.1@11.1.1'
, 'oracle.soa.b2b#11.1.1@11.1.1'
, 'oracle.soa.bpel#11.1.1@11.1.1'
, 'oracle.soa.composer.webapp#11.1.1@11.1.1'
, 'oracle.soa.ext#11.1.1@11.1.1'
, 'oracle.soa.mediator#11.1.1@11.1.1'
, 'oracle.soa.rules_dict_dc.webapp#11.1.1@11.1.1'
, 'oracle.soa.rules_editor_dc.webapp#11.1.1@11.1.1'
, 'oracle.soa.workflow#11.1.1@11.1.1'
, 'oracle.soa.workflow.wc#11.1.1@11.1.1'
, 'oracle.soa.worklist#11.1.1@11.1.1'
, 'oracle.soa.worklist.webapp#11.1.1@11.1.1'
, 'oracle.wsm.seedpolicies#11.1.1@11.1.1'
, 'orai18n-adf#11@11.1.1.1.0'
]

targetApplications=[
  'AqAdapter'
, 'DbAdapter'
, 'DefaultToDoTaskFlow'
, 'FileAdapter'
, 'FtpAdapter'
, 'JmsAdapter'
, 'MQSeriesAdapter'
, 'OracleAppsAdapter'
, 'OracleBamAdapter'
, 'SocketAdapter'
, 'b2bui'
, 'composer'
, 'soa-infra'
, 'usermessagingdriver-email'
, 'usermessagingserver'
, 'worklistapp'
, 'wsil-wls'
, 'wsm-pm'
, 'DMS Application#11.1.1.1'
]

targetJDBC=[
  'EDNDataSource'
, 'EDNLocalTxDataSource'
, 'mds-owsm'
, 'mds-soa'
, 'OraSDPMDataSource'
, 'SOADataSource'
, 'SOALocalTxDataSource'
]

targetShutdownClass=[
  'JOC-Shutdown'
, 'DMSShutdown'
]

targetStartupClass=[
  'JPS Startup Class'
, 'JRF Startup Class'
, 'ODL-Startup'
, 'Audit Loader Startup Class'
, 'AWT Application Context Startup Class'
, 'JMX Framework Startup Class'
, 'JOC-Startup'
, 'DMS-Startup'
, 'OWSM Startup class'
, 'SOAStartupClass'
]

targetWorkManagers=[
  'wm/SOAWorkManager'
]

targetSystemResources=[
  'Module-FMWDFW'
]

undeployApps=['FMW Welcome Page Application#11.1.0.0.0']

def connectToAdminsServer():
  connect('weblogic', 'welcome1', 't3://node1.vijfhuizen.com:7001')

def undeployApplications(listofobjects):
  cd ('/AppDeployments')

  listofcurrentobjects = ls(returnMap='true')
  for o in listofcurrentobjects:
    targetit = false
    for i in listofobjects:
      if i == o:
        targetit = true
    if targetit == true:
      print 'Undeploy -> ' + o
      undeploy(o)

def setNewTarget(newtarget, newobject, listofobjects):
  cd('/Clusters/' + newtarget)
  cl_target = cmo
  cd(newobject)
  listofcurrentobjects = ls(returnMap='true')
  for o in listofcurrentobjects:
    targetit = false
    cd(o)
    curobject = cmo
    curobjectname = cmo.name
    for i in listofobjects:
      if i == curobjectname:
        targetit = true
    if targetit == true:
      print curobjectname + ' -> ' + cl_target.name
      curobject.addTarget(cl_target)
    
    cd('..')

def destroyOOTBComponents():
  print "@@@ Remove dummy soa_server @@@"
  cd('/Servers/soa_server1')
  cmo.setCluster(None)
  cmo.setMachine(None)
  cd('/')
  editService.getConfigurationManager().removeReferencesToBean(getMBean('/Servers/soa_server1'))
  cmo.destroyServer(getMBean('/Servers/soa_server1'))

  print "@@@ Remove dummy machine @@@"
  editService.getConfigurationManager().removeReferencesToBean(getMBean('/Machines/LocalMachine'))
  cmo.destroyMachine(getMBean('/Machines/LocalMachine'))

  print "@@@ Remove JMS Stuff @@@"

  cmo.destroyJMSSystemResource(getMBean('/SystemResources/SOAJMSModule'))
  cmo.destroyJMSSystemResource(getMBean('/SystemResources/UMSJMSSystemResource'))
  cmo.destroyJMSServer(getMBean('/Deployments/SOAJMSServer'))
  cmo.destroyJMSServer(getMBean('/Deployments/UMSJMSServer'))
  cmo.destroyFileStore(getMBean('/FileStores/SOAJMSFileStore'))
  cmo.destroyFileStore(getMBean('/FileStores/UMSJMSFileStore'))

# 
# It starts all here
#

mycluster = 'My-Cluster'
connectToAdminsServer()
edit()
startEdit()

destroyOOTBComponents()

for lCluster in targetClusters:
  print "@@@ Remove uneeded applications @@@"
  undeployApplications(undeployApps)
  print "@@@ Retarget @@@"
  setNewTarget(mycluster, '/Libraries', targetLibraries)
  setNewTarget(mycluster, '/AppDeployments', targetApplications)
  setNewTarget(mycluster, '/JDBCSystemResources', targetJDBC)
  setNewTarget(mycluster, '/StartupClasses', targetStartupClass)
  setNewTarget(mycluster, '/ShutdownClasses', targetShutdownClass)
  setNewTarget(mycluster, '/SelfTuning/KIMPTC/WorkManagers', targetWorkManagers)
  setNewTarget(mycluster, '/SystemResources', targetSystemResources)

save()
activate()

The Linux script to call all the post actions.

#!/bin/bash

#
# Post SOA Configuration
#
$WLS_SERVER/common/bin/wlst.sh postsoa.py

#
# Recreate SOA JMS Queues
#
$WLS_SERVER/common/bin/wlst.sh $ORACLE_HOME/bin/soa-createUDD.py --domain_home /data/user_projects/domains/MYDOMAIN --soacluster My-Cluster --create_jms true

#
# Create Java Object Cache (Changes hostnames/ports!)
# Using inline input (note the empty lines!)
#
$ORACLE_HOME/common/bin/wlst.sh <<!EOF
connect('weblogic', 'welcome1', 't3://node1.vijfhuizen.com:7001')
execfile('/opt/weblogic/Middleware/oracle_common/bin/configure-joc.py')
node1.vijfhuizen.com,node2.vijfhuizen.com

KIMPTC-Cluster
9991


exit()
EOF

Change diagnostic logging

Apart of the default logging facilities in WebLogic, the Oracle products such as SOA Suite, WebCenter  are also using the custom diagnostics loggers. The normal logfiles are configured during the domain configuration by using WLST via edit/activate sessions.



The diagniostics log files that are shown Enterprise can be changed directly via EM or via WLST. By default these files are created in the log directory of the server. To change these values via WLST can be done as follows:

#
#
# Script to set ODL logging
#
# Start this script with the wlst.sh script from oracle_common
#
# $FMW_HOME/oracle_common/bin/wlst.sh -i changeodllogging.py
#

import os
import sys

username = 'weblogic'
password = ''
adminurl = 't3://node1.vijfhuizen.com:7001'

#Domain settings
domain_name=''

domain_dir='/data/user_projects/domains/' + domain_name

# Connectionsettings
def connectToServer():
  connect(username, password, adminurl)

#Definition to disconnect from a server
def disconnectFromServer():
  disconnect()
  exit()

# change the odl-handler log path
def changeODLPath():
  domainConfig()
  managedServers=cmo.getServers()

  res = ''
  for managedServer in managedServers:
    sname=managedServer.getName()
    path = '/Servers/' + sname
    cd(path)

    lh = listLogHandlers()
    for l in lh:
      lname = l.get('name')
      odlfile = '/data/logs/' + domain_name + '/' + sname + '-' + lname + '-diagnostic.log'
      configureLogHandler(target=sname,name=lname, path=odlfile)
      res = res + '\n' + odlfile

#Calling connectToServer definition with no arguments
connectToServer()

# do the change om all servers ( Admin, managed)
changeODLPath()

#Calling disconnectFromServer definition with no arguments
disconnectFromServer()

Oracle Repository Creattion Utility (RCU) and product options

In case you want to script your installtion and configuration of your WLS environment, you need to select which of the components you need from the RCU. Here is a table that shows which options you need to install for a particular application.

if you have any comments please feel free to send me an email, and I will update the table.

RCU Options	OSB	SOA	BAM	WebC	UCM	OID
Metadata Services (MDS)	X	X	X	X	X	X
Audit Services (IAU)		X	X			X
Oracle Internet Directory(OID)		O	O	O	O	X
Single Sign-On (SSO)		O	O	O	O	X
SOA Infrastructure (SOAINFRA)	X	X
Business Activity Monitoring(BAM)	X	O	X
User Messaging (ORASDPM)		X	X
Oracle WebCenter (WEBCENTER)				X
WebCenter Portlets (PORTLETS)				X
Oracle Content Server(OCSERVER)				O	X
Oracle Identity Federation(OIF)						X
Discussions (DISCUSSIONS)				O
Wiki and Blog (WIKI)				O

Monitor your Weblogic Servers

As we have running SOA Suite, Service Bus or any other application on top of weblogic. You are intereseted in the behavior of the server. I have created a script that reads for all managed servers the following information:

• Memory usage
• Datasource connections
• HTTP Session
• Threads

The script is straight forward and appends each time a line in a result file. This can be used for furthe analyzing.

The property file:

# 
# Properties file
#
username=weblogic
password=welcome1
adminurl=t3://node1.vijfhuizen.com:7001
domain=MyDomain

#
# Show Sessions
#
showsession=true
appname=HelloWorldApplication
webname=/helloworld

# Show threads
showthreads=true

# Show memory
showmemory=true

# Show datasources
showdatasources=true

#
# Result file
#
appendresultsto=/tmp/wlmon.csv

The WLST script:

import traceback
import getopt

from java.util import Calendar

import getopt
import sys

# simulate command line invocation

propFile= file('wlsmon.properties', 'r')
propDict= dict()
for propLine in propFile:
  propDef= propLine.strip()
  if len(propDef) == 0:
        continue
  if propDef[0] in ( '!', '#' ):
        continue
  punctuation= [ propDef.find(c) for c in ':= ' ] + [ len(propDef) ]
  found= min( [ pos for pos in punctuation if pos != -1 ] )
  name= propDef[:found].rstrip()
  value= propDef[found:].lstrip(":= ").rstrip()
  propDict[name]= value

propFile.close()

# Connection properties
username = propDict['username']
password = propDict['password']
adminurl = propDict['adminurl']
domain   = propDict['domain']
appname  = propDict['appname']
webname  = propDict['webname']

# Switches
showsession     = propDict['showsession']
showthreads     = propDict['showthreads']
showmemory      = propDict['showmemory']
showdatasources = propDict['showdatasources']

# Open file in append mode
appendresultsto = propDict['appendresultsto']
f = open(appendresultsto,'a')

# Connect to Adminserver
connect(username, password, adminurl)

#Get number of servers from the domain config
domainConfig()
managedServers=cmo.getServers()

heap_string       = ''
session_string    = ''
datasource_string = ''
thread_string     = ''

#Get Runtime for our server
domainRuntime()
for managedServer in managedServers:
  servername=managedServer.getName()

  path = '/ServerRuntimes/' + servername
  cd(path)

  if cmo.getState() == 'RUNNING':

    #Get all running applications
    path='/ServerRuntimes/' + servername + '/JVMRuntime/' + servername
    cd(path)

    if showmemory == 'true':
      hpfree = str(cmo.getHeapFreeCurrent())
      hpfreepercent = str(cmo.getHeapFreePercent())
      hpsizecurrent = str(cmo.getHeapSizeCurrent())
      hpsizemax = str(cmo.getHeapSizeMax())
      uptime = str(cmo.getUptime())
 
      heap_string = uptime + ';' + hpsizemax + ';' + hpsizecurrent + ';' + hpfree + ';' + hpfreepercent
    
    path = '/ServerRuntimes/' + servername
    cd(path)

    if showsession == 'true':

      apps=cmo.getApplicationRuntimes()
      for app in apps:
      
        #We are intersted only on this application
        if app.getName() == appname:
    
          #Get all components in that application
          comps=app.getComponentRuntimes()
          for comp in comps:
              
            if comp.getName() == servername + '_/' + webname:
              #We are interested in only web components
              if comp.getType() == 'WebAppComponentRuntime':
                #Get all active sessions
                sessionids=comp.getServletSessionsMonitoringIds()
                session_string = str(len(sessionids))

    path='/ServerRuntimes/' + servername + '/JDBCServiceRuntime/' + servername
    cd(path)

    if showdatasources == 'true':
      dsMBeans = cmo.getJDBCDataSourceRuntimeMBeans()
      for ds in dsMBeans:        
        name                             = ds.getName()
        state                            = ds.getState()
        currcapacity                     = str(ds.getCurrCapacity())
        activeconnectionscurrentcount    = str(ds.getActiveConnectionsCurrentCount())
        activeconnectionshighcount       = str(ds.getActiveConnectionsHighCount())
        datasource_string = datasource_string + ';' + name + ';' + state + ';' + currcapacity + ';' + activeconnectionscurrentcount + ';' + activeconnectionshighcount 

    path='/ServerRuntimes/' + servername
    cd(path)

    if showthreads == 'true':
      opensocks = cmo.getOpenSocketsCurrentCount();

      path = '/ServerRuntimes/' + servername + '/ThreadPoolRuntime/ThreadPoolRuntime/'
      cd(path)

      executethreadtotal    = str(cmo.getExecuteThreadTotalCount())
      standbythreads        = str(cmo.getStandbyThreadCount())
      idlethreads           = str(cmo.getExecuteThreadIdleCount())
      pending               = str(cmo.getPendingUserRequestCount())
      hoggingthreads        = str(cmo.getHoggingThreadCount())
      serverthroughput      = str(cmo.getThroughput())

      thread_string = str(opensocks) + ';' + executethreadtotal + ';' + standbythreads + ';' + idlethreads + ';' + pending + ';' + hoggingthreads + ';' + serverthroughput

    cal = Calendar.getInstance()
    dt = cal.getTime()
    f.write(str(dt) + ';' + servername + ';' + heap_string + ';' + session_string + datasource_string + ';' + thread_string + '\n')

#Disconnect and exit
f.close()
disconnect()
exit()

OSB Deployment with ANT

As deployment woth OSB 11g can be done via Eclise (Oracle Workshop), via the SB Console,  then ANT deployment is not straight forward.

• Edwin Biemond wrote an article.
• Eric Elzinga run into an issue.

 If you run into ANT errors with OSB, make sure:

• You delete the .metadata directory before starting de deployment. OSB will recreate the .metatdata folder.
• If errors occurs, check the .metadata/.log file

SOA 11g PS install behaviour

If you install from the Oracle SOA Suite PS2 only the SOA components, with the graphical installer or via silent install and you did the domain confguration, you will see that the 'soainfra' application has warnings. It is running, but in the log file of the managed server(s), you will see that is can not find some JNDI stuff.

• jms/bpm/PeopleQueryConnectionFactory
• jms/bpm/CubeCommandXAConnectionFactory
• jms/bpm/PeopleQueryConnectionFactory
• jms/bpm/MeasurementTopic
• jms/bpm/PeopleQueryTopic

You can ignore these errors in your log file, unless you think I need them, then you should created the JMS Connection factories and topics.

If you select the BAM option as well, you will not have these errors :-).

Let's hope they will fix it in PS3.

Long running synchronous composite

Some times you need to start a process that runs for a long time. Let say hours. Normally you would choose to create a asynchronous process. So far so good. But when you call this process from a Web application, the web application can not wait for the result for the callback response.

A simple solution is to create s synchronous composite and reply to the caller immediately. This can be achieved in the next example. This solution is based on a BPEL implementation.

Note that a after the reply an checkpoint is created. This will force the process to save the state to the database AND it force the send the reply back to the caller. Otherwise, with no checkpoint, the reply is send after the transaction is committed (XA) and case of an error, the reply is not send.

Human Worklist group authentication

As I have written an article that is using database authentication within the SOA suite for human tasks, this soultion is not complete. Apart of the pointing reference to the example "workflow-120-SQLIdentityProvider", this example is not available for download (yet).
The solution is wokring, but it does not work for groups. If you assign an task to:

• a user, you can use the task in the worklist application.
• a user AND group, you can use/claim the task in the worklist application.
• a group, an error occurs in the assignement. 

 The issue is not the SQLAuthenticator or the configuration of weblogic. But it is the custom java code that is used in the custom identity provider.

Follow the instruction in my previous article, but now use the new Java code, summary:

1. Apply the SQL Authenticator in your SOA WLS environment.
2. Test if users and groups can be viewed and edited.
3. Shutdown the admin server and managed servers.
4. Copy the dbprovider.jar to the SOA_DOMAIN/lib directory
5. make changes in the SOA FMW  JPS config file:
6. Start admin and managed servers
7. Test the worklist application with database users.
8. Configure SQL Authenticator in WebLogic Server
9. Add a new Identity Store.
10. Add a new Service Instance.
11. Change JpsContext to the new Identity store.

OSBS: A new version of the build tool

Download OSBS11g here (14MB)..

Introduction
A few years ago, when Oracle had only the BPEL component, deployment was rather complex. Therefore I create my own build tool, replacing the obant tool and ignoring the existing build.xml files, supplied within your JDeveloper project. Now with Oracle 11g the SCA composite is introduced and we are running on the Oracle Weblogic Server.

It took me some time to upgrade the Oracle SOA Build Server to the 11g version. Many new features have been added:

• Supports Oracle SOA 11g (from 11.1.1.3, aka PS2)
• Deploy/Undeploy SCA composites
• Deploy/Undeploy MDS
• Support of partitions
• Deploy to diferent targets
• Restructured project directory
  
• Restructured environment properties
  
• Updated documentation (Use Guide)

Overview
The default tools of SOA 10g SOA 11g, is only focussing on deployment of a single component (BPEL / ESB / SCA). While in real life, more components are involved with a deployment. You can think on Java application; data-model changes, etc. To overcome this issue, and create a mechanism to deploy multiple type of components, a general "build.xml" that is able to compile and deploy components.

The solution described in the next paragraphs will take this into account.

Functionality
The build tool covers the following features and can be easily change and extended to customer requirements:

• Depends on SOA 11g environment, installed soa_domain
• Using full ANT tasks features, Ant 1.7 + Ant-Contrib tasks
• Support of Multiple environments; Dev / Test / Acceptance / Production / ...
• Deploy SCA composites
• Undeploy SCA components
  
• UnDeploy MDS artifacts
• Deploy MDS artifacts
• Deploy SQL
• Integration with SubServersion; download latests or tags
• Stand-alone tool ; not depending on Oracle installation
• Compile and Deploy Java programs
• Supporting 'MakeAll'

Configuration
To use the build tool some configuration must be made. Most of all settings are located in properties files. Configure the build tool configuration file to prepare the deployment for this release. Edit the following file:

$BUILD_TOOL/build.env

In build.env, edit ANT_HOME and ANT_JAVA_EXTENSIONS, change the first part of the path to where the package was copied to ($BUILD_TOOL). Also change LOG_DIR in build.env, log files will go to this directory.

An example of this file is here

...

LOG_DIR=/home/oracle/buildtool/log
FMW_HOME=/app/oracle/products/11g/fmw
SOA_HOME=/app/oracle/products/11g/admin/domains/soa_domain

... 

Manual Deployment

Download
Download the latest version of your project. Example for SystemTest environment:

obbuild.sh -e test Download

Note: it will use the file test/replace_.properties to execute the find and replace for the endpoints.


Example for tag 2.0.23 being the current version for acceptance:

obbuild.sh -e acc Download 2.0.22

Example for revision 1254 for development:

obbuild.sh Download rev 1254

Note: it will use the file test/replace_.properties to execute the find and replace for the endpoints.

The replace.properties file is used for a general replacement of tokens in your projects. The file describes how tokens can be replaced in one or
all BPEL processes, ESB services.

Syntax:

find.seq.name=[all|]
[all|].seq.find.value=
[all|].seq.replace.value=

Example:
Replace in all the files localhost:9700 into 127.0.0.1:8001

find.1.name=all
all.1.find.value=localhost:9700
all.1.replace.value=localhost:8001

Example:
Replace in HellWorld bpel process the default domain and the hostname

find.1.name=HelloWorld
HelloWorld.1.find.value=localhost:9700
HelloWorld.1.replace.value=192.168.0.1:8888

find.2.name=HelloWorld
HelloWorld.2.find.value=/orabpel/default
HelloWorld.2.replace.value=/orabpel/systest


Deploy generic SCA composite
Deploy each BPEL process separately. Example for Test environment:

For example: Compile and Deploy Helloworld 1.0 to the default domain of node1


obbuild.sh -e test DeploySCA HelloWorldSCA 1.0 default node1

Deploy Java WebService
Deploy each Java process separately. Example for Test environment:

obbuild.sh -e test DeployJava HelloJavaWorld

Note: default-application.xml must be included in source code.

Automatic Deployment
To deploy the whole system at once, use the task 'MakeAll'.

obbuild.sh -e test MakeAll

Note: This target is using the 'makeall_test.properties' that describes the correct sequence of all the AIA components (Java, BPEL, and ESB) that will be built and deployed. The next table shows an example of the 'makeall.properties' file.

sql.make.10=car
deploy.version.10=1.0
deploy.target.10=dbnode

mds.make.15=common_capesoa
deploy.version.15=1.0
deploy.target.15=node1

sca.make.100=HelloWorldSCArefxx
deploy.version.100=1.1
deploy.target.100=node1

sca.make.200=HelloWorldSCArefxx
deploy.version.200=2.1
deploy.target.200=node2

sca.make.300=HelloWorldSCArefxx

sca.make.20=HelloWorldSCArefbb
sca.make.30=HelloWorldSCArefcc
sca.make.40=HelloWorldSCA

A target can be given to deploy to a particular node or database.

targets.properties
target.node1=node1.vijfhuizen.local:8010
target.node1.username=weblogic
target.node1.password=welcome1

target.node2=node2.vijfhuizen.local:8020
target.node2.username=weblogic
target.node2.password=welcome1

target.admin=t3://192.168.2.10:7001
target.admin.username=weblogic
target.admin.password=welcome1

target.dbnode=jdbc:oracle:thin:@database.vijfhuizen.local:1521:XE
target.dbnode.username=scott
target.dbnode.password=tiger

Command line interface
The command line interface of the obbuild tool is as follows:

./obbuild.sh -?

obbuild.sh [-[hvtV] [-e environment] [-f build-file] [-m makefile] [-r replace-file] [task] [arg1] [arg2] .. [arg6]]

 -h      Shows this help
 -V      Show the version of this program
 -t      Show all the tasks in the build file
 -e      Select envirnonment property file, default 'dev', reading dev.properties
 -f      Read a specific build file, default is build.xml in current directory

-m      Read a specific makefile
 -r      Read a specific replacement file, use for promoting BPEL process to other environments.
 -v      Show verbose output
 task    The task to be executed, default is main
 arg1..6 Optional arguments passed to the build file as properties ANT.ARG1..ANT.ARG6

./obbuild.sh -t

obbuild.sh: Tasks in ./build.xml                                         
          CreateDirs - Create directory structure                        
            Download - Download latest source code                       
                       [arg=tagged-version | rev number]                 
                main - Default task                                      
             Promote - Find/Replace on one or more processes, see replace.properties
         CompileJava - Compile Java files
                       [arg1=name of the application]
           CreateWar - Create WAR file from class files
                       [arg1=name of application]
           CreateEar - Create EAR file from WAR file
                       [arg1=name of application]
          DeployJava - Deploy an EAR file to
                       [arg1=name of the application]
        UndeployJava - Undeploy an application from target
                       [arg1=name of the application]
             MakeAll - Create a new set of the application
                       [arg1=[version] [arg2=partition]
            SendMail - Send a mail
                       [arg1=to arg2=subject arg3=text]
           DeployXSD - Deploy the XSD files: creating a ZIP file
                       [arg1=Name]
          CompileSCA - Compile SCA composite
                       [arg=sca-composite-name]
           DeploySCA - Deploy SCA composite
                       [arg=composite-name] | [arg=version|1.0] [arg=partition] [arg=target]
         UndeploySCA - Undeploy SCA composite
                       [arg=sca-composite-name arg=revision | arg=target]
           DeployMDS - Deploy MDS artifact
                       [arg=oramds-artifact] arg=[target]
         UndeployMDS - Undeploy MDS artifact
                       [arg=oramds-artifact] arg=[target]
           DeploySQL - Deploy SQL
                       [arg=sql directory] | [arg=db-target]
        DeployScript - Deploy Script
                       [arg=ant|wlst] | [arg=script-name] [arg=target]

There is no support from Oracle or myself and I expect that you have the proper Oracle licences to use the Oracle software.

ANT and MDS: Composite Compile Error

When you compile and deply a composite from ANT and you are using the MDS for retrieval of XSD and WSDL composites, you could run into compile errors. The compiler will run into an error that can not load  the files with "oramds:" protocol. For example:

MDS-00054: The file to be loaded oramds:/soa/shared/bpel/runtimeFault.wsdl does not exist.

The solution is to create an adf-config.xml file directly into your composite directory, in the following sub directories, this directory should be created:

/composite-directory/.adf/META-INF/composite

The adf-config.xml shoudl be placed there and contain a mapping to your shared storage. In the next example I choose for persistent storage in the database.

<?xml version="1.0" encoding="windows-1252" ?>
<adf-config xmlns="http://xmlns.oracle.com/adf/config"
            xmlns:adf="http://xmlns.oracle.com/adf/config/properties"
            xmlns:sec="http://xmlns.oracle.com/adf/security/config">
  <adf:adf-properties-child xmlns="http://xmlns.oracle.com/adf/config/properties">
    <adf-property name="adfAppUID" value="JIPSYEW_bpel-5393"/>
  </adf:adf-properties-child>
  <adf-mds-config xmlns="http://xmlns.oracle.com/adf/mds/config">
    <mds-config xmlns="http://xmlns.oracle.com/mds/config">
      <persistence-config>
        <metadata-namespaces>
          <namespace metadata-store-usage="mstore-usage_1" path="/soa/shared"/>
          <namespace metadata-store-usage="mstore-usage_1" path="/apps/mymdsartifacts"/>
        </metadata-namespaces>
        <metadata-store-usages>
          <metadata-store-usage id="mstore-usage_1">
<!--
            <metadata-store class-name="oracle.mds.persistence.stores.file.FileMetadataStore">
              <property value="${oracle.home}/integration" name="metadata-path"/>
              <property value="seed" name="partition-name"/>
            </metadata-store>
-->
<!--
  DATABASE MDS Connection
  User for SCA Ant Compilation
-->
            <metadata-store class-name="oracle.mds.persistence.stores.db.DBMetadataStore">
              <property name="jdbc-url" value="jdbc:oracle:thin:@database.vijfhuizen.local:1521:orcl"/>
              <property name="jdbc-userid" value="dev_mds"/>
              <property name="jdbc-password" value="welcome1"/>
              <property name="partition-name" value="soa-infra"/>
            </metadata-store>

          </metadata-store-usage>
        </metadata-store-usages>
      </persistence-config>
    </mds-config>
  </adf-mds-config>
  <sec:adf-security-child xmlns="http://xmlns.oracle.com/adf/security/config">
    <CredentialStoreContext credentialStoreClass="oracle.adf.share.security.providers.jps.CSFCredentialStore"
                            credentialStoreLocation="../../src/META-INF/jps-config.xml"/>
  </sec:adf-security-child>
</adf-config>

SOA Suite 11g R1 Developers Guide Available

As in the northern hemisphere it is holiday period, Antony Reynolds and Matt Wright have finally finished their Oracle 11g SOA Suite Release 1 Developers Guide. I was one of the technical reviewers to keep Matt and Antony sharp. Here is some info on the book. OK you might think it is some publicity for them, but it took some time to write adn to review, :-), but who cares.

From Antony's blog:

More than 40% of the book is new content, including guidance on how to use the new rules editor and the Event Delivery Network.  When we started writing together our original target was the 11g product, but along came the BEA acquisition and 11g was delayed so we re-focused the book on 10g.  Of course a few months after the 10g book was published 11g came out and so we dusted off the earliest chapters, updated them and started again.  So in some ways this book was what we meant to write when we started.
While writing the last book Matt emigrated to Australia.  So to keep up with him while writing this book I moved to Colorado.  Matt, not to be outdone, responded by leaving Oracle and starting his own company.  Our families are now very worried what we might do if we start writing another book together.
The editorial team at Packt were very good at chasing us to finish the book, without them we would probably only be half-way through.  We had some great reviewers who provided very insightful comments and often pointed us in the right direction when we had lost our way.  You may well be familiar with some of the reviewers from their blogs or participation at industry events;

• John Deeb
• Hans Forbrich
• Bill Hicks
• Marc Kelderman
• Manoj Neelapu
• ShuXuan Nie
• Hajo Normann

In addition to the reviewers we had a lot of support from product management at Oracle, I found Clemens Utschig particularly helpful.
I hope you enjoy the book, we have tried to make it into a guide for practitioners of SOA, and endeavored to explain how and why to use different parts of the SOA Suite.  Let us know what you think.

SOA 11g: Native IO must be enabled!

On a project working with Sun Solaris the server was very slow. I took 15 minutes to start and executing composites or accesing the "Human Worklist Application" or "Enterprise Manager" took ages.

I saw in both log files of the AdminServer and soa_server1, one small line:

<Native IO Disabled. Using Java IO.>

This line implies that the system will use Java code instead of O/S specific libraries for executing I/O function.

This can be fixed by setting the correct dynamic Libraries; LD_LIBRARY_PATH.

Make sure you set the LD_LIBRARY_PATH in your start scripts of the Admin and managed servers.

Add this in  the start scripts:

LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$FMW_HOME/wlserver_10.3/server/native/solaris/sparc64
export LD_LIBRARY_PATH


This will result in the following line on the log :-)

<Native IO Enabled.>

SQLAuthenticator and Human Worklist Application

Sometimes you need another authentication mechanism to verify the username and password. In WLS the default authenticator is file based and can be accessed via a lightweight LDAP interface. For simple straight forward applications, it is sufficient.

In a SOA environment, when you use the default worklist application, the users must be registered in the Weblogic Server. Legacy applications have there own mechanism of storing user data. In our Oracle world, users and passwords are often stored in tables of the database.

The goal is to re-use this table mechanism in combination with the default Worklist Application. The approach is rather simple, but it is not working.

• Add WebLogic SQLAuthenticator provider in your security realm.

When you have applied this new authenticator, the worklist application can not authenticate the users from the database. You will get an error:

Internal Error in Verification Service for user SKING. lookupUser.
Check the underlying exception and correct the error. If the error  persists, contact Oracle Support Services.                                            

ORABPEL-30504                                                                                                                                             
Internal Error  in Verification Service.

The issue is within the worklist application itself. It only authenticates via LDAP and ignoring the SQLAuthenticator.

We are able to fix this issue. The solution is based on the SOA Samples of the Human Worklfow; "workflow-120-SQLIdentityProvider". You can obtain the examples via Oracle TechNet or via your Oracle Rep.

The solution is as follows and is based on the SQL Authenticator Example of Edwin Biemond.

• Apply the SQL Authenticator in your SOA WLS environment.
• Test if users and groups can be viewed and edited.
• Shutdown the admin server and managed servers.
• Copy the dbprovider.jar to the SOA_DOMAIN/lib directory
• make changes in the SOA FMW  JPS config file:

1. Add a new Identity Store.
2. Add a new Service Instance.
3. Change JpsContext to the new Identity store.

• Start admin and managed servers
• Test the worklist application with database users.

Note: the dbprovider.jar is created with the "workflow-120-SQLIdentityProvider" example. Check the files

• DBUserSearchResponse.java
• DBRolesSearchResponse.java

If they are query the correct tables and columns. If they are not correct, compile and create a new JAR file.

Example of the jps-confix.xml file:

<serviceProviders>
  <!-- New ID provider -->
  <serviceProvider type="IDENTITY_STORE" name="custom.provider" 
class="oracle.security.jps.internal.idstore.generic.GenericIdentityStoreProvider">
     <description>Custom IdStore Provider</description>
  </serviceProvider>
  <!-- EOF New ID provider -->
...
<serviceInstances>
  <!-- NEW Service Instance -->
  <serviceInstance name="idstore.custom" provider="custom.provider" location="dumb">
      <description>Custom Identity Store Service Instance</description>
      <property name="idstore.type"            value="CUSTOM"/>
      <property name="ADF_IM_FACTORY_CLASS"    value="org.sample.providers.db.DBIdentityStoreFactory"/>
      <property name="DB_SERVER_NAME"          value="database.vijfhuizen.local"/>
      <property name="DB_SERVER_PORT"          value="1521"/>
      <property name="DB_DATABASE_NAME"        value="orcl"/>
      <property name="ST_SECURITY_PRINCIPAL"   value="demo"/>
      <property name="ST_SECURITY_CREDENTIALS" value="demo"/>
  </serviceInstance>
  <!-- EOF NEW Service Instance -->
...
<jpsContexts default="default">
  <!-- This is the default JPS context. All the mendatory services and Login Modules must be configured in this default context -->
  <jpsContext name="default">
      <serviceInstanceRef ref="credstore"/>
      <serviceInstanceRef ref="keystore"/>
      <serviceInstanceRef ref="policystore.xml"/>
      <serviceInstanceRef ref="audit"/>
      <serviceInstanceRef ref="idstore.custom"/>
  </jpsContext>

Good luck!

SOA 11g Cloning

Cloning an SOA 11g environment is rather simple. Due to the architecture of the Weblogic application server, you can do this easily. The steps are as follows.

1. Make sure you stopped the Managed Servers and Admin server.
2. Copy the BEA_HOME directory to your target destination.
3. Copy the SOADOMAIN_HOME directory to your target destination.
4. Export the SOA repository from the database.
5. Import the SOA repository in target database.
6. Manually make changes in the SOADOMAIN_HOME/config/config.xml file.
7. Manually make changes in the WLS_HOME/wls_server1/common/nodemanger/* properties.
8. Manually make changes in the SOADOMAIN_HOME/config/jdbc/* datasource file(s) to target database.
9. Make changes in start-up scripts when using WKA in Coherence clustering. 
10. Start servers
11. Check SOAP Server URL and Callback Server URL

1
Make sure that all the WLS servers are stopped in the domain, even the clustered ones. This is to make sure that no caching occurs.

2
Copy the binary tree to another destination. You could install the WLS software-only, but a copy is quicker.

3
Use the tool pack/unpack to copy a complete domain from one destination to another. Alternatively you could copy the complete file tree of your SOA_DOMAIN, including the application directory.

4
Make an export of the SOA repository, make sure you have the rights to export the various SOA schema's such as, MDS, SOA_INFRA etc.

5
Import the exported data. In this case, the SOA schema's must be already there or they will be created during the import. Again, you must have the correct privileges to create schema's and apply the correct grants.

But be ware that the users in the other database need to have to following roles:

select * from dba_role_privs where grantee like 'DEV_%' order by grantee;

GRANTEE        GRANTED_ROLE         
-------------- ---------------------
DEV_SOAINFRA   CONNECT              
DEV_SOAINFRA   RESOURCE             
DEV_SOAINFRA   AQ_USER_ROLE         
DEV_SOAINFRA   AQ_ADMINISTRATOR_ROLE
DEV_SOAINFRA   SELECT_CATALOG_ROLE  

DEV_MDS        RESOURCE             
DEV_MDS        CONNECT              

DEV_ORABAM     CONNECT              
DEV_ORABAM     RESOURCE             

Or more in detail:

select
  lpad(' ', 4*level) || granted_role "User, his roles and privileges"
from
  (
/* 
 * Users
 */ 
    select 
      null     grantee, 
      username granted_role
    from 
      dba_users
    where
      username like 'DEV_%'
/*
 * the roles to roles relations 
 */ 
  union
    select 
      grantee,
      granted_role
    from
      dba_role_privs
  /* THE ROLES TO PRIVILEGE RELATIONS */ 
  union
    select
      grantee,
      privilege
    from
      dba_sys_privs
  )
start with grantee is null
connect by grantee = prior granted_role;

6
Make changes in the config.xml file, to set new hostnames and/or IP addresses for the Admin Server, Managed Servers and node manager (machine)


7
Make changes in the nodemanger comnfiguration. If this is a new server, the nodemanager.properties should be updated. If the cloning is done on the same server, you should change the port number only, or reuse an existing node-manager.

8
Make changes in the JDBC connections. The data-sources for target database updated.

9
If you are using a clusterm verify the coherence properties, such as multicast adresses. If you are using well-known-adresses (WKA) make sure the are updated.

10
Now you are able to start the admin and managed servers.

11
If you are usingf a loadbalancer check SOAP Server URL and Callback Server URL via Enterprise manager. (EM -> SOA-INFRA -> SOA Administration  -> Common Properties)


Note: With this solution you loose existing running processes.

Oracle Policy Automation (OPA) on Linux

Apart of the business rules engine within SOA Suite, Oracle bought a few years a ago another great rule engine, Haley. This product is rebranded and optimzied to Oracle Policy Automation (OPA). The product is a real tool that can be supplied to business analysts. The rules can be defined by Microsoft Word or by Microsoft Excel. These rules are saved to OPA.

OPA is more or less just an Java application that can be deployed to Weblogic and can be accessed via SOAP or via Java API.

When you deploy OPA to Weblogic, you will run into a deployment error. Deployed applications on WLS are not unpacked, this will result into an error when you want to start the application.

EngineConfigurationException: No rulebase directory 

The issue is can be fixed to set some specific Java configuration properties. This deployment issue can be fixed as follows, assuming you have downloaded OPA and unzipped the file in a directory:

$ cd web-determinations
$ mkdir tmp
$ cd tmp
$ unzip ../web-determinations.war
$ vi WEB-INF/classes/configuration/application.properties
...
load.rulebase.from.classpath=true
rulebase.path=rulebases
...
load.resource.as.resource=true
resources.path=resources
...
load.properties.as.resource=true
properties.path=configuration
...
load.templates.as.resource=true
templates.path=templates
$ zip -r web-determinations.war *

Deploy this new 'web-determinations.war' file to WLS (10g, 11g)

Oracle SOA 11g PS2 (patchset #2) is out!

This week new/updated releases are out:

• Oracle Weblogic Server 10.3.3
• Oracle JDeveloper 11g, ps2
• Oracle SOA Suite 11g ps2
• Oracle Service Bus 11g (through eDelivery.oracle.com)

See also:


http://www.oracle.com/technology/software/products/middleware/htdocs/fmw_11_download.html

Create SOA 11g deployment only user

Creating a user that is able to deploy applications, Java and composites. Is done as follows

Logon to WLS Console

Create user

• Click on 'Security Reams'
• Click on 'myrealm'
• Click on 'Users & Groups'
• Click on 'New'
• Enter Name:SOADeployUser
• Enter Description: SOA Deployment user
• Enter Password: ****
• Enter Confirm: ****
• Click on 'OK'

Add roles to user

• Click on 'Security Reams'
• Click on 'myrealm'
• Click on 'Users & Groups'
• Click on 'SOADeployUser'
• Click on 'Groups'
• Assign the following groups to 'SOADeployUser'
• - Deployers
• - Operators
• - Monitors
• Click on 'Save'

Monitoring Java Virtual Machines

Apart of the default consoles that are delivered with the Oracle product; such as Weblogic Console, BPEL console, ESB Console, Enterprise Manager. We can also look 'under the hood' of the application server.

With the latest release of WebLogic, you proberly use Oracle JRockit as JVM. This JVM has its own console, JRockit Mission Control. If you are not using JRockit, because you are using another OS such HP or IBM, or you just stick to the Oracle/SUN JDK, then the following solution will help. A nice article is published by Olaf Heimburger describing both tools based on the 11g release.

Monitoring the behavior of the JVM can be done via the tool 'VisualVM'. The tool can be used directly on Java VM that are running on your local machine. By default it can connect to JVM's of version 1.6. If you are using a JVM of version 1.5 you should at the following Java properties during the startup.

The tool is able to connect to local JVM's as well remote JVM's. Connecting to a remote JVM, a daemon process must be running on the server to collect the data from the JVM and expose this for client applications.

Server Configuration


WLS 9.2 only

The Java parameters can be set in the arguments settings for each managed server.

Add the following Java option into the WLS 9.2 managed servers:

-Dcom.sun.management.jmxremote.port=3333 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false

After applying the Java settings, restart the managed server.

Create in the JAVA_HOME directory a policy file, that grants all permissions to the Java Statistic daemon process.

JAVA_HOME=/u01/appl/j1bplpe/product/wls/jdk150_12
export $JAVA_HOME

cd $JAVA_HOME/bin
vi jstatd.policy:
grant codebase "file:${java.home}/../lib/tools.jar" {
  permission java.security.AllPermission;
};

Now we can start the daemon process to collect the java runtime statitics.

Generic

The Jstat daemon process can now be started. In the next example it is started 'online'. I would recommend this, this process should not be running during normal production. Otherwise start the programm with 'nohup' and '&'.

cd $JAVA_HOME/bin
./jstatd -J-Djava.security.policy=jstatd.policy -p 1099

The daemon process is started, it connect to the JMX port 3333 and can be accessed with 'VisualVM' over port 1099 without having credentials or SSL.

Unix start script
cat start_jstatd.sh
#!/bin/bash

start_jstatd.sh
JAVA_HOME=/usr/lib/jvm/sun-java-1.6
export JAVA_HOME
nohup $JAVA_HOME/bin/jstatd -J-Djava.security.policy=./jstatd.policy -p 1099  &
echo $! > jstatd.pid
echo Java Stats Deamon started: `cat jstatd.pid`

stop_jstatd.sh
#!/bin/bash

JAVA_HOME=/usr/lib/jvm/sun-java-1.6
export JAVA_HOME

if [ -f "jstatd.pid" ];
then
  pid=`cat jstatd.pid`
  echo "killing jstatd with pid $pid"
  kill -9 $pid
else
  kill -9 `ps -ef | grep "jstatd" | grep -v grep | awk '{ print $2; }'`
fi

Client Configuration

At the client side, download the code latest version visualVM, this can be obtained from:

http://visualvm.dev.java.net

Unzip the ZIP file into a directory. In this directory you can start the the client tool to visualize the JVM. If Java 1.6 is installed on your workstation, the settings are already correct and we can start the application.

Linux:
        ./visualvm

Windows:
        visualvm.exe

The screen look like this.


Create a new Remote connection to the server you want. In this case we connect to the node #1 of my cluster via port 1099. This is the port number on which the jstatd daemon is listening.

Select the appropiate JVM, weblogic.Server., to view memory and thread information.

Garbage collection:

More information and documentation on this tool can be obtained via the website.

SOA 11g: Migration FAQ / Best practice

Migration projects from SOA 10g to SOA 11g is not a walk-in-the-park. You should read the migration documentation for the SOA 11g migration carefully. During your migration you would run into behaviours that you did not expect.

Use fully qualified names
During configuraton of the SOA domain, make sure you use the fully qualified name as specified in your DNS server. Some servers are using the short-name that is binded to the localhost (127.0.0.x) IP adres.

BPEL Compiler stricter
WIth SOA 11g patch set #2, the SCA runtime engine is more stricter. During testing of your SCA, that was created with patch set #1, you could run into runtime 'Selection errors'. One case is that you should check your Assign and Switch cases. You could have programmed to select the first node of an XML message:

/outputCollection/Order/Orderline/price

During runtime this selection failes, multi-node result. You should chnage your code into

/outputCollection/Order/Orderline[1]/price

Use abstract services
A SOA 10g project can be migrated via JDeveloper. But it migrates 1:1. If the BPEL process is calling another BPEL process (10g), after migration the new 11g composite is still calling the BPEL 10g process. You must change this manually.
Alternatively you could use abstract WSDL. This will decouple the calling BPEL process. Therefore you must also make some changes in your BPEL or ESB services, before you migrate.

HTTP 503 error during deployment
Follow the instructions at http://soa-bpel-esb.blogspot.com/2009/08/oracletiptoolsidefabricdeploycommonsoar.html to solve this issue.

Error: TopLink deployment descriptor generation failed
During deployment you get this Toplink error. Please verify that reported problems are corrected and that the deployment descriptor path and name are valid.
The issue is in the JDeveloper .JPR file. An oracle.toplink.addin.TopLinkProjectSettings section is this file, remove the section and the deployment should work.

Composite WSDLs are not loaded during startup of JDeveloper
In some cases JDeveloper can not load the external WSDL that is used in your composite. This occurs when an composite.xml file is 'open' when you close JDeveloper the previous time. The workaround is, off course, to close all 'compiste.xml' tabs before shutting down JDeveloper.

BPEL Migration fails with special characters
If your 10g SOA project contains special characters like é or  ï, the project fails after migration. Remove these characters before migrating your projects.

RuntimeFault.wsdlcan not be found
The http://localhost:80/orabpel/xmllib/RuntimeFault.wsdl can not be found after migration. The location of these files are moved to the MDS. Change this location manually in your code to oramds:/soa/shared/bpel/RuntimeFault.wsdl

java.lang.Exception: oracle.sysman.emSDK.webservices.wsdlapi.SoapTestException: null 
An error occurs when you are using the 'count' function with getVariableData. Use the function 'oracountNodes' instead of this one.

see also Metalink Note 1059573.1 APPLYING COUNT XPATH FUNCTION TO bpws:getVariableData RESULTS IN SELECTIONFAILURE FAULT

Selectionfailure on empty element
During a test activity such as string(bpws:getVariableData('creditcardLimit')) != '' will fail. This can be solved by using countNodes functionality: ora:countNodes('transactienummer') > 0

BPEL Preferences
Changing the BPEL preferences is now done via Enterprise Manager with the MBean browser. See the article.

http://eelzinga.wordpress.com/2009/10/28/oracle-soa-suite-11g-setting-and-getting-preferences/

java.lang.IllegalArgumentException
An error is generated:

"java.lang.IllegalArgumentException: Error occurred while attempting to retrieve message part ... from a normalized message payload with elements ..."

During migration of the ESB services that are not using a transformation for mapping the request and reply messages, the migration tool is NOT adding a Reply mapping. This only occurs when the request message and the reply message using different part names.

This can be solved by creating the Reply mapping manully via the assign 

Unable to register service.
After restart of the server you could get:

Unable to register service. oracle.fabric.common.FabricException: Error in getting XML input stream: : Response: '503: Service Unavailable' for url:
 
This occurs that the composite is pointing to a WSDL of another service. This pointer should be a abstract WSDL.

Base64 encoding/decoding does not compile
An error occurs "Error(17,39): Failed to compile bpel generated classes. failure to compile the generated BPEL classes for BPEL process ... The class path setting is incorrect. Ensure that the class path is set correctly. If this happens on the server side, verify that the custom classes or jars which this BPEL process is depending on are deployed correctly..." This can be solved by adding importing the oracle.soa.common.util packaged instead of the com.collaxa.common.util
 
FTP adapter header variable

The FTP Adpater is using the bpelx:headerVariable as directory or filename setting. This variable can be set on the Invoke or Recevice activity via a property.

If I missed something or you have anything to add, please send me a mail or put it in the comments.

AIA 2.4/2.5: Foundation Pack Quick Cluster Install

Installing AIA in a clustered environment is not a 'walk in the park'. Many documents exists from Oracle that describes various approaches to install AIA on a cluster. This article is written to have a simple straight forward installation and configuration.

Pre-requisites:

• You have a running clustered Oracle 10g SOA Suite. If not, follow my article on Oracle SOA Suite 10g Cluster installation.

The AIA cluster installation is executed in 5 steps:

• Step 1: Ensure that the SOA cluster is setup correctly
• Step 2: AIA Software only install
• Step 3: AIA Deployment and configuration preparation
• Step 4: AIA Deployment and configuration execution
• Step 5: AIA post configuration

Step 1: Ensure that the SOA cluster is setup correctly

The following are basic verification steps to ensure that your SOA cluster is setup correctly.

• Navigating to the BPEL, ESB consoles successfully using the external Load Balancer URL as well as each of the internal virtual host.
• Ensuring that the EM console shows all participating nodes in active state. There must be only one Active Design time. 
• Ensure that ESB Design-time is completely up and running by accessing the following URL:
• http://node1.vijfhuizen.com:7777/esb/dtStatus.jsp
• http://node2.vijfhuizen.com:7777/esb/dtStatus.jsp 
• These pages should return a true. Test this via both nodes. 
• Deploying a sample BPEL process and a sample ESB process. And verify via the consoles on each node that the deployments exists. 
• Change the state of the BPEL process on one node, e.x. Set state to retired. Verify on the other console that the state has been changed. 
• Undeploy the BPEL process from this node. Verify on the other console that the BPEL process does not exists anymore.

Step 2: AIA Software only install
Install on both nodes the AIA Software only! Just run the AIA installer ant install the software only! It is recommened to install the software in a different  directory as the Oracle SOA 10g software, for example:

• ORACLE_HOME: /app/oracle/products/10g/soa
• AIA_HOME: /app/oracle/products/10g/aia

Step 3: AIA Deployment and configuration preparation

Add for each node the following java option, make sure all servers are down (opmnctl shutdown). Add the java option in the 'oc4j_soa' container.

vi $ORACLE_HOME/opmn/conf/opmn.xml
...
-Daia.home=/fs01/app/oracle/esbacc01/aia
...

Make sure indexing is enabled on th both Apache HTTP servers, apply this setting twice.

vi $ORACLE_HOME/Apache/Apache/conf/httpd.conf
...
<Directory />
Options FollowSymLinks MultiViews Indexes
...
<Directory "/app/app/oracle/product/10g//soa/Apache/Apache/htdocs">
Options FollowSymLinks MultiViews Indexes
...

Check the AIA configuration. With the AIA software only install, not all settings are correct:

vi $AIA_HOME/bin/aiaenv.sh
...
AIA_HOME=aia.home
ORACLE_HOME=oracle.home
AIA_INSTANCE=aiaapp.deploy.home
PLATFORM=platform

into

AIA_HOME=/app/oracle/product/oracle/10g/aia
ORACLE_HOME=/app/oracle/product/oracle/10g/soa
JAVA_HOME=$ORACLE_HOME/jdk
AIA_INSTANCE=/app/oracle/product/oracle/10g/soa/j2ee/oc4j_soa/applications
PLATFORM=ias_10g

Execute a chmod 775 on the ANT_HOME/bin/* oon beide nodes, to make the files executable.

chmod 755 $AIA_HOME/bin/*

Execute the following script on both nodes to enable the new settings.

source aiaenv.sh

Add heartbeat parameters to teh ESB config directory at the end of the file on both nodes:

vi $ORACLE_HOME/integration/esb/config/esb_config.ini
...
PingInterval=30
PingCount=30

Restart both nodes, first node1 then node2.

$ORACLE_HOME/opmn/bin/opmnctl shutdown
$ORACLE_HOME/opmn/bin/opmnctl startup

Now we need to prepare the deployment properties of AIA. This is done on both nodes.

cp $AIA_HOME/config/deploy.properties $AIA_HOME/config/deploy.properties.org
cp $AIA_HOME/Infrastructure/install/templates/deploy.properties.tmpl $AIA_HOME/config/deploy.properties

Open this deploy.properties file, on each node, to check parameters. Make sure the hostnames are correct for each node (j2ee.hostname, iasinstancename):

vi $AIA_HOME/config/deploy.properties
FPVersion=2.5.0.0.0
aia.home=/fs01/app/oracle/esbacc01/aia
oracle.home=/fs01/app/oracle/esbacc01/soa
http.hostname=node1.vijfhuizen.com
j2ee.hostname=node1.vijfhuizen.com
http.port=80
admin.user=oc4jadmin
iasinstancename=AS.vijfhuizen.com
oc4jinstancename=oc4j_soa
opmn.port=6003
rmi.port=12401
db.hostname=node1.vijfhuizen.com
db.port=1521
dbuser=sys
SID=esbacr011
#admin.password.encrypted=
admin.password=welcome1
#dbpwd.encrypted=
dbpwd=esbacr01

Check on both nodes the paramterea of BPEL. Note, that we set the option cluster to false! And the hostnames are different:

vi $ORACLE_HOME/bpel/utilities/ant-orabpel.properties
...
platform = ias_10g
hostname = node1.vijfhuizen.nl
http.hostname = ${hostname}
http.port = 80
cluster = false
oc4jinstancename = oc4j_soa
j2ee.hostname = node1.vijfhuizen.nl
asinstancename =

Before we can start with the the AIA configuration, we need to make a jms error topic. Otherwise the ant script for creating the FPErrorHandlingInstall will fail.

• Login to EM
• Click on soa_group link
• Click on Administration
• Click on JMS Destinations
• Click on Create New
• Destination Type: Topic
• Desination Name: ESB_ERROR
• In memory: checked
• JNDI Location: jms/topic/ESB_ERROR
• Click on OK

Step 4: AIA Deployment and configuration execution
Now we are able to start in configuration of AIA into Oracle SOA Suite 10g. The approach is as follows. FIrst we do some manual install scripts on node #2 (while node #1 is down!). Then we do a full install on node #1, while node#2 is down. After this we can bring up #2.

Stop node #1

$ORACLE_HOME/opmn/bin/opmnctl shutdown

Check if node #2 is up and running (oc4j_soa, HTTP, esbdt)

$ORACLE_HOME/opmn/bin/opmnctl status

Start the first scripts:

cd $AIA_HOME/Infrastructure/install/scripts/
ant --noconfig -buildfile FPCommonConfig.xml replaceFPInstallProperties

Buildfile: FPCommonConfig.xml

...log information...

BUILD SUCCESSFUL

ant --noconfig -buildfile FPConfiguration.xml CommonConfigurations

Buildfile: FPConfiguration.xml

...log information...
BUILD SUCCESSFUL

The first step is done of AIA. Just for checking, you should be able to view the AIA compontens:

http://loadbalancer.vijfhuizen.com:7777/AIAComponents/

The out-of-the-box URI is incorrect when is is executing a deployment. Change the URI as well the password!

vi $AIA_HOME/Infrastructure/install/scripts/FPInstall.properties
...
#deployer.uri=deployer:oc4j:opmn://node1.vijfhuizen.nl:6003/oc4j_soa
deployer.uri=deployer:cluster:opmn://node1.vijfhuizen.nl:6003/soa_group
oc4j.admin.user = oc4jadmin
oc4j.admin.password = welcome1
...

Execute the next AIA scripts.

ant --noconfig -buildfile FPCommonConfig.xml CreateCommonDatasources
Buildfile: FPCommonConfig.xml
 

CreateCommonDatasources:
 

...log information... 
 
BUILD SUCCESSFUL
 

ant --noconfig -buildfile FPCAVSInstall.xml createSchedulerDataSource

Buildfile: FPCAVSInstall.xml

createSchedulerDataSource:
 

...log information... 
 

BUILD SUCCESSFUL
ant --noconfig -buildfile FPErrorHandlingInstall.xml DeployESBErrorListenerMDB

Buildfile: FPErrorHandlingInstall.xml

DeployESBErrorListenerMDB:

...log information...
BUILD SUCCESSFUL

ant --noconfig -buildfile FPCAVSInstall.xml DeployAIAUI

Buildfile: FPCAVSInstall.xml

...log information... 

BUILD SUCCESSFUL

Check that the AIA data-sources are created in the oc4j_soa container of node #2.

vi $ORACLE_HOME/j2ee/oc4j_soa/config/data-sources.xml
...
<managed-data-source connection-pool-name="JMSConnectionPool" 
jndi-name="jdbc/AIA_JMS_DS" name="AIA_JMS_DS" tx-level="local"/>
<managed-data-source connection-pool-name="AIAXAConnectionPool" 
jndi-name="jdbc/xref" name="XREFDataSource"/>
<managed-data-source connection-pool-name="AIAConnectionPool" 
jndi-name="scheduler/OracleManagedDS" name="SchedulerOracleManagedDS"/>
<connection-pool name="Example Connection Pool">
...

Shutdown node #2!

$ORACLE_HOME/opmn/bin/opmnctl shutdown

Startup node #1!

$ORACLE_HOME/opmn/bin/opmnctl startall

Now we can execute a full install of AIA on the first node. This step can take a while. Verify that the deployment properties is correct in $AIA_HOME/config. A follow the log file that is created. It it fails you can re-run this step.

Note: I had an issue on a project when we used a Oracle RAC database. DUe to sime unknown reasons, we had to restart the database, because the script could not make user. It was 'hanging'. It tries to create two user schema's:

create user aia identified by aia;
grant connect,resource to aia;
grant CREATE VIEW to aia;
grant aq_administrator_role to aia;
grant execute on dbms_aqadm to aia;
grant execute on dbms_aq to aia;
grant execute on dbms_aqin to aia;
grant execute on dbms_aqjms to aia;

create user jmsuser identified by jmsuser;
GRANT connect,resource,AQ_ADMINISTRATOR_ROLE TO jmsuser IDENTIFIED BY jmsuser;
GRANT execute ON sys.dbms_aqadm TO jmsuser;
GRANT execute ON sys.dbms_aq TO jmsuser;
GRANT execute ON sys.dbms_aqin TO jmsuser;
GRANT execute ON sys.dbms_aqjms TO jmsuser;

cd $AIA_HOME/Infrastructure/install/scripts
ant --noconfig -buildfile FPInstall.xml

Buildfile: FPInstall.xml

...log information...
BUILD SUCCESSFUL

Check that the AIA data-sources are created in the oc4j_soa container of node #1.

vi $ORACLE_HOME/j2ee/oc4j_soa/config/data-sources.xml
...
<managed-data-source connection-pool-name="JMSConnectionPool" 
jndi-name="jdbc/AIA_JMS_DS" name="AIA_JMS_DS" tx-level="local"/>
<managed-data-source connection-pool-name="AIAXAConnectionPool" 
jndi-name="jdbc/xref" name="XREFDataSource"/>
<managed-data-source connection-pool-name="AIAConnectionPool" 
jndi-name="scheduler/OracleManagedDS" name="SchedulerOracleManagedDS"/>
<connection-pool name="Example Connection Pool">
...

Startup node #2.

$ORACLE_HOME/opmn/bin/opmnctl startall

Step 5: AIA post configuration
Now we have almost finished the AIA cluster setup, we need to make a few post configuration steps.

Go to node #2, and install the taskform application.

cd $AIA_HOME/Infrastructure/ErrorHandling/AIAErrorTaskAdministrationProcess
ant --noconfig deployTaskForm

Buildfile: build.xml

deployTaskForm:

... log info...

BUILD SUCCESSFULL

On node #1 and #2, add durable subscriber for the JMS topic

cd $AIA_HOME/Infrastructure/ErrorHandling/AIAReadJMSNotificationProcess/bpel
vi consumeFaultMsg.wsdl
...
<jca:operation ActivationSpec="oracle.tip.adapter.jms.inbound.JmsConsumeActivationSpec"
 DestinationName="java:comp/resource/AIA_ERROR_JMS/Topics/AIA.AIA_ERROR_TOPIC"
 DurableSubscriber="AIAReadJMSNotificationAgent"
 UseMessageListener="false"
 MessageSelector="JMSCorrelationID LIKE '%AIA_EH_DEFAULT%'"
 PayloadType="TextMessage" OpaqueSchema="false">
</jca:operation>

Add cluster-id to the Error notification agent. This makes that only one JMS service will dequeue a message, and not both nodes.

cd $AIA_HOME/Infrastructure/ErrorHandling/AIAReadJMSNotificationProcess/bpel
vi bpel.xml
...
<activationAgent className="oracle.tip.adapter.fw.agent.jca.JCAActivationAgent" partnerLink="consumeFaultMsg">
<property name="portType">Consume_Message_ptt</property>
<property name="clusterGroupId">AIAClusterERRORJMS</property>
</activationAgent>
...

Verify that the ESB systems points to the correct cluster-id and load balancer. Check the flowwing ESB systems

• DefaultSystem
• AIASystem
• BPELSystem

Each system should have the following values:

• Cluster Name: esb
• Virtual Host: loadbalancer.vijfhuizen.com
• Port: 80
• Topic Location: ESBTopics/Topics/ESB_JAVA_DEFERRED
• Connection Factory Location: OracleOJMS/XATCF

The Oracle SOA Suite 10g is now clustered and configured for AIA Foundation Pack.

I would thank Michel Schildmeijer for giving feedback and recommendations for this article.

Done

QED

SOA 11g: Timeout settings

After configuration of your SOA 11g environment, the  (non-)clustered domain must be changed to increase the default time-out settings.

We do this directly in the config files; make sure your admin server and managed servers are down.

At Weblogic level:

Set the global transaction JTA to 3600 seconds, be default it is 40 sec:

$DOMAIN_HOME/soa_domain/config/config.xml
...
<timeout-seconds>3600</timeout-seconds>
...

At SOA 11g - BPEL level: Set the sync wait time of synchrounouse processed to 120 seconds (45 by default). In the first 11g release this can be done via the file, since patch set #1 this can be done via EM going to the BPEL Manager properties.

$DOMAIN_HOME/soa_domain/config/soa-infra/configuration/bpel-config.xml
...
<syncMaxWaitTime>120</syncMaxWaitTime>
...

The following EJB needs to be extended on their time out. This can be done in via Enterprise Manager.

Note, that the first time it will ask you to create a Plan.xml file.
Make sure that this Plan.xml is stored in his own directory, for example: $WLS_DOMAIN/soa_domain/plan/ejb

• Log into Oracle WebLogic Administration Console.
• Click Deployments.
• Expand soa-infra -> EJBs.
• You can check in the config tab for the timeout setting in the BPEL* EJBs:
  BPELEngineBean
  BPELDeliveryBean
  BPELActivityManagerBean
  BPELServerManagerBean
  BPELProcessManagerBean
  BPELInstanceManagerBean
  BPELFinderBean
• Click Save.
• Update the soa-infra application
• Restart Oracle WebLogic Server.

BPEL 10g: Clustering with JGroups on OC4J and Weblogic

Using a cluster with Oracle SOA Suite with BPEL is more or less straight forward. BPEL is using JGroups for clustering. It is used for process deployed, process state changes and process undeployment. Most likely user will use the out-of-the-box feature of JGroups. This is based on multicasting. I have written a cluster document that was based on mulicast earlier.

Using multicasting can be an issue in the network. Multicast is based on UDP and is often blocked in the network. I encountered this issue at a few customers. The solution is rather simple, instead of using UDP/multicast we use TCP to point to the nodes in the cluster.

In the next example, we have two nodes; node1 and node2. They will use port 7900 with a range of 3 to broadcast the BPEL process changes.

On each node:

cd /u01/appl/p1bplpe/product/OracleAS_1/bpel/system/config
mv jgroups-protocol.xml jgroups-protocol.xml.old
cat >> jgroups-protocol.xml << EOF 
<config> 
  <TCP start_port="7900" loopback="true"
      send_buf_size="32000" recv_buf_size="64000"/>
 <TCPPING timeout="3000" initial_hosts="node1[7900],node2[7900]" port_range="3" num_initial_members="3"/>
  <FD timeout="2000" max_tries="4"/>
  <VERIFY_SUSPECT timeout="1500" down_thread="false" up_thread="false"/>
  <pbcast.NAKACK gc_lag="100" retransmit_timeout="600,1200,2400,4800"/>
  <pbcast.STABLE stability_delay="1000" desired_avg_gossip="20000" 
down_thread="false" max_bytes="0" up_thread="false"/>
  <VIEW_SYNC avg_send_interval="60000" down_thread="false" up_thread="false" />
  <pbcast.GMS print_local_addr="true" join_timeout="5000" 
join_retry_timeout="2000" shun="true"/>
 </config> EOF 
 
cp jgroups-protocol.xml jgroups-protocol.xml.new

Restart the nodes

JDeveloper 11g: SSL handshake error

Oracle JDeveloper always connects over HTTPS (SSL) to your SOA Server. In my cluster setup I ran into an issue, that it could not setup the SSL connections.

Creating HTTPS connection to host:node2.vijfhuizen.local, port:8021

unable to find valid certification path to requested target

This issue comes that the runtime Java of JDeveloper does not have the correct certificates loaded in your keystore. You should copy these certificates in your local keystore (cacerts), this file is located in $JAVA_HOME/jre/lib/security.

Implementing this solution is written here.

$JAVA_HOME/bin/javac InstallCert.java

Then

mv $JAVA_HOME/jre/lib/security/cacerts $JAVA_HOME/jre/lib/security/cacerts.org
cp jssecacerts $JAVA_HOME/jre/lib/security

Weblogic SOA Perfomance boost

I saw a strange behaviour with SOA Suite 11g. A customer was complaining that the server was slow, it took them an amount of time to create a domain or even starting a domain. We known that it will take some time to startup SOA 11g, the application needs to be started, SCA artefacts needs to be loaded, etc.


It took some time to found out what the cause was. It was the following combination. The servers are running on VMWare ESX server with Linux.


The cause was the behaviour of the '/dev/random' and '/dev/urandom' devices. Those devices are generating random numbers. It is the combination of VMWare and the '/dev/urandom' device, that causes problems. In this case the '/dev/urandom' is locked when a process is accessing it. That resultat that other processes must and will wait until the devices is available.


Off course, there is a solution for it. At last we found out that someone else also run into this issues! Credits and kuddos to Oleg Shpak!



We can solve this as follows, four solutions:

1) Change the java configuration in a way that '/dev/urandom' is not mapping directly to '/dev/random'. Change the file $JAVA_HOME/jre/lib/security/java.security:

securerandom.source=file:/dev/urandom

into 

securerandom.source=file:/dev/./urandom

2) Add an Java option during startup of the JVM: .

-Djava.security.egd=file:/dev/./urandom

3) Make sure that the /dev/random is always filled, via the rngd daemon. This solution works for all application running on the OS.

4) Not nice, but it works:

mv /dev/random /dev/random.org
ln -s /dev/urandom /dev/random

SOA 11g PS2: Patch set 2, new features

Read this:

http://blogs.oracle.com/soabpm/2010/03/11gr1_patchset_2_111130_soa_fe.html

Summary:

• Re-Introduction of BPEL domains, now called partitions
• Spring Component Implementation
• Full and complete BPEL 2.0
• Improved audit trailing
• Improved transaction handling
• Support of Get/Post binding level
• Full BPMN 2.0 support

SOA still a hype?

Introduction

Some people are thinking dat Service Oriented Architecture is a hype while for others it's common sense. I often hear this comment from customers not from partners. I heard this signal at an Oracle Event "Oracle Fusion Middleware 11g: Weblogic or Wait".

As I work as consultant with this Oracle SOA technology for many years, for me this is common knowledge. As this is also the case by partners and colleagues, I assume this is also the case at the customer.

So is SOA a hype or not? How about technologies as complex Event Procesessing (CEP), Service Composed Architecture (SCA). It depends from which perspective you look at it. Every customer is different and unique. Depending on on the technology that the customer is using, the knowledge of the staff , the past experiences the SOA technology and the branch of the customer. Some of them will find it a hype, while for others it is obvious.

SOA is not a just walk in the park. Every one carries his own bag of experience. Starting with SOA technolgy will take through a learning curve. This curve is as follows:
At the beginning you will be dissapointed on on the new technology. You need to implement more steps for a simple action, for example an insert in a table. But during time and experience you will see the benefits; such as re-usability, business process flow implementation. What I see at most customers using the SOA technolgoy, they need to step out of the current view to the application and think in business process flow. Most users run monolithic applications, or silo apoplication, for many many years. The often result thar functional owners of these application are think in entities and screens. The business process flow is within their head and implemnented over different applications. This is were SOA comes in. It can/must/will/is able to take over the process control of this functional business flow.


So how do we implement a service oriented architecture successfully? There are three subjects to cover:

• Organisation
• People
• Project

Organisation
The organisation must be ready for these type of projects. Therefore the organisation should execute a SOA assesment. In this assessment various parts of the organisation is measured on their maturity on SOA. This assessment show the current status to SOA and what steps to take to increase the maturity level.



People
The people should be trained, from technical point of view as well from functional point of view. Technical training can be done via the normal ways; Oracle University; hire consultants to train the technical people. The functional people should also be trained. They should now the new way of approaching this new implementation and usage of this architecture. The should be trained to think and describe in processes and business flows.

Project
In the beginning, start with a small project that is easy to implement and add values to the business. This project should create a win-win situation for every one. Because these type of projects is new, everyone should learn and benefit from it. Design and Implement this project with expiernced people. The are able to succeed in these kind of projects and they can guide and train the organisation.