Tuesday, August 17, 2010

OSBS: A new version of the build tool

Download OSBS11g here (14MB)..

Introduction
A few years ago, when Oracle had only the BPEL component, deployment was rather complex. Therefore I create my own build tool, replacing the obant tool and ignoring the existing build.xml files, supplied within your JDeveloper project. Now with Oracle 11g the SCA composite is introduced and we are running on the Oracle Weblogic Server.

It took me some time to upgrade the Oracle SOA Build Server to the 11g version. Many new features have been added:
  • Supports Oracle SOA 11g (from 11.1.1.3, aka PS2)
  • Deploy/Undeploy SCA composites
  • Deploy/Undeploy MDS
  • Support of partitions
  • Deploy to diferent targets
  • Restructured project directory
  • Restructured environment properties
  • Updated documentation (Use Guide)
Overview
The default tools of SOA 10g SOA 11g, is only focussing on deployment of a single component (BPEL / ESB / SCA). While in real life, more components are involved with a deployment. You can think on Java application; data-model changes, etc. To overcome this issue, and create a mechanism to deploy multiple type of components, a general "build.xml" that is able to compile and deploy components.

The solution described in the next paragraphs will take this into account.

Functionality
The build tool covers the following features and can be easily change and extended to customer requirements:
  • Depends on SOA 11g environment, installed soa_domain
  • Using full ANT tasks features, Ant 1.7 + Ant-Contrib tasks
  • Support of Multiple environments; Dev / Test / Acceptance / Production / ...
  • Deploy SCA composites
  • Undeploy SCA components
  • UnDeploy MDS artifacts
  • Deploy MDS artifacts
  • Deploy SQL
  • Integration with SubServersion; download latests or tags
  • Stand-alone tool ; not depending on Oracle installation
  • Compile and Deploy Java programs
  • Supporting 'MakeAll'

Configuration
To use the build tool some configuration must be made. Most of all settings are located in properties files. Configure the build tool configuration file to prepare the deployment for this release. Edit the following file:

$BUILD_TOOL/build.env

In build.env, edit ANT_HOME and ANT_JAVA_EXTENSIONS, change the first part of the path to where the package was copied to ($BUILD_TOOL). Also change LOG_DIR in build.env, log files will go to this directory.

An example of this file is here
...
LOG_DIR=/home/oracle/buildtool/log
FMW_HOME=/app/oracle/products/11g/fmw
SOA_HOME=/app/oracle/products/11g/admin/domains/soa_domain
... 

Manual Deployment

Download
Download the latest version of your project. Example for SystemTest environment:

obbuild.sh -e test Download

Note: it will use the file test/replace_.properties to execute the find and replace for the endpoints.


Example for tag 2.0.23 being the current version for acceptance:

obbuild.sh -e acc Download 2.0.22

Example for revision 1254 for development:

obbuild.sh Download rev 1254

Note: it will use the file test/replace_.properties to execute the find and replace for the endpoints.

The replace.properties file is used for a general replacement of tokens in your projects. The file describes how tokens can be replaced in one or
all BPEL processes, ESB services.

Syntax:

find.seq.name=[all|]
[all|].seq.find.value=
[all|].seq.replace.value=

Example:
Replace in all the files localhost:9700 into 127.0.0.1:8001

find.1.name=all
all.1.find.value=localhost:9700
all.1.replace.value=localhost:8001


Example:
Replace in HellWorld bpel process the default domain and the hostname

find.1.name=HelloWorld
HelloWorld.1.find.value=localhost:9700
HelloWorld.1.replace.value=192.168.0.1:8888


find.2.name=HelloWorld
HelloWorld.2.find.value=/orabpel/default
HelloWorld.2.replace.value=/orabpel/systest



Deploy generic SCA composite
Deploy each BPEL process separately. Example for Test environment:

For example: Compile and Deploy Helloworld 1.0 to the default domain of node1


obbuild.sh -e test DeploySCA HelloWorldSCA 1.0 default node1

Deploy Java WebService
Deploy each Java process separately. Example for Test environment:

obbuild.sh -e test DeployJava HelloJavaWorld

Note: default-application.xml must be included in source code.

Automatic Deployment
To deploy the whole system at once, use the task 'MakeAll'.

obbuild.sh -e test MakeAll

Note: This target is using the 'makeall_test.properties' that describes the correct sequence of all the AIA components (Java, BPEL, and ESB) that will be built and deployed. The next table shows an example of the 'makeall.properties' file.


sql.make.10=car
deploy.version.10=1.0
deploy.target.10=dbnode

mds.make.15=common_capesoa
deploy.version.15=1.0
deploy.target.15=node1

sca.make.100=HelloWorldSCArefxx
deploy.version.100=1.1
deploy.target.100=node1

sca.make.200=HelloWorldSCArefxx
deploy.version.200=2.1
deploy.target.200=node2

sca.make.300=HelloWorldSCArefxx

sca.make.20=HelloWorldSCArefbb
sca.make.30=HelloWorldSCArefcc
sca.make.40=HelloWorldSCA

A target can be given to deploy to a particular node or database.

targets.properties
target.node1=node1.vijfhuizen.local:8010
target.node1.username=weblogic
target.node1.password=welcome1

target.node2=node2.vijfhuizen.local:8020
target.node2.username=weblogic
target.node2.password=welcome1

target.admin=t3://192.168.2.10:7001
target.admin.username=weblogic
target.admin.password=welcome1

target.dbnode=jdbc:oracle:thin:@database.vijfhuizen.local:1521:XE
target.dbnode.username=scott
target.dbnode.password=tiger

Command line interface
The command line interface of the obbuild tool is as follows:

./obbuild.sh -?

obbuild.sh [-[hvtV] [-e environment] [-f build-file] [-m makefile] [-r replace-file] [task] [arg1] [arg2] .. [arg6]]

 -h      Shows this help
 -V      Show the version of this program
 -t      Show all the tasks in the build file
 -e      Select envirnonment property file, default 'dev', reading dev.properties
 -f      Read a specific build file, default is build.xml in current directory
-m      Read a specific makefile
 -r      Read a specific replacement file, use for promoting BPEL process to other environments.
 -v      Show verbose output
 task    The task to be executed, default is main
 arg1..6 Optional arguments passed to the build file as properties ANT.ARG1..ANT.ARG6

./obbuild.sh -t

obbuild.sh: Tasks in ./build.xml                                         
          CreateDirs - Create directory structure                        
            Download - Download latest source code                       
                       [arg=tagged-version | rev number]                 
                main - Default task                                      
             Promote - Find/Replace on one or more processes, see replace.properties
         CompileJava - Compile Java files
                       [arg1=name of the application]
           CreateWar - Create WAR file from class files
                       [arg1=name of application]
           CreateEar - Create EAR file from WAR file
                       [arg1=name of application]
          DeployJava - Deploy an EAR file to
                       [arg1=name of the application]
        UndeployJava - Undeploy an application from target
                       [arg1=name of the application]
             MakeAll - Create a new set of the application
                       [arg1=[version] [arg2=partition]
            SendMail - Send a mail
                       [arg1=to arg2=subject arg3=text]
           DeployXSD - Deploy the XSD files: creating a ZIP file
                       [arg1=Name]
          CompileSCA - Compile SCA composite
                       [arg=sca-composite-name]
           DeploySCA - Deploy SCA composite
                       [arg=composite-name] | [arg=version|1.0] [arg=partition] [arg=target]
         UndeploySCA - Undeploy SCA composite
                       [arg=sca-composite-name arg=revision | arg=target]
           DeployMDS - Deploy MDS artifact
                       [arg=oramds-artifact] arg=[target]
         UndeployMDS - Undeploy MDS artifact
                       [arg=oramds-artifact] arg=[target]
           DeploySQL - Deploy SQL
                       [arg=sql directory] | [arg=db-target]
        DeployScript - Deploy Script
                       [arg=ant|wlst] | [arg=script-name] [arg=target]

There is no support from Oracle or myself and I expect that you have the proper Oracle licences to use the Oracle software.

ANT and MDS: Composite Compile Error

When you compile and deply a composite from ANT and you are using the MDS for retrieval of XSD and WSDL composites, you could run into compile errors. The compiler will run into an error that can not load  the files with "oramds:" protocol. For example:

MDS-00054: The file to be loaded oramds:/soa/shared/bpel/runtimeFault.wsdl does not exist.

The solution is to create an adf-config.xml file directly into your composite directory, in the following sub directories, this directory should be created:

/composite-directory/.adf/META-INF/composite

The adf-config.xml shoudl be placed there and contain a mapping to your shared storage. In the next example I choose for persistent storage in the database.

<?xml version="1.0" encoding="windows-1252" ?>
<adf-config xmlns="http://xmlns.oracle.com/adf/config"
            xmlns:adf="http://xmlns.oracle.com/adf/config/properties"
            xmlns:sec="http://xmlns.oracle.com/adf/security/config">
  <adf:adf-properties-child xmlns="http://xmlns.oracle.com/adf/config/properties">
    <adf-property name="adfAppUID" value="JIPSYEW_bpel-5393"/>
  </adf:adf-properties-child>
  <adf-mds-config xmlns="http://xmlns.oracle.com/adf/mds/config">
    <mds-config xmlns="http://xmlns.oracle.com/mds/config">
      <persistence-config>
        <metadata-namespaces>
          <namespace metadata-store-usage="mstore-usage_1" path="/soa/shared"/>
          <namespace metadata-store-usage="mstore-usage_1" path="/apps/mymdsartifacts"/>
        </metadata-namespaces>
        <metadata-store-usages>
          <metadata-store-usage id="mstore-usage_1">
<!--
            <metadata-store class-name="oracle.mds.persistence.stores.file.FileMetadataStore">
              <property value="${oracle.home}/integration" name="metadata-path"/>
              <property value="seed" name="partition-name"/>
            </metadata-store>
-->
<!--
  DATABASE MDS Connection
  User for SCA Ant Compilation
-->
            <metadata-store class-name="oracle.mds.persistence.stores.db.DBMetadataStore">
              <property name="jdbc-url" value="jdbc:oracle:thin:@database.vijfhuizen.local:1521:orcl"/>
              <property name="jdbc-userid" value="dev_mds"/>
              <property name="jdbc-password" value="welcome1"/>
              <property name="partition-name" value="soa-infra"/>
            </metadata-store>

          </metadata-store-usage>
        </metadata-store-usages>
      </persistence-config>
    </mds-config>
  </adf-mds-config>
  <sec:adf-security-child xmlns="http://xmlns.oracle.com/adf/security/config">
    <CredentialStoreContext credentialStoreClass="oracle.adf.share.security.providers.jps.CSFCredentialStore"
                            credentialStoreLocation="../../src/META-INF/jps-config.xml"/>
  </sec:adf-security-child>
</adf-config>