Enterprise Messaging with Synapse, WSO2 ESB and WebSphere MQ

I've been recently playing a lot with Apache Synapse, WSO2 ESB and IBM WebShpere MQ (WMQ). My intention was to use WMQ as a JMS provider for Synapse and WSO2 ESB. IBM WebSphere MQ being a very popular and matured messaging solution, I figured that getting Synapse and WSO2 ESB to work with WMQ and document the integration process would really help the IBM, Apache and WSO2 communities. Originally I anticipated this application integration to be a very comlpex and tedious task. But for much of my delight, it turned out to be a very easy and simple process in the end. What was little complicated was to get WMQ properly installed and get its JMS features configured. But the integration with Synapse and WSO2 ESB was indeed a piece of cake.

So here I'm going to list the steps hat one should follow to get WMQ integrated with Synapse or WSO2 ESB. Start by downloading the required software. Binary distributions of Apache Synapse and WSO2 ESB can be downloaded from their respective websites. A trial version of the IBM WMQ V7.0 can be downloaded from here. In addition to these application you will also require Java 5 or higher along with Apache ANT to run some of the samples described here.

Installing Apache Synapse or WSO2 ESB doesn't require any additional steps. You simply need to extract the downloaded archives and that's all. However WMQ installation is not so easy. It is a fairly long and time consuming process which requires carefulness and patience. Anyway the installation process is well documented and so it shouldn't be a problem. For your convenience I'm describing the WMQ installation procedure for Linux here. The necessary commands to be executed at each step are given in italic font.

Installing WMQ

1. Create an empty directory and extract the downloaded WMQ archive into it

2. Add a new user group called 'mqm' to the system

groupadd mqm

3. Add a new user called 'mqm' to the system. Add the user to the 'mqm' group. Set the user's home directory to /var/mqm

useradd -d /var/mqm -g mqm mqm

4. Create the directory /var/mqm and change its owner to mqm

mkdir /var/mqm

chown mqm:mqm mqm

5. Create the directory /opt/mqm

mkdir /opt/mqm

6. Create another directory anywhere on the disk and set a symlink to it from /opt/mqm

mkdir /home/hiranya/mqm

ln -s /home/hiranya/mqm /opt/mqm

7. Install libstdc++5 library on the system (using Synaptic or apt-get on Debian/Ubuntu systems) if not already installed

8. As root go to the directory where WMQ is extracted and run the license display script

./mqlicense.sh

9. Install the MQ runtime and the server using rpm

rpm -ivh MQSeriesRuntime-7.0.0-0.i386.rpm MQSeriesServer-7.0.0-0.i386.rpm --nodeps

10. Install the MQ samples

rpm -ivh MQSeriesSamples-7.0.0-0.i386.rpm --nodeps

11. Install the MQ client

rpm -ivh MQSeriesClient-7.0.0-0.i386.rpm --nodeps

That wasn't too bad, was it? Now it's time to verify the installation. Here's what you got to do.

Verifying the Installation

1. Login as user mqm (ssh mqm@localhost)

2. Create a new queue manager

crtmqm -q venus.queue.manager

3. Start the queue manager

strmqm

4. Start MQSC program (a new shell will appear - without a prompt)

runmqsc

5. Define a local queue in the MQSC shell

define qlocal (orange.queue)

6. Terminate MQSC

end

7. Go to /opt/mqm/samp/bin and run the amqsput sample program to place a message on to the queue

./amqsput ORANGE.QUEUE

8. The above command should be followed up by some sample message text of your choice and a blank line

9. Run the following command to pull the message back from the queue

./amqsget ORANGE.QUEUE

10. So far so good! We got the server properly installed. Now on to the client!

11. Create a new queue manager

crtmqm -q saturn.queue.manager

12. Start the queue manager

strmqm

13. Start MQSC

runmqsc

14. Create a new queue

define qlocal (queue1)

15. Create a connection channel to the queue

define channel (channel1) chltype (svrconn) trptype (tcp) mcauser ('mqm')

16. Create a listener (specify a port)

define listener (listener1) trptype (tcp) control (qmgr) port (8585)

17. Start the listener

start listener (listener1)

18. Terminate MQSC

end

19. Create a system environment variable to point to the channel created above

export MQSERVER='CHANNEL1/TCP/localhost(8585)'

20. Go to /opt/mqm/samp/bin and run the amqsputc to place a message in the queue (using the client API)

./amqputc QUEUE1 saturn.queue.manager

21. Now run amqsgetc to receive the message

./amqgetc QUEUE1 saturn.queue.manager

22. Excellent! We got the client installed properly too :)

Now that we got WMQ installed let's try to get its JMS features enabled.

Setting Up JMS

1. Install WMQ classes for JMS

rpm -ivh MQSeriesJava-7.0.0-0.i386.rpm --nodeps

2. Start MQSC (This will start mqsc for the last created queue manager - saturn.queue.manager)

runmqsc

3. Define a JMS channel for the queue manager

define channel(java.channel) chltype(svrconn) trptypr(tcp)

4. Define a listener

define listener(listener.tcp) trptype(tcp) port(1414)

5. Start the listener

start listener(listener.tcp)

6. Setup the classpath variable and the MQ_JAVA_LIB_PATH to run the IVT programt

export CLASSPATH=/opt/mqm/java/lib/com.ibm.mqjms.jar:/opt/mqm/samp/jms/

export MQ_JAVA_LIB_PATH=/opt/mqm/java/lib

7. Go to /opt/mqm/java/bin and invoke the IVT sample program

./IVTRun -nojndi

./IVTRun -nojndi -client -m saturn.queue.manager -host localhost -channel JAVA.CHANNEL

8. If the IVT program can recieve and send JMS messages we are done! Now let's proceed to enabling JNDI support

9. Go to the /opt/mqm/java/bin directory and open the JMSAdmin.config file. Edit the PROVIDER_URL property to point to an empty directory of your choice on the file system. This directory will be used the JNDI provider source (eg: /var/mqm/jndi). If the specified directory does not exist in the file system create it.

10. Run the IVTSetup tool to create the default set of JNDI bindings

./IVTSetup

11. Now run the IVTRun tool as follows

./IVTRun -url "file:/var/mqm/jndi" -icf com.sun.jndi.fscontext.RefFSContextFactory

12. By now we have enabled and verified JNDI support. Let's use the JMSAdmin tool to make some modifications in the JNDI bindings

13. Fireoff /opt/java/bin/JMSAdmin and run the following commands:

ALTER QCF(ivtQCF) TRANSPORT(CLIENT)

ALTER QCF(ivtQCF) QMGR(saturn.queue.manager)

14. Run the IVTRun tool again as mentioned in step 11. This time messages will be sent to the saturn queue manager.

Now we are all set. It's time to get WMQ integrated with Synapse and WSO2 ESB. Let's start with Apache Synapse.

Synapse Integration

Follow the steps given below as the user 'mqm'. Trying to run Synapse as a different user caused some JMS security exceptions.

1. Copy the following jar files from /opt/mqm/java/lib to SYNAPSE_HOME/lib

• com.ibm.mqjms.jar
• fscontext.jar
• providerutil.jar
• com.ibm.mq.jmqi.jar
• dhbcore.jar

2. Enable the JMS listener in the axis2.xml. Configure the default connection factory as follows.

<parameter name="default">
  <parameter name="java.naming.factory.initial">com.sun.jndi.fscontext.RefFSContextFactory</parameter>
  <parameter name="java.naming.provider.url">file:/var/mqm/jndi</parameter>
  <parameter name="transport.jms.ConnectionFactoryJNDIName">ivtQCF</parameter>
  <parameter name="transport.jms.ConnectionFactoryType" locked="false">queue</parameter>
  <parameter name="transport.jms.Destination">ivtQ</parameter>
</parameter>

Note that we are using the JNDI configuration used by the IVT sample program. We are using the same queue connection factory and the JMS queue to define the default connection factory.

3. Create the following proxy service in the synapse.xml (this is a simplified version of sample 250)

<proxy name="JMSProxy" transports="jms">
  <target>
     <inSequence>                             
        <log level="full"/>
        <drop/>
     </inSequence>
  </target>
  <parameter name="transport.jms.ContentType">
       <rules>
           <jmsProperty>contentType</jmsProperty>
           <default>application/xml</default>
       </rules>
  </parameter>
</proxy>

4. Start Synapse - go to SYNAPSE_HOME/bin and run ./synapse.sh

5. Go to SYNAPSE_HOME/samples/axis2Client/src/samples/userguide and open the GenericJMSClient.java source file. Make the following changes in the code.

• Set the jms_dest property default value to "ivtQ" (line 45)
• Set the java.naming.provider.url to "file:/var/mqm/jndi" (line 82)
• Set the java.naming.factory.initial to "com.sun.jndi.fscontext.RefFSContextFactory" (line 85)
• Set the lookup key to "ivtQCF" (line 89)

6. Now from the SYNAPSE_HOME/samples/axis2Client directory run the following command to run the JMS client application which will send messages to the ivtQ.

ant jmsclient -Djms_type=pox -Djms_payload=IBM

7. At this point Synapse will pick the messages from the queue and log them on the console.

WSO2 ESB 2.1 Integration

Follow the steps given below as user 'mqm'

1. Copy the Websphere MQ client jars (mentioned above under Synapse Integration) to ESB_HOME/repository/components/lib

2. Enable the JMS listener in axis2.xml and configure the default JMS connection factory as follows

<parameter name="default">              
        <parameter name="java.naming.factory.initial" >com.sun.jndi.fscontext.RefFSContextFactory</parameter>                      
        <parameter name="java.naming.provider.url" >file:/var/mqm/jndi</parameter>
        <parameter name="transport.jms.ConnectionFactoryJNDIName" >ivtQCF</parameter>
        <parameter name="transport.jms.ConnectionFactoryType" >queue</parameter>
        <parameter name="transport.jms.Destination">BOGUSQ</parameter>
</parameter>

Note that we have specified a queue named 'BOGUSQ' as the default destination. This is because we need to use ivtQ for our proxy service only. If we use 'ivtQ' here all the services deployed in ESB (XKMS, echo, wso2carbon-sts) will start listening on the same queue. You also need to login to JMSAdmin and create the queue named 'BOGUSQ'. In JMSAdmin shell run the following commands.

DEFINE Q(BOGUSQ) QMGR(saturn.queue.manager)

ALTER Q(BOGUSQ) QUEUE(QUEUE1)

If the QUEUE1 does not exist, first login to MQSC and create it.

3. Add the proxy service we used with Synapse to ESB_HOME/conf/synapse.xml

4. Start WSO2 ESB

./wso2server.sh -DuseSynapseXML

5. Use the sample client we used in Synapse to send messages to the queue