The Tech Feast

High Volume Web API management with WSO2 ESB

2012-01-20T05:21:00.001-08:00

During last year's WSO2Con, Paul Fremantle and I co-presented on the topic "High Volume Web API management with WSO2 ESB". There we discussed how WSO2 ESB is specifically designed to handle large volumes of traffic received over many concurrent connections. We explained the multithreading, streaming and non-blocking IO aspects of the ESB while looking at some of the high performance extensions we offer such as the message relay and the pass through HTTP transport. We also went through some ESB and JVM performance tuning basics taking the WSO2 ESB deployment at eBay as a case study.

WSO2 ESB Tips & Tricks 08: Message Stores and Processors

2012-01-08T00:54:00.000-08:00

Ability to store a message in a temporary storage is a fundamental requirement in many enterprise integration scenarios. There are many occasions where a systems integrator would have to design a message flow which simply dumps all the messages into some kind of a storage facility for later retrieval. Here are several scenarios that I’ve encountered while working on real-life integration projects which required me to consider using an intermediate message store:

Matching Request Rates

Some services can only process requests at a given rate. But the service consumers, who are generally unaware of the service implementation details, may send requests at any rate they prefer. In this case we should not directly expose the service to consumers who might overwhelm the service by sending too many requests. Therefore consumers should be provided with a virtual service interface (a proxy service) which stores all incoming requests in a persistent store. A separate component would retrieve these messages from the temporary store and play them at the actual service at the preferred rate. This type of a messaging system is generally known as a store-and-forward messaging system.

Guaranteed Delivery

In certain integration scenarios it is of utmost importance that all requests are received and processed by the back-end services (i.e. no message loss). However this is a rather tricky requirement to implement as the back-end services could be unavailable at times due to various reasons like maintenance activities and hardware faults. Here also we can use a persistent message store to solve the problem. We can configure the messaging system or the middleware to detect communication errors that may occur while sending requests and then save the failed requests in a message store for later delivery. Depending on the availability of tools an automated retry mechanism or a manual retry mechanism can be implemented for the failed messages. In this case the persistent store is acting as a dead letter channel.

In-Order Delivery

Some scenarios require that systems consume messages in the exact order they were produced. In such situations respective middleware should be programmed to deliver the requests in the received order. If a request fails due to a temporary outage, the middleware should not send the rest of the messages because that would disrupt the order at which the messages are consumed. The middleware should retry and make sure the failed message is indeed received by the back-end system before the rest of the messages can be sent. A temporary message store is a natural fit for implementing a solution of this nature. However unlike in the previous cases, the message store should be capable of retaining the order in which messages are stored. In general it should be a message queue which provides basic FIFO semantics. The middleware would enqueue all the incoming requests in the store and a different component will pick them up, one at a time, for delivery. In order to preserve the order of messages under error conditions, the delivery component should not remove a message from the store until a satisfactory response has been received from the back-end system for each message sent. This type of systems also falls under the store-and-forward category.

Separation of Concerns

Sometimes it is necessary to run messages through expensive logging and audit routines. However such additional processing can put a lot of overhead on message flows thus contributing to huge delays and sluggish user experience. Using a message store, it is possible to decouple concerns like logging from the fundamental requirements like routing. For an example middleware can be configured to create a copy of each incoming message and dump that into a temporary store. The original message can be routed through the middleware to the desired endpoint without any additional processing. The logging and audit systems can asynchronously consume the messages in the store and perform all the necessary processing on them off-line. This is one of the simplest as well as most common uses of a message store. In fact the standard enterprise integration pattern named "Message Store pattern" describes this very specific use case of intermediate message stores.

As seen from the above examples, the concept of message stores can be used to solve countless enterprise integration problems. The asynchronous and persistent messaging patterns that result from message stores are at the crux of modern message-oriented middleware (MOM). It is an accepted norm that message stores are ideal for constructing complex distributed systems while keeping the overall architecture cohesive but loosely coupled.

Starting from version 4.0.0, WSO2 ESB has built-in support for creating and managing message stores. In fact we took a revolutionary step forward in the enterprise integration space by introducing the “message store – message processor” architecture to WSO2 ESB. At high level a message store is a logical entity in which messages can be stored for later retrieval. Additionally a message store may provide some QoS features such as persistence, message order retention and random access support. The exact runtime behavior and the features offered by a message store are determined by the message store implementation being used. WSO2 ESB currently ships with following two message store implementations.

In-Memory Message Store	Stores messages in an in-memory queue. Not persistent. Supports random access. Retains message order.
JMS Message Store	Stores message in a JMS queue (works with any JMS/JNDI compliant broker). Persistent. Does not support random access. Retains message order.

WSO2 ESB also provides an API using which a user can develop his/her own message store implementations. For an example one can come up with a file system based message store, a JDBC based message store or a NoSQL based message store and use them in WSO2 ESB. XML syntax for configuring a message store is as follows:

<messageStore name="string" class="classname" >
  <parameter name="string" > "string" </parameter>*
</messageStore>

The store mediator is used to enqueue messages in a given message store. This mediator can be used in any sequence or proxy service. Therefore any message flow in WSO2 ESB runtime can be linked up with a message store. By using the store mediator in a fault sequence a message store can be used as a dead letter channel.

<store messageStore="storename"/>

Message stores in WSO2 ESB can be managed via the ESB management console or JMX. One may list the available stores, browse the objects stored in them and perform various operations on the stored messages using the above techniques.

Message processors work hand-in-hand with message stores. They are used to consume messages stored in message stores. Every message processor must be associated with an existing message store. Similar to the case with message stores, there can be multiple message processor implementations. The exact behavior and functionality provided by a processor is determined by its type. Currently WSO2 ESB ships with following two message processor implementations:

Message Forwarding Processor	Retrieves the messages stored in a message store and reliably forwards them to a specified endpoint. This processor attempts to send one message at a time and it does not dequeue a message from the store until it receives a response from the target endpoint. Therefore this processor is ideal for implementing in-order delivery scenarios and guaranteed delivery scenarios.
Sampling Processor	Retrieves the messages stored in a message store and injects them to a given sequence at specified intervals. This processor utilizes the Quartz scheduler framework for periodically processing messages. This can be used to implement message rate throttling scenarios.

Message processors also come with an API using which any custom message processor can be implemented. A message processor is configured as follows in the WSO2 ESB configuration:

<messageProcessor name="string" class="classname" messageStore="storename">
  <parameter name="string" > "string" </parameter>*
</messageProcessor>

As you can see the message store-message processor architecture and the associated API form a very powerful framework on which a wide range of enterprise messaging scenarios can be implemented. Generally speaking, any integration scenario that involves asynchronous or persistent messaging can be easily realized using these components of WSO2 ESB. We have already received some very positive feedback on this architecture from our users and I believe in the long run this will become one of the most widely used features of our middleware stack.

If you are interested in leaning more about the "message store-message processor" architecture of WSO2 ESB following resources will provide some useful details:

Official documentation: Message stores
Official documentation: Message processors
Article: Implementing store and forward messaging patterns with WSO2 ESB - part 1
Article: Implementing store and forward messaging patterns with WSO2 ESB - part 2

WSO2 ESB 4.0.3 is Out with REST API Support

2012-01-04T06:02:00.000-08:00

WSO2 ESB 4.0.3 is now available for download. This contains several bug fixes over WSO2 ESB 4.0.0 and 4.0.2 releases. In addition to that this release comes with the highly anticipated REST API management support. This is a very powerful feature which I believe will be a game changer. I'm planning to do a series of blog posts and articles in the near future describing various use cases and capabilities of this brand new feature. At WSO2 we have already done some public webinars, workshops and development work for actual customers using the REST API support in the new ESB release and the feedback we have received so far has been phenomenal.

REST support is nothing new for WSO2 ESB. Even though we are primarily based on Apache Axis2, which is a SOAP engine, we have always had support for exposing RESTful interfaces and mediating RESTful invocations. However REST support in WSO2 stack was always a bit of a black sheep in the family. More often than not REST was perceived as a second class citizen in the WSO2 stack. But the new REST API support we have implemented changes all that. Now REST fans can easily define REST APIs and associated resources on the ESB and combine them with the powerful mediation features provided by the underlying messaging framework. Instead of worrying about mapping REST concepts to SOAP (which was the old way of handling REST calls with proxy services), developers can now configure REST endpoints in the ESB by directly specifying HTTP verbs, URL patterns (servlet styled), URI templates, HTTP media types and other related headers. Most REST fans will find themselves at home when working with the new API support in WSO2 ESB.

That's only a beginning. We are also looking into combining some of our best products together (including ESB of course) to come up with a comprehensive API management solution. This will be a complete platform where developers will be able to develop and expose REST APIs and users can subscribe and consume them.

As shown in the above diagram, core API management features such as API key management, user management, security and SLA monitoring are all provided by the solution itself. We are also looking at how to leverage our multi-tenant support to make this solution work seamlessly in the cloud.

If you are interested in learning more about the new REST API support in WSO2 stack and our future plans in this area, we are conducting a series of workshops on this very topic in US later this month. I'm sure you will find these sessions both interesting and eye opening. Also stay tuned for my own blog posts and articles on the subject which will give a more in-depth technical insight on the matter.

ESB: The Swiss Army Knife of SOA

2011-11-13T15:51:00.000-08:00

My talk at WSO2Con 2011 is now available on YouTube. There I discussed why an ESB is essential for SOA, what kind of problems it can solve and what it is not.

JMS Proxy Services with Fail-Over Support

2011-10-22T22:38:00.001-07:00

Rajika has written a very comprehensive article about setting up WSO2 ESB with JMS in a fail-over scenario.

The concept of fail-over endpoints is integral to WSO2 ESB. The user can define a fail-over endpoint in the ESB which in turns consists of two or more child endpoints. ESB will select one of the child endpoints to be the primary endpoint and will keep forwarding all messages to this primary endpoint. If the ESB detects some error while communicating with the primary endpoint, it will automatically fail-over to one of the available backup endpoints. User can also configure the endpoint error codes upon which to trigger a fail-over, endpoint suspend durations and timeout durations. However this is not what Rajika is talking about in his article. His article is about setting up a JMS proxy service with fail-over support in the ESB .

For an example imagine a situation where you have setup a proxy service to receive messages from a JMS queue. The ESB proxy service polls messages out of the queue and forwards them to a backend application for further processing. Now what if the ESB instance which hosts the above mentioned proxy service happens to fail (say due to a hardware failure). Actually in this case there's both good news and bad news. Good news is that no messages will get lost due to the persistent behavior of JMS messaging. If some messages were in flight within the ESB when the failure occurred those messages might get lost. But even that can be avoided by using something like JTA. The bad news however is that the backend application will not receive further requests to be processed. So as far as your overall system is concerned, your applications and services will stay idle while messages keep piling up on the JMS queue.

Rajika discusses a possible architecture to avoid such system downtime by having a backup proxy service (ideally deployed in a separate ESB instance) which listens on the same JMS queue. The backup proxy will be disabled by default to prevent contention between the two proxy services. His architecture also utilizes a scheduled task deployed on the backup ESB which sends periodic heartbeats to the other ESB node. If the backup ESB detects that the primary ESB instance is not responding, it will activate the backup proxy service which will start polling for JMS messages and resume operations in the system.

This is actually fairly simple to implement in WSO2 ESB and can be even further customized to suit the exact enterprise requirements. All the source code and relevant configurations are provided in the article itself.

Rediscover SOA with WSO2 Carbon and WS-Discovery

2011-10-18T09:47:00.000-07:00

WS-Discovery is a standard protocol for discovering services and endpoints in an SOA based environment. Using WS-Discovery, servers can publish service metadata to a centralized repository known as the discovery proxy. Clients and other interested parties can query the discovery proxy to find a required service or an endpoint.

WSO2 Carbon platform has exceptional support for WS-Discovery. Service hosting products like WSO2 Application Server can publish their services to a user defined discovery proxy. WSO2 Governance Registry can be used as this discovery proxy and products like WSO2 ESB can query a discovery proxy to locate backend service endpoints.

I recently wrote an article explaining how WS-Discovery is supported in the Carbon platform. The article starts by explaining the basics of WS-Discovery and then describes how to build a complete practical solution based on the discovery protocol. The sample solution described in the article, an inventory management system for Trader Paul's department store, uses 4 WSO2 products. Almost all the core business logic of the solution resides in WSO2 Application Server and WSO2 Data Services Server. The web application which provides the UI layer of the solution uses WS-Discovery to dynamically discover the business services and data services. WSO2 Governance Registry acts as the discovery proxy where all the service metadata are stored. The solution also uses WSO2 ESB to expose some of the core services to the public through a secured channel. The article describes how to use the discovery support in ESB to create these proxy services. Further it describes an advanced functionality of the ESB (discovery mediation extensions), using which 100% dynamic proxy services can be created. These dynamic proxy services use the WS-Discovery protocol to discover the backend service endpoints at runtime.

This article shows the real power of the Carbon framework and what we can achieve with it when the appropriate components/products are combined into a single distributed solution. I certainly enjoyed a lot putting up all the samples and writing content for this article. I hope you will get a similar feeling reading it. All the code and configurations used for the samples are also provided so you can run the Trader Paul's Inventory Management System on your own and try it out. I'm currently working on porting this sample to the StratosLive environment. So it will be soon available on the cloud as well. If you are just looking to learn about WS-Discovery, or just trying to understand how to build solutions with WSO2 products, this article would be a great starting point.

WSO2 ESB Documentation Revamped

2011-10-16T02:22:00.000-07:00

No software application is complete without documentation. According to Ian Sommerville (author of the well known textbook on Software Engineering), software is not just the computer programs, but also all associated documentation and configuration data that is needed to make the programs operate correctly. According to this definition documentation is not only a key component of software, but it is one of the factors which differentiates software from computer programs. But unfortunately many open source projects are getting hammered for their poorly structured and less comprehensive documentation. WSO2 ESB documentation also used to suffer from a number of such strictures which made it look slightly obfuscated and disorganized to the reader. While the samples and other configuration related documentation was fairly well organized, the overall documentation offering had several gaps in terms of content, user friendliness and ability to search.

However I'm pleased to announce that those days are now behind us. WSO2 ESB now has a brand new documentation package with a whole lot of new content, an exquisite look and feel and a ton of new features. You will find that almost all the old content has been rewritten, reformatted and reshaped to make it more comprehensive and easier to follow. In contrast to the old documentation, the new offering contains many diagrams, sample configurations and much easier navigation capabilities between different sections. Samples are no longer a separate section in the documentation but they are baked into the relative explanatory sections in the documentation. This way the reader can seamlessly move between content and relevant samples.

Perhaps the most significant feature in the new documentation package is its ability to search for content. All the content items have been properly labeled and indexed so the reader can find out what he's looking for by doing a search on the documentation site. Try out following search links to see for yourself.

As of now you will find documentation related to WSO2 ESB 3.0.1 and WSO2 ESB 4.0.0 in the new documentation site. Please note that this is still very much an ongoing effort. We are continuously working on making our documentation even more comprehensive and easier to follow. Feel free to help us out by pointing out mistakes, limitations and better yet providing new content that we can include in our documentation offering.

WSO2: The Mission, Vision and the Future

2011-10-15T08:26:00.000-07:00

That was without a doubt a perfect week. I'm of course talking about the WSO2Con 2011 which was held last month at Colombo, Sri Lanka. We are still trying to cope up with the continuous flow of complements, well wishes, congratulations and thank you notes. There's no doubt in my mind that it was the most elegant and most successful technical event Sri Lanka (and perhaps even Asia) has ever witnessed. We had over 20 technical presentations conducted by people from various countries and organizations. We had participants from all corners of the world; USA, UK, Russia, India, China, New Zealand, Mexico...you name it. If you are doing anything related to web services or SOA that was the place to be. (If you missed it this year, start planning for next year right now)

The opening keynote of WSO2Con delivered by Dr. Sanjiva Weerawarana is now available on-line. This speech is about WSO2, WSO2 history and WSO2 vision. It's about what we have achieved, how we operate and what we intend to conquer in the years to come.

Within just a matter of 6 years, WSO2 has managed to develop a comprehensive SOA middleware platform which consists of around 15 products and a very powerful cloud platform. WSO2 products are used by some of the largest organizations in the world to handle billions of business transactions. WSO2 offers the only complete SOA PaaS in the cloud which is capable of hosting web services, web applications, data services, mashups, business processes, gadgets, integration flows and more. And WSO2 has managed to do all this while adhering to true open source principles and open standards.

This speech is about these great achievements, what motivated us to do what we do and how we are going to scale the game up in the future.

SOA & Business: Thoughts from a Pro

2011-09-12T08:44:00.000-07:00

Most developers and architects today understand the fundamentals of SOA. SOA concepts and related technologies such as WS-* have been around for several years now and they are not exactly the new kids in town. But it's surprising to see that many business organizations are still not harnessing the true power and potential of SOA. SOA can help make a business more agile and flexible to change while reducing lot of development and maintenance overheads. It can also address many IT management problems by implanting proper SOA governance practices and processes within an organization. So then why many organizations have yet failed to realize these goals with SOA?

While many companies have been struggling to implement proper SOA, eBay, the world's largest on-line market place, has done wonders with it. Most of eBay's IT infrastructure is based on SOA principles. They have also developed a fantastic SOA platform called eBay Turmeric, which they open sourced a few months ago (this framework uses WSO2 Governance Registry inside). Sastri Malladi, who is a distinguished architect at eBay has been a driving force behind these SOA adoption efforts. In a couple of days time he's going to deliver a keynote speech titled "Service Orientation - Why is it good for business?" at WSO2Con. If you are wondering why business organizations should consider SOA or why they are failing to implement proper SOA, this is the session to attend. I'm sure it's going to be a very exciting session.

SOA & Security: Keeping the Evil Away

2011-09-10T03:03:00.000-07:00

SOA is about opening up. Instead of writing tightly coupled code, you implement your business logic and data access logic as reusable services which can be accessed over a network. Applications, clients and other stakeholders consume these services in varying manners to interact with your business. But more data and business logic you open up, more insecure and vulnerable you organization becomes to various types of attacks. Therefore you need to think about the security aspects of your SOA deployment right from the beginning. Security should be an integrated part of your solution and not a patched up layer that lives elsewhere. We all know what happens to systems with poorly designed security aspects. Need I remind you the well known case of Cablegate?

Fortunately for developers and architects, there are tons of security mechanisms and technologies out there. However selecting the right set of security technologies is usually a quite cumbersome task. You need to carefully weigh in many factors including level of security required, performance, hardware availability and user experience. You should also take into account the strengths and weaknesses of each security mechanism before deploying any of them in your organization.

Prabath Siriwardena, one of our security Gurus, is preparing to deliver a speech titled “Security in Practice” next week at WSO2Con. He will be explaining some of the above mentioned challenges associated with SOA security and how to overcome them using widely accepted security technologies such as WS-Security, WS-Trust, SAML, XACML, OpenID and OAuth. Prabath will discuss each of these technologies in detail, while exploring the pros and cons of each approach. If you are looking to develop a system based on SOA or just looking around to learn the security best practices in the industry, this is the session to attend. So don’t miss this opportunity of a lifetime.

SOA & ESB: Food for Thought

2011-09-08T22:44:00.001-07:00

With the immense popularity of Web Services and SOA, the term "Enterprise Service Bus" has become a well known technical buzz word. Many people consider Enterprise Service Bus (ESB) to be a magic wand by which any problem in enterprise integration and SOA can be solved at a whim. But what is the real technical definition of ESB? What are the problems it's supposed to solve? What are the fundamental services and facilities provided by an ESB? What are its limitations and what are the problems it's not supposed to solve? Many people don't really know the answers to these important questions and end up using the wrong tool to solve the wrong problem.

Some people consider ESB as a shortcut to implementing SOA. And why not? Most ESB solutions out their (including our own WSO2 ESB) provide excellent support for SOAP, WSDL and a plethora of other WS-* standards. But does adding an ESB into a solution architecture really bring any SOA aspects into it? On the other hand is it possible to implement a SOA without using an ESB? Food for thought...

Now don't get me wrong. ESB is a fantastic piece of technology. The number of problems it can solve and the number of use cases it can support is mind boggling. Having worked with a number of customers in the past, I've learnt that regardless of the application of SOA principles, it's almost impossible to implement a useful integration solution without an ESB. However as developers and architects we must have a good understanding of what an ESB is and what it is capable of so we can put the ESB technology to use in a more effective manner. Knowing the answers to above questions will help us make better choices when planning out an integration project and selecting middleware for a solution implementation.

I'm going to make an attempt at answering some of the above mentioned questions in my WSO2Con talk scheduled for next Tuesday. It would be interesting to listen to the feedback other developers and architects in the audience have to offer regarding these all important issues. So if you're keen on learning the fundamentals of enterprise integration and ESB feel free to drop in to my talk titled "ESB: The Swiss Army Knife of SOA". If you're not interested in this particular topic, please do join us at WSO2Con anyway. We have 20+ other professionals lined up to deliver some amazing tech talks. I'm sure you will find it interesting and useful.

WSO2Con 2011... An Event Not to Miss

2011-09-07T01:55:00.000-07:00

WSO2Con 2011 is happening next week at Colombo. It's going to be a parade of tech talks conducted by 20+ brilliant speakers from all around the world. WSO2 also has plenty of tutorials, hands-on technical sessions and entertainment items planned for the attendees. This is definitely not an event to miss.

No Business Like Show Business

2011-07-24T09:00:00.000-07:00

Some highlights from my visit to the Wax Museum at Hollywood, LA....

The Fantastic 4

2011-06-20T10:18:00.001-07:00

As usual I'm late with this, but better later than never :)

After almost an year worth of planning, developing, bug fixing and testing, WSO2 ESB 4.0 is finally out. This is the first WSO2 ESB release for year 2011 and follows the 3.0.1 release which went out in September 2010. WSO2 ESB 4.0 comes with an array of new features targeting enterprise users:

Message stores and processors (for implementing advanced QoS levels such as in-order delivery and exactly once delivery)
Concept of functions for the ESB configuration language (define sequences and endpoints as parameterized reusable entities)
Deployment synchronizer (easily synchronize clusters of ESB nodes via the registry)
Built-in Qpid broker (can be used as a JMS provider or an event broker for pub-sub messaging)
LDAP based user store powered by Apache DS
HTTP relay transport (a new non-blocking HTTP transport that supports pure streaming)

In addition you will find the following new mediators in this release:

URL rewrite mediator (rewrite and modify URL strings)
Event mediator (fire events to predefined topics)
Conditional router mediator (implement complex routing scenarios and rules easily)

Apart from these improvements, our UI gurus have given a complete make over to the ESB management console with collapsible menus, improved input validation, and new styling. You will also find that many of the recurring bugs in the 3.x series have been fixed in this latest release.

If you are looking for a high performance, user friendly solution for enterprise integration, then WSO2 ESB is for you. It's completely free and open source under the ASL 2.0. Download today and let us know how it goes. You can get in touch with us over mail, forums and issue tracker.

WSO2 ESB Tips & Tricks 07: Cron Legacy

2011-06-10T22:46:00.000-07:00

SOA architects and system integrators often come across situations where they want to schedule a particular task to be executed periodically. It could be a task which polls a message queue or a database at regular intervals to see whether any new data has been received. It could be a task which invokes a web service to retrieve a live feed or to check the status of an external system. Or it could be a task which performs regular house keeping activities such as cleaning up log files, updating configuration files and obtaining data backups.

WSO2 ESB provides a simple but powerful framework based on Quartz for scheduling and managing periodic tasks. The user can program tasks using Java and deploy them to the ESB as jar files. Such custom developed tasks must implement the Task interface of Apache Synapse which consists of a single method named execute(). Within this method user can implement his/her business logic which will then get executed periodically by the ESB. The compiled jar files containing the task implementation classes should be placed in the repository/components/lib directory of the ESB. These jar files will get converted into OSGi bundles at server startup and get deployed into the ESB runtime.

When it comes to scheduling tasks, the user has two options. The user can configure a simple trigger by simply specifying the number of times the task should be executed along with the period between successive runs. The following example shows how the task named foo has been scheduled to be executed every 5 seconds for 1000 times. Here the task implementation class is named org.wso2.testing.tasks.FooTask (to keep the task running until the server is shutdown, just get rid of the count attribute).

<task class="org.wso2.testing.tasks.FooTask" name="foo">
 <trigger interval="5" count="1000"/>
</task>

Alternatively user can configure tasks using the well known ‘Cron’ syntax. This way it is possible to specify more complex timing requirements. The following example shows how the task foo has been scheduled to be executed every Friday at 5pm. (To learn more about Cron syntax refer this tutorial)

<task class="org.wso2.testing.tasks.FooTask" name="foo">
 <trigger cron="0 0 17 ? ? FRI"/>
</task>

WSO2 ESB ships with a sample task implementation class named MessageInjector. This can be used to inject a message into the service bus periodically. Injected messages will be handled by the ‘main’ sequence so they can be routed and mediated as if they were received from a remote client. The same technique can be extended to implement a scheduled task which periodically invokes a web service. Sample 300 of WSO2 ESB documentation shows how to put the MessageInjector task to use.

<task class="org.apache.synapse.startup.tasks.MessageInjector" name="CheckPrice">
 <property name="to" value="http://localhost:9000/services/SimpleStockQuoteService"/>
 <property name="soapAction" value="urn:getQuote"/>
 <property name="message">
               <m0:getQuote xmlns:m0="http://services.samples">
                   <m0:request>
                       <m0:symbol>IBM</m0:symbol>
                   </m0:request>
               </m0:getQuote>
       </property>
       <trigger interval="5"/>
</task>

Note how a set of properties are passed into the task implementation class from the XML configuration. This can be done with user developed custom tasks as well. Let’s assume the task implementation class has a private attribute named ‘studentName’. If you want to pass a value for this attribute from the task configuration you should implement a getter method and a setter method for the above attribute following the standard Java bean conventions (ie getStudentName and setStudentName). Then you can have the following property definition under the task configuration to initialize the studentName attribute.

<property name="studentName" value="Hiranya"/>

This technique can be used to initialize any primitive type attribute of the task implementation class.

WSO2 ESB scheduled tasks also support a concept called ‘pinned servers’. This comes in handy when deploying scheduled tasks in a cluster of ESB servers. In such an environment it might be required that the task get deployed only on some of the nodes and not all the nodes. But since nodes in a cluster usually share all the configuration files this becomes little tricky. The solution is to specify the pinned servers attribute in the task configuration. With that the task will get deployed only on the specified set of servers. Basically this feature allows you to pin a scheduled task to a subset of the nodes in a cluster.

<task class="org.wso2.testing.tasks.FooTask" name="foo" pinnedServers="host1.wso2.org, host2.wso2.org">
 <trigger cron="0 0 17 ? ? FRI"/>
</task>

In the above example the task will get deployed only if the server hostnames match the ones specified in the pinnedServers attribute.

WSO2 ESB management console can be used to create and manage scheduled tasks. The ESB artifact uploader UI (added in ESB 3.x) can be used to upload jar files containing tasks to the ESB without restarting the server. Unfortunately current releases of WSO2 ESB do not enable the user to temporarily suspend the execution of a task. To do that user must remove the task from the ESB configuration. This limitation will be addressed in a future release.

Best Train Ride Ever

2011-03-20T09:07:00.000-07:00

I was in Europe with a colleague, working with some customers and we got the rare opportunity to travel from Switzerland to Austria by train. The initial plan was to take a flight, but my CEO, Dr. Sanjiva Weerawarana suggested that we take the train and enjoy the scenery along the way. Boy oh boy, wasn’t he right about that?

Starting from Zurich, the train runs several hundred kilometers parallel with the Alps, until it turns South at Salzburg to finally cross the magnificent mountain range to reach the city of Graz, our destination. We had to make two connections along the way and the trip took almost 10 hours, but it totally was worth it and we enjoyed every minute of the journey. A couple of days later, we again crossed the Alps by train, this time to get to the city of Vienna.

I just downloaded all the pictures from my camera to the computer and apparently I have taken several hundred photographs along the way. I have a hard time picking the best of them, and therefore I’m just going to upload a few random photographs here…

WSO2 ESB Tips & Tricks 06: Error Handling Semantics

2011-02-16T01:49:00.000-08:00

Sequence error handling is one of the most useful yet undervalued features of WSO2 ESB. Sequence error handling brings Java try-catch like semantics into the ESB mediation flow. However, error handling semantics of WSO2 ESB slightly varies, depending on how the sequences are setup in the ESB. This seems to generate lot of confusion around the concept of ESB error handling. For those who find it difficult to understand the error handling capabilities of WSO2 ESB, following guidelines might help:

If a sequence explicitly defines a fault handler using the ‘onError’ attribute, the specified fault handler will be invoked, whenever an error occurs in the sequence. This is true even if the sequence is invoked by a proxy service.
If a request arrives through the main sequence and if it happens to fail within a sequence which does not explicitly define a fault handler, the default ‘fault’ sequence will be invoked.
If a request arrives through a proxy service and if it happens to fail within a sequence which does not explicitly define a fault handler, the fault sequence of the proxy service will be invoked. If the proxy service does not define a fault sequence, then no fault handler will be invoked.
When there is a fault handler engaged at proxy service level, and another error handler engaged at the sequence level, the sequence level error handler gets invoked in case of an error (as per rule 1). In this case the proxy service fault sequence is ignored.

To learn more about WSO2 ESB error handling, please refer the previous installment of the tips and tricks series.

Among Stars and Planets

2011-01-23T07:28:00.000-08:00

From my visit to NASA Ames Research Center:

It's a great place to revise your old lessons on sun, moon, earth and the other planets. If you ever visit this place, check out the cool 3D visual model of the solar system they have on the giant screen. It is controlled by a Wi-Fi operated joystick. They sometimes let the visitors operate the model for a while. I was among the lucky few :)

Apache Synapse 2.0 Released

2011-01-15T07:12:00.000-08:00

The Apache Synapse team reached a very important milestone last week. After a long but very busy 2 and half years, we announced the general availability of Apache Synapse version 2.0. The previous release of Synapse was version 1.2 which was released way back in year 2008. The Apache Synapse project has grown a lot in terms of code, features and community over the past 2 years and therefore you will find a horde of new features, improvements, bug fixes and samples in the latest release. Some of the noteworthy new features in this release are:

New, fine-grained configuration model
Hot deployment and hot update support for configuration artifacts
Priority based mediation support
Comprehensive eventing capabilities with WS-Eventing support
Secure vault for encrypting passwords in configuration files
File locking support in the VFS transport for concurrent polling
URLRewrite mediator for fast and simple URL rewriting
Synapse configuration observer API
Multiple identity support in the HTTPS transport
Enhanced JMX monitoring support for the NHTTP transport
Dead letter channel implementation (experimental)
Synapse XAR Maven plug-in for generating configuration artifacts

Apache Synapse is a lightweight Enterprise Service Bus (ESB) released under the Apache Software License. It is based on the tried and tested Apache Axis2 SOAP engine. It has excellent support for SOAP, REST, POX, JSON and a variety of wire level transports (HTTP/S, JMS, File transport, FIX...). The non-blocking HTTP transport of Synapse enables it to handle very high volumes of HTTP traffic over thousands of concurrent connections.

Apache Synapse also supports many WS-* standards including WS-Security, WS-ReliableMessaging, and WS-Policy. It can be easily linked up with any SOA registry for metadata management and governance purposes. Synapse has a very flexible configuration model built into it, which makes it one of the easiest ESB applications to learn and adopt.

If you are involved in any SOA projects or system integration activities, Synapse may have something to offer. So please feel free to grab the binary distro and take it for a spin. Feed back welcome on Synapse user list.

From Fleet Week Air Show

2011-01-02T04:58:00.000-08:00

Here are some pictures from the Fleet Week air show held on last October at San Francisco port. The main attraction of the event were the renowned Blue Angels.

(Sorry about the sloppy pictures. I’m not a pro photographer and taking still pictures of some fighter jets travelling close to the speed of sound is probably not the easiest thing in the world.)

Exploring the History of the Silicon Chip

2010-12-28T01:45:00.000-08:00

A few weeks back I visited the Intel Musuem along with my colleagues Sumedha and Nuwan. The museum is located at Intel’s headquarters in Santa Clara, California- the heart of Silicon Valley. It showcases Intel’s products and processes while giving a deep insight to the history of microprocessors. They exhibit almost all their land mark processor designs from the past, equipment and materials used to manufacture silicon chips and some of Intel’s innovative creations such as ‘Arti’ the talking robot.

Here are some of my favorite pictures from the trip....

Apache Xerces2/J 2.11.0 Released

2010-12-07T21:41:00.000-08:00

The Apache Xerces project team is pleased to announce that version 2.11.0 of Apache Xerces-J is now available.

Xerces-J 2.11.0 can be downloaded at: http://xerces.apache.org/mirrors.cgi

This release expands on Xerces' experimental support for XML Schema 1.1 (see Xerces-J-bin.2.11.0-xml-schema-1.1-beta.zip) by providing implementations for the simplified complex type restriction rules (also known as subsumption), xs:override and a few other XML Schema 1.1 features. Xerces-J 2.11.0 also introduces experimental support for XML Schema Component Designators (SCD). It fixes several bugs which were present in Xerces-J 2.10.0 and also includes a few other minor enhancements.

Specifically, the changes introduced in this release are:

Implemented XML Schema 1.1's simplified complex type restriction rules (also known as subsumption)
Added support for XML Schema 1.1's overriding component definitions (<xs:override>)
Added experimental support for a parser and evaluator for XML Schema Component Designators (SCD)
Implemented support for the vc:typeAvailable, vc:typeUnavailable, vc:facetAvailable and vc:facetUnavailable attributes for conditional inclusion processing in XML Schema 1.1 documents
Implemented support for allowing xs:group as a child of xs:all in XML Schema 1.1 documents
Made several enhancements to the support for assertions in the XML Schema 1.1 implementation
Improved the ways in which the XML Schema API exposes values and identity constraint information
Fixed a bug where XMLSchemaValidator.findSchemaGrammar() was not getting called for substitution groups
Fixed a bug in the regular expression support which was introduced in the previous release. See JIRA Issue XERCESJ-1456 for details
Fixed multiple issues with decimal precision in the processing of xs:precisionDecimal value
Fixed various bugs and made various improvements

A complete list of JIRA issues resolved in this release is available here: http://s.apache.org/pI

For more information please visit: http://xerces.apache.org/xerces2-j/

SOA Meets the Cloud

2010-11-15T11:04:00.000-08:00

We just released WSO2 Stratos v1.0.0, the SOA PaaS solution based on WSO2 Carbon. Stratos enables you to run WSO2 middleware on a cloud infrastructure in a completely multi tenanted environment. This release brings you following SOA middleware services:

WSO2 Enterprise Service Bus as a Service
WSO2 Application Server as a Service
WSO2 Data as a Service
WSO2 Governance as a Service
WSO2 Identity as a Service
WSO2 Business Activity Monitoring as a Service
WSO2 Business Processes as a Service
WSO2 Business Rules as a Service
WSO2 Mashups as a Service
WSO2 Gadgets as a Service

With Stratos you can deploy webapps, web services, mediation flows, gadgets and business processes on the cloud. Just like all WSO2 products, Stratos is also free and open source. Try out Stratos for free at cloud.wso2.com.

ASF Says "It's On"

2010-11-11T15:00:00.000-08:00

As most of you know there's an ensuing battle between the Apache Software Foundation (ASF) and Oracle over Java and Apache Harmony. The roots of this conflict go all the way back to the time when Java was mainly driven by Sun Microsystems. However with ASF getting re-elected as a ratified JCP executive committee member, things are starting to take an interesting turn. A couple of days back ASF made a bold statement indicating that they are going to vote down the upcoming Java SE 7, which Oracle is pushing forward so hard. ASF is also urging other JCP members to back ASF on this. ASF has indicated that they are not afraid to terminate their ties with JCP if needed. This is pretty huge for a group that maintains an overwhelming number of world class Java projects including Tomcat, Maven, Geronimo, Ant and Derby. (I also happen to be a committer for a couple of Java projects at ASF)

There are currently so many news articles on the web covering this conflict end-to-end. I found the news report from The Register to be quite interesting and amusing at the same time. Here's a little excerpt:

Oracle was forced to eat crow and congratulate ASF for its near unanimous election back onto the JCP's SE and EE executive committee. It was an election that saw its own nominee - the little-known-outside-Oracle-circles Hologic - soundly rejected by JCP members.
Congratulating ASF, Oracle's spokesperson on Java SE, Henrik Ståhl, claimed Oracle still respected ASF and wanted to work with the group. "Our disagreement around TCK licensing does in no way lower our respect for and desire to continue to work with Apache," Ståhl said here.
On Tuesday, the ASF told Ståhl and Oracle just exactly where they could shove their "respect" with a statement saying it is walking out of the JCP unless Harmony gets a license.

Another interesting news report is available on IT World.

Taming the Java Garbage Collector

2010-11-04T16:19:00.000-07:00

Tuning the garbage collection in JVM is one of those things that developers often tend to ignore and overlook. But if done properly, it can save you hundreds of megabytes worth precious memory, without making a significant impact on the application performance. Also GC tuning becomes an absolute necessity in certain applications due to various QoS requirements such as real-time processing, low response time and high throughput. So today I'm going to discuss a little bit about GC in JVM and how to properly tune up GC for best application performance. My discussion is entirely focused on Java 5 and 6, so if you are using an older JVM you are at the wrong place.

Before we jump into the discussion on GC tuning we need to have a basic understanding on the various concepts associated with GC. So here goes...

What is garbage collection?

Garbage collection is the process of finding objects which are no longer reachable from references in the executing code, and reclaiming the memory occupied by such objects. An object which is reachable by at least one reference in the executing code is considered to be 'live'. Objects which cannot be reached by any references are called 'dead' objects. So the process of garbage collection can be also defined as the process of finding dead objects in the memory and reclaiming the memory used by them. In general, a garbage collector is responsible for 3 tasks:

Allocating memory for new objects
Ensuring that any referenced objects (live objects) remain in memory
Recovering memory used by objects that are no longer reachable (dead objects)

The following characteristics govern the notion of a 'good' garbage collector:

Safe (never reclaims a live object)
Comprehensive (a dead object does not remain unclaimed for more than a small number of collection cycles)
Efficient (does not introduce too long pauses)
Defragmentation (does not leave the memory fragments all over the place)
Scalable (allocation and deallocation scales well in multithreaded systems)

It is not possible and necessary for a garbage collector to display all these desirable characteristics at once. Usually each garbage collector has its own strengths and weaknesses.

GC Classification

Garbage collectors can be classified based on a number of factors.

Serial vs Parallel:

In serial collection only one collection occurs at a time (even with multiple CPU cores). In parallel collection, the task of collection is divided into subtasks and executed in parallel, possibly on multiple CPUs. This speeds up collection but is more complex and leads to potential fragmentation.

Stop-the-world vs Concurrent:

Stop-the-world collectors suspend the entire application during collections. Concurrent collectors run concurrently with the application (there could be occasional stop-the-world collections). With concurrent collection, freeze times are shorter but it has to operate on the objects which are being used by the running application. This adds more overhead on the collector and requires more CPU power and heap.

Compacting vs Non-compacting vs Copying:

Compacting collectors arrange all the live objects together in contiguous memory blocks. Then the remaining space can be considered free. This way the collection is slow but the allocations are faster. Non-compacting collectors free dead objects in-place. This leads to faster collections but also a recipe for fragmentation. Copying collectors copy (in contrast to moving) all the live objects to a different area in the memory. Then the source area can be considered free. This leads to slower and expensive collections but provides better allocation performance.

Generational Collection

Modern garbage collectors follow a scheme known as 'generational collection'. With this approach, the heap memory is allocated to two or more regions known as generations. A generation is a block of memory which contains objects of a certain age. In many implementations there are two generations, one for young (new and reasonably new) objects and one for old objects. Young generation is usually much smaller compared to the old generation. Generational collection allows employing different GC algorithms in different generations. This enables selecting a suitable algorithm based on the maturity of the objects.

Generational collection makes use of an interesting characteristic of applications, often referred to as the "generational hypothesis". This hypothesis states:

Most allocated objects are not referenced for long (they die young - sometimes also stated as infant mortality)
Few references from older objects to younger objects exist

Based on this hypothesis, modern garbage collectors run collections on the young generation frequently. This is fast and efficient because the young generation is small and likely to contain many dead objects. Objects that survive some number of young generation collections are moved (tenured) to the old generation. Because old generation is much larger, its occupancy grows very slowly. Therefore old generation collections are infrequent. But when they do occur, they tend to take a much longer time to complete.

In new HotSpot JVMs, the garbage collector divides the heap into 3 generations:

Young generation - contains newly allocated objects
Old generation - objects that has survived some number of young gen collections and some very large objects that were directly allocated in old gen
Permanent generation (perm gen) - contains classes, methods and associated descriptors which are managed by the JVM

The young generation is further divided into 3 regions. The larger division is known as the Eden. This is where almost all the new object allocations take place (under special circumstances, large objects may get allocated in the old generation). The other smaller spaces are known as survivor spaces. One of the survivor spaces are always kept empty until the next young generation collection.

When an old generation fills up a full collection (major collection) is performed. All generations are collected during a full collection. First the young generation is collected using the young generation collection algorithm. Then the old generation collection algorithm is run on the old generation and permanent generation. If compaction occurs, each generation is compacted separately. During a full collection if the old generation is too full to accept tenured objects from the young generation, the old generation collection algorithm is run on the entire heap (except with CMS collector).

Available Garbage Collectors

HotSpot JVM contains 3 garbage collectors:

Serial collector
Parallel collector
Concurrent mark-sweep collector

In addition to that there is a special enhanced version of the parallel collector known as the parallel compacting collector. Let's see how each of these collectors work.

Serial Collector (Mark-Sweep-Compact collector)

This is the collector used by default on Java HotSpot client JVM. It is a serial, stop-the-world, copying collector. Because it is serial and operates in the stop-the-world mode it is not a very efficient collector.

Young generation collection:

Live objects in Eden are copied to the empty survivor space (objects that won't fit are directly copied to the old generation)
Live objects in the occupied survivor space are copied to the empty survivor space (relatively old objects are copied to old generation)
If the free survivor space becomes full during the process, remaining live objects in Eden and occupied survivor space are tenured
At this point Eden and the occupied survivor space contains only dead objects and so can be considered empty - The previously free survivor space contains some live objects
The two survivor spaces switch roles

Old/Permanent generation collection:

A two phase Mark-and-sweep algorithm is used to clean up dead objects
After that a sliding compaction (live objects are slided towards the beginning of the generation) is performed to compact the generations

Parallel Collector (Throughput Collector)

This is very similar to the serial collector in many ways. In fact the only notable difference is that parallel collector uses multiple threads to perform the young generation collection. Other than that it uses the same algorithms as the serial collector. The number of threads used for collection is equal to the number of CPUs available. Because of the parallel collection feature, this collector usually results in much shorter pauses and higher application throughput. However note that the old generation collection is still carried out using a single thread in serial fashion. This is the default collector used in Java HotSpot server JVM.

Parallel Compacting Collector

This is an enhanced version of the parallel collector. It uses multiple threads to perform the old generation collection as well. The old generation collection divides the generations into regions and operate on individual regions in parallel. The algorithm used for old generation collection is also slightly different from what's used in serial and parallel collectors.

Concurrent Mark-Sweep Collector (CMS Collector)

While the parallel collectors give prominence to application throughput, this collector gives prominence to low response time. It uses the same young generation collection algorithm as the parallel collectors. But the old generation collection is performed concurrently with the application instead of going to stop-the-world mode (at least most of the time). A collection cycle starts with a short pause known as the initial mark. This identifies the initial set of live objects directly reachable from the application code. Then during the concurrent marking phase, collector marks all live objects transitively reachable from the initially marked set. Because this happens concurrently with the application not all live objects get marked up. To handle this, the application stops again for a second pause for the remark phase. Remark phase is often run using multiple threads for efficiency. After this marking process a concurrent sweep phase is initiated.

CMS collector is not a compacting collector. Therefore it uses a set of free-lists when it comes to allocation. Therefore the allocation overhead is higher. Also CMS collector is best suited for large heaps. Because collection happens concurrently, the old generation will continue to grow even during collection. So the heap should be large enough to accommodate that growth. Another issue with CMS is floating garbage. That is objects considered as live may become garbage towards the end of the collection cycle. These will not get immediately cleaned up but will definitely get cleaned up during the next collection cycle. CMS collector requires lot of CPU power as well.

Unlike other collectors, CMS collector does not wait till the old generation becomes full to start a collection. Instead it starts collecting early so it can avoid old generation getting filled up to the capacity. If the old generation gets filled up before CMS kicks in, it resorts to the serial stop-the-world collection mode used by serial and parallel collectors. To avoid this CMS uses some statistics regarding previous collection times and the time taken to fill up the old generation. CMS also kicks in if the old generation occupancy exceeds a predefined threshold known as the initiating occupancy fraction. This is a configurable parameter and defaults to 68%.

There is a special mode of operation for the CMS collector known as the incremental mode. In the incremental mode, concurrent collection cycles are broken down into smaller chunks. Therefore during a concurrent collection cycle, the collector will suspend itself to give full CPU to the running application. This in turns reduces the impact of long concurrent collection phases. This mode is particularly useful in cases where the number of CPUs is small.

Selecting the Appropriate Collector

Most of the time JVM is smart enough to select the right collector for the situation by analyzing the system configuration (see the next section). But since the JVM has no knowledge of the application requirements, sometimes the JVM chosen collector will not be good enough. When it comes to manually selecting a collector for an application, there are no hard and fast rules that say a particular collector is suitable for a given scenario. So there are only a set of general guidelines and recommendations. Most of the time they will work, but there could be exceptions. Hope you find them useful:

Serial collector is the collector of choice for most applications that run on client-style machines and do not have low pause requirements. On modern hardware this collector can handle a wide range of applications with heaps as small as 64MB.
If the application has a small data set select the serial collector.
If the application has no low pause requirements and runs on a machine with a single CPU select the serial collector.
Parallel collector is best for applications that run on machines with multiple CPUs and do not have low pause requirements.
If application throughput is the main consideration and slightly longer pauses are acceptable select the parallel collector.
The parallel compacting collector can be used in almost any scenario where the parallel collector seems suitable. In fact the parallel compacting collector is almost always preferred over the parallel collector. However with this collector a single application can hog the CPU for an extended period of time. Therefore it's not very suitable in cases where multiple JVMs reside on a single large machine.
CMS collector should be the collector of choice whenever there is a requirement for low pauses and low response time. However this collector makes use of lot of CPU resources. Therefore impact on CPU usage must be carefully evaluated. Usually applications that have large sets of long-lived data (a large old generation), which run on machines with multiple CPUs, tend to benefit from this collector. Web servers and most interactive applications often fall into the category of CMS benefactors.
If it is needed to run an application with low pause requirements on a machine which doesn't have too many CPU resources, consider using the CMS incremental mode.

Note: Command line options needed to enable different collectors are given at the end of the post

Ergonomics

Modern JVMs has the ability to select a suitable garbage collector, heap size and JVM type by looking at the host platform and the OS. In addition to that, a new way of dynamically tuning memory management has been introduced to the parallel collectors. This way, a user can specify the desired behavior and the collector dynamically tunes the sizes of the heap regions in an attempt to achieve the requested behavior. The combination of platform-dependent default selections and dynamic GC tuning is referred to as ergonomics.

The ergonomics default selections are done based on the 'class' of the machine. A machine is considered to be server-class if it has:

2 or more processors
2 GB or more physical memory

This applies to all platforms except for 32-bit machines running Windows.

If the machine is classified as non server-class, ergonomics will choose the following settings:

HotSpot client JVM
serial collector
initial heap size of 4 MB
maximum heap size of 64 MB

For server-class machines it is slightly complicated. However in general it is something like:

HotSpot server JVM
parallel collector
initial heap size of 1/64 th of physical memory
maximum heap size of 1/4 th of physical memory under an upper limit of 1 GB

One can run the HotSpot client JVM on a server-class machine by explicitly enabling the -client command line option. In that case ergonomics will select the serial collector. Also when the collector selected by ergonomics is not the parallel collector the initial heap size and maximum heap size will be 4MB and 64MB respectively.

Sizing the Heap

There are lots of options available when it comes to sizing the heap, its regions and generations. Most of the time we don't have to meddle with these settings, but there could be exceptional situations where we have to tune them up to obtain optimal performance and avoid out of memory errors. The most common settings related to heap size are the initial heap size and maximum heap size which are set by -Xms and -Xmx command line options respectively. If you set a lower value to the initial heap size than the maximum heap size, JVM will try to start with a heap of initial size and then grow the heap as and when required. If you set equal values to both parameters, JVM will start with the specified maximum heap size.

Another important parameter is the NewRatio value which governs the ratio between the old generation size and young generation size. On most server class systems this defaults to 8. That means the old generation is 8 times larger than the young generation. If the application tends to do lot of new allocations then reducing this ratio to a reasonable value may not be a bad idea. Reducing this ratio will generally result in less minor collections. The size of the young generation can be further controlled by the NewSize and MaxNewSize options.

We can get the JVM to dynamically grow or shrink the generations based on how they are utilized by setting the MinHeapFreeRatio and MaxHeapFreeRatio parameters. These parameters try to impose some restrictions on the amount of free heap space in each generation. If the free space percentage in a given generation is about to drop below the MinHeapFreeRatio, JVM will expand the generation to meet the lower limit. Similarly generations will be contracted if the free space percentage crosses the MaxHeapRatio.

Command Line Options

Collector selection

Serial collector: -XX:+UseSerialGC
Parallel collector: -XX:+UseParallelGC
Parallel compacting collector: -XX:+UseParallelOldGC (combine with -XX:+UseParallelGC)
CMS collector: -XX:+UseConcMarkSweepGC

Parallel collector settings

Parallel GC thread count: -XX:ParallelGCThreads=n
Desired maximum pause length: -XX:MaxGCPauseMilis=n
Throughput (percentage of CPU time spent on application - defaults to 99): -XX:GCTimeRatio=n

CMS collector settings

Enable incremental mode: -XX:+CMSIncrementalMode
Parallel GC thread count: -XX:+ParallelGCThreads=n
Old gen occupancy threshold that triggers collections: -XX:CMSInitiatingOccupancyFraction=n

Heap sizing options

Initial size: -Xms
Maximum size: -Xmx
Initial size of the new generation: -XX:NewSize=n
Maximum size of the perm gen space: -XX:MaxPermSize=n
Ratio between old and new generation sizes: -XX:NewRatio=n

Debug options

Print basic GC info: -XX:+PrintGC
Print verbose GC info: -XX:+PrintGCDetails
Print details with time: -XX:+PrintGCTimeStamps

References

Rediscover SOA with WSO2 Carbon & WS-Discovery

2010-11-03T16:15:00.000-07:00

One of the hottest features of new WSO2 Carbon 3.0 based releases is the out-of-the-box WS-Discovery support. WS-Discovery is a standard protocol for discovering services and service endpoints. This enables service clients to search for services based on a given criteria and bind with the discovered services. The WS-Discovery specification defines two modes of operation:

1. Adhoc mode

In the adhoc mode, servers advertise the services they have using a UDP multicast protocol. Similarly client applications can search for available services by sending out probe requests over UDP multicast. Servers listening for such probe requests can then send the service information to the client over a unicast channel.

2. Managed mode

In the managed mode, servers and clients use an intermediary known as the discovery proxy for all service discovery purposes. Servers will register the available services with the discovery proxy by sending notifications over HTTP. Then clients can directly probe the discovery proxy to discover the registered services. This mode of operation does not use UDP multicast for any sort of communication. All interactions take place over regular HTTP channels and if needed, HTTPS can be used to provide transport level security.

Starting from version 3.0, WSO2 Carbon framework has full support for WS-Discovery managed mode. The following products ship the necessary WS-Discovery libraries with them by default:

WSO2 Web Services Application Server (WSAS)
WSO2 Enterprise Service Bus (ESB)
WSO2 Data Services Server (DSS)
WSO2 Governance Registry (G-Reg)

WSO2 G-Reg has the ability to act as a discovery proxy. This mode is enabled by default and user doesn't have to configure anything in the G-Reg side. In products like WSAS and DSS, WS-Discovery support should be manually enabled by pointing them to an already existing discovery proxy. Once configured, these server applications will automatically register the services they have with the discovery proxy. Service information will be synchronized with the discovery proxy at server start up, shutdown and service level updates. This ensures that the information in the discovery proxy are always up-to-date. WSO2 ESB can query one or more discovery proxy instances to find the necessary services and service endpoints. The ESB UI also enables creating proxy services and mediation endpoints using the discovered endpoints.

There are lot of other cool things you can do in Carbon platform with WS-Discovery. There are several plug-ins and extensions available to download and try out as well. I'm planning to post a series of blogs addressing various aspects of WS-Discovery in the next few months. So stay tuned! In the meantime read this article to learn how Jos Dirksen integrated Mule with WSO2 G-Reg using the WS-Discovery capabilities of Carbon framework.

WSO2 Excels at NBQSA

2010-10-03T19:18:00.000-07:00

WSO2 bagged four awards at the 12th National Best Quality Software Awards (NBQSA) held last week. NBQSA is an annual competition organized by the Sri Lankan chapter of the BCS. Every year many corporate firms, government organizations, tertiary education institutes and individuals take part in this event. The competition is pretty intense and the winners are also given the opportunity to take part in international events such as APICTA.

This year WSO2 really made their presence felt at NBQSA by winning the following four awards:

Bronze award for tools and infrastructure apps category - WSO2 Data Services Server
Silver award for research and development category - WSO2 Gadget Server
Gold award for tools and infrastructure apps category - WSO2 Enterprise Service Bus
Glod award for overall victory - WSO2 Enterprise Service Bus

I feel truly proud to be a part of such a dynamic team as the WSO2. Three cheers to WSO2 and all the folks who worked with WSO2 (in past and present) and made these achievements possible.

WSO2 ESB 3.0.1 Up for Grabs

2010-09-12T02:04:00.000-07:00

Last week we released WSO2 ESB 3.0.1. This release contains a number of important bug fixes and we highly recommend everyone using WSO2 ESB 3.0.0 to upgrade to the latest release as soon as possible. Please note that this is only a bug fix release and as such it doesn't contain any new features over 3.0.0. Therefore any configuration can be easily migrated to the 3.0.1 from 3.0.0.

Some of the significant fixes available in this release are:

Fixes to the hot deployment/hot update implementation
Fixes to the built-in patch model
Transport fixes (particularly in the NHTTP and FIX transports)
An Axis2 security fix

To make a long story short, there are more than one reason to migrate to WSO2 ESB 3.0.1. So download your copy today.

From Quadricycle to the Prius

2010-09-04T22:52:00.000-07:00

I went to see the California Automobile Museum a few days back. It's located in Sacramento and displays a wide range of classic cars and a few modern cars. If you are an automobile fanatic, this is the place to be. I don't consider myself to be a total car fanatic, but even I loved the place. It's a great place to learn and experience the history of automobile industry and admire the long journey the human race has come in search of faster and safer ways of transportation.

Here are some highlights from my visit:

Experiencing the Nature at Lake Tahoe

2010-09-02T09:01:00.001-07:00

A couple of weeks back I was in Auburn, California. During my stay, I got the opportunity to visit the lake Tahoe. It's only a 2-3 hours drive to the magnificent lake from Auburn. Here are some of the breath taking sceneries around the lake.....

Truly awesome experience!

FIX Your SOA with WSO2 ESB

2010-07-02T01:06:00.000-07:00

I believe WSO2 ESB is the ONLY complete enterprise service bus which supports the FIX (Financial Information eXchange) protocol. The FIX transport of WSO2 ESB enables the service bus to communicate with FIX acceptors and initiators. FIX messages received by the service bus can be routed, filtered or transformed using the existing set of mediators. If needed, FIX messages can be transformed and sent to external systems over different protocols such as HTTP and JMS.

I published an article titled "FIX Your SOA with WSO2 ESB" which explains how to setup the FIX transport in WSO2 ESB 3.0 and how to use it to implement a variety of integration scenarios. It gives a good overview on how to develop high performance finance data routing systems using WSO2 ESB. Hope you'll like it!

Axis2 TCP Transport Revamped

2010-06-27T06:24:00.000-07:00

I recently did some major improvements to the Axis2 TCP transport. The TCP transport enables Axis2 services and clients to send/receive SOAP messages over TCP, using Java TCP sockets. The old TCP transport was very simple and it must be configured globally in the axis2.xml file. Due to this limitation, a service could not open up its own port to listen to incoming TCP messages. All the TCP requests were captured by a single, globally configured port and WS-Addressing headers were used to dispatch the requests to the appropriate services.

The new transport implementation is fairly advanced with a wide range of configuration options. It can be configured globally in the axis2.xml file, or it can be configured at the service level in the corresponding services.xml files. Only the port number was configurable in the previous TCP transport implementation. The new implementation supports all the parameters described below:

transport.tcp.port - Port number (mandatory parameter)
transport.tcp.hostname - The host name to which the server socket should be bound
transport.tcp.backlog - The length of the message back log for the server socket (defaults to 50)
transport.tcp.contentType - Content type of requests (defaults to text/xml)

In the new transport, if a request is received by a port configured at the service level, it is pre-dispatched to the corresponding service. If the global port receives a TCP message, WS-Addressing headers will be looked up while dispatching.

These improvements are now available in the Axis2 trunk. So feel free to take it for a ride and give us your feedback. I also have some plans to make some improvements to the Axis2 UDP transport. I intend to add multicast support to the existing UDP transport and with that we will be able to support multicast request – unicast response message exchange pattern in Axis2.

WSO2 ESB Mediation Statistics - Let Numbers Do the Talking

2010-06-19T21:37:00.000-07:00

WSO2 ESB comes with a mediation statistics component, which allows users to collect and monitor statistics on proxy services, sequences and endpoints. Data gathered by this component can be viewed through the ESB management console or one can even develop some custom code to pump the data into an external monitoring tool.

I just published an article titled "WSO2 ESB Mediation Statistics: What Can Numbers Tell About Your SOA?" which explains the ins and outs of the mediation statistics component along with how to process the collected statistical information by writing custom code. Enjoy!

New Configuration Model of Synapse

2010-06-12T02:31:00.000-07:00

If you are trying out a latest Apache Synapse build off the SVN trunk, you will notice that the Synapse configuration model has gone through some significant changes lately. In the past, Synapse was programmed to load the entire mediation configuration (sequences, endpoints, proxy services, tasks etc) from a single XML file, named synapse.xml. Synapse would parse this XML file at startup and construct an object model known as the SynapseConfiguration which contains the definitions of all active mediation components at runtime. While this approach was simple and clean, it had a number of limitations:

Configuration management becomes a nightmare as the size of the synapse.xml grows. A typical heavy duty configuration would consist of dozens of proxy services, sequences and endpoints. When all these are packed into a single XML file, it becomes difficult to locate a single configuration item quickly.
With all configuration artifacts stored in a single flat file, it is almost impossible to develop satisfactory tooling support for configuration development and maintenance.
A team of developers cannot work on configuring a single Synapse instance at the same time.
Hot deployment is a feature that Synapse has been lacking for years. Being able to hot deploy a proxy service or a sequence into Synapse without having to restart the service bus is a great convenience at development time. But a configuration model based on a single XML file is not at all capable of handling such requirements.

Considering these drawbacks, I implemented a new configuration model for Synapse, known as the multi XML configuration builder. The new model, which is now the default configuration model in Synapse, loads the configuration from a structured file hierarchy, instead of loading the mediation configuration from a single XML file. With this model in place, each endpoint, sequence and proxy service has to be defined in separate XML files. The directory structure for storing these individual configuration files is as follows:

synapse-config
   |
   +-- registry.xml
   +-- endpoints
   +-- event-sources
   +-- local-entries
   +-- proxy-services
   +-- sequences
   `-- tasks

As you can see there are separate dedicated directories for each type of artifacts. Each of these directories can house zero or more XML configuration files. Each file must have the .xml extension to be recognized by the Synapse configuration builder. If you take a look at the source code you will notice that I have enforced this restriction using a Java FileFilter:

    private static FileFilter filter = new FileFilter() {
       public boolean accept(File pathname) {
           return (pathname.isFile() && pathname.getName().endsWith(".xml"));
       }
   };

   private static void createProxyServices(SynapseConfiguration synapseConfig, String rootDirPath)
           throws XMLStreamException {

       File proxyServicesDir = new File(rootDirPath, PROXY_SERVICES_DIR);
       if (proxyServicesDir.exists()) {
           if (log.isDebugEnabled()) {
               log.debug("Loading proxy services from : " + proxyServicesDir.getPath());
           }
           File[] proxyDefinitions = proxyServicesDir.listFiles(filter);
           for (File file : proxyDefinitions) {
               try {
                   OMElement document = parseFile(file);
                   ProxyService proxy = SynapseXMLConfigurationFactory.defineProxy(
                           synapseConfig, document);
                   proxy.setFileName(file.getName());
                   SynapseArtifactDeploymentStore.getInstance().addArtifact(
                           file.getAbsolutePath(), proxy.getName());
               } catch (FileNotFoundException ignored) {}
           }
       }
   }

The mediation registry is defined in the registry.xml file which should be placed at the top level of the file hierarchy.

So does the multi XML configuration builder solve the problems in the old configuration model? Let’s consider the facts:

With Synapse configuration broken down into smaller, manageable pieces the whole configuration becomes easier to manage and keep track of. As long as the XML files are named appropriately, it is extremely easy to quickly locate a particular configuration item. We recommend using the artifact names to name the corresponding XML files. For an example the file containing the definition of the FooProxy can be named FooProxy.xml.
With the multi XML configuration builder, developing powerful and elegant tools for creating pieces of the service bus configuration becomes a trivial task. Also one can use conventional configuration management tools and version controlling systems such as Subversion to store and manage the configuration artifacts.
A team of developers can now work on configuring Synapse. Each developer in the team can work on his own configuration file or set of files.
Supporting hot deployment is now feasible. As a matter of fact, Ruwan implemented hot deployment and hot update support for Synapse based on the multi XML configuration builder a few weeks back. This feature is now available in the Synapse trunk and will be available for the next Synapse release.

So there you go. Target achieved. There is one little glitch in this approach though. That is how do we handle backward compatibility with older Synapse versions? For instance how can a Synapse 1.2 user, who has a single synapse.xml file, migrate to a new Synapse version? We have provided a solution for that as well. In the new Synapse configuration file hierarchy you can place a synapse.xml file at the top level (alongside with registry.xml). All the mediation components defined in this synapse.xml will be loaded to the service bus at startup along with any other components defined inside the individual directories. So a Synapse 1.2 user can simply copy the existing synapse.xml file to the synapse-config directory in a new Synapse distribution, and it will be picked up by the service bus. In addition to this convenience feature, we are planning on developing some migration tools that can help users to easily migrate an old Synapse configuration file onto a newer version of Synapse.

As far as Synapse is concerned each configuration builder should be associated with a corresponding configuration serializer implementation. Serializers are used to convert the SynapseConfiguration object model back to the textual/XML form. So for the multi XML configuration builder I developed a matching multi XML configuration serializer which can save a SynapseConfiguration object model to a file hierarchy. Similar to the configuration builder this implementation was also heavily dependent on Java file IO APIs. However, after a while we realized that the serializer is not working as expected on certain platforms; most notably on Windows. After running some debug sessions and doing some on-line reading I realized that the Java file IO operations are not consistent on every platform. As a result sometimes the serializer would encounter trouble creating a new file or moving an existing file on Windows.

At this point, my colleague, Rajika suggested using Apache Commons IO API for file manipulation. Commons IO API provides a nice layer of abstraction on top of the standard Java IO APIs. It handles all file IO operations in a consistent and platform independent manner. So I got rid of almost all the Java file IO code in the multi XML configuration serializer and replaced them with corresponding calls to the Commons IO API. Some sample code fragments are shown below:

    private void cleanUpDirectory()  throws Exception {
       // If the target directory already exists and contains any files simply rename it to
       // create a backup - This method does not delete the target directory
       if (rootDirectory.exists() && rootDirectory.isDirectory() &&
               rootDirectory.listFiles().length > 0) {

           if (log.isDebugEnabled()) {
               log.debug("The directory :" + rootDirectory.getPath() + " already exists. " +
                       "Creating a backup.");
           }

           backupDirectory = new File(rootDirectory.getParentFile(), "__tmp" +
                   new GregorianCalendar().getTimeInMillis());
           FileUtils.moveDirectory(rootDirectory, backupDirectory);
       }

       // Create a new target directory
       FileUtils.forceMkdir(rootDirectory);
   }

   private void writeToFile(OMElement content, File file) throws Exception {
       File tempFile = File.createTempFile("syn_mx_", ".xml");
       OutputStream out = new FileOutputStream(tempFile);
       XMLPrettyPrinter.prettify(content, out);
       out.flush();
       out.close();

       FileUtils.copyFile(tempFile, file);
       FileUtils.deleteQuietly(tempFile);
   }

FileUtils is the entry point for file IO operations in the Commons IO API. It provides you with a range of useful methods for managing and manipulating files.

Note that when writing to a file we first write the whole thing to a temporary file and once completed we copy the temp file to the final location. This was required to handle certain edge cases in the hot update implementation. Without that the hot updater will attempt to pickup and process half baked files (Again mostly on Windows – It seems Windows writes to files in chunks).

All in all, Apache Synapse is now equipped with a powerful new configuration model, a matching configuration serializer and hot deployment capabilities that leverage the new configuration model. Speaking of hot deployment, it is also based on the Axis2 hot deployment framework. Therefore it is configured in the axis2.xml file. We are yet to do a Synapse release with all these new exciting features but if you are itching to try them out feel free to grab a nightly build. Also be aware that WSO2 ESB 3.0, which is based on Synapse, has been released and that release contains all these new improvements.

Attention Open Source Developers...

2010-05-28T23:28:00.000-07:00

When you keep contributing to an open source project, the project community will eventually reward you by granting you committer rights. However most open source committers don’t realize that committership is a big responsibility. The moment you become a committer of a project, you become responsible for each and every line of code in the project code base. Every single change you make to the project source code affects other committers in the project and all the people around the world who use that software. Therefore as a committer you should stick to some kind of a policy or a code of conduct which governs how you make changes to the project source code. The following is a list of items that I think should be in such a policy. I’m not implying that I have always followed this policy in the past. Rather, I always try my best to stick by it:

1. Before anything else, if your contribution involves API changes, you MUST take the matter to the project mailing list. You MUST NOT make any API changes without notifying your fellow committers and the user base. Sometimes you might feel that the API change you are about to do is rather small, unimportant and trivial. For an example you might spot a spelling mistake in a method signature and feel the urge to correct it immediately. But still, an API change is an API change. If somebody’s using the misspelled method, your change will break his/her code. Also some developers tend to think that certain interfaces in the code base are “not so important” and nobody is using them, and therefore it is ok to make changes to them. Developers like that don’t really understand open source. The truth is there are no unimportant APIs in open source projects. When you make project source code open, you are basically inviting the rest of the world to use that code, develop on top of the APIs to do whatever they want with it. So even though you don’t know it, there could be hundreds of people out there who use the “unimportant” interface you are about to change.

2. Always crate an entry in the project issue tracker to keep track of your contributions. Almost every open source community maintains an issue tracker. At Apache and at WSO2 we mainly use Atlassian JIRA. Whenever you are about to make a change to the source code, consider creating a new entry in the issue tracker before you check-in the patch. In my opinion there are only two excuses for not creating an entry in the issue tracker:

Adding some descriptive comments to the code (note that comments and Javadocs are two different things)
Refactoring some private attributes or private methods of a class (eg: renaming a local variable)

Once you have created the entry in the issue tracker, make sure you get it assigned to yourself. Once the relevant changes are checked-in to the code base, mark the entry as resolved. It doesn’t look very professional when you have dozens of open tickets which are not assigned to anyone. It is also not nice to have tickets in the work-in-progress mode, even after it has been fixed on the source code.

3. While making changes to the source code, always follow the coding conventions accepted by your project community. If you come across any badly written or poorly formatted code fix them then and there. Refactor as you go. However do not make any functional changes, which are not relevant to the problem you are attempting to solve. If you come across old functional issues while fixing some other issue, report them on the issue tracker and leave them to be fixed in a future iteration. Generally it’s not a very good idea to solve multiple problems by one commit. That can make things harder to keep track of.

4. When you are actually committing a patch to the code base, make it a point to properly log the changes you are about to check-in. I often commit code into the Apache Subversion repository and WSO2 Subversion repository. Whenever I make a contribution, I always add a couple of lines to the SVN commit log, describing my changes. If it is like a rather large patch, I would mention a summary of the changes in bulleted point form. Also in your commit log message, you MUST mention the ID of the corresponding issue tracker entry. This can really help you track down changes in the future. At Apache and at WSO2 we have integrated JIRA with the Subversion repository. Whenever someone checks a bit of code into the SVN, with a JIRA ID in the commit log message, the relevant JIRA entry gets updated with the information related to the commit (see example). This makes it really easy to identify all the changes done in the source code to resolve a particular issue.

So there you have it! In my opinion, every open source developer should adhere to these key guidelines, whenever making a change to the project source code.

Having said that, an open source committer should never feel uneasy, to make changes to the project source (unless it is specifically restricted by the community - as in the case of a release). If you have done some changes to the code, and if you are confident that the changes you have made are accurate and necessary, you should feel free to check them into the code base. Same applies for patches contributed by the non-committers as well. If a patch solves the problem at hand to a satisfactory level without violating the boundaries of the current design, committers should take necessary steps to check it in. The solution doesn’t have to be “perfect”. At Apache we follow a commit-then-review (CTR) policy, most of the time. This model enables developers to get code changes into the project as early as possible, without worrying too much about the impact they may have on the project. Once the changes have been checked-in, they can be reviewed and fine-tuned as necessary.

However, during my career I have come across developers who at times leave issues open for weeks just because the available patches are not ‘perfect’. While the patches provide a satisfactory solution to the problem at hand, these developers always manage to find some small, irrelevant limitation in the patch, which can be easily fixed in a future iteration. They make it an excuse to not commit the patch. These developers, unfortunately, haven’t grasped the true spirit of open source. They don’t realize that the whole point of open source is to attract as much contributions as possible and correct any problems they might have, as we go on.

Before I wrap up, I’d like to reemphasize that open source committership gives you great power and authority over your project. We all know what comes with great power. Yes, it’s “great responsibility”. So use your powers with care and caution!

A Different Experience...

2010-05-20T08:14:00.000-07:00

Last week, I got the opportunity to visit yet another interesting part of the world – the Middle East. I have been to the Dubai airport before, while on my way to United States. But transits don’t count, right? So last week I visited the city of Amman, the capital of the Hashemite kingdom of Jordan. I was there for a whole week. It was not the trip I expected, but in a really good way.

Let’s start with the weather. I was expecting the weather to be burning hot with tons of scorching sunlight. It turns out the weather over there is actually better and far more comfortable compared to the usual weather in Colombo. The temperature is almost as same as in Colombo (around 30 degrees Celsius) but the humidity is nearly 0. So it’s actually very comfy. And during the nights the temperature would drop to something around 18 degrees Celsius resulting in some very cold nights and mornings. I also learnt that they have the usual four seasons over there at Jordan. They even get some snow showers during the winter. Who knew?

The city of Amman is indeed one of the largest, most developed and beautiful cities I’ve ever been to. It’s very clean and tidy for a capital of a country. I was kind of surprised to not see much pedestrians walking around the town. But the roads are always full of vehicles all the time (even at midnight). So I guess pretty much everyone owns a vehicle. I also got to see quite a number of Ferraris and Porches storming the highways. I also realized that travel costs are actually very low in Amman. You can get a taxi anywhere and travel about 4 km for 0.75 Dinars (~ 1 USD).

Amman is also full of great hotels, restaurants, cafes and various shops. Truly great place for tourism I should say. Somewhere close to down town they have a street called the ‘Rainbow Street’. The street full of night clubs, restaurants, coffee shops and book shops is a great place to go and get something to eat and chill down for a while. They also have these huge shopping malls at Amman– larger than any mall I’ve ever been to. If you ever get to the city of Amman please make some time to go and visit the Mecca mall or the City mall.

Something has to be mentioned about the people of Jordan too. They are really nice and friendly. People at Amman are always full of smiles and have nothing but nice things to say. During my short stay I was even offered a couple of lunches and one great dinner.

Unfortunately for me I didn’t actually get anytime to go and visit the real tourist hotspots in Jordan, like the city of Petra or the Dead Sea. But then again, it’s always nice to have something to look forward to.

WSO2 ESB Tips & Tricks 05: Error Handling with Sequences and Proxy Services

2010-04-24T04:52:00.000-07:00

The job of an Enterprise Service Bus is to act as the backbone of an organization’s SOA. It is the spine through which all the systems and applications within the enterprise communicate with each other. At times an ESB might even help to integrate internal systems with external applications so that complex cross cutting business tasks such as supply chain management and customer relationship management can be carried out with precision and in perfect synchronism. As such an ESB often has to deal with many wire level protocols, messaging standards and remote APIs. But applications and networks are full of errors. Applications often crash due to various bugs. Network routers and links often get into states where they cannot pass messages through with the expected efficiency. These error conditions are very likely to cause a fault or trigger a runtime exception in the ESB. A good ESB should provide simple yet powerful mechanisms for dealing with such errors. So today, I’m going to discuss a little bit about error handling options provided in WSO2 ESB.

WSO2 ESB gives you the concept of fault sequences. A fault sequence is just like any other sequence, a collection of mediators. A fault sequence can be associated with another sequence or a proxy service. When the sequence or the proxy service encounters an error during mediation or while forwarding a message out, the message which triggered the error will be delegated to the specified fault sequence. Using the available mediators we can log the erroneous message, forward it to a special error tracking service, send a SOAP fault back to the client indicating the error or even send a mail to the system admin. It is not mandatory to associate each sequence and proxy service with a fault sequence. In situations where a fault sequence is not specified explicitly, a default fault sequence will be used to handle errors. Sample 4 in WSO2 ESB documentation shows how to specify a fault sequence with a regular mediation sequence.

Whenever an error occurs in WSO2 ESB, the mediation engine attempts to provide as much information as possible on the error to the user. This is done by initializing a set of property values on the erroneous message. These properties are:

ERROR_CODE
ERROR_MESSAGE
ERROR_DETAIL
ERROR_EXCEPTION

Within the fault sequence we can access these property values using the get-property XPath function. Sample 4 uses the log mediator as follows to log the actual error message:

<log level="custom">
  <property name="text" value="An unexpected error occured"/>
  <property name="message" expression="get-property('ERROR_MESSAGE')"/>       
</log>

Note how the ERROR_MESSAGE property is being used to get the error message text. You can also write special custom mediators for error handling tasks. In such cases you can use the MessageContext API to retrieve above mentioned property values from the message.

String errorMsg = (String) messageContext.getProperty(“ERROR_MESSAGE”);
Exception e = (Exception) messageContext.getProperty(“ERROR_EXCEPTION”);

WSO2 ESB also makes it possible to send a SOAP fault back to the client whenever an exception occurs in the ESB. This is done using the makeFault mediator. The makeFault mediator converts the current message into a SOAP fault. The user can specify the error code and the reason statement to be included in the fault message. Once the SOAP fault is constructed it can be sent to the client using the send mediator. Sample 5 in WSO2 ESB documentation shows exactly how to do this. Here’s the makeFault mediator configuration from sample 5:

<makefault>
  <code value="tns:Receiver" xmlns:tns="http://www.w3.org/2003/05/soap-envelope"/>           
  <reason expression="get-property('ERROR_MESSAGE')"/>
</makefault>

Note how the ERROR_MESSAGE property is used to set the original error message as the reason phrase of the SOAP fault. Keep in mind that the makeFault mediator only converts the message into a fault message. To send it to a client or an endpoint you need to use the send mediator. In sample 5 we mark the message as a response using the property mediator and invoke the send mediator so that it is sent back to the client which sent the original request.

Assume you want to send an E-Mail to the system admin whenever the ESB encounters an error. How do you do it? You simply need to enable the mail transport sender in the ESB configuration (by editing the axis2.xml file or using the UI). Configure the transport sender with the appropriate mail server settings. Then in your fault sequence you can simply use the send mediator as follows to send an E-Mail out.

<property name="Subject" value="An Error Occurred in the ESB" scope="transport"/>
<property name="OUT_ONLY" value="true"/>
<send>   
  <endpoint>       
     <address uri=”mailto:admin@yourdomain.com”/>
  </endpoint>
</send>

So these are the error handling options available for sequences and proxy services in WSO2 ESB. It also provides some more error handling options for endpoints. That however is a different matter entirely and hence I’m leaving it for a future blog post.

Apache Attacks: A Real Eye Opener

2010-04-14T05:38:00.000-07:00

It seems year 2010 is gradually turning into the year of cyber crimes. The year started off with news reports on a very sophisticated and targeted attack on Google corporate infrastructure. According to Google, the attackers were successful in stealing valuable intellectual property. Now after about four months from the Google incident, a massive attack has been carried out on the Apache Software Foundation. Attackers have exploited a previously unknown vulnerability in the Atlassian JIRA instance used by the ASF, to gain root access to the server hosting the JIRA instance. Attackers also messed up the JIRA instance to install some sniffers that can capture and log user passwords. According to the Apache infra team a whole bunch of user passwords have been compromised as a result of the attack. This attack was later followed by another attack, this time directly on the Atlassian IT infrastructure, which has also exploited the same security hole in JIRA.

The Apache infra team did a pretty amazing job to mitigate the threat and take control of the situation. They have also taken some additional security measures to prevent such disasters in the future while Atlassian has rolled out patches for the aforementioned security hole.

It is hard to imagine what makes somebody attack the ASF IT infrastructure. The whole world knows that we are just doing non-profit, voluntary work at ASF. Nobody gains a monetary benefit by attacking the ASF. May be it is just to compromise the passwords and get to the users/committers. Most people use the same password to login to multiple systems (Gmail, Facebook, Yahoo, Hotmail etc). So if the attacker can get the password for one system, he can gain access to all the other systems.

All in all, this incident is a real eye opener to all of us surfing the Internet. No system is 100% safe and no system is invulnerable. An attacker with sufficient patience and skill will always find a way in. It is up to the users to be careful and minimize the chance of something horrible happening.

Moral of the Story:

We should use multiple passwords to login to different systems as much as possible. We should use strong passwords at all times. And we should definitely change the passwords in a regular basis to mitigate the effects of a possible brute force attack. (We hear these stuff everyday but how many of us actually do it? That's the problem. We should actually put these guidelines into action.)

PS: If you are an Apache committer and did not change the JIRA password yet, please do it NOW!!!

WSO2 ESB Tips & Tricks 04: Troubleshooting JMS

2010-04-12T04:31:00.000-07:00

WSO2 ESB works really well with JMS. The JMS transport adapter used by WSO2 ESB comes from the Apache Axis2 project and it makes use of JNDI to connect to various JMS brokers. Therefore WSO2 ESB can work with pretty much any JMS broker that offers JNDI support. The product ships with sample configurations for Apache ActiveMQ integration, but it can be easily modified for integration with a JMS broker like IBM MQ or SwiftMQ.

Enabling JMS in WSO2 ESB is a simple 2-step process.

Deploy the JMS client libraries to the repository/components/lib directory (these libraries will be used by the ESB to connect to the JMS broker)
Enable JMS transport receiver and the sender by modifying the axis2.xml file or using the management console

Having followed the above mentioned process you can start creating JMS endpoints to forward messages and JMS proxy services to receive messages. However while implementing certain JMS related scenarios you might run into some error conditions. The rest of this blog post focuses on a couple of such commonly encountered problems and how to get rid of them, once and for all.

CNF? NCDF?

If you ever run into any ClassNotFoundExceptions or NoClassDefFoundExceptions while using JMS support in WSO2 ESB, chances are you haven’t deployed all the required client libraries. Please inspect what you have copied into repository/components/lib directory and make sure the required class is available in one of the deployed jars.

Also note that WSO2 ESB ships with geronimo-jms library which contains the javax.jms packages. Therefore you do not have to deploy them again.

HTTP Header Conversion Problem

When forwarding HTTP traffic to a JMS queue using WSO2 ESB, you might at times run into an error similar to the one given below.

ERROR JMSSender Error creating a JMS message from the axis message context
javax.jms.MessageFormatException: MQJMS1058: Invalid message property name: Content-Type

This exception is thrown by the JMS client libraries used to connect with your JMS broker and the problem is specific to the JMS broker being used. I have encountered this issue mostly with IBM Websphere MQ, but I guess there might be other broker implementations that may display the same issue. So here’s what happens….

The incoming HTTP message contains a bunch of HTTP headers that contain the ‘-‘ character. Some notable examples are ‘Content-length’ header and the ‘Transfer-encoding’ header. When ESB attempts to forward the message over JMS it sets the headers of the incoming message to the outgoing JMS message as JMS properties. According to the JMS spec the ‘-‘ character is prohibited in JMS property names. Some JMS brokers like ActiveMQ do not strictly check for this and hence they won’t complain. But certain other brokers will throw exceptions similar to the one shown above.

Even though the problem sounds very complex and tricky the solution is almost trivial. We just need to get rid of the troublemaking HTTP headers from the message before it is delivered over JMS. You can make use of the property mediator as follows to achieve this:

<property action="remove" name="Content-Length" scope="transport">
<property action="remove" name="Accept-Encoding" scope="transport">
<property action="remove" name="User-Agent" scope="transport">
<property action="remove" name="Content-Type" scope="transport">

JMS Property Data Type Mismatch

Sometimes when the ESB attempts to forwards a message over JMS, the client libraries may complain, saying that the data type of a particular message property is invalid. You may often run into this problem if you are using the property mediator to manipulate property values set on the message. This is because certain implementations of JMS, have restrictions on data types of properties. The property mediator always sets property values as strings.

To overcome this problem you will have to revise your mediation sequences and avoid manipulating property values which should contain non-string values. If you must set a non-string property value you will have to write a simple custom mediator to get the job done. Starting from WSO2 ESB 3.0 (scheduled to go out this month), the property mediator is type aware. Therefore to get rid of this problem you can set properties with specific data types. For an example if you want to set a property named foo with the integer value 12345 use the property mediator as follows:

<property name="foo" value="12345" type="INTEGER" scope="transport/">

If you leave the type attribute out the property value will be set as a string.

Too Many Threads? Out of Memory?

With certain JMS brokers WSO2 ESB tends to spawn new worker threads indefinitely until it runs out of memory and crashes. I have observed this behavior with SwiftMQ. This problem is caused by a bug in the underlying Axis2 engine and there is a simple workaround to the problem. In your mediation sequence engage the property mediator as follows:

<property action="remove" name="transportNonBlocking" scope="axis2">

This will prevent the ESB from creating new worker threads unlimitedly and everything will be back to normal. Note that you can always use a JMX client like jconsole to monitor the active threads and memory consumption of the ESB.

"I See You"

2010-01-08T21:22:00.000-08:00

I saw it and it’s a beauty!

Went to see Avatar last week with some friends, and I got only two words about the movie – “Frikkin Awesome”. It’s undoubtedly one of the best movies I’ve ever watched and definitely the best movie I’ve watched in last couple of years. Everything about Avatar is so fascinating. The plot, sceneries, special effects, music and just about everything else is near perfection. I also liked the message that movie attempts to give. Avatar is about people who are at the two ends of the spectrum. On one end we have people who are willing to engage in dirty act, commit any crime, kill anyone and destroy anything to earn a quick buck. On the other end of the spectrum there are people who are willing to sacrifice their lives to defend what is considered correct and uphold the moral values. The movie takes you through the conflict between these two types of people.

If you still didn’t watch it you are not even a person. So make haste. It’s science fiction and fantasy at their very best. Also please do yourselves a favor by watching it on the big screen in a real theatre, rather than watching it on your home theatre system.

Mr. James Cameron sir, you are a genius. Thank you loads to the actors and the behind screen team of Avatar for the great entertainment. And congratulations on breaking the record previously held by Return of the King for the second highest-grossing movie ever. With the way things have been going, may be even the first place (which is currently held by Titanic) is a possibility right now.

WSO2 ESB Tips & Tricks 03: Transport Switching

2009-12-13T03:04:00.000-08:00

Back to WSO2 ESB Tips and Tricks!

Today we are going to see how to implement a transport switching scenario with WSO2 ESB. WSO2 ESB supports receiving and sending messages over following transport and application layer protocols.

HTTP/HTTPS
Mail protocols (POP3, IMAP, SMTP)
JMS / AMQP
VFS (Virtual File System)
FIX (Financial Information eXchange)
TCP
UDP

With WSO2 ESB having support for such a wide range of transports, it is easy to implement a scenario which involves receiving messages over one protocol and forwarding them over another. As an example I’m going to explain how to receive messages over HTTP and forward them over JMS, using Apache Active MQ as the JMS provider. Looking at how most system integrators use WSO2 ESB it seems this is one of the most common use cases for ESB products.

The first thing we have to do is enable the JMS transport sender in the ESB. By default only the HTTP and HTTPS transports are enabled in the ESB configuration. But configuring and enabling other transports is a trivial task. Transports can be enabled by uncommenting the corresponding entries in the axis2.xml file or by using the ESB management console.

First, shutdown the ESB if it is already running. Then we need to deploy the Active MQ JMS client libraries into the ESB runtime. Download and extract an Apache Active MQ binary distribution (I prefer v5.2). Copy the following jar files from the Active MQ lib directory to the ESB_HOME/repository/components/lib directory.

geronimo-j2ee_management.jar
activemq-core.jar

Also this would be a good time to start the Active MQ broker.

Now start the ESB. The underlying Carbon platform will detect the newly deployed jar files and automagically convert them into OSGi bundles on the fly. Once the server has started go to the ESB_HOME/repository/components/dropins directory and do a file listing. You will see a set of OSGi bundles created out of the jar files you deployed.

Now that the server is up and running login to the management console as an admin. Click on ‘Transports’ under ‘Manage’ on the left navigation bar. A list of available transport receivers and senders will be displayed.

Select the ‘Enable’ option for the JMS sender. Click on ‘Enable’ to start the JMS transport sender. Once started, you can further configure the transport by specifying various parameters. But for this demonstration we can go ahead with the default settings (ie no parameters).

If you prefer to enable the JMS sender the old school way open up the axis2.xml file in the conf directory and uncomment the following entry before starting the ESB.

<transportSender name="jms" class="org.apache.axis2.transport.jms.JMSSender"/>

Having enabled the JMS sender, we should now create a proxy service which will accept messages over HTTP and send them to a JMS endpoint. Click on ‘Add’ under ‘Proxy Services’ to start the proxy service creation wizard. On the first step simply specify a suitable name for the service (eg: JMSBridge) and click on ‘Next’.

On the second step of the wizard, create an anonymous in-sequence consisting of a property mediator. Configure the property mediator to set the ‘OUT_ONLY’ property to the value ‘true’. Save the in-sequence and create an anonymous endpoint for the proxy service. Specify the following JMS EPR as the target address. This EPR points to a dynamic queue named ‘SimpleStockQuoteService’ in Apache Active MQ broker.

jms:/SimpleStockQuoteService?transport.jms.ConnectionFactoryJNDIName=QueueConnectionFactory&java.naming.factory.initial=org.apache.activemq.jndi.ActiveMQInitialContextFactory&java.naming.provider.url=tcp://localhost:61616&transport.jms.DestinationType=queue

In this scenario we will be doing a one-way invocation using the sample client application. In other words, the client sends a message to the ESB but does not expect a response in return. So the ESB should not expect a response from the backend JMS service either. We indicate this to the ESB by setting the OUT_ONLY property in the in-sequence of the proxy service. ESB will not register any call backs for messages containing the OUT_ONLY property thus no response would be expected.

Having specified the in-sequence and the target endpoint click on ‘Next’ to proceed to the third step of the wizard. Leave the out-sequence and the fault-sequence settings unchanged and simply click on ‘Finish’ to save and exit the wizard. A new proxy service will be deployed on the ESB at this point.

If you prefer to create the above proxy using the ESB configuration language you can use the XML snippet given below.

<syn:proxy name="JMSBridge" transports="https http" startOnLoad="true" trace="disable">
<syn:target>
   <syn:endpoint>
      <syn:address uri="jms:/SimpleStockQuoteService?transport.jms.ConnectionFactoryJNDIName=QueueConnectionFactory&java.naming.factory.initial=org.apache.activemq.jndi.ActiveMQInitialContextFactory&java.naming.provider.url=tcp://localhost:61616&transport.jms.DestinationType=queue"/>
   </syn:endpoint>
   <syn:inSequence>
      <syn:property name="OUT_ONLY" value="true" scope="default"/>
   </syn:inSequence>
</syn:target>
</syn:proxy>

On the management console, click on the ‘Synapse’ link under ‘Configure’ and add the above XML as a child of the top level ‘definitions’ element. Click ‘Update’ to apply the changes.

Now we are all set to run the scenario. Deploy the SimpleStockQuoteService into the sample Axis2 server. Open up the axis2.xml file for the sample Axis2 server and enable the JMS transport receiver in it. This file can be found at samples/axis2Server/repository/conf directory. To enable the JMS listener uncomment the following XML fragment.

<transportReceiver name="jms" class="org.apache.axis2.transport.jms.JMSListener">
......
</transportReceiver>

Finally start the sample Axis2 server. Since we have enabled the JMS receiver for the sample server, the SimpleStockQuote service will be exposed over JMS and the server will start listening on a JMS queue named ‘SimpleStockQuoteService’. When you haven’t specified a queue name in the service configuration, the Axis2 JMS transport will by default listen on a queue with the same name as the service. To explicitly specify the queue name for a service one must set the transport.jms.Destination parameter as a service parameter (ie in the services.xml file).

Finally go to samples/axisClient and run the following command to send a one-way message to the ESB proxy service over HTTP. (Here we invoke the in-only placeOrder operation on the backend service)

ant stockquote –Daddurl=http://localhost:8280/services/JMSBridge –Dmode=placeorder

The ESB will take the message and forward it over JMS to the queue named ‘SimpleStockQuoteService’. The sample Axis2 server which is currently listening on this queue will pick up the message and process it. Note the following log entry in the Axis2 server log when it takes the message from the JMS queue.

Sun Dec 13 16:15:39 IST 2009 samples.services.SimpleStockQuoteService  :: Accepted order for : 5789 stocks of IBM at $ 89.61765128551305

So there you have it. The proxy service abstraction in WSO2 ESB makes it extremely simple to perform protocol conversions and route messages back and forth. I suggest you also have a look at the set of transport switching samples that come with WSO2 ESB distribution. We got samples covering all the major protocols and even some of the domain specific transports like FIX and AMQP.

In my next post, I will focus on common problems that most developers encounter while implementing transport switching scenarios involving JMS and possible solutions to such pitfalls.

Introduction to Apache Synapse @ Apache Asia 2009

2009-12-04T08:08:00.000-08:00

Yesterday (3rd December 2009) I conducted a technical presentation titled “Introduction to Apache Synapse” at the Apache Asia Roadshow 2009 held at Colombo, Sri Lanka. For everyone’s benefit I have published the presentation slides on SlideShare. The presentation went well, and so far I have received pretty good feedback from the participants. There were a few very interesting questions from the audience at the end of the session too.

In this presentation I have addressed the areas like problems of enterprise integration, ESB pattern, key features of Synapse, Synapse architecture and the configuration model. I hope you will find it interesting and useful.

Introduction to Apache Synapse

View more documents from Hiranya Jayathilaka.

WSO2 ESB Tips & Tricks 02: Content Based Routing

2009-11-25T05:03:00.000-08:00

Content based routing (CBR) is one of the most fundamental and widely used features of an ESB. Most integration scenarios involve situations where we need to analyze the content of a message and route it to a predefined endpoint based on the results. Depending on the actual requirements the ESB will have to inspect the message headers, message body or both.

For an example take a situation where you have two servers, server-A and server-B, that host an order processing service. Assume that server-A takes an hour to process an order whereas server-B takes over two hours. You could use something like WSO2 ESB to do CBR and provide a better service to the clients who are placing order requests on the servers. You can configure the ESB to analyze the request content and somehow identify the high priority requests based on a certain criteria. Such requests can be forwarded to server-A and the other requests can be forwarded to server-B. This way you can assure a better service to the clients who are placing high priority orders.

So how exactly do we implement a CBR scenario with WSO2 ESB? WSO2 ESB provides mediators like the filter mediator and the switch mediator for this task. I will now explain how to employ the filter mediator and get a simple CBR scenario up and running with WSO2 ESB.

Start the server and login to the management console. Click on “Sequences” and select the “Edit” option for the “main” sequence. Start by deleting all the child mediators already present in the sequence. To delete a mediator simply click on it and select the delete option from the little popup thingy that appears.

Now add an “In” mediator and an “Out” mediator to the sequence. Then add a “Filter” mediator as the first child of the “In” mediator. “Filter” mediator will appear on the editor with two more child mediators – “Then” mediator and the “Else” mediator. Let’s leave them as they are for the moment and configure the “Filter” mediator as follows.

On the “Filter” mediator configuration form (just below the editor window) select the “Source and Regular Expression” option. If you cannot see this form on your browser just click on the “Filter” mediator icon on the editor and it will appear on the screen. Specify the following parameters for the source and regular expression fields.

Source: //m0:getQuote/m0:request/m0:symbol
Regex: IBM

Now click on the “Namespaces” link and define the following namespace.

Prefix: m0
URL: http://services.samples

Exit the namespace editor popup and click on the “Update” button to save the changes made to the “Filter” mediator configuration. Now click on “Then” mediator and select the “Add Child” option from the little popup menu. Add a “Send” mediator as the child of the “Then” mediator. Specify an anonymous endpoint for the send mediator. Provide the address http://localhost:9000/services/SimpleStockQuoteService for the anonymous endpoint.

Once you are done with that click on “Else” mediator. Again add a “Send” mediator as a child. Specify the address http://localhost:9001/services/SimpleStockQuoteService as an anonymous endpoint for the send mediator.

Finally add another “Send” mediator as a child of the “Out” mediator which we added earlier. Do not specify any endpoints for that.

Save and close the sequence editor. We are now ready to run the scenario.

If you prefer to work with the Synapse configuration language and want to implement this scenario by manually editing the configuration you can use the following XML sequence configuration. Simply click on the “Synapse” link under “Configure” in the management console and locate the configuration for the “main” sequence. Replace it with the following XML snippet.

<syn:sequence name="main">
<syn:in>
   <syn:filter xmlns:m0="http://services.samples" source="//m0:getQuote/m0:request/m0:symbol" regex="IBM">
       <syn:then>
           <syn:send>
               <syn:endpoint>
                   <syn:address uri="http://localhost:9000/services/SimpleStockQuoteService"/>
               </syn:endpoint>
           </syn:send>
       </syn:then>
       <syn:else>
           <syn:send>
               <syn:endpoint>
                   <syn:address uri="http://localhost:9001/services/SimpleStockQuoteService"/>
               </syn:endpoint>
           </syn:send>
       </syn:else>
   </syn:filter>
</syn:in>
<syn:out>
   <syn:send/>
</syn:out>
</syn:sequence>

Before we run this scenario let’s try to figure out what it is supposed to do. The “In” mediator will take all the incoming requests and hand them over to the “Filter” mediator which evaluates an XPath expression on the content (Yes – this is the XPath we specified as the 'Source' parameter earlier. It simply extracts a text value from the SOAP payload). We extract a value from the message payload and perform a regular expression match on it (Out regex is “IBM”). If the regular expression matches, the message will be handed to the “Then” mediator or else it will be given to the “Else” mediator. Both these mediators invoke the “Send” mediator but on two different endpoints, so that the incoming requests are sent to one of two endpoints depending on the result of the XPath evaluation and the regular expression match. The “Out” mediator grabs all the responses coming back from the endpoints and invokes the “Send” mediator. Since we haven’t specified a target endpoint for this “Send” mediator, responses will be sent back to the clients based on the addressing information available on the message.

That sounds simple enough. Let’s run this scenario. First go to the directory ESB_HOME/samples/axis2Server/src/SimpleStockQuoteService and deploy the sample service by running the command “ant” (you need to have Apache ANT installed). Then go to ESB_HOME/samples/axis2Server and run the following two commands on two different shells/command prompts. (Run the commands applicable to your platform)

Unix/Linux:
./axis2server.sh –http 9000 –https 9005
./axis2server.sh –http 9001 –https 9006

Windows:
axis2server.bat –http 9000 –https 9005
axis2server.bat –http 9001 –https 9006

These two commands will start two Axis2 instances, one running on port 9000 and the other running on port 9001. To invoke the sample client go to ESB_HOME/samples/axis2Client and execute the following command.

ant stockquote –Daddurl=http://localhost:8280 –Dsymbol=IBM

This will send a message to the ESB with the string “IBM” in the payload. This will match the regular expression and hence will be forwarded to the server running on port 9000. Check out the console where you started that server instance for confirmation. You should see something like this.

Wed Nov 25 18:48:57 IST 2009 samples.services.SimpleStockQuoteService :: Generating quote for : IBM

Now again invoke the client as follows.

ant stockquote –Daddurl=http://localhost:8280 –Dsymbol=MSFT

This will send a message to the ESB with the string “MSFT” in the payload and voila! The regular expression match will fail causing the message to be forwarded to port 9001. Check the Axis2 log for confirmation.

Wed Nov 25 18:48:49 IST 2009 samples.services.SimpleStockQuoteService :: Generating quote for : MSFT

So there you go – CBR made easy by WSO2 ESB. The filter mediator brings if-else like semantics into the mediation logic. The ESB also has a switch mediator. Just guess what semantics that can bring into the mediation. And then of course we have the router mediator to implement complex CBR scenarios. If you are interested in learning more about CBR support available in WSO2 ESB then you can have a look at the following samples. These 2 samples are shipped with WSO2 ESB so you can try them out all by yourself.

WSO2 ESB Tips & Tricks 01: Sequences and Proxy Services

2009-11-20T20:42:00.000-08:00

As promised, here is the first article of the “WSO2 ESB Tips & Tricks” series. The main objective of this post is to introduce two very important functional components of the ESB, namely sequences and proxy services. These are perhaps the most commonly used functional components of WSO2 ESB. They can be used to implement even the most complex of messaging systems and enterprise integration patterns using the service bus. During the course of this article I will briefly explain some of the use cases of sequences and proxy services along with some useful information on how to use them in WSO2 ESB.

WSO2 ESB supports four modes of operation.

Service mediation
Message mediation
Task scheduling
Eventing

In the service mediation mode, WSO2 ESB acts as a proxy to a real Web Service hosted in a Web Services container like Apache Axis2 or WSO2 WSAS. The ESB exposes a virtual Web Service which can accept requests from clients. The requests are processed, mediated and forwarded to the actual service implementation by the ESB. Any responses coming from the service implementation are mediated and forwarded to the clients. Proxy services are used to get the ESB operating in this mode.

In the message mediation mode, WSO2 ESB functions as a message router. It can filter, transform, drop messages or forward them to remote endpoints for further processing when operating in this mode. Sequences are used to define the message mediation behavior of the ESB. A sequence is a series of mediators, where each mediator is an entity that can accept messages and carry out a predefined task on them. WSO2 ESB provides you with a wide range of mediators designed to carry out various processing tasks on the messages. You can mix and match these available mediators to develop sequences. For instance a sequence comprising of the log mediator and the send mediator will act as a simple log-and-forward message flow. You can have any number of mediators in a sequence and a sequence can also dispatch messages to other sequences if needed.

A proxy service is a combination of three sequences and a target endpoint. The target endpoint is for the actual service implementation to which the messages will be forwarded after mediation. The three sequences are as follows.

In-sequence: All the incoming requests to the proxy service are dispatched to the in-sequence. This sequence defines how the requests should be processed before forwarding them to the target endpoint.
Out-sequence: All the responses coming back from the backend service implementation are dispatched to this sequence. It defines how the responses should be handled before sending them back to the clients. The out sequence can also forward the responses to a given service endpoint, thus effectively linking up multiple services. This way a single request to the proxy service will trigger multiple service invocations. The response from the 1st service is fed to the 2nd service as the request.
Fault sequence: If an error occurs during service mediation the faulty message is handed to the fault sequence for error handling work. Depending on our requirements the fault sequence can be used log the error, ignore it or send a SOAP fault back to the client indicating that something went wrong.

Now that you have a basic understanding of sequences and proxy services let’s take a look at some of the use cases where these components come in handy. Proxy services can be used to expose an existing service over a different schema (message format). In scenarios like this one, proxy services can transform the messages flowing back and forth. Also since proxy services can be exposed over multiple transports they can be used to expose an existing service over a different transport. For an example we can take a simple HTTP service and expose it over JMS by creating a proxy service for the HTTP service and exposing it over JMS. The ESB will take care of switching the communication protocols. In addition, proxy services are useful in adding QoS features (security, RM etc) to ordinary message flows. As an example, we can take an unsecured service, create a proxy service for it and engage security on the proxy service to add security to the backend service implementation.

Sequences can be used to audit, filter, control and transform messages flowing through a network. Certain advanced features like access control, load balancing and fail over are also possible with sequences. In WSO2 ESB all the incoming messages which are not destined to a proxy service are dispatched to the “main” sequence. The main sequence can further distribute them to other sequences and they in turns can call even more sequences. This is somewhat analogous to how a C program works. The application runtime invokes the main procedure and it in turns can call other procedures.

Enough with the theory! Let’s get practical and see how to create sequences and proxy services in WSO2 ESB. Here actually you have several options. You can either use the XML based Synapse configuration language to define sequences and proxy services, or use the set of graphical tools that come with WSO2 ESB. To create a sequence graphically, first start the ESB server and logon to the management console. (By default the web based console is accessible at https://localhost:9443/carbon) Sign in as an administrator (By default the username and the password are both “admin”). Click on the “Sequences” option in the left panel. This will show you a list of existing sequences. By default the “main” sequence and the global “fault” sequence is listed on this page.

To create a new sequence, click on “Add Sequence”. This will bring up the on-line sequence editor. Start by giving a name to the sequence. Then go ahead and start adding mediators.

To further customize the behavior of a mediator instance click on the mediator and a dialog box will appear just below the sequence editor panel. Once you are done creating the sequence click on “Save” to save and close the editor. The newly created sequence will now show up on the list of available sequences.

Ceating a proxy service is just as easy. Click on the Proxy Service link in the left panel to start the proxy service creation wizard.

Simply go through the 3-step wizard and specify an in-sequence, target endpoint, out-sequence and a fault sequence for the proxy service. When specifying these items you can either import an already existing sequence/endpoint or create one on the spot by selecting the “Anonymous” option.

Once you finish the wizard the new service will get created and deployed on the server. If you gave the name “FooProxy” to the service it will be exposed on the HTTP endpoint http://localhost:8280/services/FooProxy.

Now if you click on the “Synapse” link on the left navigation bar you can see that the overall ESB configuration has been updated with your changes.

The UI also allows you to enable statistics collection on sequences and proxy services, enable tracing on them and modifying their configuration. Play around with the various options available to you and get familiar with sequences, proxy services and the ESB management console. We will be using these a lot in the days to come.

New Kids in Town

2009-11-19T21:24:00.000-08:00

The team at WSO2 just released the latest version of WSO2 Carbon (v2.0.2) along with a whole bunch of Carbon based products. Log on to the WSO2 Oxygen Tank to lay your hands on the following smoking hot releases, right from the WSO2 oven ;)

WSO2 Enterprise Service Bus v2.1.2
WSO2 Web Services Application Server v3.1.2
WSO2 Business Process Server v1.1.0
WSO2 Mashup Server v2.0.1
WSO2 Identity Server v2.0.2

You will be particularly interested in our Business Process Server and Mashup Server releases since we haven't done any releases of them for a fairly long time. Needless to say that all new releases have many new features, bug fixes and enhancements over the previous versions. So download today and experience the power of SOA.

Introducing "WSO2 ESB Tips and Tricks"

2009-11-15T06:04:00.000-08:00

WSO2 ESB has kept me quite busy for the last couple of months. The ESB 2.1 release in July was soon followed by a 2.1.1 release and work is now underway for the 2.1.2 release. That’s way too much activity in such a short period, for an open source project. Anyway, since I’m spending a lot of time working on WSO2 ESB, I thought I might as well blog about my work in a regular basis. So in the next few months, I will be publishing a series of blog posts regarding WSO2 ESB and how to use it to implement real life integration scenarios. This series of blogs, which I have named “WSO2 ESB Tips and Tricks”, will surely benefit many folks who are either just learning SOA concepts or looking to integrate a bunch of apps using an ESB.

You can consider this post to be the 0th article of the “WSO2 ESB Tips and Tricks” series. So before I move on to the 1st post of the series I think I should briefly describe what WSO2 ESB is and why you should consider it as an option for enterprise integration. So here goes:

WSO2 ESB is a fast and lightweight, enterprise service bus. It is based on Apache Synapse, the lightweight ESB from the ASF (speaking of which, “Happy Birthday ASF”). It supports many messaging standards including SOAP, WS-* standards and REST as well as a variety of application and transport layer protocols like HTTP/S, Mail, JMS, VFS, AMQP, TCP and FIX. WSO2 ESB comes with a rich collection of mediators and other functional components which can be used out of the box to implement even the most complex integration patterns. Message routing, transformation, protocol switching, load balancing, clustering and service chaining are some of the common features supported by WSO2 ESB. Controlling and managing WSO2 ESB is also a trivial task thanks to the Web based management. Many of the common system administration tasks such as user management, certificate management and statistics collection are all available as inbuilt features of the server and the management console. WSO2 ESB is shipped with an embedded WSO2 Governance Registry, which makes it easy to store and manage SOA metadata. In addition to that the ESB can be configured to work with an externally hosted WSO2 Governance Registry if needed.

WSO2 ESB is also based on WSO2 Carbon, the OSGi based components framework for SOA. WSO2 Carbon makes it possible to easily install and configure additional features into the ESB runtime. Custom code and third part libraries can also be deployed into the server without any hassle.

Perhaps the best thing about WSO2 ESB (apart from it being super fast) is that it is totally free and open source. Binary and source distributions are available for download through the WSO2 Oxygen Tank. All artifacts are released under the business friendly Apache Software License 2.0. WSO2 also offers training and commercial support for users that require them.

As I have already mentioned a couple of times, WSO2 ESB is well known as a fast ESB. It can easily handle over 2500 transactions per second while maintaining constant memory usage. This level of performance is neither a coincidence nor an accident, but by clever and careful design. Two of the most significant features of WSO2 ESB architecture can be listed as follows:

Non-blocking HTTP transport : The HTTP server worker threads of the ESB do not get blocked over network I/O. The Apache HTTP Core-NIO based transport makes it possible to accept many concurrent connections and process more messages than any other Java HTTP transport implementation.
Pull Parsing and Streaming Model: WSO2 ESB uses Apache AXIOM, the StAX based XML infoset model. Therefore it does not build the object model for incoming messages, unless it has to. Incoming bytes are streamed through the ESB without touching the payload. This reduces memory usage significantly and also saves many valuable CPU cycles.

These architectural elements combine with the Carbon framework to deliver the best performance imaginable along with seamless modularization.

Some of the latest features that we have added to WSO2 ESB like rule based mediation and EDA (Eventing) support make WSO2 ESB suitable for implementing even the most complicated SOA platforms.

I think this explanation pretty much justifies why you should give WSO2 ESB a try. It is fast, feature rich and user friendly by all means. And don’t forget, it’s free and open source too. So grab a copy of the latest binary distribution today and get started. If you need any help we got tons of free documentation and articles on the WSO2 Oxygen Tank. This is in addition to our mailing lists and user forums.

I hope you will enjoy the “WSO2 ESB Tips and Tricks”.

A Tour to the Past

2009-09-18T23:03:00.000-07:00

Something non-technical for a change today.....

I've always wanted to visit a museum and see some artifacts from the Jurassic age. I'm very enthusiastic about learning more about the giant lizards that roamed the earth millions of years ago and I've always wished that I could see those gigantic dinosuar fossils live, without a TV screen. Finally I got the opportunity when I was in Salt Lake City, Utah, a few weeks back. The Utah Museum of Natural History is a fascinating place with lots of things to see, learn and do. They exhibit some very rare artifacts illustrating the Jurassic ages, world history, American history, geography and plant and wildlife. What I liked most was that instead of just seeing stuff, the museum has lots of facilities where you can watch educational videos, read books and managazines and take part in interactive games and activities. The use of modern technology in the musuem is mind blowing.

The musuem has special admittance rates for University students (you just need to show your student ID) and they have a special gallery named "Toadally Frogs", dedicated to kids, featuring some rare species of toads and frogs.

It indeed was a wonderful and fulfilling experience vsiting the natural history musuem of Utah. If you are visiting Salt Lake City, make sure you allocate some time to go and visit this exceptional place. Oh and did I mention that they have a huge collection of natural minerals and rare stones along with a gallery full of live insects? You don't wanna miss those.

Enterprise Messaging with Synapse, WSO2 ESB and WebSphere MQ

2009-09-12T23:26:00.002-07:00

I've been recently playing a lot with Apache Synapse, WSO2 ESB and IBM WebShpere MQ (WMQ). My intention was to use WMQ as a JMS provider for Synapse and WSO2 ESB. IBM WebSphere MQ being a very popular and matured messaging solution, I figured that getting Synapse and WSO2 ESB to work with WMQ and document the integration process would really help the IBM, Apache and WSO2 communities. Originally I anticipated this application integration to be a very comlpex and tedious task. But for much of my delight, it turned out to be a very easy and simple process in the end. What was little complicated was to get WMQ properly installed and get its JMS features configured. But the integration with Synapse and WSO2 ESB was indeed a piece of cake.

So here I'm going to list the steps hat one should follow to get WMQ integrated with Synapse or WSO2 ESB. Start by downloading the required software. Binary distributions of Apache Synapse and WSO2 ESB can be downloaded from their respective websites. A trial version of the IBM WMQ V7.0 can be downloaded from here. In addition to these application you will also require Java 5 or higher along with Apache ANT to run some of the samples described here.

Installing Apache Synapse or WSO2 ESB doesn't require any additional steps. You simply need to extract the downloaded archives and that's all. However WMQ installation is not so easy. It is a fairly long and time consuming process which requires carefulness and patience. Anyway the installation process is well documented and so it shouldn't be a problem. For your convenience I'm describing the WMQ installation procedure for Linux here. The necessary commands to be executed at each step are given in italic font.

Installing WMQ

1. Create an empty directory and extract the downloaded WMQ archive into it

2. Add a new user group called 'mqm' to the system

groupadd mqm

3. Add a new user called 'mqm' to the system. Add the user to the 'mqm' group. Set the user's home directory to /var/mqm

useradd -d /var/mqm -g mqm mqm

4. Create the directory /var/mqm and change its owner to mqm

mkdir /var/mqm

chown mqm:mqm mqm

5. Create the directory /opt/mqm

mkdir /opt/mqm

6. Create another directory anywhere on the disk and set a symlink to it from /opt/mqm

mkdir /home/hiranya/mqm

ln -s /home/hiranya/mqm /opt/mqm

7. Install libstdc++5 library on the system (using Synaptic or apt-get on Debian/Ubuntu systems) if not already installed

8. As root go to the directory where WMQ is extracted and run the license display script

./mqlicense.sh

9. Install the MQ runtime and the server using rpm

rpm -ivh MQSeriesRuntime-7.0.0-0.i386.rpm MQSeriesServer-7.0.0-0.i386.rpm --nodeps

10. Install the MQ samples

rpm -ivh MQSeriesSamples-7.0.0-0.i386.rpm --nodeps

11. Install the MQ client

rpm -ivh MQSeriesClient-7.0.0-0.i386.rpm --nodeps

That wasn't too bad, was it? Now it's time to verify the installation. Here's what you got to do.

Verifying the Installation

1. Login as user mqm (ssh mqm@localhost)

2. Create a new queue manager

crtmqm -q venus.queue.manager

3. Start the queue manager

strmqm

4. Start MQSC program (a new shell will appear - without a prompt)

runmqsc

5. Define a local queue in the MQSC shell

define qlocal (orange.queue)

6. Terminate MQSC

end

7. Go to /opt/mqm/samp/bin and run the amqsput sample program to place a message on to the queue

./amqsput ORANGE.QUEUE

8. The above command should be followed up by some sample message text of your choice and a blank line

9. Run the following command to pull the message back from the queue

./amqsget ORANGE.QUEUE

10. So far so good! We got the server properly installed. Now on to the client!

11. Create a new queue manager

crtmqm -q saturn.queue.manager

12. Start the queue manager

strmqm

13. Start MQSC

runmqsc

14. Create a new queue

define qlocal (queue1)

15. Create a connection channel to the queue

define channel (channel1) chltype (svrconn) trptype (tcp) mcauser ('mqm')

16. Create a listener (specify a port)

define listener (listener1) trptype (tcp) control (qmgr) port (8585)

17. Start the listener

start listener (listener1)

18. Terminate MQSC

end

19. Create a system environment variable to point to the channel created above

export MQSERVER='CHANNEL1/TCP/localhost(8585)'

20. Go to /opt/mqm/samp/bin and run the amqsputc to place a message in the queue (using the client API)

./amqputc QUEUE1 saturn.queue.manager

21. Now run amqsgetc to receive the message

./amqgetc QUEUE1 saturn.queue.manager

22. Excellent! We got the client installed properly too :)

Now that we got WMQ installed let's try to get its JMS features enabled.

Setting Up JMS

1. Install WMQ classes for JMS

rpm -ivh MQSeriesJava-7.0.0-0.i386.rpm --nodeps

2. Start MQSC (This will start mqsc for the last created queue manager - saturn.queue.manager)

runmqsc

3. Define a JMS channel for the queue manager

define channel(java.channel) chltype(svrconn) trptypr(tcp)

4. Define a listener

define listener(listener.tcp) trptype(tcp) port(1414)

5. Start the listener

start listener(listener.tcp)

6. Setup the classpath variable and the MQ_JAVA_LIB_PATH to run the IVT programt

export CLASSPATH=/opt/mqm/java/lib/com.ibm.mqjms.jar:/opt/mqm/samp/jms/

export MQ_JAVA_LIB_PATH=/opt/mqm/java/lib

7. Go to /opt/mqm/java/bin and invoke the IVT sample program

./IVTRun -nojndi

./IVTRun -nojndi -client -m saturn.queue.manager -host localhost -channel JAVA.CHANNEL

8. If the IVT program can recieve and send JMS messages we are done! Now let's proceed to enabling JNDI support

9. Go to the /opt/mqm/java/bin directory and open the JMSAdmin.config file. Edit the PROVIDER_URL property to point to an empty directory of your choice on the file system. This directory will be used the JNDI provider source (eg: /var/mqm/jndi). If the specified directory does not exist in the file system create it.

10. Run the IVTSetup tool to create the default set of JNDI bindings

./IVTSetup

11. Now run the IVTRun tool as follows

./IVTRun -url "file:/var/mqm/jndi" -icf com.sun.jndi.fscontext.RefFSContextFactory

12. By now we have enabled and verified JNDI support. Let's use the JMSAdmin tool to make some modifications in the JNDI bindings

13. Fireoff /opt/java/bin/JMSAdmin and run the following commands:

ALTER QCF(ivtQCF) TRANSPORT(CLIENT)

ALTER QCF(ivtQCF) QMGR(saturn.queue.manager)

14. Run the IVTRun tool again as mentioned in step 11. This time messages will be sent to the saturn queue manager.

Now we are all set. It's time to get WMQ integrated with Synapse and WSO2 ESB. Let's start with Apache Synapse.

Synapse Integration

Follow the steps given below as the user 'mqm'. Trying to run Synapse as a different user caused some JMS security exceptions.

1. Copy the following jar files from /opt/mqm/java/lib to SYNAPSE_HOME/lib

com.ibm.mqjms.jar
fscontext.jar
providerutil.jar
com.ibm.mq.jmqi.jar
dhbcore.jar

2. Enable the JMS listener in the axis2.xml. Configure the default connection factory as follows.

<parameter name="default">
  <parameter name="java.naming.factory.initial">com.sun.jndi.fscontext.RefFSContextFactory</parameter>
  <parameter name="java.naming.provider.url">file:/var/mqm/jndi</parameter>
  <parameter name="transport.jms.ConnectionFactoryJNDIName">ivtQCF</parameter>
  <parameter name="transport.jms.ConnectionFactoryType" locked="false">queue</parameter>
  <parameter name="transport.jms.Destination">ivtQ</parameter>
</parameter>

Note that we are using the JNDI configuration used by the IVT sample program. We are using the same queue connection factory and the JMS queue to define the default connection factory.

3. Create the following proxy service in the synapse.xml (this is a simplified version of sample 250)

<proxy name="JMSProxy" transports="jms">
  <target>
     <inSequence>                             
        <log level="full"/>
        <drop/>
     </inSequence>
  </target>
  <parameter name="transport.jms.ContentType">
       <rules>
           <jmsProperty>contentType</jmsProperty>
           <default>application/xml</default>
       </rules>
  </parameter>
</proxy>

4. Start Synapse - go to SYNAPSE_HOME/bin and run ./synapse.sh

5. Go to SYNAPSE_HOME/samples/axis2Client/src/samples/userguide and open the GenericJMSClient.java source file. Make the following changes in the code.

Set the jms_dest property default value to "ivtQ" (line 45)
Set the java.naming.provider.url to "file:/var/mqm/jndi" (line 82)
Set the java.naming.factory.initial to "com.sun.jndi.fscontext.RefFSContextFactory" (line 85)
Set the lookup key to "ivtQCF" (line 89)

6. Now from the SYNAPSE_HOME/samples/axis2Client directory run the following command to run the JMS client application which will send messages to the ivtQ.

ant jmsclient -Djms_type=pox -Djms_payload=IBM

7. At this point Synapse will pick the messages from the queue and log them on the console.

WSO2 ESB 2.1 Integration

Follow the steps given below as user 'mqm'

1. Copy the Websphere MQ client jars (mentioned above under Synapse Integration) to ESB_HOME/repository/components/lib

2. Enable the JMS listener in axis2.xml and configure the default JMS connection factory as follows

<parameter name="default">              
        <parameter name="java.naming.factory.initial" >com.sun.jndi.fscontext.RefFSContextFactory</parameter>                      
        <parameter name="java.naming.provider.url" >file:/var/mqm/jndi</parameter>
        <parameter name="transport.jms.ConnectionFactoryJNDIName" >ivtQCF</parameter>
        <parameter name="transport.jms.ConnectionFactoryType" >queue</parameter>
        <parameter name="transport.jms.Destination">BOGUSQ</parameter>
</parameter>

Note that we have specified a queue named 'BOGUSQ' as the default destination. This is because we need to use ivtQ for our proxy service only. If we use 'ivtQ' here all the services deployed in ESB (XKMS, echo, wso2carbon-sts) will start listening on the same queue. You also need to login to JMSAdmin and create the queue named 'BOGUSQ'. In JMSAdmin shell run the following commands.

DEFINE Q(BOGUSQ) QMGR(saturn.queue.manager)

ALTER Q(BOGUSQ) QUEUE(QUEUE1)

If the QUEUE1 does not exist, first login to MQSC and create it.

3. Add the proxy service we used with Synapse to ESB_HOME/conf/synapse.xml

4. Start WSO2 ESB

./wso2server.sh -DuseSynapseXML

5. Use the sample client we used in Synapse to send messages to the queue

WSO2 Carbon Shines in Open Source Space

2009-09-09T05:48:00.000-07:00

Awesome news folks!

WSO2 Carbon, the open source SOA platform, the team of which I'm also a part of, has won an award in "InfoWorld Best of Open Source Software (Bossie) 2009". Bossie awards cover a wide range of software categories and WSO2 Carbon won the award in the "Platforms and Middleware" category. With this, WSO2 Carbon has been recognized by InfoWorld as one of top 40 open source products available for business and IT. Here I quote the official announcement from InfoWorld which declares the award winners.

"Two leading lights of open source SOA are MuleSource and WSO2. The Mule ESB takes our prize for the finest enterprise service bus in the open source pantheon, while WSO2 receives a Bossie for its Carbon framework, the basis of a completely componentized SOA platform."

Fantastic.... Given the fact that WSO2 Carbon is still a very new software platform, this is a great achivement indeed. It just shows how fast WSO2 Carbon has matured as a complete SOA platform and how fast its community has expanded in the last few months.

Read the complete publication from InfoWorld and the official news article from WSO2 for more details on this great achievement by the WSO2 Carbon team.

Integrating Apache ActiveMQ with WSO2 ESB

2009-07-22T02:40:00.000-07:00

When going through some of the recent posts in the WSO2 ESB user forum, I got the impression that some folks are having trouble getting Apache ActiveMQ to work with WSO2 ESB. So today I'm going to explain how to integrate Apache ActiveMQ with WSO2 ESB and run one of the JMS sample scenarios.

I will be using WSO2 ESB 2.1 and Apache ActiveMQ 5.2. First download WSO2 ESB 2.1 and Apache ActiveMQ 5.2 from the respective websites if you already haven't done so. Apache ActiveMQ has different distributions for Windows platform and Unix/Linux platform. So remember to download the distribution that matches your target platform. To run the sample scenario described here you will also require Apache ANT. Once you have extracted the downloaded archives to some location in your local disk. Let's refer to WSO2 ESB installation directory as ESB_HOME and ActiveMQ installation directory as AMQ_HOME.

Now we are all set to go.....

First go to AMQ_HOME/bin directory and start the ActiveMQ broker. On Linux you simply have to execute the script named 'activemq'.

Then you need to copy the following 3 jar files into ESB_HOME/repository/components/lib directory. (With WSO2 ESB 2.1 this is the location where most of the 3rd party dependencies should be deployed)

activemq-core-5.2.0.jar
geronimo-j2ee-management_1.0_spec-1.0.jar
geronimo-jms_1.1_spec-1.1.1.jar

You will find all these jars in AMQ_HOME/lib directory.

Then open up ESB_HOME/conf/axis2.xml file in your favorite text editor and uncomment the JMS listener configuration. JMS listener configuration should look something like this.


<transportReceiver name="jms" class="org.apache.axis2.transport.jms.JMSListener">
      <parameter name="myTopicConnectionFactory" locked="false">
              <parameter name="java.naming.factory.initial" locked="false">
                  org.apache.activemq.jndi.ActiveMQInitialContextFactory
              </parameter>
              <parameter name="java.naming.provider.url" locked="false">
                  tcp://localhost:61616
              </parameter>
              <parameter name="transport.jms.ConnectionFactoryJNDIName" locked="false">
                  TopicConnectionFactory
              </parameter>
              <parameter name="transport.jms.ConnectionFactoryType" locked="false">topic</parameter>
      </parameter>

      <parameter name="myQueueConnectionFactory" locked="false">
              <parameter name="java.naming.factory.initial" locked="false">
                  org.apache.activemq.jndi.ActiveMQInitialContextFactory
              </parameter>
              <parameter name="java.naming.provider.url" locked="false">
                  tcp://localhost:61616
              </parameter>
              <parameter name="transport.jms.ConnectionFactoryJNDIName" locked="false">
                  QueueConnectionFactory
              </parameter>
              <parameter name="transport.jms.ConnectionFactoryType" locked="false">queue</parameter>
      </parameter>

      <parameter name="default" locked="false">
              <parameter name="java.naming.factory.initial" locked="false">
                  org.apache.activemq.jndi.ActiveMQInitialContextFactory
              </parameter>
              <parameter name="java.naming.provider.url" locked="false">
                  tcp://localhost:61616
              </parameter>
              <parameter name="transport.jms.ConnectionFactoryJNDIName" locked="false">
                  QueueConnectionFactory
              </parameter>
              <parameter name="transport.jms.ConnectionFactoryType" locked="false">queue</parameter>
      </parameter>
</transportReceiver>

Now go to ESB_HOME/bin and start sample 250 which demonstrates how to receive messages over JMS and forward them over HTTP. To start the sample go to ESB_HOME/bin and run the command ./wso2esb-samples.sh -sn 250 if you are on Unix/Linux or run the command ./wso2esb-samples.bat -sn 250 if you are on Windows. This will bring up WSO2 ESB with sample 250 configuration.

The ESB startup may take some time depending on your system. However note the following lines on the console when it starts up.

INFO - JMSConnectionFactory JMS ConnectionFactory : myTopicConnectionFactory initialized
INFO - JMSConnectionFactory JMS ConnectionFactory : myQueueConnectionFactory initialized
INFO - JMSConnectionFactory JMS ConnectionFactory : default initialized
INFO - JMSListener JMS Transport Receiver/Listener initialized...

This confirms that the JMS transport has successfully started by connecting to the ActiveMQ broker. When the ESB has fully started up you should get a message similar to the following line on the console.

INFO - StartupFinalizerServiceComponent WSO2 Carbon started in 33 sec

Now we are ready to run the sample. Go to ESB_HOME/samples/axis2Server/src/SimpleStockQuoteService and run the command ant to build the sample service and deploy in the sample Axis2 server that comes with WSO2 ESB (This is where you need Apache ANT. So make sure you have properly installed ANT before trying to build the service and proceed any further.). Then switch back to ESB_HOME/samples/axis2Server and start the sample server by running axis2server.sh or axis2server.bat startup script.

Finally to run the sample client head over to ESB_HOME/samples/axis2Client directory and run the following command.

ant jmsclient -Djms_type=pox -Djms_dest=dynamicQueues/StockQuoteProxy -Djms_payload=MSFT

This will send a message to a JMS queue named StockQuoteProxy. The ESB will receive the message from the queue and forward it to the sample Axis2 server over HTTP. When the Axis2 server is done with processing the message it will print the following line on the console.

samples.services.SimpleStockQuoteService  :: Accepted order for : 9351 stocks of MSFT at $ 158.6569470528259

Congratulations! You just got Apache ActiveMQ integrated and working with WSO2 ESB. Now that wasn't too hard, was it? The key thing you have to remember is the way 3rd party dependencies are deployed into WSO2 ESB. In older versions of WSO2 ESB you had to copy the jar files into ESB_HOME/lib. But that does not work with the latest version of WSO2 ESB. The deployment model of WSO2 ESB has slightly changed with the introduction of WSO2 Carbon and Equinox P2 into the ESB.

So that's all about getting ActiveMQ to work with WSO2 ESB. With this knowledge now you may try other JMS related samples that come with WSO2 ESB.

Head First Security in SOA

2009-07-15T09:00:00.000-07:00

No, O'Reilly Media hasn't published a book on 'Security in SOA' in their world famous Head First series (at least not that I know of). This post is about a wonderful presentation on the above mentioned subject, conducted by Prabath SiriWardena, couple of weeks back at the WSO2 Summer School. Prabath is one of my colleagues at WSO2 and he is one of the most experienced folks we got. Prabath's expertise is on computer security and at WSO2 he leads all security related projects including WSO2 Identity Server. In this summer school presentation Prabath has started simple, by explaining the fundamental concepts of computer security like confidentiality, integrity and availability and goes on to more complex topics such as public key cryptography, transport level and message level secuity, WS-Security specs and Username-Token authentication. He gives a glimpse on various options available to SOA architects and Web Service authors to secure their applications at different levels while emphasizing on the importance of interoperability.

Throughout the presentation he has kept things simple yet extremely interesting. You will find the entire presentation sort of follows the storyline based, fun-filled teaching method which is a very effective technique commonly used in the books of the Head First series (and hence the title).

If you want to learn the fundamentals of Security in SOA and how it is used in the enterprise (or how it should be used in the enterprise), this presentation would be a great starting point. So start flipping through the slides now and see for yourself.

Summer School - Security in SOA

View more documents from wso2.org.

Amplify Your SOA with WSO2 ESB 2.1

2009-07-12T23:52:00.000-07:00

If you have been following my blog, then you already know that WSO2 Carbon 2.0 and a host of other Carbon based WSO2 SOA products were released last week. WSO2 Enterprise Service Bus 2.1, which is one of the released products, must be highlighted as a high quality, feature rich and extremely user friendly piece of SOA middleware for a number of reasons. Like all its predecessors, this version of WSO2 ESB is also based on Apache Synapse, the lightweight, ultra-fast ESB. WSO2 ESB is generally popular among SOA enthusiasts because of the following set of key features provided by the ESB.

Proxy services - facilitating synchronous/asynchronous transport, interface (WSDL/Schema/Policy), message format (SOAP 1.1/1.2, POX/REST, Text, Binary), QoS (WS-Addressing/WS-Security/WS-RM) and optimization switching (MTOM/SwA).
Non-blocking HTTP/S transports based on Apache HttpCore for ultrafast execution and support for thousands of connections at high concurreny with constant memory usage.
Built in Registry/Repository, facilitating dynamic updating and reloading of the configuration and associated resources (e.g. XSLTs, XSD, WSDL, Policies, JS, Configurations ..)
Easily extendable via custom Java class (mediator and command)/Spring mediators, or BSF Scripting languages (Javascript, Ruby, Groovy, etc.)
Built in support for scheduling tasks using the Quartz scheduler.
Load-balancing (with or without sticky sessions)/Fail-over, and clustered Throttling and Caching support
WS-Security, WS-Reliable Messaging, Caching & Throttling configurable via (message/operation/service level) WS-Policies
Lightweight, XML and Web services centric messaging model
Support for industrial standards (Hessian binary web service protocol/ Financial Information eXchange protocol and optional Health Level-7 protocol)
Enhanced support for the VFS(File/FTP/SFTP)/JMS/Mail transports with optional TCP/UDP transports and transport switching for any of the above transports
Support for message splitting & aggregation using the EIP and service callouts
Database lookup & store support with DBMediators with reusable database connection pools
WS-Eventing support with event sources and event brokering
Rule based mediation of the messages using the Drools rule engine
Transactions support via the JMS transport and Transaction mediator for database mediators
Internationalized GUI management console with user/permission management for configuration development and monitoring support with statistics, configurable logging and tracing
JMX monitoring support and JMX management capabilities like, Gracefull/Forcefull shutdown/restart

Wow! That's a lot of features for a software product developed openly and distributed free of charge under Apache Software License 2.0. As you would imagine, it is mainly the performance and the lightweight operation model of WSO2 ESB which makes it stand out from the rest. In addition to the above mentioned key features the latest ESB 2.1 release brings you the following set of new features.

Rule based mediation via Drools
Fine grained authorization for services via the Entitlement mediator
Reliable-Messaging specification 1.1 support
Enhanced WS-Eventing support and Event Sources making it an even broker
Enhanced AJAX based sequence, endpoint and proxy service editors
Enhanced transport configuration management through the graphical console
Enhanced integrated registry and search functionalities with versioning, notifications, rating of resources, and commenting
Enhanced remote registry support
Default persistence to the registry for the configuration elements
Enhanced permission model with the user management
Enhanced REST/GET and other HTTP method support
P2 based OSGi feature support, for optional features like service management, runtime governance

The coolest thing about WSO2 ESB 2.1 is that it is 100% OSGi based (thanks to the Carbon platform of course!). All the features are packed into OSGi bundles and therefore adding new features and removing unnecessary features cannot get any easier. The newly introduced provisioning support based on Equinox P2 makes it particularly easy to deploy new features and third party libraries into the ESB. With WSO2 ESB 2.1, you can deploy only the features you want and only them. You are not forced to load any features/libraries that you never use. Why waste memory and other resources on features never used, right?

With ESB 2.1 registry integration support has improved vastly. WSO2 ESB 2.1 comes with an embedded WSO2 G-Reg instance but you can easily point the ESB to a remotely hosted registry instance in a matter of seconds. ESB 2.1 also has the ability to export its entire configuration to the registry and reload the configuration back from the registry at the server startup.

User interfaces and context sensitive help system have gone through lot of rework. You will find that most of the web interfaces are now fully AJAX compliant making it easy and fun to work with WSO2 ESB. All UIs are fully internationalized and can be even separated from the backend system to be hosted as different web application.

WSO2 ESB 2.1 is a giant step forward by the WSO2 folks to make their ESB even more elegant and enterprise ready. It gives you a combination of high performance, modularity and user friendliness. It is completely free and open source, with a very active and supportive community of developers to back up all the development work. WSO2 also offers user training, development support and production support to any party which requires such facilities.

If you are looking for a robust mediation solution to power your enterprise SOA or if you are tired of trying out expensive proprietary ESB solutions, it’s high time you give WSO2 ESB 2.1 a spin. It will be totally worth it!!!

WSO2 Carbon Platform Takes Another Giant Step Forward

2009-07-10T07:40:00.000-07:00

Weeks have passed by since I did my last blog post. The reason for my prolonged silence in the blog sphere was due to the fact that I was working hard on a very important point release of WSO2 Carbon 2.0 and a host of WSO2 Java products based on the Carbon platform. Ideally these releases should have come out bit early but realizing the importance of implementing some new features, we decided to push the releases back by a few weeks. Anyway WSO2 Carbon 2.0 and four of the Carbon based WSO2 products (WSAS 3.1, ESB 2.1, G-Reg 3.0 and IS 2.0) were released on 8th July 2009 and you can now download them from the download page of the WSO2 Oxygen Tank.

WSO2 Carbon is an OSGi based application framework for SOA and Web Services. All WSO2 Java products including WSO2 ESB, WSO2 WSAS, WSO2 Governance Registry and WSO2 Identity Server are based on this revolutionary SOA platform. We have made some very important architectural and functional enhancements to WSO2 Carbon recently. Therefore the users of Carbon 2.0 based products will be able to reap the benefits of those improvements in the forms of improved performance, reliability, extensibility, user friendliness and stability.

One of the most important architectural changes we made to the Carbon platform this time around is getting rid of OSGi bundle activators and instead, adapting OSGi declarative services to handle bundle initialization and cleanup. OSGi declarative services are considered the best solution when it comes to managing systems with a large number of OSGi bundles. Declarative services allow bundles to startup and initialize properly while sharing required data among various bundles without following a pre configured bundle startup order. Declarative services also helped us improve the dynamic nature of the overall Carbon platform, so now starting bundles and stopping bundles at runtime is easier and safer.

Another key enhancement we did for this release of Carbon is the introduction of the P2 based provisioning system. This work is still at its early stages but it already enables Carbon and Carbon based product users to easily deploy Carbon components, user developed OSGi bundles, mediators and third party dependencies to the system easily, without having to write a single line of code to handle OSGi initialization or cleanup work.

In General Carbon 2.0 and Carbon 2.0 products have better registry integration support, better transport management support, better user management support, better security features and better user interfaces. We gave most of the Carbon component UIs a fresh look through the extended use of AJAX. Persistence of configuration data is another area where we have done a lot of work lately. WSO2 ESB 2.1 in particular has the ability to store the entire ESB configuration in a registry and then load the configuration back from the registry during server startup.

Documentation is yet another area that’s been largely improved with this release. We paid special attention to improving our context sensitive help documents (accessed via product user interfaces) and other downloadable guides and tutorials. In the previous releases of Carbon and Carbon based products we were mostly focused on system architecture and functionality. But now the Carbon framework is largely stabilized we were able to spare some cycles for improving our documentation too. We have added lots of new guides and catalogs including mediator catalog, transports catalog, server administration guide and the endpoints guide. Our plan is to make our products easy to learn and use via making our user interfaces simple, integrating help with the user interfaces and making ample documentation freely available to all.

In addition to the key improvements mentioned above Carbon 2.0 and related products come with tons of new features, minor bug fixes, performance enhancements and usability improvements which add lot of value to the WSO2 Carbon SOA platform. If you are interested in SOA and Web Services middleware you must give some of the new Carbon 2.0 based WSO2 products a try. After all it’s totally free and open source so it doesn’t cost you a penny to try our software. I guarantee that it would be a great experience and after that you will love WSO2 products.

On a finishing note I would like to quote the WSO2 Carbon 2.0 release note here….

WSO2 Carbon 2.0.0 - Middleware a' la carte

Welcome to the WSO2 Carbon 2.0.0 release. This release is available for download at http://wso2.org/projects/carbon

WSO2 Carbon is the base platform for all WSO2 Java products. Built on OSGi, Carbon encapsulates major SOA functionality such as data services, business process management, ESB routing/transformation, rules, security, throttling, caching, logging and monitoring. These product capabilities are no longer tied to individual products, but are available as components.

A Carbon feature consists of one or more Carbon components. These are nothing but Eclipse equinox P2 features. In order to extend the functionality of you Carbon server, simply install new features onto you server as outlined in https://wso2.org/wiki/display/carbon/p2-based-provisioning-support

Key Features

Extensible OSGi component based architecture
P2 provisioning based feature enhancement of the server
Unified server management framework
Unified Management Console
Web service, JMX & Equinox OSGi console based administration
Integrated security & permissions management

New Features In This Release

Experimental Equinox P2 based provisioning support - extend your Carbon instance by installing new P2 features (See https://wso2.org/wiki/display/carbon/p2-based-provisioning-support)
Fixed start levels eliminated
Performance improvements to the Registry
Ability to make normal jar files into OSGi bundle (Just copy the jar files into CARBON_HOME/repository/components/lib)
Various bug fixes and enhancements including architectural improvements to Apache Axis2, Apache Rampart, Apache Sandesha2 , WSO2 Carbon and other projects, including security fixes

How to Run

Extract the downloaded zip
Go to the bin directory in the extracted folder
Run the wso2server.sh or wso2server.bat as appropriate
Point you browser to the URL https://localhost:9443/carbon/
Use "admin", "admin" as the username and password.
If you need to start the OSGi console with the server use the property -DosgiConsole when starting the server

For more details, run, wso2server.sh (wso2server.bat) --help

How to Install Additional Features

You can build your own server by selecting only the features that you require

For further details refer to

https://wso2.org/wiki/display/carbon/p2-based-provisioning-support

Known Issues

All known issues have been recorded at

https://wso2.org/jira/browse/CARBON

Training

WSO2 Inc. offers a variety of professional Training Programs, including training on general web services as well as WSO2 Carbon, Apache Axis2, Data Services and a number of other products.

For additional support information please refer to

http://wso2.com/training/course-catalog/

Support

WSO2 Inc. offers a variety of development and production support programs, ranging from web-based support up through normal business hours, to premium 24x7 phone support.

For additional support information please refer to

http://wso2.com/support/

For more information on WSO2 Carbon, visit the WSO2 Oxygen Tank (http://wso2.org)

Terminator is Here! Make Your Judgement!

2009-05-08T07:14:00.000-07:00

I recently installed 'Terminator' on my laptop which is running Ubuntu Gutsy. Terminator is a neat little shell client for Linux and many other platforms, which enables you to split the shell screen horizontally and vertically into multiple shell screens. This is a feature that I've always longed for.

As a software developer I usually have to work with multiple applications and multiple shell instances at the same time. For example, very often I fire off my IDE from one shell instance, run my Maven builds in another shell instance, interact with SVN from another shell instance and browse the local file system from another shell instance. Most traditional shell clients attempt to address this requirement by providing tabs. Tabs are great but keeping track of which application is running in which tab and switching between them accordingly can be a real pain. It becomes an unavoidable annoyance when you open up more than 3 tabs.

In my opinion, Terminator gives you the most optimal solution for this problem by giving the ability to split a single shell screen into multiple shell screens. You can see all the shell instances you have opened up on your screen at one glance and switching between them is lot more easier than having to switch between tabs. Splitting the shell screen is also easy with Terminator. Simply right click on the Terminator shell and select one of the split options available from the context menu that appears. Following is a little screen shot of Terminator that I took when working with four shell instances. It clearly illustrates the power of Terminator.

Installing Terminator on Ubuntu systems is also pretty easy. Simply add the following line to your sources.list file, update the apt cache and type 'sudo apt-get install terminator' to install the application. Piece of cake!!

deb http://ppa.launchpad.net/gnome-terminator/ubuntu gutsy main restricted universe multiverse

(Depending on the version of your Ubuntu installation, you might have to edit this line)

If you often happen to work with many shell screens at once you'll definitely love 'Terminator'. So install 'Terminator' today and join the fun!!

A 'Good' ESB Should....

2009-05-02T22:44:00.000-07:00

As technologies like SOA and Web Services continue to become more and more dominant mechanisms for implementing complex distributed systems, the demand for efficient and reliable enterprise service bus middleware is becoming larger and larger. There are dozens of potential ESB solutions out there, some open source and some proprietary, but the cold hearted truth is most of these products have a number of short comings, which makes them totally useless in production environments. Production environments don't require fast ESBs. They need ultra fast ESBs that can handle thousands of concurrent user requests. such deployments require middleware which can satisfy scalability and availability requirements through features such as load balancing, clustering and fail over support. A good ESB should also be capable of dealing with many communication protocols, transport mechanisms and messaging standards.

Fortunately we are not totally out of hope. There are some really good products out there which give you all the above mentioned features and deliver 100% in production deployments. WSO2 ESB is certainly one of those middleware solutions which is fast, reliable and feature rich. It is based on Apache's tried and tested Apache Synapse light weight ESB and starting from version 2.0 it is also based on WSO2's revolutionary Carbon framework.

WSO2 folks recently published a cool flash presentation which walks us through all the basic features of WSO2 ESB within a few minutes. Have a look and see whether your ESB delivers at least half of the things the WSO2 ESB provides.

Summer of Code is Back

2009-04-25T07:40:00.000-07:00

Google Summer of Code 2009, the annual open source feast organized by the Internet giant Google is now underway. I also applied for the contest and I'm pleased to announce that my project proposal to implement SQL authorization support for Apache Derby dblook has been accepted. So that makes me a GSoC winner, two times in a row. Last year I implemented XML Schema 1.1 type alternatives support for Apache Xerces2/J under the guidance of Khaled Noaman. This year Dag Wanvik will be mentoring my project.

I'm eagerly looking forward to complete this task in time and of course in classy fashion. Apache Derby community is really active, dynamic and supportive. I feel gifted to get an opportunity to work with such a passionate group of committers. And not to mention this is going to be a great learning experience as well as a genuine test of skill for me.

On the sad side, this will probably be the last GSoC I'm going to take part as a student of the University of Moratuwa. I will be officially passing out from the University in a couple of months time. University of Moratuwa became the no. 1 ranked institute in last year's GSoC which made us really happy and proud of ourselves. I'm confident that my University has ranked pretty high this year too (Google haven't released the numbers as of yet). So that means this is the last time I get to enjoy being on top of the world in GSoC :(

Committed to Apache Xerces

2009-04-09T07:46:00.000-07:00

The Apache Xerces community granted me committer status for Apache Xerces2-J. The vote was called in just a few days back and it was officially closed on 8th of April (yesterday). Xerces2-J is an open source library for parsing, validating and manipulating XML documents. It is the most matured and most standard complaint XML parser available for Java applications today.

Apache Xerces used to be a subproject of the Apache XML project but due to the immense popularity and high adoption rate that Xerces continued to exhibit, it became a top level project in year 2004. Presently Apache Xerces is a collection of four sub projects.

Apache Xerces C++
Apache Xerces2 Java
Apache Xerces Perl
Apache XML Commons

The original code base which formed the foundation of Xerces C++ and Xerces-J was developed at IBM and donated to ASF in year 1999. Therefore Apache Xerces happens to be one of the oldest projects of the ASF as well. In fact Xerces is nearly as old as the ASF itself and will be celebrating its 10th anniversary this November. Given all the facts and figures, it is indeed a great honour to be a part of such a majestic open source project :)

Talking about Apache Xerces2-J, it supports a large number of XML and XML related standards and APIs. The latest released version (2.9.1) supports the following specs.

XML 1.0 (4th Edition)
Namespaces in XML 1.0 (2nd Edition)
XML 1.1 (2nd Edition)
Namespaces in XML 1.1 (2nd Edition)
W3C XML Schema 1.0 (2nd Edition)
XInclude 1.0 (2nd Edition)
OASIS XML Catalogs 1.1
SAX 2.0.2
DOM Level 3 Core, Load and Save
DOM Level 2 Core, Events, Traversal and Range
JAXP 1.3

Due to the performance levels achievable through Xerces and its high level of standard compliancy, hundreds of open source projects and other commercial software applications make use of Xerces2-J. It is fair enough to say that Xerces2-J is the most widely used XML parser in the Java world. If you have a Java application which does some XML related stuff it's very likely that Xerces2-J is somewhere in your application's classpath. People sometimes use it even without knowing that they are actually using Xerces2-J. Sun Microsystems ships Xerces2-J with their JDK. This enables Sun JDK users to develop XML parsing applications powered by Xerces2-J without having to explicitly put the Xerces2-J jars in the project classpaths. However folks at Sun Microsystems have changed the Xerces2-J package name from org.apache.xerces.* to com.sun.org.apache.xerces.* which in my opinion is being a little ungrateful :(

Nowadays we are working on getting Xerces2-J to support the new XML Schema 1.1 specification from W3C. In fact I started contributing to Xerces2-J by implementing a XML Schema 1.1 feature as a Google Summer of Code participant, last year. I've been taking part in mailing list discussions and fixing bugs ever since. The Xerces2-J community is rather small but there's a very helpful and friendly bunch of experienced developers there, who have dedicated their lives and careers to Xerces. That encourages young developers like me to do more and more towards the betterment of the project, as and when the time permits.

MOINC and Scalability

2009-04-04T07:26:00.001-07:00

My colleague Aravinda just published a cool post on his blog regarding the scalability testing round we carried out on MOINC a few weeks back. He has also published some charts illustrating the results of our tests. Have a look and see for yourself.

WSO2 ESB 2.0.2 Released...Come and Get it

2009-04-04T07:14:00.000-07:00

The WSO2 ESB team is pleased to announce the release of version 2.0.2 of the Open Source Enterprise Service Bus (ESB). This is a bug fix release of the 2.0 release of WSO2 ESB

WSO2 ESB is a lightweight and easy-to-use Open Source Enterprise Service Bus (ESB) available under the Apache Software License v2.0. WSO2 ESB allows administrators to simply and easily configure message routing, intermediation, transformation, logging, task scheduling, etc.. The runtime has been designed to be completely asynchronous, non-blocking and streaming based on the Apache Synapse core.

WSO2 ESB 2.0.2 is developed on top of the revolutionary Carbon platform (Middleware a' la carte), and is based on the OSGi framework to achieve the better modularity for your SOA architecture. This also contains a lots of new features and many other optional components to customize the behavior of the server. Further, if you do not want any of the built in features, you can uninstall those features without any trouble. In other words, this ESB can be customized to your SOA needs.

You can download this distribution from http://wso2.org/downloads/esb and give it a try.

How to Run

Extract the downloaded zip
Go to the bin directory in the extracted folder
Run the wso2server.sh or wso2server.bat as appropriate
Point your browser to the URL https://localhost:9443/carbon
Use "admin", "admin" as the username and password to login as an admin and create a user account
Assign the required permissions to the user through a role
If you need to start the OSGi console with the server use the property -DosgiConsole when starting the server
Samples configurations can be specified by passing the property -Desb.sample=${SAMPLE_NUMBER}

Key Features

Proxy services - facilitating transport, interface (WSDL/Schema/Policy), message format (SOAP 1.1/1.2, POX/REST, Text, Binary), QoS (WS-Addressing/WS-Security/WS-RM) and optimization switching (MTOM/SwA).
Non-blocking HTTP/S transports based on Apache HttpCore for ultrafast execution and support for thousands of connections at high concurreny with constant memory usage.
Built in Registry/Repository, facilitating dynamic updating and reloading of the configuration and associated resources (e.g. XSLTs, XSD, JS, ..)
Easily extended via custom Java class (mediator and command)/Spring mediators, or BSF Scripting languages (Javascript, Ruby, Groovy, etc.)
Built in support for scheduling tasks using the Quartz scheduler.
Load-balancing (with or without sticky sessions) /Fail-over, and clustered Throttling and Caching support
WS-Security, WS-Reliable Messaging, Caching and Throttling configurable via (message/operation/service level) WS-Policies
Lightweight, XML and Web services centric messaging model
Support for industrial standards (Hessian binary web service protocol/Financial information exchange protocol)
Enhanced support for the VFS/JMS/Mail transports
Support for message splitting and aggregation using the EIP
Database lookup and store support with DBMediators with reusable database connection pools
JMX monitoring support

New Features of the WSO2 ESB 2.0 (2.0.2)

This ESB release is based on Carbon "Middleware a' la carte" which is an OSGi based SOA platform by WSO2 Inc.
Transactional JMS transport and the Transaction mediators
Integrated graphical user management
Integrated graphical key store management
Configurable logging through the management console
Graphical data source declaration
WS-Eventing support and Event Sources making it an event broker
Enhanced sequence and proxy service editor
Module management capability
Transport configuration management through the graphical console
Graceful/Forced shutdown/restart
Enhanced integrated registry and search functionalities
User permissions support
Enhanced monitoring tools for statistics and tracing
Try-It tool to try an existing service
Graphical policy editor
Administration console fully internationalized
Better modularity and extendability through OSGi component architecture

How You Can Contribute....

Mailing Lists

Join our mailing list and correspond with the developers directly.

Developer List : esb-java-dev@wso2.org
User List : esb-java-user@wso2.org

Reporting Issues

WSO2 encourages you to report issues and your enhancement requests for the WSO2 ESB using the public JIRA.

You can also watch how they are resolved, and comment on the progress..

Discussion Forums

Alternatively, questions could be raised using the forums available.

WSO2 ESB Forum : Discussion forum for WSO2 ESB developers/users

Training

WSO2 Inc. offers a variety of professional Training Programs, including training on general Web services as well as WSO2 ESB, Apache Synapse, Apache Axis2 and number of other products.

For additional support information please refer to http://wso2.com/training/course-catalog/

Support

WSO2 Inc. offers a variety of development and production support programs, ranging from Web-based support up through normal business hours, to premium 24x7 phone support.

For additional support information please refer to http://wso2.com/support/

For more information on WSO2 ESB, visit the WSO2 Oxygen Tank (http://wso2.org)

We welcome your feedback on this implementation. Thank you for your interest in WSO2 ESB.

-- The WSO2 ESB Team --

ApacheCon EU 2009....Finishing Touches

2009-03-27T14:20:00.000-07:00

The ApacheCon EU 2009 (Amsterdam, Netherlands) came to an end today evening. We had a BarCamp on 23rd followed by a Hackathon on 24th. The conference commenced on 25th and continued until today evening. We had two keynotes and around 64 technical presentations during the last three days. I was fortunate enough to take part in the following presentations.

Embrace OSGi (by Carsten Ziegeler, member of ASF)
Tales from OSGi trenches (by Bertrand Delacretaz, member of ASF)
OSGi as a framework for building a product line (by Ruwan Linton and Afkham Azeez, WSO2)
HBasics: Hadoop's big database (by Michael Stack, Microsoft)
Apache license as a business model (by Paul Fremantle, WSO2)
Clustered Web services for high availability and scalability (by Ruwan Linton, WSO2)
EDA with Apache Synapse (by Paul Fremantle, WSO2)
Performance tuning Apache Tomcat (by Filip Hanik, SpringSource)
How to become a project at ASF (by Martijn Dashorst)
Introduction to NIO 2.0 (by Jeanfrancois Arcand)
Using MINA 2.0 (by Emmanuel Lecharmy, IKTEK)
Becoming a Tomcat super user (by mark Thomas, SpringSource)
High availability != High cost (by Norman Maurer, HEAG MediaNet)
Apache POI (by Nick Burch, Torchbox)
Enterprise build & test in the cloud (Carlos Sanchez, G2iX)
Shindig for blogs and wikis (Dave Johnson, VP Apache Roller)

I also got the opportunity to chair most of the above sessions thanks to the organizers and folks of the Apache TAC. Thanks guys! You rock!

Out of the two keynotes, I really enjoyed the keynote on 'open sourcing the analyst business' by James Governor. It was a real blast!

In addition to all the technical knowledge I gathered during the conference I also made quite a lot of new friends. I met some very interesting people working at various levels and projects of the ASF. All in all it was a very successful ApacheCon, not just to me but to everybody that participated. (I have a lot more to mention but unfortunately I have to cut it short. I have lot of packing left to do before I start the return journey tomorrow morning.)

Happy Birthday ASF

2009-03-25T13:53:00.000-07:00

The Apache Software Foundation (ASF) celebrates its 10th anniversary today. Having finished the regular activities of the ApacheCon scheduled for the day, all the committers, PMC members and members got together to celebrate this great occasion. Jim Jagielski, the chairman of the ASF board of directors cut the giant birth day cake. It was indeed a fun filled and proud moment for everyone at Apache. Another important event that took place during the reception was the key signining party where around 20 folks including myself, got the opportunity to get signed PGP keys from ASF.

All in all it was a great birth day party with loads of fun and excitement. Happy 10th anniversary ASF!!!

ApacheCon EU 2009 Begins

2009-03-23T13:21:00.000-07:00

After a 3 hour flight from Sri Lanka to India followed by an 8 hour flight from India to Netherlands, I finally arrived at beautiful Amsterdam city, to take part in the ApacheCon Europe 2009. The event is now being held at the Movenpick hotel at the Amsterdam city center. The main attraction of the first day was BarCamp Apache where a number of developers, committers and open source geeks like me presented on a variety of topics. I myself conducted a presentation on Apache Synapse and how it can be used to scale up Web Services deployments. Among the other presentations I really liked the presentation by Ricardo Varela on software for mobile platforms and the presentation by Ross Gardler on teaching open source. Sri Lankan student community was credited for their achievements in the last Google Summer of Code during the presentation by Ross :)

All in all it was a really awesome BarCamp with lots of great talks, t-shirts, sweets and coffee. Let's see how the things turn out tomorrow.

Web Services for Mission Critical Apps

2009-03-19T08:41:00.000-07:00

While reading the “Beautiful Code” from O'Reilly Media I came across a chapter compiled by Ronald Mak. He was a senior scientist at the Research Institute for Advanced Computer Science and he has been contracted twice by the NASA to develop their enterprise applications to manage space missions. In the chapter 20 of “Beautiful Code” Ronald describes the internals of the Collaborative Information Portal (CIP) which he and his team developed for the NASA for the Mars Exploration Rover (MER) mission which commenced in year 2003. The development of the project has taken nearly two years and needless to say being a highly mission critical ERP solution the requirements of the system have been very demanding. Among the most fundamental features of the system were,

Time management (Managing all time zones on Earth and two time zones on Mars)
Personnel management (Managing all mission personnel) and,
Data management (Managing all the data sent from the two automated Rovers on Mars)

So how did Ronald and his team tackled this mammoth task? They used a 3-tiered service oriented architecture (SOA). Most of the code has been written in Java and they have followed J2EE standards. Also the developers of the CIP have used commercial off-the-shelf (COTS) software as much as possible to avoid re-inventing the wheel thus saving loads of time and effort. The client tier of the system consists mainly of standalone GUI applications developed mostly with Swing. These client applications communicate with a set of Web Services hosted on a J2EE compliant application server. This forms the middleware tier of the CIP. Finally there is a data tier which consists of mission file servers, databases and other metadata repositories. Ronald explains that using an SOA based on J2EE, was one of the main driving forces behind their success with the CIP.

The Web Services in the middleware tier are perhaps the most important components of the system. These services have been implemented using Enterprise Java Beans (EJB). They have developed two types of beans, namely stateless session beans and stateful session beans. Stateless session beans act as the service providers by receiving all the client requests and routing them to relevant stateful session beans which implement the business logic of the system. Ronald further elaborates that use of Web Services made CIP modular, language independent and loosely coupled.

All-in-all Ronald and his team have manged to deliver the solution on schedule. The end system has recorded a 99.9% uptime, indicating how reliable the system was. In addition to using Web Services and SOA to make the system highly maintainable and scalable, they have employed a number of other techniques to shape up the CIP into a highly reliable ERP solution. According to Roland, logging was one such technique. They have used a system wide logging mechanism based on Apache Log4j to log all the important (and even not so important) activities that take place within the system. They have also developed services to specifically support the monitoring function of the system which means that they have considered monitoring as a fundamental requirement of the system. In addition, Ronald and the team made the CIP dynamically configurable and hot swappable to minimize the down time maximize the robustness.

I believe that CIP of NASA is one of many examples which effectively demonstrates the power of Web Services and SOA. These technologies can help even huge government agencies like NASA to develop large complex enterprise software systems on time while meeting all the strict reliability, availability and scalability requirements as desired. As the time goes on these state of the art technologies will prove to be even more powerful and useful. I'm just glad that I'm into further exercising SOA as a Software Engineer.

Social Networking: Bandwidth Eater?

2009-03-18T04:18:00.000-07:00

Me and a group of my colleagues at University of Moratuwa, conducted a small research to analyze the impact of social networking on bandwidth usage in corporate networks. Under the guidance of Dr. Chandana Gamage, we studied the network resource utilization patterns triggered by browsing social networking sites such as Facebook and YouTube. We then put our observations and findings into a short research paper titled, “Network Resource Utilization and Social Networking”. Our research was rather small and probably not highly analytical but I think anybody who runs a corporate network should have a quick look at it. However if you are a sworn social networking fan, I would say it’s best not to read it since our findings might seriously piss you off ;)

On a finishing note let me just say that bandwidth on corporate networks is a valuable resource that is generally shared among many people. So use it carefully and use it responsibly. Having some fun with social networking is fine. But always know your limits and ethical responsibilities.

Pushing to the Limits

2009-03-17T01:12:00.000-07:00

Just a couple of days back, me and the rest of the MOINC team conducted a large scale performance testing round for MOINC at the advanced computing lab of the Department of CSE, University of Moratuwa. We used over 20 PCs for our tests and the resulting numbers we obtained were pretty amazing. My friend and colleague Aravinda is already in the process of analyzing the statistics collected and we hope to compile a research paper based on the results. We will be publishing the details regarding the experiments as well as the collected data soon on our website in a more aesthetically pleasing manner. In the meantime I would like to say that we as the developers are very happy regarding the level of scalability the prototype MOINC platform has displayed during the tests.

Following are some of the pictures that were taken during the performance testing round. The sight of the MOINC screen saver coming up in about 20 PCs at the same time was truly a treat to the eye :)

I thank our department head, Mrs. Vishaka Nanayakkara and officer in charge of the advanced computing lab, Mr. Ananda Fransiscu for all the support in carrying out this round of testing.

Get Ready to Get MOINCed

2009-03-16T06:05:00.000-07:00

Few weeks back me and my team managed to finalize developing the first prototype of our dream project, MOINC. If you are clueless as to what project MOINC is all about, it is an attempt to combine the Web Services paradigm with grid computing. The goal of the project is to be able to deploy enterprise Web Services for high availability and high scalability using commodity hardware. So far we have done three formal presentations describing each of the major components of the MOINC platform and we have written three research papers which we hope to publish very soon. In addition to the presentations we also conducted a formal demonstration of MOINC at University of Moratuwa, using 6 computers which effectively gave a preview of MOINC in action to a panel headed by our project supervisor Dr. Sanjiva Weerawarana and our senior lecturer Dr. Chathura De Silva. Presentations were attended by Dr. Sanath Jayasena, Dr. Chandana Gamage, Dr. Chathura De Silva and Mr. Shantha Fernando of the department of CSE, University of Moratuwa.

We got pretty good feedback from all those who attended the presentations and the demonstration, clearly indicating that we are on the right track to make MOINC into a useful software solution for business organizations worldwide. Some of the core features of the MOINC platform that we demonstrated are listed below.

Deploy, manage and undeploy service artifacts (Service artifacts are uploaded as Axis archives - *.aar files)
Track down idling computers in the local network and add them to the MOINC grid dynamically
Download service artifacts into idling computers from a centralized registry/repository
Action script based screen saver for client PCs
Load balance the incoming service requests among all the active nodes connected to the MOINC grid (Powered by Apache Synapse)
Monitor the grid via an AJAX based Web interface (Powered by WSO2 WSF/AJAX)
Collect statistics related to computers connected to the MOINC grid and use them in the intelligent load balance mode
MOINC community portal and forums (backed by WSO2 Registry and JForum)

That’s certainly a lot of features for a mere prototype. No wonder we got pretty good feedback. However there is certainly lot more work to be done. We need to improve the overall security of the platform. Currently there is a couple of security loop holes in MOINC SMM that we need to close off. Also Dr. Sanjiva suggested writing a Java security manager for MOINC client agent which ensures the security of PCs connected to the MOINC server. We need to start working on that soon as well. Also we have to finish our Maven2 integration stuff and get the release artifacts for MOINC 0.1-alpha out soon. We still haven’t cut the release artifacts for all the components of MOINC but the full source code of the prototype can be checked out from our SVN. Also checkout the developer resources section on our website for all the slides we used for the above mentioned three presentations.

I will also make arrangements for all of you humble readers to have a sneak peak at our research papers before we actually publish them. I promise. Meanwhile enjoy the presentations and other design documents :)

Carbonizing SOA

2009-03-12T22:55:00.000-07:00

It's probably bit late to blog regarding this subject, but what the heck? After all, late is better than never.

WSO2 folks released their flagship product, the mother of all open source SOA solutions, WSO2 Carbon – Middleware a la carte. It is a componentized, customizable SOA platform built on Equinox and OSGi. The platform can be configured and tailored to suit the enterprise architecture of any business organization. Among the major components of the Carbon platform are;

1.WSO2 Web Services Application Server 3.0
2.WSO2 ESB 2.0
3.WSO2 Registry 2.0
4.WSO2 Business Process Server 1.0

All these high level components share the same set of core Carbon modules. It is really exciting to see how those high level middleware solutions combine with the core Carbon framework to deliver the results. Sounds interesting? Feel like learning more? Here you can find an introductory presentation on WSO2 Carbon. It gives a good insight into this revolutionary product and its architecture. Also don't forget to checkout other related articles and presentations on WSO2 Oxygen Tank.

Going Undercover....

2008-12-09T21:02:00.000-08:00

I recently completed the video game 'Need for Speed Undercover', the latest member of the popular NFS series by EA Games. The game was so exciting and thrilling that I had to blog about it.

I have been a big fan of the NFS series since NFS 2. I really liked NFS Underground, NFS Underground 2 and NFS Most Wanted. NFS Underground was the first game in the NFS series to have a base storyline. Underground 2 and Most Wanted took the NFS series to a whole new level by making the storyline even more exciting and adding more features to the game. However I was a bit disappointed when NFS Carbon came out. It certainly did not live up to the hype and I found the game rather short and less challenging compared to its predecessors. Things got even worse when they released NFS Pro Street. To me Pro Street is the most unsuccessful game in the whole series. The whole point of playing a racing game on your PC or PS3 is to drive extremely fast and crash into things (which is something you cannot do in the real world without hurting someone or getting brain damage). But Pro Street does not allow the player to do that since going fast and crashing into objects would cause your car to get totaled and immobilized. Also fixing damaged cars will consume large sums of money the player has earned in the career mode of the game. And the biggest problem I had with NFS Pro Street was that it took the game back to the world of professional racing on closed racing circuits. For somebody who liked the virtual street racing experience offered by NFS Underground and NFS Most Wanted getting used to racing on closed tracks is almost like getting tortured.

However it seems the folks at EA have learned their lesson from the feedback they got from the fans for the NFS Pro Street. With NFS Undercover they have taken the action back into the open streets. Player gets to ride in a virtual city in some of the finest auto mobiles of the world. Couple of cars worth mentioning are Bugatti Veryon and the McLaren F1 which was last available in NFS 6. Cars are categorized into three, namely Japanese, European and American (see complete car list). However NFS Carbon fans will realize that these three categories effectively map to the three car categories in NFS Carbon (tuners, exotics, and muscles). The city has plenty of wide open roads where the player can really push the cars into their limits. Most of the tracks enable the player to accelerate the car to its top speed and hold that velocity.

Another aspect of the game I liked most was the soundtracks. Predecessors of NFS Undercover had mostly hard rock and technical death metal oriented sound tracks. But Undercover brings you a little bit of pop, a little bit of Latin and even a little bit of Spanish in addition to the rock stuff.

The underlying story of the game is perhaps the most interesting feature. In the previous versions of NFS the storyline was simple and very much predictable. In those games there is some kind of a bad ass, champion racer who bosses the other racers in the town (for example Razor in Most Wanted and Darius in Carbon). The player enters the world of racing as an amateur and starts to work his way up to become the new racing king of the town. In the process the player will have to confront many other racers, companions of the existing champion and sometimes even the police. But NFS Undercover brings you a totally different storyline. The story is full of action, suspense and even a bit of romance. The storyline clearly has some traces of 'Gone in 60 Seconds', 'Fast and the Furious' and 'CSI Miami'. Also the game enables the player to play the roles of the street racer, gang member, car thief as well as the federal agent.

The game is pretty fast paced and action packed. Graphics and the related effects are so natural and realistic. It's everything the racing gamers ever dreamed of. If you are like me and if you haven't played Undercover yet then go for it. It's totally worth it. Well done EA. You guys rock!

Also don't forget to check out NFS Planet for some cool Undercover wallpapers and movies.

You are not good
and you are not bad.
You blend and you listen
and you don't trust anyone.
Once you go undercover
you are on your own.

MOINC - The Future Web Services!!!

2008-10-23T00:57:00.000-07:00

As Web services continue to become the dominant paradigm in the arena of distributed computing, more and more people around the world are starting to get interested in improving the performance and scalability achievable from Web services deployments. An enterprise grade Web services deployment must be high available and scalable in order for it to be useful to the clients and provide a competitive advantage for the service provider. Due to this reason it has become a must for the Web services middleware providers to support high availability and scalability via general techniques such as clustering.

Me and some of my colleagues recently started a venture to explore the potential of grid computing as a means of improving the availability and scalability of Web services deployments. Our plan is to design and develop a complete open source Java Web services deployment platform which makes use of grid computing as the basis for clustering. This exciting concept is a brain child of one of our lecturers at the Department of Computer Science and Engineering, Dr. Sanjiva Weerawarana, who also happens to be the chairman and CEO of WSO2. This research project focuses on not only improving the scalability of Web services deployments via grid computing but also putting the much wasted electrical energy to good use by getting idling computers to do some useful work.

We have named the project 'MOINC' which stands for Mora Open Infrastructure for Network Computing. (If you are confused what 'Mora' is all about it's a little nick name for the University of Moratuwa. 'Mora' also means shark in Sinhala.) So far we have managed to successfully complete the requirements gathering and design phases of this massive R&D effort. We have identified four components in our target platform and four groups of undergrads are working on implementing the components. The four components are;

MOINC Server
MOINC Client Agent
MOINC Server Manager
Thisara Communication Framework

You can learn more about our project and its four main components from our official website. You will also find links to our design documents, proposals, specifications and SVN repositories there. We recently released the 0.1-alpha version of the Thisara messaging framework which is one of the four components of the MOINC platform. You can download the binaries from our website too.

Google Chrome, Looking Good

2008-10-10T07:34:00.000-07:00

It seems almost everybody is taking a strong liking into Google Chrome these days and almost everybody who has downloaded it seems to be appreciating it a lot. So I myself thought of giving Google Chrome a trial run.

However I have to say that my first impression on Google Chrome was not a very good one mainly due to the fact that it’s only available for the Windows platform at the moment. I’m an open source fan and that makes me naturally dislike anything that won’t run on Linux out of the box. However I think I can forgive Google for that. I believe they just wanted to keep the things simple by focusing on one target platform since this is only a beta release. Also they probably wanted to get the maximum possible test coverage out of this release. In that case Windows is the only logical solution since whether we like it or not Windows still dominates the client side system software market. (and it will probably remain that way for a long time :-( )

Other than the above mentioned glitch my experience with Google Chrome has been a fairly good one. It’s easy to install and configure. I really like the wording and labeling convention used in Chrome. It’s simple and casual English that anybody can understand. Not a single technical word is used so that even someone who is browsing the Internet for the first time can easily get used to Google Chrome. Even the buttons that are usually rendered as ‘browse’ buttons in other browsers will be rendered as ‘choose file’ buttons.

The user interfaces are pretty cool too. Chrome UI designers have clearly dumped the approaches taken by IE developers and Firefox developers to come up with their own style. It has no menu bars and no tool bars. All it has is the address bar with few more additional controls embedded into it. The address bar can be used to type in both URLs and search queries. One advantage of this organization is that it makes the browser’s main Window large so that the user can see more content without having to scroll much. But the problem is for someone who is so used to IE or Firefox, figuring out how to control the browser is going to be a bit of a pain.

In my opinion it has slightly better performance compared to Firefox-3 and IE-7. (well at least I feel that way) I have experienced frequent browser crashes with both IE-7 and Firefox-3. But I’m yet to experience that in Chrome.

Perhaps the most enticing feature of Chrome is that every time a new tab is opened a new process is forked off by the browser. This way each tab gets its own set of resources. This will hopefully rectify most of the memory related issues experienced by other browsers. If you are a regular Firefox user you know that as we continue to open new tabs in Firefox the overall performance of the browser degrades significantly. This is because all the tabs have to share the same set of resource (memory in particular). By dedicating a separate process for each tab Google Chrome effectively deals with this issue.

I will continue to test Google Chrome in the days to come. I just started with Chrome and I will be in a better position to give a comprehensive feedback after another couple of weeks. Until then ‘good job Google’!

Lex It! Yacc It!

2008-10-08T23:57:00.000-07:00

Lately I have been working a lot with the two legendary parser generator tools, ‘Lex’ and ‘Yacc’. (Well it’s actually ‘Flex’ and ‘Bison’ to be precise) To be honest I didn’t have any experience with Lex and Yacc two weeks back. But the two applications are so easy to learn and master that anyone could become an expert compiler developer in no time. Practically speaking one could start developing parsers and compilers without any prior knowledge on compiler theory or language parsing. All you need to get started is some experience in C programming. However I have to admit that some knowledge on regular expressions, context free grammars and push down automatons can make your life lot easier since that would help you better understand the syntax of the two configuration languages used by Lex and Yacc.

Lex (or Flex) generates a C function called yylex which is capable of scanning and tokenizing a given input. The input to Lex primarily consists of regular expressions and C code segments to be executed on detection of strings that match the regular expression definitions. Yacc (or Bison) generates a C function called yyparse which accepts a sequence of tokens from yylex and parses them. The YACC input would include the productions of the language grammar and C code segments to be executed on application of the productions.

It’s truly amazing how easy to develop parsers to accept complex languages with Lex and Yacc. The more I use it more I like it. The syntax of the configuration files are well structured and so easy to understand. I was so inspired by these two applications that I started developing my own command-line mathematics package for Unix/Linux systems. My plan is to develop something which is similar to the command interpreter that comes with MATLAB. I have already implemented support for executing simple mathematical expressions, trigonometric functions, logarithms and variables. I’m currently trying to implement the support for user defined functions. Once that is done I intend to implement support for matrix algebra and simple statistics.

The above sounds like a lot of work but thanks to Lex and Yacc I have to write only a very small amount of code. The two magic tools do the job by generating hundreds of lines of C code for me. Of course needless to say that developing something similar from the scratch would surely take months.

I’ve always wondered how come there are so many programming languages. But now I think I know the answer. It’s so freaking easy to develop parsers and compilers for a language with tools like Lex and Yacc. Once you have a language expressed in terms of a set of productions it takes only a few minutes to develop a fully functional compiler. So anyone can come up with an idea for a new language and turn it into reality in a flash. In fact I’m thinking about doing the same in the near future :-)

If you are like me and want to learn and master Lex and Yacc here is a cool reference guide. It helped me out a lot and I’m sure it will do the same for you.

GSoC 2008 ....... Done

2008-09-01T05:50:00.000-07:00

You may have noticed that my blog has gone rather silent in the last couple of months. Well, the reason was my Google Summer of Code (GSoC) project. The final deadline was on 18th of August so I worked really hard to deliver the goods on time and finally I can proudly state that I managed to successfully complete my project on time.

The objective of my GSoC project was to implement XML schema type alternatives support for Apache Xerces2/J, the legendary open source XML parser for Java applications. Type alternatives is the answer from W3C XML schema working group, for conditional type assignment problem. This feature was first introduced in the XML schema 1.1 structures specification. Type alternatives allow a type to be assigned to an element dynamically at validation time based on one or more conditions. Conditions are expressed as XPath 2.0 expressions. Here is an example element declaration which uses XML schema type alternatives.

As stated in the above code snippet the declared type of the message element is the complex type called messageType. However the element declaration also contains a few type alternatives. These type alternatives allow the type of the message elements to be determined at the validation time. Based on the conditions stated as the test attribute values the schema validator will assign a type for the message element prior to validating its content. The expressions '@kind' and '@code' refer to two attributes of the message element. The first type alternative will assign the type called messageTypeString if the kind attribute has the value 'string' and the code attribute has a value greater than 1000.

When the schema validator encounters an element declaration with one or more type alternatives it will evaluate the test expressions one by one until an expression which evaluates to true is found. When such a matching type alternative is found the corresponding type will be assigned to the element. If none of the type alternatives match then a default type will be assigned.

My project mainly consisted of two main sections. First section of the project was to implement the type alternatives traversal support so that Xerces2/J can properly traverse an XML schema document which contains type alternatives and add the corresponding information to the schema grammar. Implementing this was fairly easy and I managed to complete it prior to th GSoC mid term evaluations. The second part of the project was to implement type alternatives validation. This was fairly difficult since I had to develop a bare minimal XPath 2.0 implementation for Xerces2/J. Developing the XPath processor actually covered a significant portion of the entire project.

My workings will be fully available in the Xerces2/J code base (even now the code related to traversal part is in one of the SVN branches) very soon. All in all it was a great learning experience as it was a great opportunity for me to learn a whole bunch of cool technologies like XML, XML schema, and XPath 2.0. I also got the opportunity to put some of my knowledge on theory of computing into action and sharpen my programming skills. I would like to give my heartiest gratitude to the Google, the Apache community and very specially to my mentor Khaled Noaman for being a very supportive guide right from the start of my project.

Financial Applications with WSO2 ESB

2008-08-19T20:07:00.000-07:00

It's getting better and better folks. FIX protocol support in WSO2 ESB is evolving in a blistering rate. This article by Asanka Abeysinghe shows how an entire broker trading platform can be re-engineered using WSO2 ESB. It combines a set of well known usecases of the FIX transport of WSO2 ESB and demonstrates how the ESB can be used to build complete financial applications.

However when looking back at the way things have gone this is nothing to be surprised of. Few months back Paul Fremantle, CTO of WSO2, infact predicted that FIX protocol support in WSO2 ESB will soon evolve to this stage.

FIX Support in WSO2 ESB

2008-08-10T00:08:00.000-07:00

WSO2 ESB is an ultra fast, lightweight, open source ESB based on Apache Synapse. The FIX transport implementation of Apache Synapse is fully operational in WSO2 ESB 1.7. Asanka Abeysinghe from WSO2 has recently published a comprehensive article on using the FIX transport in WSO2 ESB. He starts by giving a short introduction to the FIX protocol and goes onto explaining a number of interesting usecases for the FIX transport in WSO2 ESB.

If you are looking for a powerful SOA based solution to deal with your FIX transactions, this is a must read for you. While you are at it don't forget to take a peek at the article titled 'Apache Synapse FIX'ed', and article I wrote few months back introducing the FIX transport implementation of Apache Synapse.

Fixing the FIX Transport

2008-08-09T05:54:00.001-07:00

The 1.2 version of Apache Synapse, the open source lightweight ESB was released on June 2008. One of the most striking features of this release was the FIX transport implementation, mainly due to the fact that most ESBs in the world still do not support the FIX protocol. The Synapse FIX transport implementation which is only two months old at the moment has already made a lot of hype in the world of Software Engineering and SOA. It seems we already got a few interested people (potential clients?) who are at the moment testing and playing with the transport.

Synapse 1.2 ships with two samples that demonstrate the FIX transport module. One of them demonstrates how two FIX endpoints can be bridged using Apache Synapse. The other samples shows how to bridge an HTTP client with a FIX endpoint. Over the last couple of months the Synapse community has worked really hard to further improve the FIX transport implementation and also to identify more exciting usecases for the transport module. As a result we managed to add four new samples demonstrating the FIX transport to the Synapse documentation. These samples are currently available in the Apache Synapse Snapshot and will be most likely included in the next release.

The first of the four newly added samples (sample 259) shows how to bridge a FIX client with an HTTP endpoint. The sample effectively demonstrates how Banzai, the sample FIX blotter that ships with Quickfix/J can be used to send order requests via Synapse to a service listening on HTTP. The sample uses the XSLT mediator to convert the FIX messages into a SOAP messages.

The second of the new samples (sample 260) shows how Synapse can be used to bridge a FIX endpoint with an AMQP endpoint. Here once the FIX message is converted into XML it will be bound to a JMS payload and sent to an AMQP consumer. Since AMQP is used widely in business applications, I believe that this sample will open up the door way to a ton of really cool usecases.

The third sample (sample 261) demonstrates how Synapse can be used to switch between FIX sessions with different versions (BeginString values). The sample successfully bridges a FIX 4.0 session with a FIX 4.1 session but the underlying concepts can be used to bridge virtually any two FIX sessions. One thing worth mentioning here is that the FIX transport implementation of Synapse initially did not support bridging FIX sessions of different versions. But considering the potential usecases, we implemented that feature very recently.

The fourth newly added sample (sample 262) demonstrates how CBR (Content Based Routing) can be done with FIX messages using Apache Synapse. The sample configuration causes Synapse to accept FIX messages over a session, read a certain symbol in the messages and based on the symbol value route the messages to different endpoints.

We also added namespace support to the FIX transport module so that it can properly parse and validate XML based FIX messages with namespaces. Another recent feature addition was improving the way FIX sessions are initialized in Synapse. The initial implementation lazy initializes all the FIX sessions for outgoing FIX messages. That means an outgoing session will not be created until a message arrives for that particular session. However since this leads to fairly large delays we improved the transport module so that the outgoing sessions are also initialized at the startup along with the incoming sessions. (the old way of initializing sessions is also supported)

Currently we are working on adding support for FIX repeated groups. (we already have a feature request for this on the JIRA from one of our users) All in all it seems that the FIX transport module for Synapse is increasingly becoming a very powerful and matured piece of software in a blistering rate.

Be Careful Using Ubuntu Update Manager

2008-06-26T06:43:00.000-07:00

Ubuntu Linux comes with a pretty cool Update Manager tool which searches for software updates and patches on the Internet and provides options to install them. I've been using this tool for months to install various updates on my system. The best thing about the Ubuntu Update Manager is that, it not only updates the OS but also updates the drivers and other utilities installed on the system. Everything seemed to work fine until I installed some kernel updates on my system using the Update Manager.

After applying the updates the Update Manager required a system restart to complete the update process. This has happened before so I patiently complied. But when the system rebooted I noticed that my system has gone dumb. Sounds were not working at all. And then I tried to connect to the Internet via Wi-Fi, only to find that my wireless network device is also screwed up. Luckily for me I have some friends who are professional Linux experts. I hooked up my computer to a wired network and contacted them over the Internet.

Fixing the wireless connectivity issue was not a big deal. The very Update Manager which got me into trouble helped me out. I installed a bunch of driver updates listed by the Update Manager. Then after a system reboot my Wi-Fi device was back up and running.

The difficult part was fixing the audio related issue. I had to fire up the synaptic package manager and install a couple of Linux backport modules and restart the system to get sounds working again.

However all in all it was yet another learning experience for me. If you are a regular Ubuntu Linux user like me, my advice to you is be careful when installing kernel patches using the Update Manager. Know what you are doing. Be ready to deal with some undesirable results.

WSO2 WSAS 2.3 Released

2008-06-11T03:55:00.000-07:00

The WSO2 WSAS team is pleased to announce the release of the WSO2 WSAS 2.3. WSO2 WSAS is an enterprise ready Web services engine powered by Apache Axis2 release under the Apache Software License 2.0.

This release can be downloaded from;
http://wso2.org/projects/wsas/java

From the WSO2 WSAS 2.3 - Release Note - 10th June 2008

WSO2 WSAS is an enterprise ready Web services engine powered by Apache Axis2 which offers a complete middleware solution. It is a lightweight, high performing platform for Service Oriented Architectures, enabling business logic and applications.

Bringing together a number of Apache Web services projects, WSO2 WSAS provides a secure, transactional and reliable runtime for deploying and managing Web services.

Key Features

* Data services support - Expose you enterprise data as a services in a jiffy
* WSAS IDE - Eclipse IDE integration
* Clustering support for High Availability & High Scalability
* Full support for WS-Security, WS-Trust, WS-Policy and WS-Secure Conversation and XKMS
* EJB service provider support - Expose your EJBs as services
* Axis1 backward compatibility - Deploy Axis1 services on WSAS & Engage advanced WS-* protocols in front of legacy services
* JMX & Web interface based monitoring and management
* WS-* & REST support
* GUI, command line & IDE based tools for Web service development

New Features In This Release

* Improved interoperability
* Improved Data Services support
* Various bug fixes to Apache Axis2, Apache Rampart & WSAS
* WSO2 Mercury Integration - A new WS-RM Implementation

Data Services - Bringing Enterprise Data to Web

* Service enable data locked in relational databases, CSV & Excel files in no time
* Zero code. Simple descriptor file describes the data to service mapping
* Controlled access to your data
* Customizable XML output
* Benefit from REST & WS-* support
* Built-in Connection pooling support
* Supports exposing Stored procedures & functions
* Built-in caching
* Throttling - to ensure your database is never overloaded.
* Easy configuration via graphical console
* Test your services via Try-it tool

Training

WSO2 Inc. offers a variety of professional Training Programs, including

training on general Web services as well as WSO2 WSAS, Apache Axis2, Data Services and a number of other products.

For additional support information please refer to
http://wso2.com/training/course-catalog/

Support

WSO2 Inc. offers a variety of development and production support programs, ranging from Web-based support up through normal business hours, to premium 24x7 phone support.

For additional support information please refer to http://wso2.com/support/

For more information on WSO2 WSAS, visit the WSO2 Oxygen Tank (http://wso2.org)

WSO2 ESB 1.7 Released

2008-06-10T03:32:00.000-07:00

The WSO2 Enterprise Service Bus (ESB) team is pleased to announce the release of its version 1.7 of the Open Source ESB.

The WSO2 ESB is an ultra fast, light-weight and versatile Enterprise Service Bus based on the Apache Synapse ESB. It allows you to Connect, Manage and Transform service interactions between Web services, REST/POX services and Legacy systems. You can easily switch transports between http/s, JMS, File Systems, Mail, FIX etc, or read/write from Databases, split, aggregate or clone messages and support declarative enforcement of QoS aspects such as WS-Security, WS-Reliable Messaging etc, and also switch between message formats such as SOAP 1.1/1.2, PoX/REST, Hessian, Text, Binary, MTOM and SwA.

The WSO2 ESB is released under the Apache Software License v2.0, and ships with a graphical management and administration console and enhanced JMX management/monitoring support, and integrates seamlessly with the WSO2 Registry.

Webinar series introducing the WSO2 ESB v1.7:
In this Webinar series Paul Fremantle, CTO of WSO2, will introduce the new features and capabilities of the WSO2 ESB. The first session will recap on the overall approach and benefits of the WSO2 ESB solution and the underlying Apache Synapse project, and then go into the added functionality and benefits of the 1.7 release. The series will include details of the newly released support for Hessian, FIX, AMQP and also discuss the improvements in performance and stability.

* For more details on the Webinar series, and to register,
visit http://wso2.com/about/news/esb-webinar-june-17/

Core features of the WSO2 ESB includes:
* Proxy services / Service mediation and Message mediation
* Support for Non-blocking http/s, JMS, FIX, Apache VFS (s/ftp, file,
zip/tar/gz, webdav, cifs..), POP3/IMAP/SMTP, AMQP transports
* Support for SOAP 1.1/1.2, PoX/REST, Hessian, Text and Binary payloads
* Support for scheduled task execution and management
* Support for custom extensions in Java through custom mediators, POJO
Classes and Java Command classes
* Support for Apache BSF Scripting languages such as (Javascript, Ruby,
Groovy..etc)
* Support for clustered deployment with pinned services and tasks
* Throttling, Caching, Load balancing and Failover support
* Support for declarative WS-Reliable Messaging, WS-Security and
WS-Policy attachment
* Integrated WSO2 Registry with support for external Registries
* Ability to stop, re-start and gracefully shutdown the ESB through JMX
* Cluster aware sticky load balancing support

New features of the v.1.7 release includes:
* Support for Hessian binary messages
* FIX (Financial Information eXchange) protocol transport
* WS-Reliable Messaging support with WSO2 Mercury
* Ability to stop, re-start and gracefully shutdown the ESB through JMX
* Integrated WSO2 Registry shipped, with ability to connect to a remote
WSO2 Registry
* Support for re-usable database connection pools for DB report/lookup
mediators
* Support for GZip encoding and HTTP 100 continue
* Natural support for dual channel messaging with WS-Addressing
* Cluster aware sticky load balancing support
* Non-blocking streaming of large messages at high concurreny with
constant memory usage
* Support for an ELSE clause for the Filter mediator
* Ability to specify XPath expressions relative to the envelope or body
* Support for separate policies for incoming/outgoing messages
* Support for a mandatory sequence before mediation
* New Router mediator
* Ability to re-deploy proxy services

Useful Links
Download WSO2 ESB - http://wso2.org/downloads/esb/
Quickstart Guide
Installation Guide
Administration Guide
Samples Guide
Documentation Index

Contribute to WSO2 ESB
SVN: http://wso2.org/repos/wso2/trunk/esb/java/
JIRA: http://wso2.org/jira/browse/ESBJAVA
User list: esb-java-user@wso2.org
Developer list: esb-java-dev@wso2.org
Web Forum: http://wso2.org/forum/187

Training
WSO2 Inc. offers a variety of professional Training Programs, including training on general Web services as well as WSO2 ESB, Apache Synapse and Axis2, Data Services and a number of other products. For additional support information please refer to http://wso2.com/training/course-catalog/

Support
WSO2 Inc. offers a variety of development and production support programs, ranging from Web-based support up through normal business hours, to premium 24x7 phone support. For additional support information please refer to http://wso2.com/support/ For more information on WSO2 ESB visit the WSO2 Oxygen Tank (http://wso2.org)

Apache Synapse 1.2 Released

2008-06-09T21:53:00.000-07:00

The Apache Synapse team is pleased to announce the release of version 1.2 of the Open Source Enterprise Service Bus (ESB).

Apache Synapse is an lightweight and easy-to-use Open Source Enterprise Service Bus (ESB) available under the Apache Software License v2.0. Apache Synapse allows administrators to simply and easily configure message routing, intermediation, transformation and logging task scheduling, etc.. The runtime has been designed to be completely asynchronous, non-blocking and streaming.

The Apache Synapse project and the 1.2 release can be found here:
http://synapse.apache.org

Apache Synapse offers connectivity and integration with a range of legacy systems, XML-based services and SOAP Web Services. It supports non-blocking HTTP and HTTPS using the Apache HTTPCore (http://hc.apache.org) components, as well as supporting JMS (v1.0 and higher) and a range of file systems and FTP sources including SFTP, FTP, File, ZIP/JAR/TAR/GZ via the Apache VFS project (http://commons.apache.org/vfs/filesystems.html).

At the same time Synapse 1.2 release adds the support for the Financial Information eXchange (FIX) an industry driven messaging standard through QuickFixJ, Hessian binary web service protocol, as well as other functional, stability and performance improvements. Synapse supports transformation and routing between protocols without any coding via configurable virtual services.

Synapse provides first class support for standards such as WS-Addressing, Web Services Security (WSS), Web Services Reliable Messaging (WSRM), Throttling and caching, configurable via WS-Policy upto message level, as well as efficient binary attachments (MTOM/XOP).

The 1.2 release contains a set of enhancements based on feedback from the user community, including:

* Support for Hessian binary web service protocol
* FIX (Financial Information eXchange) protocol for messaging
* WS-Reliable Messaging support with WSO2 Mercury
* Support for re-usable database connection pools for DB report/lookup mediators
* Support for GZip encoding and HTTP 100 continue
* Natural support for dual channel messaging with WS-Addressing
* Cluster aware sticky load balancing support
* Non-blocking streaming of large messages at high concurreny with constant memory usage
* Support for an ELSE clause for the Filter mediator
* Ability to specify XPath expressions relative to the envelope or body
* Support for separate policies for incoming/outgoing messages
* Support for a mandatory sequence before mediation

The combination of XML streaming and asynchronous support for HTTP and HTTPS using Java NIO ensures that Synapse has very high scalability under load. Performance tests show that Synapse can scale to support thousands of concurrent connections with constant memory on standard server hardware.

Apache Synapse ships with over 50 samples (http://synapse.apache.org/Synapse_Samples.html) designed to demonstrate common integration patterns "out-of-the-box", along with supporting sample services, and service clients that demonstrate these scenarios. Apache Synapse is configured using a straightforward XML configuration syntax
(http://synapse.apache.org/Synapse_Configuration_Language.html).

Apache Synapse is openly developed by a community that welcomes all forms of input, ranging from suggestions and bug reports to patches and code contributions. Your comments and feedback on the project and release are welcome.

The Apache Synapse code and binaries are available from the website at http://synapse.apache.org

Having Issues with Ubuntu Visual Effects?

2008-06-03T23:12:00.000-07:00

I have been experiencing issues activating Ubuntu visual effects on my laptop. I installed Ubuntu Gutsy on my laptop few months back (yes, I'm yet to upgrade to Ubuntu Hardy) and it works fine except that it never allowed me to turn the visual effects on. I have seen these visual effects in action on other computers and they are really exciting. I quite haven't seen anything similar even in the Windows world. So I badly wanted to use visual effects on my personal laptop but strangely it never allowed me. Whenever I tried to activate visual effects, Ubuntu gave me an error simply saying something like 'unable to turn visual effects on'. (wow! what a descriptive error message, isn't it?)

So after a bit of googling and reading some forum entries I managed to diagnose the problem. The issue was with my VGA card which is treated as 'blacklisted' by Linux. This particular type of VGA (Intel Corporation Mobile GM965/GL960 Integrated Graphics Controller) is being used in a variety of laptops these days and hence I thought of blogging a bit about this and let the world know. If you don't know how to find the type of VGA in your computer simply enter the following command in a shell.

$ lspci | grep VGA

In order to find whether your VGA is blacklisted or not run the following command.

$ compiz --replace ccp &

If your VGA is blacklisted you will see something like this on the terminal.

[1] 6002
Checking for Xgl: not present
Blacklisted PCIID '8086:2a02' found
aborting and using fallback: /usr/bin/metacity

If this is the case you can do a simple override so that Linux won't care whether your VGA is blacklisted or not. Then you can turn the visual effects on and off at will. To perform the override run the following command. (this workaround has been tested on the above mentioned VGA only, so beware and attempt at your own risk!)

$ mkdir -p ~/.config/compiz; echo SKIP_CHECKS=yes >> ~/.config/compiz/compiz-manager

This will create a small configuration file named compiz-manager in the ~/.config/compiz directory. This file will instruct Linux to skip certain tests which cause the failures in turning the visual effects on.

And that's basically it. If you need more information please take a look at this forum post.

University of Moratuwa Climbs to the Top with GSoC 2008

2008-05-18T23:18:00.000-07:00

For years University of Moratuwa in Sri Lanka has been doing a kick-ass job in the process of converting talented youngsters in Sri Lanka into world class computer engineers and IT professionals. It is not a secret that out of all the IT education centers in the country University of Moratuwa is the best in the business. Graduates passed out from the University of Moratuwa can be found virtually in any software firm in Sri Lanka and every year a significant number of University of Moratuwa graduates end up joining some of the finest organizations and universities in the world.

Talking about kicking butt, this time around University of Moratuwa has managed to kick some really serious butt big time. Being able to secure the first place in the Google Summer of Code 2008 (GSoC 2008) top participants list is no ordinary thing. According to the latest statistics from Google it is from the University of Moratuwa, Sri Lanka that the largest number of GSoC 2008 applications has been received. The blistering number of applications is 93! The second place of this list goes to University of Campinas, Brazil. 29 applications have been received from University of Campinas. So clearly University of Moratuwa is way ahead from the rest of the world with this regard.

Also Google has declared that University of Moratuwa has forwarded the largest number of winning GSoC 2008 applications. There are 23 winning applications from University of Moratuwa this year from which 18 are from the department of Computer Science and Engineering. I personally feel very proud of this achievement and feel really happy about myself for being able to be one of those 18 winners.

Sri Lanka is among the top 10 GSoC 2008 participant countries this year. Sri Lanka is holding this position along with countries like USA, Canada and India who are at the frontier of the global IT industry. However out of all the IT education institutes in Sri Lanka only University of Moratuwa has earned a position in the top 10 GSoC 2008 participant institutes list. This is an clear indication of the level of excellence University of Moratuwa has reached over the past couple of decades in terms of quality of education. In addition to that this achievement by the University of Moratuwa is an indication of the fact that a new generation of computer engineers and IT professionals who appreciate and support open source software is being created at Sri Lanka.