High Volume Web API management with WSO2 ESB

During last year's WSO2Con, Paul Fremantle and I co-presented on the topic "High Volume Web API management with WSO2 ESB". There we discussed how WSO2 ESB is specifically designed to handle large volumes of traffic received over many concurrent connections. We explained the multithreading, streaming and non-blocking IO aspects of the ESB while looking at some of the high performance extensions we offer such as the message relay and the pass through HTTP transport. We also went through some ESB and JVM performance tuning basics taking the WSO2 ESB deployment at eBay as a case study.

WSO2 ESB Tips & Tricks 08: Message Stores and Processors

Ability to store a message in a temporary storage is a fundamental requirement in many enterprise integration scenarios. There are many occasions where a systems integrator would have to design a message flow which simply dumps all the messages into some kind of a storage facility for later retrieval. Here are several scenarios that I’ve encountered while working on real-life integration projects which required me to consider using an intermediate message store:

Matching Request Rates

Some services can only process requests at a given rate. But the service consumers, who are generally unaware of the service implementation details, may send requests at any rate they prefer. In this case we should not directly expose the service to consumers who might overwhelm the service by sending too many requests. Therefore consumers should be provided with a virtual service interface (a proxy service) which stores all incoming requests in a persistent store. A separate component would retrieve these messages from the temporary store and play them at the actual service at the preferred rate. This type of a messaging system is generally known as a store-and-forward messaging system.

Guaranteed Delivery

In certain integration scenarios it is of utmost importance that all requests are received and processed by the back-end services (i.e. no message loss). However this is a rather tricky requirement to implement as the back-end services could be unavailable at times due to various reasons like maintenance activities and hardware faults. Here also we can use a persistent message store to solve the problem. We can configure the messaging system or the middleware to detect communication errors that may occur while sending requests and then save the failed requests in a message store for later delivery. Depending on the availability of tools an automated retry mechanism or a manual retry mechanism can be implemented for the failed messages. In this case the persistent store is acting as a dead letter channel.

In-Order Delivery

Some scenarios require that systems consume messages in the exact order they were produced. In such situations respective middleware should be programmed to deliver the requests in the received order. If a request fails due to a temporary outage, the middleware should not send the rest of the messages because that would disrupt the order at which the messages are consumed. The middleware should retry and make sure the failed message is indeed received by the back-end system before the rest of the messages can be sent. A temporary message store is a natural fit for implementing a solution of this nature. However unlike in the previous cases, the message store should be capable of retaining the order in which messages are stored. In general it should be a message queue which provides basic FIFO semantics. The middleware would enqueue all the incoming requests in the store and a different component will pick them up, one at a time, for delivery. In order to preserve the order of messages under error conditions, the delivery component should not remove a message from the store until a satisfactory response has been received from the back-end system for each message sent. This type of systems also falls under the store-and-forward category.

Separation of Concerns

Sometimes it is necessary to run messages through expensive logging and audit routines. However such additional processing can put a lot of overhead on message flows thus contributing to huge delays and sluggish user experience. Using a message store, it is possible to decouple concerns like logging from the fundamental requirements like routing. For an example middleware can be configured to create a copy of each incoming message and dump that into a temporary store. The original message can be routed through the middleware to the desired endpoint without any additional processing. The logging and audit systems can asynchronously consume the messages in the store and perform all the necessary processing on them off-line. This is one of the simplest as well as most common uses of a message store. In fact the standard enterprise integration pattern named "Message Store pattern" describes this very specific use case of intermediate message stores.

As seen from the above examples, the concept of message stores can be used to solve countless enterprise integration problems. The asynchronous and persistent messaging patterns that result from message stores are at the crux of modern message-oriented middleware (MOM). It is an accepted norm that message stores are ideal for constructing complex distributed systems while keeping the overall architecture cohesive but loosely coupled.

Starting from version 4.0.0, WSO2 ESB has built-in support for creating and managing message stores. In fact we took a revolutionary step forward in the enterprise integration space by introducing the “message store – message processor” architecture to WSO2 ESB. At high level a message store is a logical entity in which messages can be stored for later retrieval. Additionally a message store may provide some QoS features such as persistence, message order retention and random access support. The exact runtime behavior and the features offered by a message store are determined by the message store implementation being used. WSO2 ESB currently ships with following two message store implementations.

In-Memory Message Store	Stores messages in an in-memory queue. Not persistent. Supports random access. Retains message order.
JMS Message Store	Stores message in a JMS queue (works with any JMS/JNDI compliant broker). Persistent. Does not support random access. Retains message order.

WSO2 ESB also provides an API using which a user can develop his/her own message store implementations. For an example one can come up with a file system based message store, a JDBC based message store or a NoSQL based message store and use them in WSO2 ESB. XML syntax for configuring a message store is as follows:

<messageStore name="string" class="classname" >
  <parameter name="string" > "string" </parameter>*
</messageStore>

The store mediator is used to enqueue messages in a given message store. This mediator can be used in any sequence or proxy service. Therefore any message flow in WSO2 ESB runtime can be linked up with a message store. By using the store mediator in a fault sequence a message store can be used as a dead letter channel.

<store messageStore="storename"/>

Message stores in WSO2 ESB can be managed via the ESB management console or JMX. One may list the available stores, browse the objects stored in them and perform various operations on the stored messages using the above techniques.

Message processors work hand-in-hand with message stores. They are used to consume messages stored in message stores. Every message processor must be associated with an existing message store. Similar to the case with message stores, there can be multiple message processor implementations. The exact behavior and functionality provided by a processor is determined by its type. Currently WSO2 ESB ships with following two message processor implementations:

Message Forwarding Processor	Retrieves the messages stored in a message store and reliably forwards them to a specified endpoint. This processor attempts to send one message at a time and it does not dequeue a message from the store until it receives a response from the target endpoint. Therefore this processor is ideal for implementing in-order delivery scenarios and guaranteed delivery scenarios.
Sampling Processor	Retrieves the messages stored in a message store and injects them to a given sequence at specified intervals. This processor utilizes the Quartz scheduler framework for periodically processing messages. This can be used to implement message rate throttling scenarios.

Message processors also come with an API using which any custom message processor can be implemented. A message processor is configured as follows in the WSO2 ESB configuration:

<messageProcessor name="string" class="classname" messageStore="storename">
  <parameter name="string" > "string" </parameter>*
</messageProcessor>

As you can see the message store-message processor architecture and the associated API form a very powerful framework on which a wide range of enterprise messaging scenarios can be implemented. Generally speaking, any integration scenario that involves asynchronous or persistent messaging can be easily realized using these components of WSO2 ESB. We have already received some very positive feedback on this architecture from our users and I believe in the long run this will become one of the most widely used features of our middleware stack.

If you are interested in leaning more about the "message store-message processor" architecture of WSO2 ESB following resources will provide some useful details:

• Official documentation: Message stores
• Official documentation: Message processors
• Article: Implementing store and forward messaging patterns with WSO2 ESB - part 1
• Article: Implementing store and forward messaging patterns with WSO2 ESB - part 2

WSO2 ESB 4.0.3 is Out with REST API Support

WSO2 ESB 4.0.3 is now available for download. This contains several bug fixes over WSO2 ESB 4.0.0 and 4.0.2 releases. In addition to that this release comes with the highly anticipated REST API management support. This is a very powerful feature which I believe will be a game changer. I'm planning to do a series of blog posts and articles in the near future describing various use cases and capabilities of this brand new feature. At WSO2 we have already done some public webinars, workshops and development work for actual customers using the REST API support in the new ESB release and the feedback we have received so far has been phenomenal.

REST support is nothing new for WSO2 ESB. Even though we are primarily based on Apache Axis2, which is a SOAP engine, we have always had support for exposing RESTful interfaces and mediating RESTful invocations. However REST support in WSO2 stack was always a bit of a black sheep in the family. More often than not REST was perceived as a second class citizen in the WSO2 stack. But the new REST API support we have implemented changes all that. Now REST fans can easily define REST APIs and associated resources on the ESB and combine them with the powerful mediation features provided by the underlying messaging framework. Instead of worrying about mapping REST concepts to SOAP (which was the old way of handling REST calls with proxy services), developers can now configure REST endpoints in the ESB by directly specifying HTTP verbs, URL patterns (servlet styled), URI templates, HTTP media types and other related headers. Most REST fans will find themselves at home when working with the new API support in WSO2 ESB.

That's only a beginning. We are also looking into combining some of our best products together (including ESB of course) to come up with a comprehensive API management solution. This will be a complete platform where developers will be able to develop and expose REST APIs and users can subscribe and consume them.

As shown in the above diagram, core API management features such as API key management, user management, security and SLA monitoring are all provided by the solution itself. We are also looking at how to leverage our multi-tenant support to make this solution work seamlessly in the cloud.

If you are interested in learning more about the new REST API support in WSO2 stack and our future plans in this area, we are conducting a series of workshops on this very topic in US later this month. I'm sure you will find these sessions both interesting and eye opening. Also stay tuned for my own blog posts and articles on the subject which will give a more in-depth technical insight on the matter.