Is Subversion Going to Make a Come Back?

The Apache Software Foundation (ASF) announced the release of Subversion 1.8 yesterday. As I started to read the release note, I started wondering how come Subversion is still alive. The ASF heavily use Subversion for pretty much everything. In fact the source code of Subversion is also managed using a Subversion repository. But outside the ASF I've seen a strong push towards switching from Subversion to Git. Most startups and research groups that I know of have been using Git from day one. WSO2, the company I used to work for, is in the process of moving their code to Git. Being an Apache committer I obviously have to use Subversion regularly. But about a year ago I started using Git (GitHub to be exact) for my other development activities, and I absolutely adore it. It scales well for large code bases and large development teams, and it makes common tasks such as merging, reverting, reviewing other people's work and branching so much easier and intuitive. 

But as it turns out Subversion is still the world's most widely used source version control system. As declared in the official blog post rolled out by the ASF yesterday, a number of tech giants including WordPress heavily use Subversion. According to Ohloh, the percentage of open source projects that use Subversion is around 53%, compared to the 29% that use Git. Looks like Subversion has managed to capture quite a share of the market making it a very hard-to-kill technology. It would be interesting to see how the competition between Subversion and Git would unfold in the future. It seems the new release comes with a bunch of new features, which indicates that the project is very much alive and kicking and the Subversion community is not even close to giving up on the project.

How the World's Fastest ESB was Made

A couple of years ago, at WSO2 we implemented a new HTTP transport for WSO2 ESB. Requirements for this new transport can be summarized as follows:

1. Ultra-fast, low latency mediation of HTTP requests.
2. Supporting a very large number of inbound (client-ESB) and outbound (ESB-server) connections concurrently (we were looking at several thousand concurrent connections).
3. Automatic throttling and graceful performance degradation in the presence of slow or faulty clients and servers.

The default non-blocking HTTP (NHTTP) transport from Apache Synapse, which we were also using in WSO2 ESB, supported the above requirements up to a certain extent but we wanted to do better. The default transport was very generic and it was designed to offer reasonable performance in all the integration scenarios the ESB could potentially participate in. However HTTP load balancing, HTTP URL routing (URL rewriting) and HTTP header-based routing are some of the most widely used integration patterns in the industry and to support these use cases well, we needed a specialized transport. 

The old NHTTP transport was based on a dual buffer model. Incoming message content was placed in a SharedInputBuffer and the outgoing message content was placed in a SharedOutputBuffer. Apache Axiom, Apache Axis2 and the Synapse mediation engine sit between the two buffers, reading from the input buffer and writing to the output buffer. This architecture is illustrated in the following diagram.

The key advantage of this architecture is that it enables the ESB (mediators) to intercept all the messages and manipulate them in any way necessary. The main downside is every message happens to go through the Axiom layer, which is not really necessary in cases like HTTP load balancing and HTTP header-based routing. Also the overhead of moving data from one buffer to another was not always justifiable in this model. So when we started working on the new HTTP transport we wanted to get rid of these limitations. We knew that this might result in a not-so-generic HTTP transport, but we were willing to pay that price at the time.

So after some very interesting brainstorming sessions, an exciting 1-week long hackathon followed by several months of testing, bug-fixing and refactoring we came up with what’s today known as the HTTP pass-through transport. This transport was based on a single buffer model and completely bypassed the Axiom layer. The resulting architecture is illustrated below.

The HTTP pass-through transport was first released in June 2011 along with WSO2 ESB 4.0. Back then it was disabled by default and the user had to enable it by uncommenting a few entries in the axis2.xml file. The performance numbers we were seeing with the new transport were simply remarkable. WSO2 also published some of these benchmarking results in a March 2012 article. However at this point the 2 main limitations in the new transport were starting to give us headaches.

1. Configuration overhead (Users had to explicitly enable the transport depending on their target use cases)
2. Cannot support any integration scenario that requires HTTP content manipulation (because Axiom was bypassed, any mediator attempting to access the message payload would not get anything useful to work with)

In addition to these technical issues there were other process related issues that we had to deal with. For instance maintaining two separate HTTP transports was twice as work for the developers and testers. We found that because the pass-through transport was not used as the default, it often lagged behind the default NHTTP transport in terms of features and stability. So after a few brainstorming sessions we decided to try and make the pass-through transport the default HTTP transport in Apache Synapse/WSO2 ESB. But this required making the content manipulation use cases (content aware use cases) work with the new transport. This implied bringing Axiom back into the picture, the very thing we wanted to avoid in our initial implementation. So in order to balance out our performance and heterogeneous integration requirements we came up with the idea of “on-demand message parsing in the mediation engine”.

In this new model, each mediator instance belongs to one of two classes.

1. Content-unaware mediators – Mediators that never access the message content in anyway (eg: drop mediator)
2. Content-aware mediators – Mediators that always access the message content (eg: xslt mediator)

We also identified a third class known as conditionally content-aware mediators. These mediators could be either content-aware or content-unaware depending on their exact instance configuration. For an example a simple log mediator instance, configured as <log/> is content-unaware. However a log mediator configured as <log level=”full”/> would be content-aware since it’s expected to log the message payload. Similarly a simple property mediator instance such as <property name=”foo” value=”bar”/> is content-unaware but <property name=”foo” expression=”/some/xpath”/> could be content-aware depending on what the XPath expression does. In order to capture this content-awareness characteristic of mediator instances at runtime, we introduced a new method (isContentAware) to the top level Mediator interface of Synapse. The default implementation in AbstractMediator class returns true by default so as to maintain backward compatibility. 

With this change in place we modified the mediation engine to check the content-awareness of property of each mediator at runtime before submitting a message to it. List mediators such as the SequenceMediator would run the check recursively on its child mediators to obtain the final value. Assuming that messages are always received through the pass-through HTTP transport, the mediation engine would invoke a special message parsing routine whenever a mediator is detected to be content-aware. It is in this special routine that we bring Axiom into the picture. Therefore if none of the mediators in a given flow or a service is content-aware, the pass-through transport works as it usually does without ever engaging Axiom. But whenever a content-aware mediator is involved, we bring Axiom in. This way we can reap the performance benefits of the pass-through transport while supporting all integration scenarios of the ESB. Since we engage Axiom on-demand we get the best possible outcome for all scenarios. For instance a simple pass through proxy would always work without any Axiom interactions. An XSLT proxy that transforms requests would engage Axiom only in the request flow. Response flow would operate without parsing the messages.

Another tricky problem we encountered was dealing with message parsing itself. For instance how do we parse a message and then send it out when there is only one buffer provided by the underlying pass-through transport? Ideally we need two buffers to read the incoming message from and write the outgoing message to. Also the fact that the Axis2 message builder framework can only handle streams posed a few problems. The buffer we maintained in the pass-through transport was a Java NIO ByteBuffer instance. So we needed to adapt the buffer into a stream implementation whenever the mediation engine engages Axiom. We solved the first problem by implementing our message builder routine to create a second output buffer whenever Axiom is dragged into the picture. The outgoing messages are serialized into this second buffer and the pass-through transport was modified to pick the outgoing content from the second buffer when it’s available. Writing an InputStream implementation that can wrap a ByteBuffer instance solved the second problem.

One last problem that needed to be solved was handling security. In Synapse/WSO2 ESB, security is handled by Apache Rampart, which runs as an Axis2 module that intercepts the messages before they hit the mediation engine. So on-demand parsing at the mediation engine doesn’t work in this scenario. We need to parse the messages before Rampart intercepts them. We solved this issue by introducing a new smart handler to the Axis2 handler chain, which intercepts every message and performs an early parse if security is engaged on the flow. The same solution can be extended to support other modules that require parsing message payload in the Axis2 handler chain.

The reason I decided to compile this blog is because WSO2 folks just released WSO2 ESB 4.6. And this release is based on the new model I’ve described here. Pass-through transport is what the users now get by default. The WSO2 team has also published some performance figures that clearly indicate what the new design is capable of. It turns out the latest release of WSO2 ESB outperforms all the major open source ESB vendors by a significant margin. This release also comes with a new XSLT mediator (Fast XSLT) that operates on the top of the pass-through model of the underlying transport and a new streaming XPath implementation based on Antlr.

The next step of this effort would be to get these improvements integrated into the Apache Synapse code base. This work is already underway and you can monitor its progress through SYNAPSE-913 and SYNAPSE-920.

Busting Synapse and WSO2 ESB Myths

Paul Fremantle, PMC chair of the Apache Synapse project and CTO of WSO2, has written a very interesting blog post addressing some of the myths concerning Apache Synapse and WSO2 ESB. As of now both projects are quite popular, mature and have a very large user base including some of the largest organizations in the world. Surprisingly there are still some people who believe that these projects do not fall under the category of ESB (Enterprise Service Bus) implementations. In his latest post, Paul gives a clear and complete answer to all these misbeliefs, and backs it up with a wide range of facts.

ESB is one of those things in the IT world which don't have a proper standard definition. The best definitions I've come across attempt to align the term along the following cues:

• An architectural construct that provides fundamental services to complex architectures
• An entity that acts as a hub connecting many diverse systems
• A central driver that facilitates Enterprise Application Integration (EAI)

Apache Synapse and WSO2 ESB pass with flying colors on all the above criteria. They provide an array of fundamental services to the systems and architectures that rely on them.  Some of these basic services are:

• Message passing, routing and filtering
• Message transformation
• Protocol conversion
• QoS enforcement (security, reliable delivery etc)
• Logging, auditing and monitoring

Because Synapse and WSO2 ESB do such a good job providing these fundamental services, they can be used to integrate a large number of heterogeneous systems in an enterprise setting. As Paul has also pointed out in his post, Synapse and WSO2 ESB are currently used in hundreds of production deployments all around the world to connect various applications, implemented using various technologies (both open source and proprietary) running on various platforms (Windows, Linux, .NET, J2EE, LAMP, cloud..you name it). In other words Synapse and WSO2 ESB are widely used as centralized drivers that facilitate EAI. The configuration model of Synapse and WSO2 ESB is so agile and powerful that practically any EAI pattern can be implemented on top of them. In fact there are tons of samples, articles and tutorials that explain how various well-known EAI patterns can be implemented using these 'ESB implementations'.

One thing that I've learnt from writing code to Synapse is that it has a very flexible enterprise messaging model. Support for any wire level protocol or any message format can be easily implemented on top of this model and can be deployed as a separate pluggable module. During the last few years, I myself have contributed to the implementation of following adapters/connectors on various occasions:

• FIX transport
• SAP transport (IDoc and BAPI support)
• MLLP transport and HL7 message formats
• CSV and various other office document formats
• Thrift connector
• Numerous other custom binary protocols based on TCP/IP

This is just a bunch of stuff that I've had the privilege of implementing for Synapse/WSO2 ESB. I know for a fact that other committers of Synapse and WSO2 ESB have been working on supporting dozens of other protocols, message formats and mediators. Thanks to all this hard work Synapse and WSO2 ESB are currently two of the most powerful and feature-complete ESB implementations anyone will ever come across. Also the existence of connectors for so many protocols and applications is a testament to the agility and flexibility that Synapse and WSO2 ESB can bring in as ESB products.

Another aspect of Synapse/WSO2 ESB that has been questioned many times is their ability to support RESTful integrations (Paul also addresses this issue in his post). This confusion stems from the fact that Synapse uses SOAP as its intermediary message format. Without going into too many technical details, I'd just like to point out that one of the largest online marketplace and auctioning providers in the world uses Synapse/WSO2 ESB to process several hundred millions of REST calls in a daily basis. The new API support we have implemented in Synapse makes it absolutely simple to design, implement and expose RESTful APIs on Synapse/WSO2 ESB. In fact I recently published an article which demonstrates through practical examples how powerful RESTful applications can be implemented using Synapse/WSO2 ESB while supporting advanced REST semantics such as HATEOAS. The recently released WSO2 API Manager product which supports exposing rich web APIs with support for API key management is also based on Synapse/WSO2 ESB.

I think I have made my point. Both Synapse and WSO2 ESB are two excellent ESB choices if you're looking to adopt SOA or enterprise integration within your organization. Their wide range of features is only second to the very high level of performance they offer in terms of high throughput and low resource utilization. Please also go through the post made by Paul, where he has explained some of the above issues with low level technical details. I particularly like his analogy concerning Heisenberg's principle of uncertainty :) 

Apache Synapse 2.0 Released

The Apache Synapse team reached a very important milestone last week. After a long but very busy 2 and half years, we announced the general availability of Apache Synapse version 2.0. The previous release of Synapse was version 1.2 which was released way back in year 2008. The Apache Synapse project has grown a lot in terms of code, features and community over the past 2 years and therefore you will find a horde of new features, improvements, bug fixes and samples in the latest release. Some of the noteworthy new features in this release are:

• New, fine-grained configuration model
• Hot deployment and hot update support for configuration artifacts
• Priority based mediation support
• Comprehensive eventing capabilities with WS-Eventing support
• Secure vault for encrypting passwords in configuration files
• File locking support in the VFS transport for concurrent polling
• URLRewrite mediator for fast and simple URL rewriting
• Synapse configuration observer API
• Multiple identity support in the HTTPS transport
• Enhanced JMX monitoring support for the NHTTP transport
• Dead letter channel implementation (experimental)
• Synapse XAR Maven plug-in for generating configuration artifacts

Apache Synapse is a lightweight Enterprise Service Bus (ESB) released under the Apache Software License. It is based on the tried and tested Apache Axis2 SOAP engine. It has excellent support for SOAP, REST, POX, JSON and a variety of wire level transports (HTTP/S, JMS, File transport, FIX...). The non-blocking HTTP transport of Synapse enables it to handle very high volumes of HTTP traffic over thousands of concurrent connections.

Apache Synapse also supports many WS-* standards including WS-Security, WS-ReliableMessaging, and WS-Policy. It can be easily linked up with any SOA registry for metadata management and governance purposes. Synapse has a very flexible configuration model built into it, which makes it one of the easiest ESB applications to learn and adopt.

If you are involved in any SOA projects or system integration activities, Synapse may have something to offer. So please feel free to grab the binary distro and take it for a spin. Feed back welcome on Synapse user list.

Apache Xerces2/J 2.11.0 Released

The Apache Xerces project team is pleased to announce that version 2.11.0 of Apache Xerces-J is now available.

Xerces-J 2.11.0 can be downloaded at: http://xerces.apache.org/mirrors.cgi

This release expands on Xerces' experimental support for XML Schema 1.1 (see Xerces-J-bin.2.11.0-xml-schema-1.1-beta.zip) by providing implementations for the simplified complex type restriction rules (also known as subsumption), xs:override and a few other XML Schema 1.1 features. Xerces-J 2.11.0 also introduces experimental support for XML Schema Component Designators (SCD). It fixes several bugs which were present in Xerces-J 2.10.0 and also includes a few other minor enhancements.

Specifically, the changes introduced in this release are:

• Implemented XML Schema 1.1's simplified complex type restriction rules (also known as subsumption)
• Added support for XML Schema 1.1's overriding component definitions (<xs:override>)
• Added experimental support for a parser and evaluator for XML Schema Component Designators (SCD)
• Implemented support for the vc:typeAvailable, vc:typeUnavailable, vc:facetAvailable and vc:facetUnavailable attributes for conditional inclusion processing in XML Schema 1.1 documents
• Implemented support for allowing xs:group as a child of xs:all in XML Schema 1.1 documents
• Made several enhancements to the support for assertions in the XML Schema 1.1 implementation
• Improved the ways in which the XML Schema API exposes values and identity constraint information
• Fixed a bug where XMLSchemaValidator.findSchemaGrammar() was not getting called for substitution groups
• Fixed a bug in the regular expression support which was introduced in the previous release. See JIRA Issue XERCESJ-1456 for details
• Fixed multiple issues with decimal precision in the processing of xs:precisionDecimal value
• Fixed various bugs and made various improvements

A complete list of JIRA issues resolved in this release is available here: http://s.apache.org/pI

For more information please visit: http://xerces.apache.org/xerces2-j/

ASF Says "It's On"

As most of you know there's an ensuing battle between the Apache Software Foundation (ASF) and Oracle over Java and Apache Harmony. The roots of this conflict go all the way back to the time when Java was mainly driven by Sun Microsystems. However with ASF getting re-elected as a ratified JCP executive committee member, things are starting to take an interesting turn. A couple of days back ASF made a bold statement indicating that they are going to vote down the upcoming Java SE 7, which Oracle is pushing forward so hard. ASF is also urging other JCP members to back ASF on this. ASF has indicated that they are not afraid to terminate their ties with JCP if needed. This is pretty huge for a group that maintains an overwhelming number of world class Java projects including Tomcat, Maven, Geronimo, Ant and Derby. (I also happen to be a committer for a couple of Java projects at ASF)

There are currently so many news articles on the web covering this conflict end-to-end. I found the news report from The Register to be quite interesting and amusing at the same time. Here's a little excerpt:

Oracle was forced to eat crow and congratulate ASF for its near unanimous election back onto the JCP's SE and EE executive committee. It was an election that saw its own nominee - the little-known-outside-Oracle-circles Hologic - soundly rejected by JCP members.

Congratulating ASF, Oracle's spokesperson on Java SE, Henrik Ståhl, claimed Oracle still respected ASF and wanted to work with the group. "Our disagreement around TCK licensing does in no way lower our respect for and desire to continue to work with Apache," Ståhl said here.

On Tuesday, the ASF told Ståhl and Oracle just exactly where they could shove their "respect" with a statement saying it is walking out of the JCP unless Harmony gets a license.

Another interesting news report is available on IT World.

Axis2 TCP Transport Revamped

I recently did some major improvements to the Axis2 TCP transport. The TCP transport enables Axis2 services and clients to send/receive SOAP messages over TCP, using Java TCP sockets. The old TCP transport was very simple and it must be configured globally in the axis2.xml file. Due to this limitation, a service could not open up its own port to listen to incoming TCP messages. All the TCP requests were captured by a single, globally configured port and WS-Addressing headers were used to dispatch the requests to the appropriate services.

The new transport implementation is fairly advanced with a wide range of configuration options. It can be configured globally in the axis2.xml file, or it can be configured at the service level in the corresponding services.xml files. Only the port number was configurable in the previous TCP transport implementation. The new implementation supports all the parameters described below:

• transport.tcp.port - Port number (mandatory parameter)
• transport.tcp.hostname - The host name to which the server socket should be bound
• transport.tcp.backlog - The length of the message back log for the server socket (defaults to 50)
• transport.tcp.contentType - Content type of requests (defaults to text/xml)

In the new transport, if a request is received by a port configured at the service level, it is pre-dispatched to the corresponding service. If the global port receives a TCP message, WS-Addressing headers will be looked up while dispatching.

These improvements are now available in the Axis2 trunk. So feel free to take it for a ride and give us your feedback. I also have some plans to make some improvements to the Axis2 UDP transport. I intend to add multicast support to the existing UDP transport and with that we will be able to support multicast request – unicast response message exchange pattern in Axis2.

New Configuration Model of Synapse

If you are trying out a latest Apache Synapse build off the SVN trunk, you will notice that the Synapse configuration model has gone through some significant changes lately. In the past, Synapse was programmed to load the entire mediation configuration (sequences, endpoints, proxy services, tasks etc) from a single XML file, named synapse.xml. Synapse would parse this XML file at startup and construct an object model known as the SynapseConfiguration which contains the definitions of all active mediation components at runtime. While this approach was simple and clean, it had a number of limitations:

• Configuration management becomes a nightmare as the size of the synapse.xml grows. A typical heavy duty configuration would consist of dozens of proxy services, sequences and endpoints. When all these are packed into a single XML file, it becomes difficult to locate a single configuration item quickly.
• With all configuration artifacts stored in a single flat file, it is almost impossible to develop satisfactory tooling support for configuration development and maintenance.
• A team of developers cannot work on configuring a single Synapse instance at the same time.
• Hot deployment is a feature that Synapse has been lacking for years. Being able to hot deploy a proxy service or a sequence into Synapse without having to restart the service bus is a great convenience at development time. But a configuration model based on a single XML file is not at all capable of handling such requirements.

Considering these drawbacks, I implemented a new configuration model for Synapse, known as the multi XML configuration builder. The new model, which is now the default configuration model in Synapse, loads the configuration from a structured file hierarchy, instead of loading the mediation configuration from a single XML file. With this model in place, each endpoint, sequence and proxy service has to be defined in separate XML files. The directory structure for storing these individual configuration files is as follows:

synapse-config
   |
   +-- registry.xml
   +-- endpoints
   +-- event-sources
   +-- local-entries
   +-- proxy-services
   +-- sequences
   `-- tasks

As you can see there are separate dedicated directories for each type of artifacts. Each of these directories can house zero or more XML configuration files. Each file must have the .xml extension to be recognized by the Synapse configuration builder. If you take a look at the source code you will notice that I have enforced this restriction using a Java FileFilter:

    private static FileFilter filter = new FileFilter() {
       public boolean accept(File pathname) {
           return (pathname.isFile() && pathname.getName().endsWith(".xml"));
       }
   };

   private static void createProxyServices(SynapseConfiguration synapseConfig, String rootDirPath)
           throws XMLStreamException {

       File proxyServicesDir = new File(rootDirPath, PROXY_SERVICES_DIR);
       if (proxyServicesDir.exists()) {
           if (log.isDebugEnabled()) {
               log.debug("Loading proxy services from : " + proxyServicesDir.getPath());
           }
           File[] proxyDefinitions = proxyServicesDir.listFiles(filter);
           for (File file : proxyDefinitions) {
               try {
                   OMElement document = parseFile(file);
                   ProxyService proxy = SynapseXMLConfigurationFactory.defineProxy(
                           synapseConfig, document);
                   proxy.setFileName(file.getName());
                   SynapseArtifactDeploymentStore.getInstance().addArtifact(
                           file.getAbsolutePath(), proxy.getName());
               } catch (FileNotFoundException ignored) {}
           }
       }
   }

The mediation registry is defined in the registry.xml file which should be placed at the top level of the file hierarchy.

So does the multi XML configuration builder solve the problems in the old configuration model? Let’s consider the facts:

• With Synapse configuration broken down into smaller, manageable pieces the whole configuration becomes easier to manage and keep track of. As long as the XML files are named appropriately, it is extremely easy to quickly locate a particular configuration item. We recommend using the artifact names to name the corresponding XML files. For an example the file containing the definition of the FooProxy can be named FooProxy.xml.
• With the multi XML configuration builder, developing powerful and elegant tools for creating pieces of the service bus configuration becomes a trivial task. Also one can use conventional configuration management tools and version controlling systems such as Subversion to store and manage the configuration artifacts.
• A team of developers can now work on configuring Synapse. Each developer in the team can work on his own configuration file or set of files.
• Supporting hot deployment is now feasible. As a matter of fact, Ruwan implemented hot deployment and hot update support for Synapse based on the multi XML configuration builder a few weeks back. This feature is now available in the Synapse trunk and will be available for the next Synapse release.

So there you go. Target achieved. There is one little glitch in this approach though. That is how do we handle backward compatibility with older Synapse versions? For instance how can a Synapse 1.2 user, who has a single synapse.xml file, migrate to a new Synapse version? We have provided a solution for that as well. In the new Synapse configuration file hierarchy you can place a synapse.xml file at the top level (alongside with registry.xml). All the mediation components defined in this synapse.xml will be loaded to the service bus at startup along with any other components defined inside the individual directories. So a Synapse 1.2 user can simply copy the existing synapse.xml file to the synapse-config directory in a new Synapse distribution, and it will be picked up by the service bus. In addition to this convenience feature, we are planning on developing some migration tools that can help users to easily migrate an old Synapse configuration file onto a newer version of Synapse.

As far as Synapse is concerned each configuration builder should be associated with a corresponding configuration serializer implementation. Serializers are used to convert the SynapseConfiguration object model back to the textual/XML form. So for the multi XML configuration builder I developed a matching multi XML configuration serializer which can save a SynapseConfiguration object model to a file hierarchy. Similar to the configuration builder this implementation was also heavily dependent on Java file IO APIs. However, after a while we realized that the serializer is not working as expected on certain platforms; most notably on Windows. After running some debug sessions and doing some on-line reading I realized that the Java file IO operations are not consistent on every platform. As a result sometimes the serializer would encounter trouble creating a new file or moving an existing file on Windows.

At this point, my colleague, Rajika suggested using Apache Commons IO API for file manipulation. Commons IO API provides a nice layer of abstraction on top of the standard Java IO APIs. It handles all file IO operations in a consistent and platform independent manner. So I got rid of almost all the Java file IO code in the multi XML configuration serializer and replaced them with corresponding calls to the Commons IO API. Some sample code fragments are shown below:

    private void cleanUpDirectory()  throws Exception {
       // If the target directory already exists and contains any files simply rename it to
       // create a backup - This method does not delete the target directory
       if (rootDirectory.exists() && rootDirectory.isDirectory() &&
               rootDirectory.listFiles().length > 0) {

           if (log.isDebugEnabled()) {
               log.debug("The directory :" + rootDirectory.getPath() + " already exists. " +
                       "Creating a backup.");
           }

           backupDirectory = new File(rootDirectory.getParentFile(), "__tmp" +
                   new GregorianCalendar().getTimeInMillis());
           FileUtils.moveDirectory(rootDirectory, backupDirectory);
       }

       // Create a new target directory
       FileUtils.forceMkdir(rootDirectory);
   }

   private void writeToFile(OMElement content, File file) throws Exception {
       File tempFile = File.createTempFile("syn_mx_", ".xml");
       OutputStream out = new FileOutputStream(tempFile);
       XMLPrettyPrinter.prettify(content, out);
       out.flush();
       out.close();

       FileUtils.copyFile(tempFile, file);
       FileUtils.deleteQuietly(tempFile);
   }

FileUtils is the entry point for file IO operations in the Commons IO API. It provides you with a range of useful methods for managing and manipulating files.

Note that when writing to a file we first write the whole thing to a temporary file and once completed we copy the temp file to the final location. This was required to handle certain edge cases in the hot update implementation. Without that the hot updater will attempt to pickup and process half baked files (Again mostly on Windows – It seems Windows writes to files in chunks).

All in all, Apache Synapse is now equipped with a powerful new configuration model, a matching configuration serializer and hot deployment capabilities that leverage the new configuration model. Speaking of hot deployment, it is also based on the Axis2 hot deployment framework. Therefore it is configured in the axis2.xml file. We are yet to do a Synapse release with all these new exciting features but if you are itching to try them out feel free to grab a nightly build. Also be aware that WSO2 ESB 3.0, which is based on Synapse, has been released and that release contains all these new improvements.

Apache Attacks: A Real Eye Opener

It seems year 2010 is gradually turning into the year of cyber crimes. The year started off with news reports on a very sophisticated and targeted attack on Google corporate infrastructure. According to Google, the attackers were successful in stealing valuable intellectual property. Now after about four months from the Google incident, a massive attack has been carried out on the Apache Software Foundation. Attackers have exploited a previously unknown vulnerability in the Atlassian JIRA instance used by the ASF, to gain root access to the server hosting the JIRA instance. Attackers also messed up the JIRA instance to install some sniffers that can capture and log user passwords. According to the Apache infra team a whole bunch of user passwords have been compromised as a result of the attack. This attack was later followed by another attack, this time directly on the Atlassian IT infrastructure, which has also exploited the same security hole in JIRA.

The Apache infra team did a pretty amazing job to mitigate the threat and take control of the situation. They have also taken some additional security measures to prevent such disasters in the future while Atlassian has rolled out patches for the aforementioned security hole.

It is hard to imagine what makes somebody attack the ASF IT infrastructure. The whole world knows that we are just doing non-profit, voluntary work at ASF. Nobody gains a monetary benefit by attacking the ASF. May be it is just to compromise the passwords and get to the users/committers. Most people use the same password to login to multiple systems (Gmail, Facebook, Yahoo, Hotmail etc). So if the attacker can get the password for one system, he can gain access to all the other systems.

All in all, this incident is a real eye opener to all of us surfing the Internet. No system is 100% safe and no system is invulnerable. An attacker with sufficient patience and skill will always find a way in. It is up to the users to be careful and minimize the chance of something horrible happening.

Moral of the Story:

We should use multiple passwords to login to different systems as much as possible. We should use strong passwords at all times. And we should definitely change the passwords in a regular basis to mitigate the effects of a possible brute force attack. (We hear these stuff everyday but how many of us actually do it? That's the problem. We should actually put these guidelines into action.)

PS: If you are an Apache committer and did not change the JIRA password yet, please do it NOW!!!

Introduction to Apache Synapse @ Apache Asia 2009

Yesterday (3rd December 2009) I conducted a technical presentation titled “Introduction to Apache Synapse” at the Apache Asia Roadshow 2009 held at Colombo, Sri Lanka. For everyone’s benefit I have published the presentation slides on SlideShare. The presentation went well, and so far I have received pretty good feedback from the participants. There were a few very interesting questions from the audience at the end of the session too.

In this presentation I have addressed the areas like problems of enterprise integration, ESB pattern, key features of Synapse, Synapse architecture and the configuration model. I hope you will find it interesting and useful.

Introduction to Apache Synapse

View more documents from Hiranya Jayathilaka.

Introducing "WSO2 ESB Tips and Tricks"

WSO2 ESB has kept me quite busy for the last couple of months. The ESB 2.1 release in July was soon followed by a 2.1.1 release and work is now underway for the 2.1.2 release. That’s way too much activity in such a short period, for an open source project. Anyway, since I’m spending a lot of time working on WSO2 ESB, I thought I might as well blog about my work in a regular basis. So in the next few months, I will be publishing a series of blog posts regarding WSO2 ESB and how to use it to implement real life integration scenarios. This series of blogs, which I have named “WSO2 ESB Tips and Tricks”, will surely benefit many folks who are either just learning SOA concepts or looking to integrate a bunch of apps using an ESB.

You can consider this post to be the 0th article of the “WSO2 ESB Tips and Tricks” series. So before I move on to the 1st post of the series I think I should briefly describe what WSO2 ESB is and why you should consider it as an option for enterprise integration. So here goes:

WSO2 ESB is a fast and lightweight, enterprise service bus. It is based on Apache Synapse, the lightweight ESB from the ASF (speaking of which, “Happy Birthday ASF”). It supports many messaging standards including SOAP, WS-* standards and REST as well as a variety of application and transport layer protocols like HTTP/S, Mail, JMS, VFS, AMQP, TCP and FIX. WSO2 ESB comes with a rich collection of mediators and other functional components which can be used out of the box to implement even the most complex integration patterns. Message routing, transformation, protocol switching, load balancing, clustering and service chaining are some of the common features supported by WSO2 ESB. Controlling and managing WSO2 ESB is also a trivial task thanks to the Web based management. Many of the common system administration tasks such as user management, certificate management and statistics collection are all available as inbuilt features of the server and the management console. WSO2 ESB is shipped with an embedded WSO2 Governance Registry, which makes it easy to store and manage SOA metadata. In addition to that the ESB can be configured to work with an externally hosted WSO2 Governance Registry if needed.

WSO2 ESB is also based on WSO2 Carbon, the OSGi based components framework for SOA. WSO2 Carbon makes it possible to easily install and configure additional features into the ESB runtime. Custom code and third part libraries can also be deployed into the server without any hassle.

Perhaps the best thing about WSO2 ESB (apart from it being super fast) is that it is totally free and open source. Binary and source distributions are available for download through the WSO2 Oxygen Tank. All artifacts are released under the business friendly Apache Software License 2.0. WSO2 also offers training and commercial support for users that require them.

As I have already mentioned a couple of times, WSO2 ESB is well known as a fast ESB. It can easily handle over 2500 transactions per second while maintaining constant memory usage. This level of performance is neither a coincidence nor an accident, but by clever and careful design. Two of the most significant features of WSO2 ESB architecture can be listed as follows:

• Non-blocking HTTP transport : The HTTP server worker threads of the ESB do not get blocked over network I/O. The Apache HTTP Core-NIO based transport makes it possible to accept many concurrent connections and process more messages than any other Java HTTP transport implementation.
• Pull Parsing and Streaming Model: WSO2 ESB uses Apache AXIOM, the StAX based XML infoset model. Therefore it does not build the object model for incoming messages, unless it has to. Incoming bytes are streamed through the ESB without touching the payload. This reduces memory usage significantly and also saves many valuable CPU cycles.

These architectural elements combine with the Carbon framework to deliver the best performance imaginable along with seamless modularization.

Some of the latest features that we have added to WSO2 ESB like rule based mediation and EDA (Eventing) support make WSO2 ESB suitable for implementing even the most complicated SOA platforms.

I think this explanation pretty much justifies why you should give WSO2 ESB a try. It is fast, feature rich and user friendly by all means. And don’t forget, it’s free and open source too. So grab a copy of the latest binary distribution today and get started. If you need any help we got tons of free documentation and articles on the WSO2 Oxygen Tank. This is in addition to our mailing lists and user forums.

I hope you will enjoy the “WSO2 ESB Tips and Tricks”.

Enterprise Messaging with Synapse, WSO2 ESB and WebSphere MQ

I've been recently playing a lot with Apache Synapse, WSO2 ESB and IBM WebShpere MQ (WMQ). My intention was to use WMQ as a JMS provider for Synapse and WSO2 ESB. IBM WebSphere MQ being a very popular and matured messaging solution, I figured that getting Synapse and WSO2 ESB to work with WMQ and document the integration process would really help the IBM, Apache and WSO2 communities. Originally I anticipated this application integration to be a very comlpex and tedious task. But for much of my delight, it turned out to be a very easy and simple process in the end. What was little complicated was to get WMQ properly installed and get its JMS features configured. But the integration with Synapse and WSO2 ESB was indeed a piece of cake.

So here I'm going to list the steps hat one should follow to get WMQ integrated with Synapse or WSO2 ESB. Start by downloading the required software. Binary distributions of Apache Synapse and WSO2 ESB can be downloaded from their respective websites. A trial version of the IBM WMQ V7.0 can be downloaded from here. In addition to these application you will also require Java 5 or higher along with Apache ANT to run some of the samples described here.

Installing Apache Synapse or WSO2 ESB doesn't require any additional steps. You simply need to extract the downloaded archives and that's all. However WMQ installation is not so easy. It is a fairly long and time consuming process which requires carefulness and patience. Anyway the installation process is well documented and so it shouldn't be a problem. For your convenience I'm describing the WMQ installation procedure for Linux here. The necessary commands to be executed at each step are given in italic font.

Installing WMQ

1. Create an empty directory and extract the downloaded WMQ archive into it

2. Add a new user group called 'mqm' to the system

groupadd mqm

3. Add a new user called 'mqm' to the system. Add the user to the 'mqm' group. Set the user's home directory to /var/mqm

useradd -d /var/mqm -g mqm mqm

4. Create the directory /var/mqm and change its owner to mqm

mkdir /var/mqm

chown mqm:mqm mqm

5. Create the directory /opt/mqm

mkdir /opt/mqm

6. Create another directory anywhere on the disk and set a symlink to it from /opt/mqm

mkdir /home/hiranya/mqm

ln -s /home/hiranya/mqm /opt/mqm

7. Install libstdc++5 library on the system (using Synaptic or apt-get on Debian/Ubuntu systems) if not already installed

8. As root go to the directory where WMQ is extracted and run the license display script

./mqlicense.sh

9. Install the MQ runtime and the server using rpm

rpm -ivh MQSeriesRuntime-7.0.0-0.i386.rpm MQSeriesServer-7.0.0-0.i386.rpm --nodeps

10. Install the MQ samples

rpm -ivh MQSeriesSamples-7.0.0-0.i386.rpm --nodeps

11. Install the MQ client

rpm -ivh MQSeriesClient-7.0.0-0.i386.rpm --nodeps

That wasn't too bad, was it? Now it's time to verify the installation. Here's what you got to do.

Verifying the Installation

1. Login as user mqm (ssh mqm@localhost)

2. Create a new queue manager

crtmqm -q venus.queue.manager

3. Start the queue manager

strmqm

4. Start MQSC program (a new shell will appear - without a prompt)

runmqsc

5. Define a local queue in the MQSC shell

define qlocal (orange.queue)

6. Terminate MQSC

end

7. Go to /opt/mqm/samp/bin and run the amqsput sample program to place a message on to the queue

./amqsput ORANGE.QUEUE

8. The above command should be followed up by some sample message text of your choice and a blank line

9. Run the following command to pull the message back from the queue

./amqsget ORANGE.QUEUE

10. So far so good! We got the server properly installed. Now on to the client!

11. Create a new queue manager

crtmqm -q saturn.queue.manager

12. Start the queue manager

strmqm

13. Start MQSC

runmqsc

14. Create a new queue

define qlocal (queue1)

15. Create a connection channel to the queue

define channel (channel1) chltype (svrconn) trptype (tcp) mcauser ('mqm')

16. Create a listener (specify a port)

define listener (listener1) trptype (tcp) control (qmgr) port (8585)

17. Start the listener

start listener (listener1)

18. Terminate MQSC

end

19. Create a system environment variable to point to the channel created above

export MQSERVER='CHANNEL1/TCP/localhost(8585)'

20. Go to /opt/mqm/samp/bin and run the amqsputc to place a message in the queue (using the client API)

./amqputc QUEUE1 saturn.queue.manager

21. Now run amqsgetc to receive the message

./amqgetc QUEUE1 saturn.queue.manager

22. Excellent! We got the client installed properly too :)

Now that we got WMQ installed let's try to get its JMS features enabled.

Setting Up JMS

1. Install WMQ classes for JMS

rpm -ivh MQSeriesJava-7.0.0-0.i386.rpm --nodeps

2. Start MQSC (This will start mqsc for the last created queue manager - saturn.queue.manager)

runmqsc

3. Define a JMS channel for the queue manager

define channel(java.channel) chltype(svrconn) trptypr(tcp)

4. Define a listener

define listener(listener.tcp) trptype(tcp) port(1414)

5. Start the listener

start listener(listener.tcp)

6. Setup the classpath variable and the MQ_JAVA_LIB_PATH to run the IVT programt

export CLASSPATH=/opt/mqm/java/lib/com.ibm.mqjms.jar:/opt/mqm/samp/jms/

export MQ_JAVA_LIB_PATH=/opt/mqm/java/lib

7. Go to /opt/mqm/java/bin and invoke the IVT sample program

./IVTRun -nojndi

./IVTRun -nojndi -client -m saturn.queue.manager -host localhost -channel JAVA.CHANNEL

8. If the IVT program can recieve and send JMS messages we are done! Now let's proceed to enabling JNDI support

9. Go to the /opt/mqm/java/bin directory and open the JMSAdmin.config file. Edit the PROVIDER_URL property to point to an empty directory of your choice on the file system. This directory will be used the JNDI provider source (eg: /var/mqm/jndi). If the specified directory does not exist in the file system create it.

10. Run the IVTSetup tool to create the default set of JNDI bindings

./IVTSetup

11. Now run the IVTRun tool as follows

./IVTRun -url "file:/var/mqm/jndi" -icf com.sun.jndi.fscontext.RefFSContextFactory

12. By now we have enabled and verified JNDI support. Let's use the JMSAdmin tool to make some modifications in the JNDI bindings

13. Fireoff /opt/java/bin/JMSAdmin and run the following commands:

ALTER QCF(ivtQCF) TRANSPORT(CLIENT)

ALTER QCF(ivtQCF) QMGR(saturn.queue.manager)

14. Run the IVTRun tool again as mentioned in step 11. This time messages will be sent to the saturn queue manager.

Now we are all set. It's time to get WMQ integrated with Synapse and WSO2 ESB. Let's start with Apache Synapse.

Synapse Integration

Follow the steps given below as the user 'mqm'. Trying to run Synapse as a different user caused some JMS security exceptions.

1. Copy the following jar files from /opt/mqm/java/lib to SYNAPSE_HOME/lib

• com.ibm.mqjms.jar
• fscontext.jar
• providerutil.jar
• com.ibm.mq.jmqi.jar
• dhbcore.jar

2. Enable the JMS listener in the axis2.xml. Configure the default connection factory as follows.

<parameter name="default">
  <parameter name="java.naming.factory.initial">com.sun.jndi.fscontext.RefFSContextFactory</parameter>
  <parameter name="java.naming.provider.url">file:/var/mqm/jndi</parameter>
  <parameter name="transport.jms.ConnectionFactoryJNDIName">ivtQCF</parameter>
  <parameter name="transport.jms.ConnectionFactoryType" locked="false">queue</parameter>
  <parameter name="transport.jms.Destination">ivtQ</parameter>
</parameter>

Note that we are using the JNDI configuration used by the IVT sample program. We are using the same queue connection factory and the JMS queue to define the default connection factory.

3. Create the following proxy service in the synapse.xml (this is a simplified version of sample 250)

<proxy name="JMSProxy" transports="jms">
  <target>
     <inSequence>                             
        <log level="full"/>
        <drop/>
     </inSequence>
  </target>
  <parameter name="transport.jms.ContentType">
       <rules>
           <jmsProperty>contentType</jmsProperty>
           <default>application/xml</default>
       </rules>
  </parameter>
</proxy>

4. Start Synapse - go to SYNAPSE_HOME/bin and run ./synapse.sh

5. Go to SYNAPSE_HOME/samples/axis2Client/src/samples/userguide and open the GenericJMSClient.java source file. Make the following changes in the code.

• Set the jms_dest property default value to "ivtQ" (line 45)
• Set the java.naming.provider.url to "file:/var/mqm/jndi" (line 82)
• Set the java.naming.factory.initial to "com.sun.jndi.fscontext.RefFSContextFactory" (line 85)
• Set the lookup key to "ivtQCF" (line 89)

6. Now from the SYNAPSE_HOME/samples/axis2Client directory run the following command to run the JMS client application which will send messages to the ivtQ.

ant jmsclient -Djms_type=pox -Djms_payload=IBM

7. At this point Synapse will pick the messages from the queue and log them on the console.

WSO2 ESB 2.1 Integration

Follow the steps given below as user 'mqm'

1. Copy the Websphere MQ client jars (mentioned above under Synapse Integration) to ESB_HOME/repository/components/lib

2. Enable the JMS listener in axis2.xml and configure the default JMS connection factory as follows

<parameter name="default">              
        <parameter name="java.naming.factory.initial" >com.sun.jndi.fscontext.RefFSContextFactory</parameter>                      
        <parameter name="java.naming.provider.url" >file:/var/mqm/jndi</parameter>
        <parameter name="transport.jms.ConnectionFactoryJNDIName" >ivtQCF</parameter>
        <parameter name="transport.jms.ConnectionFactoryType" >queue</parameter>
        <parameter name="transport.jms.Destination">BOGUSQ</parameter>
</parameter>

Note that we have specified a queue named 'BOGUSQ' as the default destination. This is because we need to use ivtQ for our proxy service only. If we use 'ivtQ' here all the services deployed in ESB (XKMS, echo, wso2carbon-sts) will start listening on the same queue. You also need to login to JMSAdmin and create the queue named 'BOGUSQ'. In JMSAdmin shell run the following commands.

DEFINE Q(BOGUSQ) QMGR(saturn.queue.manager)

ALTER Q(BOGUSQ) QUEUE(QUEUE1)

If the QUEUE1 does not exist, first login to MQSC and create it.

3. Add the proxy service we used with Synapse to ESB_HOME/conf/synapse.xml

4. Start WSO2 ESB

./wso2server.sh -DuseSynapseXML

5. Use the sample client we used in Synapse to send messages to the queue

Summer of Code is Back

Google Summer of Code 2009, the annual open source feast organized by the Internet giant Google is now underway. I also applied for the contest and I'm pleased to announce that my project proposal to implement SQL authorization support for Apache Derby dblook has been accepted. So that makes me a GSoC winner, two times in a row. Last year I implemented XML Schema 1.1 type alternatives support for Apache Xerces2/J under the guidance of Khaled Noaman. This year Dag Wanvik will be mentoring my project. 

I'm eagerly looking forward to complete this task in time and of course in classy fashion. Apache Derby community is really active, dynamic and supportive. I feel gifted to get an opportunity to work with such a passionate group of committers. And not to mention this is going to be a great learning experience as well as a genuine test of skill for me.

On the sad side, this will probably be the last GSoC I'm going to take part as a student of the University of Moratuwa. I will be officially passing out from the University in a couple of months time. University of Moratuwa became the no. 1 ranked institute in last year's GSoC which made us really happy and proud of ourselves. I'm confident that my University has ranked pretty high this year too (Google haven't released the numbers as of yet). So that means this is the last time I get to enjoy being on top of the world in GSoC :(

Committed to Apache Xerces

The Apache Xerces community granted me committer status for Apache Xerces2-J. The vote was called in just a few days back and it was officially closed on 8th of April (yesterday). Xerces2-J is an open source library for parsing, validating and manipulating XML documents. It is the most matured and most standard complaint XML parser available for Java applications today. 

Apache Xerces used to be a subproject of the Apache XML project but due to the immense popularity and high adoption rate that Xerces continued to exhibit, it became a top level project in year 2004. Presently Apache Xerces is a collection of four sub projects.

• Apache Xerces C++
  
• Apache Xerces2 Java
  
• Apache Xerces Perl
  
• Apache XML Commons
  

The original code base which formed the foundation of Xerces C++ and Xerces-J was developed at IBM and donated to ASF in year 1999. Therefore Apache Xerces happens to be one of the oldest projects of the ASF as well. In fact Xerces is nearly as old as the ASF itself and will be celebrating its 10th anniversary this November. Given all the facts and figures, it is indeed a great honour to be a part of such a majestic open source project :)

Talking about Apache Xerces2-J, it supports a large number of XML and XML related standards and APIs. The latest released version (2.9.1) supports the following specs.

• XML 1.0 (4th Edition)
  
• Namespaces in XML 1.0 (2nd Edition)
  
• XML 1.1 (2nd Edition)
  
• Namespaces in XML 1.1 (2nd Edition)
  
• W3C XML Schema 1.0 (2nd Edition)
  
• XInclude 1.0 (2nd Edition)
  
• OASIS XML Catalogs 1.1
  
• SAX 2.0.2
  
• DOM Level 3 Core, Load and Save
  
• DOM Level 2 Core, Events, Traversal and Range
  
• JAXP 1.3
  

Due to the performance levels achievable through Xerces and its high level of standard compliancy, hundreds of open source projects and other commercial software applications make use of Xerces2-J. It is fair enough to say that Xerces2-J is the most widely used XML parser in the Java world. If you have a Java application which does some XML related stuff it's very likely that Xerces2-J is somewhere in your application's classpath. People sometimes use it even without knowing that they are actually using Xerces2-J. Sun Microsystems ships Xerces2-J with their JDK. This enables Sun JDK users to develop XML parsing applications powered by Xerces2-J without having to explicitly put the Xerces2-J jars in the project classpaths. However folks at Sun Microsystems have changed the Xerces2-J package name from org.apache.xerces.* to com.sun.org.apache.xerces.* which in my opinion is being a little ungrateful :(

Nowadays we are working on getting Xerces2-J to support the new XML Schema 1.1 specification from W3C. In fact I started contributing to Xerces2-J by implementing a XML Schema 1.1 feature as a Google Summer of Code participant, last year. I've been taking part in mailing list discussions and fixing bugs ever since. The Xerces2-J community is rather small but there's a very helpful and friendly bunch of experienced developers there, who have dedicated their lives and careers to Xerces. That encourages young developers like me to do more and more towards the betterment of the project, as and when the time permits.

WSO2 ESB 2.0.2 Released...Come and Get it

The WSO2 ESB team is pleased to announce the release of version 2.0.2 of the Open Source Enterprise Service Bus (ESB). This is a bug fix release of the 2.0 release of WSO2 ESB

WSO2 ESB is a lightweight and easy-to-use Open Source Enterprise Service Bus (ESB) available under the Apache Software License v2.0. WSO2 ESB allows administrators to simply and easily configure message routing, intermediation, transformation, logging, task scheduling, etc.. The runtime has been designed to be completely asynchronous, non-blocking and streaming based on the Apache Synapse core.

WSO2 ESB 2.0.2 is developed on top of the revolutionary Carbon platform (Middleware a' la carte), and is based on the OSGi framework to achieve the better modularity for your SOA architecture. This also contains a lots of new features and many other optional components to customize the behavior of the server. Further, if you do not want any of the built in features, you can uninstall those features without any trouble. In other words, this ESB can be customized to your SOA needs.

You can download this distribution from http://wso2.org/downloads/esb and give it a try.

How to Run

1. Extract the downloaded zip
2. Go to the bin directory in the extracted folder
3. Run the wso2server.sh or wso2server.bat as appropriate
4. Point your browser to the URL https://localhost:9443/carbon
5. Use "admin", "admin" as the username and password to login as an admin and create a user account
6. Assign the required permissions to the user through a role
7. If you need to start the OSGi console with the server use the property -DosgiConsole when starting the server
8. Samples configurations can be specified by passing the property -Desb.sample=${SAMPLE_NUMBER}

Key Features

• Proxy services - facilitating transport, interface (WSDL/Schema/Policy), message format (SOAP 1.1/1.2, POX/REST, Text, Binary), QoS (WS-Addressing/WS-Security/WS-RM) and optimization switching (MTOM/SwA).
• Non-blocking HTTP/S transports based on Apache HttpCore for ultrafast execution and support for thousands of connections at high concurreny with constant memory usage.
• Built in Registry/Repository, facilitating dynamic updating and reloading of the configuration and associated resources (e.g. XSLTs, XSD, JS, ..)
• Easily extended via custom Java class (mediator and command)/Spring mediators, or BSF Scripting languages (Javascript, Ruby, Groovy, etc.)
• Built in support for scheduling tasks using the Quartz scheduler.
• Load-balancing (with or without sticky sessions) /Fail-over, and clustered Throttling and Caching support
• WS-Security, WS-Reliable Messaging, Caching and Throttling configurable via (message/operation/service level) WS-Policies
• Lightweight, XML and Web services centric messaging model
• Support for industrial standards (Hessian binary web service protocol/Financial information exchange protocol)
• Enhanced support for the VFS/JMS/Mail transports
• Support for message splitting and aggregation using the EIP
• Database lookup and store support with DBMediators with reusable database connection pools
• JMX monitoring support

New Features of the WSO2 ESB 2.0 (2.0.2)

• This ESB release is based on Carbon "Middleware a' la carte" which is an OSGi based SOA platform by WSO2 Inc.
• Transactional JMS transport and the Transaction mediators
• Integrated graphical user management
• Integrated graphical key store management
• Configurable logging through the management console
• Graphical data source declaration
• WS-Eventing support and Event Sources making it an event broker
• Enhanced sequence and proxy service editor
• Module management capability
• Transport configuration management through the graphical console
• Graceful/Forced shutdown/restart
• Enhanced integrated registry and search functionalities
• User permissions support
• Enhanced monitoring tools for statistics and tracing
• Try-It tool to try an existing service
• Graphical policy editor
• Administration console fully internationalized
• Better modularity and extendability through OSGi component architecture

How You Can Contribute....

Mailing Lists

Join our mailing list and correspond with the developers directly.

• Developer List : esb-java-dev@wso2.org
• User List : esb-java-user@wso2.org

Reporting Issues

WSO2 encourages you to report issues and your enhancement requests for the WSO2 ESB using the public JIRA.

You can also watch how they are resolved, and comment on the progress..

Discussion Forums

Alternatively, questions could be raised using the forums available.

WSO2 ESB Forum : Discussion forum for WSO2 ESB developers/users

Training

WSO2 Inc. offers a variety of professional Training Programs, including training on general Web services as well as WSO2 ESB, Apache Synapse, Apache Axis2 and number of other products.

For additional support information please refer to http://wso2.com/training/course-catalog/

Support

WSO2 Inc. offers a variety of development and production support programs, ranging from Web-based support up through normal business hours, to premium 24x7 phone support.

For additional support information please refer to http://wso2.com/support/

For more information on WSO2 ESB, visit the WSO2 Oxygen Tank (http://wso2.org)

We welcome your feedback on this implementation. Thank you for your interest in WSO2 ESB.

-- The WSO2 ESB Team --

ApacheCon EU 2009....Finishing Touches

The ApacheCon EU 2009 (Amsterdam, Netherlands) came to an end today evening. We had a BarCamp on 23rd followed by a Hackathon on 24th. The conference commenced on 25th and continued until today evening. We had two keynotes and around 64 technical presentations during the last three days. I was fortunate enough to take part in the following presentations.

• Embrace OSGi (by Carsten Ziegeler, member of ASF)
• Tales from OSGi trenches (by Bertrand Delacretaz, member of ASF)
• OSGi as a framework for building a product line (by Ruwan Linton and Afkham Azeez, WSO2)
• HBasics: Hadoop's big database (by Michael Stack, Microsoft)
• Apache license as a business model (by Paul Fremantle, WSO2)
• Clustered Web services for high availability and scalability (by Ruwan Linton, WSO2)
• EDA with Apache Synapse (by Paul Fremantle, WSO2)
• Performance tuning Apache Tomcat (by Filip Hanik, SpringSource)
• How to become a project at ASF (by Martijn Dashorst)
• Introduction to NIO 2.0 (by Jeanfrancois Arcand)
• Using MINA 2.0 (by Emmanuel Lecharmy, IKTEK)
• Becoming a Tomcat super user (by mark Thomas, SpringSource)
• High availability != High cost (by Norman Maurer, HEAG MediaNet)
• Apache POI (by Nick Burch, Torchbox)
• Enterprise build & test in the cloud (Carlos Sanchez, G2iX)
• Shindig for blogs and wikis (Dave Johnson, VP Apache Roller)

I also got the opportunity to chair most of the above sessions thanks to the organizers and folks of the Apache TAC. Thanks guys! You rock!

Out of the two keynotes, I really enjoyed the keynote on 'open sourcing the analyst business' by James Governor. It was a real blast!

In addition to all the technical knowledge I gathered during the conference I also made quite a lot of new friends. I met some very interesting people working at various levels and projects of the ASF. All in all it was a very successful ApacheCon, not just to me but to everybody that participated. (I have a lot more to mention but unfortunately I have to cut it short. I have lot of packing left to do before I start the return journey tomorrow morning.)

Happy Birthday ASF

The Apache Software Foundation (ASF) celebrates its 10th anniversary today. Having finished the regular activities of the ApacheCon scheduled for the day, all the committers, PMC members and members got together to celebrate this great occasion. Jim Jagielski, the chairman of the ASF board of directors cut the giant birth day cake. It was indeed a fun filled and proud moment for everyone at Apache. Another important event that took place during the reception was the key signining party where around 20 folks including myself, got the opportunity to get signed PGP keys from ASF.

All in all it was a great birth day party with loads of fun and excitement. Happy 10th anniversary ASF!!!

ApacheCon EU 2009 Begins

After a 3 hour flight from Sri Lanka to India followed by an 8 hour flight from India to Netherlands, I finally arrived at beautiful Amsterdam city, to take part in the ApacheCon Europe 2009. The event is now being held at the Movenpick hotel at the Amsterdam city center. The main attraction of the first day was BarCamp Apache where a number of developers, committers and open source geeks like me presented on a variety of topics. I myself conducted a presentation on Apache Synapse and how it can be used to scale up Web Services deployments. Among the other presentations I really liked the presentation by Ricardo Varela on software for mobile platforms and the presentation by Ross Gardler on teaching open source. Sri Lankan student community was credited for their achievements in the last Google Summer of Code during the presentation by Ross :)

All in all it was a really awesome BarCamp with lots of great talks, t-shirts, sweets and coffee. Let's see how the things turn out tomorrow.

GSoC 2008 ....... Done

You may have noticed that my blog has gone rather silent in the last couple of months. Well, the reason was my Google Summer of Code (GSoC) project. The final deadline was on 18th of August so I worked really hard to deliver the goods on time and finally I can proudly state that I managed to successfully complete my project on time.

The objective of my GSoC project was to implement XML schema type alternatives support for Apache Xerces2/J, the legendary open source XML parser for Java applications. Type alternatives is the answer from W3C XML schema working group, for conditional type assignment problem. This feature was first introduced in the XML schema 1.1 structures specification. Type alternatives allow a type to be assigned to an element dynamically at validation time based on one or more conditions. Conditions are expressed as XPath 2.0 expressions. Here is an example element declaration which uses XML schema type alternatives.

As stated in the above code snippet the declared type of the message element is the complex type called messageType. However the element declaration also contains a few type alternatives. These type alternatives allow the type of the message elements to be determined at the validation time. Based on the conditions stated as the test attribute values the schema validator will assign a type for the message element prior to validating its content. The expressions '@kind' and '@code' refer to two attributes of the message element. The first type alternative will assign the type called messageTypeString if the kind attribute has the value 'string' and the code attribute has a value greater than 1000.

When the schema validator encounters an element declaration with one or more type alternatives it will evaluate the test expressions one by one until an expression which evaluates to true is found. When such a matching type alternative is found the corresponding type will be assigned to the element. If none of the type alternatives match then a default type will be assigned.

My project mainly consisted of two main sections. First section of the project was to implement the type alternatives traversal support so that Xerces2/J can properly traverse an XML schema document which contains type alternatives and add the corresponding information to the schema grammar. Implementing this was fairly easy and I managed to complete it prior to th GSoC mid term evaluations. The second part of the project was to implement type alternatives validation. This was fairly difficult since I had to develop a bare minimal XPath 2.0 implementation for Xerces2/J. Developing the XPath processor actually covered a significant portion of the entire project.

My workings will be fully available in the Xerces2/J code base (even now the code related to traversal part is in one of the SVN branches) very soon. All in all it was a great learning experience as it was a great opportunity for me to learn a whole bunch of cool technologies like XML, XML schema, and XPath 2.0. I also got the opportunity to put some of my knowledge on theory of computing into action and sharpen my programming skills. I would like to give my heartiest gratitude to the Google, the Apache community and very specially to my mentor Khaled Noaman for being a very supportive guide right from the start of my project.

Fixing the FIX Transport

The 1.2 version of Apache Synapse, the open source lightweight ESB was released on June 2008. One of the most striking features of this release was the FIX transport implementation, mainly due to the fact that most ESBs in the world still do not support the FIX protocol. The Synapse FIX transport implementation which is only two months old at the moment has already made a lot of hype in the world of Software Engineering and SOA. It seems we already got a few interested people (potential clients?) who are at the moment testing and playing with the transport.

Synapse 1.2 ships with two samples that demonstrate the FIX transport module. One of them demonstrates how two FIX endpoints can be bridged using Apache Synapse. The other samples shows how to bridge an HTTP client with a FIX endpoint. Over the last couple of months the Synapse community has worked really hard to further improve the FIX transport implementation and also to identify more exciting usecases for the transport module. As a result we managed to add four new samples demonstrating the FIX transport to the Synapse documentation. These samples are currently available in the Apache Synapse Snapshot and will be most likely included in the next release.

The first of the four newly added samples (sample 259) shows how to bridge a FIX client with an HTTP endpoint. The sample effectively demonstrates how Banzai, the sample FIX blotter that ships with Quickfix/J can be used to send order requests via Synapse to a service listening on HTTP. The sample uses the XSLT mediator to convert the FIX messages into a SOAP messages.

The second of the new samples (sample 260) shows how Synapse can be used to bridge a FIX endpoint with an AMQP endpoint. Here once the FIX message is converted into XML it will be bound to a JMS payload and sent to an AMQP consumer. Since AMQP is used widely in business applications, I believe that this sample will open up the door way to a ton of really cool usecases.

The third sample (sample 261) demonstrates how Synapse can be used to switch between FIX sessions with different versions (BeginString values). The sample successfully bridges a FIX 4.0 session with a FIX 4.1 session but the underlying concepts can be used to bridge virtually any two FIX sessions. One thing worth mentioning here is that the FIX transport implementation of Synapse initially did not support bridging FIX sessions of different versions. But considering the potential usecases, we implemented that feature very recently.

The fourth newly added sample (sample 262) demonstrates how CBR (Content Based Routing) can be done with FIX messages using Apache Synapse. The sample configuration causes Synapse to accept FIX messages over a session, read a certain symbol in the messages and based on the symbol value route the messages to different endpoints.

We also added namespace support to the FIX transport module so that it can properly parse and validate XML based FIX messages with namespaces. Another recent feature addition was improving the way FIX sessions are initialized in Synapse. The initial implementation lazy initializes all the FIX sessions for outgoing FIX messages. That means an outgoing session will not be created until a message arrives for that particular session. However since this leads to fairly large delays we improved the transport module so that the outgoing sessions are also initialized at the startup along with the incoming sessions. (the old way of initializing sessions is also supported)

Currently we are working on adding support for FIX repeated groups. (we already have a feature request for this on the JIRA from one of our users) All in all it seems that the FIX transport module for Synapse is increasingly becoming a very powerful and matured piece of software in a blistering rate.