Attention Open Source Developers...

When you keep contributing to an open source project, the project community will eventually reward you by granting you committer rights. However most open source committers don’t realize that committership is a big responsibility. The moment you become a committer of a project, you become responsible for each and every line of code in the project code base. Every single change you make to the project source code affects other committers in the project and all the people around the world who use that software. Therefore as a committer you should stick to some kind of a policy or a code of conduct which governs how you make changes to the project source code. The following is a list of items that I think should be in such a policy. I’m not implying that I have always followed this policy in the past. Rather, I always try my best to stick by it:

1. Before anything else, if your contribution involves API changes, you MUST take the matter to the project mailing list. You MUST NOT make any API changes without notifying your fellow committers and the user base. Sometimes you might feel that the API change you are about to do is rather small, unimportant and trivial. For an example you might spot a spelling mistake in a method signature and feel the urge to correct it immediately. But still, an API change is an API change. If somebody’s using the misspelled method, your change will break his/her code. Also some developers tend to think that certain interfaces in the code base are “not so important” and nobody is using them, and therefore it is ok to make changes to them. Developers like that don’t really understand open source. The truth is there are no unimportant APIs in open source projects. When you make project source code open, you are basically inviting the rest of the world to use that code, develop on top of the APIs to do whatever they want with it. So even though you don’t know it, there could be hundreds of people out there who use the “unimportant” interface you are about to change.

2. Always crate an entry in the project issue tracker to keep track of your contributions. Almost every open source community maintains an issue tracker. At Apache and at WSO2 we mainly use Atlassian JIRA. Whenever you are about to make a change to the source code, consider creating a new entry in the issue tracker before you check-in the patch. In my opinion there are only two excuses for not creating an entry in the issue tracker:

• Adding some descriptive comments to the code (note that comments and Javadocs are two different things)
• Refactoring some private attributes or private methods of a class (eg: renaming a local variable)

Once you have created the entry in the issue tracker, make sure you get it assigned to yourself. Once the relevant changes are checked-in to the code base, mark the entry as resolved. It doesn’t look very professional when you have dozens of open tickets which are not assigned to anyone. It is also not nice to have tickets in the work-in-progress mode, even after it has been fixed on the source code.

3. While making changes to the source code, always follow the coding conventions accepted by your project community. If you come across any badly written or poorly formatted code fix them then and there. Refactor as you go. However do not make any functional changes, which are not relevant to the problem you are attempting to solve. If you come across old functional issues while fixing some other issue, report them on the issue tracker and leave them to be fixed in a future iteration. Generally it’s not a very good idea to solve multiple problems by one commit. That can make things harder to keep track of.

4. When you are actually committing a patch to the code base, make it a point to properly log the changes you are about to check-in. I often commit code into the Apache Subversion repository and WSO2 Subversion repository. Whenever I make a contribution, I always add a couple of lines to the SVN commit log, describing my changes. If it is like a rather large patch, I would mention a summary of the changes in bulleted point form. Also in your commit log message, you MUST mention the ID of the corresponding issue tracker entry. This can really help you track down changes in the future. At Apache and at WSO2 we have integrated JIRA with the Subversion repository. Whenever someone checks a bit of code into the SVN, with a JIRA ID in the commit log message, the relevant JIRA entry gets updated with the information related to the commit (see example). This makes it really easy to identify all the changes done in the source code to resolve a particular issue.

So there you have it! In my opinion, every open source developer should adhere to these key guidelines, whenever making a change to the project source code.

Having said that, an open source committer should never feel uneasy, to make changes to the project source (unless it is specifically restricted by the community - as in the case of a release). If you have done some changes to the code, and if you are confident that the changes you have made are accurate and necessary, you should feel free to check them into the code base. Same applies for patches contributed by the non-committers as well. If a patch solves the problem at hand to a satisfactory level without violating the boundaries of the current design, committers should take necessary steps to check it in. The solution doesn’t have to be “perfect”. At Apache we follow a commit-then-review (CTR) policy, most of the time. This model enables developers to get code changes into the project as early as possible, without worrying too much about the impact they may have on the project. Once the changes have been checked-in, they can be reviewed and fine-tuned as necessary.

However, during my career I have come across developers who at times leave issues open for weeks just because the available patches are not ‘perfect’. While the patches provide a satisfactory solution to the problem at hand, these developers always manage to find some small, irrelevant limitation in the patch, which can be easily fixed in a future iteration. They make it an excuse to not commit the patch. These developers, unfortunately, haven’t grasped the true spirit of open source. They don’t realize that the whole point of open source is to attract as much contributions as possible and correct any problems they might have, as we go on.

Before I wrap up, I’d like to reemphasize that open source committership gives you great power and authority over your project. We all know what comes with great power. Yes, it’s “great responsibility”. So use your powers with care and caution!

A Different Experience...

Last week, I got the opportunity to visit yet another interesting part of the world – the Middle East. I have been to the Dubai airport before, while on my way to United States. But transits don’t count, right? So last week I visited the city of Amman, the capital of the Hashemite kingdom of Jordan. I was there for a whole week. It was not the trip I expected, but in a really good way.

Let’s start with the weather. I was expecting the weather to be burning hot with tons of scorching sunlight. It turns out the weather over there is actually better and far more comfortable compared to the usual weather in Colombo. The temperature is almost as same as in Colombo (around 30 degrees Celsius) but the humidity is nearly 0. So it’s actually very comfy. And during the nights the temperature would drop to something around 18 degrees Celsius resulting in some very cold nights and mornings. I also learnt that they have the usual four seasons over there at Jordan. They even get some snow showers during the winter. Who knew?

The city of Amman is indeed one of the largest, most developed and beautiful cities I’ve ever been to. It’s very clean and tidy for a capital of a country. I was kind of surprised to not see much pedestrians walking around the town. But the roads are always full of vehicles all the time (even at midnight). So I guess pretty much everyone owns a vehicle. I also got to see quite a number of Ferraris and Porches storming the highways. I also realized that travel costs are actually very low in Amman. You can get a taxi anywhere and travel about 4 km for 0.75 Dinars (~ 1 USD).

Amman is also full of great hotels, restaurants, cafes and various shops. Truly great place for tourism I should say. Somewhere close to down town they have a street called the ‘Rainbow Street’. The street full of night clubs, restaurants, coffee shops and book shops is a great place to go and get something to eat and chill down for a while. They also have these huge shopping malls at Amman– larger than any mall I’ve ever been to. If you ever get to the city of Amman please make some time to go and visit the Mecca mall or the City mall.

Something has to be mentioned about the people of Jordan too. They are really nice and friendly. People at Amman are always full of smiles and have nothing but nice things to say. During my short stay I was even offered a couple of lunches and one great dinner.

Unfortunately for me I didn’t actually get anytime to go and visit the real tourist hotspots in Jordan, like the city of Petra or the Dead Sea. But then again, it’s always nice to have something to look forward to.

WSO2 ESB Tips & Tricks 05: Error Handling with Sequences and Proxy Services

The job of an Enterprise Service Bus is to act as the backbone of an organization’s SOA. It is the spine through which all the systems and applications within the enterprise communicate with each other. At times an ESB might even help to integrate internal systems with external applications so that complex cross cutting business tasks such as supply chain management and customer relationship management can be carried out with precision and in perfect synchronism. As such an ESB often has to deal with many wire level protocols, messaging standards and remote APIs. But applications and networks are full of errors. Applications often crash due to various bugs. Network routers and links often get into states where they cannot pass messages through with the expected efficiency. These error conditions are very likely to cause a fault or trigger a runtime exception in the ESB. A good ESB should provide simple yet powerful mechanisms for dealing with such errors. So today, I’m going to discuss a little bit about error handling options provided in WSO2 ESB.

WSO2 ESB gives you the concept of fault sequences. A fault sequence is just like any other sequence, a collection of mediators. A fault sequence can be associated with another sequence or a proxy service. When the sequence or the proxy service encounters an error during mediation or while forwarding a message out, the message which triggered the error will be delegated to the specified fault sequence. Using the available mediators we can log the erroneous message, forward it to a special error tracking service, send a SOAP fault back to the client indicating the error or even send a mail to the system admin. It is not mandatory to associate each sequence and proxy service with a fault sequence. In situations where a fault sequence is not specified explicitly, a default fault sequence will be used to handle errors. Sample 4 in WSO2 ESB documentation shows how to specify a fault sequence with a regular mediation sequence.

Whenever an error occurs in WSO2 ESB, the mediation engine attempts to provide as much information as possible on the error to the user. This is done by initializing a set of property values on the erroneous message. These properties are:

• ERROR_CODE
• ERROR_MESSAGE
• ERROR_DETAIL
• ERROR_EXCEPTION

Within the fault sequence we can access these property values using the get-property XPath function. Sample 4 uses the log mediator as follows to log the actual error message:

<log level="custom">
  <property name="text" value="An unexpected error occured"/>
  <property name="message" expression="get-property('ERROR_MESSAGE')"/>       
</log>

Note how the ERROR_MESSAGE property is being used to get the error message text. You can also write special custom mediators for error handling tasks. In such cases you can use the MessageContext API to retrieve above mentioned property values from the message.

String errorMsg = (String) messageContext.getProperty(“ERROR_MESSAGE”);

Exception e = (Exception) messageContext.getProperty(“ERROR_EXCEPTION”);

WSO2 ESB also makes it possible to send a SOAP fault back to the client whenever an exception occurs in the ESB. This is done using the makeFault mediator. The makeFault mediator converts the current message into a SOAP fault. The user can specify the error code and the reason statement to be included in the fault message. Once the SOAP fault is constructed it can be sent to the client using the send mediator. Sample 5 in WSO2 ESB documentation shows exactly how to do this. Here’s the makeFault mediator configuration from sample 5:

<makefault>
  <code value="tns:Receiver" xmlns:tns="http://www.w3.org/2003/05/soap-envelope"/>           
  <reason expression="get-property('ERROR_MESSAGE')"/>
</makefault>

Note how the ERROR_MESSAGE property is used to set the original error message as the reason phrase of the SOAP fault. Keep in mind that the makeFault mediator only converts the message into a fault message. To send it to a client or an endpoint you need to use the send mediator. In sample 5 we mark the message as a response using the property mediator and invoke the send mediator so that it is sent back to the client which sent the original request.

Assume you want to send an E-Mail to the system admin whenever the ESB encounters an error. How do you do it? You simply need to enable the mail transport sender in the ESB configuration (by editing the axis2.xml file or using the UI). Configure the transport sender with the appropriate mail server settings. Then in your fault sequence you can simply use the send mediator as follows to send an E-Mail out.

<property name="Subject" value="An Error Occurred in the ESB" scope="transport"/>
<property name="OUT_ONLY" value="true"/>
<send>   
  <endpoint>       
     <address uri=”mailto:admin@yourdomain.com”/>
  </endpoint>
</send>

So these are the error handling options available for sequences and proxy services in WSO2 ESB. It also provides some more error handling options for endpoints. That however is a different matter entirely and hence I’m leaving it for a future blog post.

Apache Attacks: A Real Eye Opener

It seems year 2010 is gradually turning into the year of cyber crimes. The year started off with news reports on a very sophisticated and targeted attack on Google corporate infrastructure. According to Google, the attackers were successful in stealing valuable intellectual property. Now after about four months from the Google incident, a massive attack has been carried out on the Apache Software Foundation. Attackers have exploited a previously unknown vulnerability in the Atlassian JIRA instance used by the ASF, to gain root access to the server hosting the JIRA instance. Attackers also messed up the JIRA instance to install some sniffers that can capture and log user passwords. According to the Apache infra team a whole bunch of user passwords have been compromised as a result of the attack. This attack was later followed by another attack, this time directly on the Atlassian IT infrastructure, which has also exploited the same security hole in JIRA.

The Apache infra team did a pretty amazing job to mitigate the threat and take control of the situation. They have also taken some additional security measures to prevent such disasters in the future while Atlassian has rolled out patches for the aforementioned security hole.

It is hard to imagine what makes somebody attack the ASF IT infrastructure. The whole world knows that we are just doing non-profit, voluntary work at ASF. Nobody gains a monetary benefit by attacking the ASF. May be it is just to compromise the passwords and get to the users/committers. Most people use the same password to login to multiple systems (Gmail, Facebook, Yahoo, Hotmail etc). So if the attacker can get the password for one system, he can gain access to all the other systems.

All in all, this incident is a real eye opener to all of us surfing the Internet. No system is 100% safe and no system is invulnerable. An attacker with sufficient patience and skill will always find a way in. It is up to the users to be careful and minimize the chance of something horrible happening.

Moral of the Story:

We should use multiple passwords to login to different systems as much as possible. We should use strong passwords at all times. And we should definitely change the passwords in a regular basis to mitigate the effects of a possible brute force attack. (We hear these stuff everyday but how many of us actually do it? That's the problem. We should actually put these guidelines into action.)

PS: If you are an Apache committer and did not change the JIRA password yet, please do it NOW!!!

WSO2 ESB Tips & Tricks 04: Troubleshooting JMS

WSO2 ESB works really well with JMS. The JMS transport adapter used by WSO2 ESB comes from the Apache Axis2 project and it makes use of JNDI to connect to various JMS brokers. Therefore WSO2 ESB can work with pretty much any JMS broker that offers JNDI support. The product ships with sample configurations for Apache ActiveMQ integration, but it can be easily modified for integration with a JMS broker like IBM MQ or SwiftMQ.

Enabling JMS in WSO2 ESB is a simple 2-step process.

1. Deploy the JMS client libraries to the repository/components/lib directory (these libraries will be used by the ESB to connect to the JMS broker)
2. Enable JMS transport receiver and the sender by modifying the axis2.xml file or using the management console

Having followed the above mentioned process you can start creating JMS endpoints to forward messages and JMS proxy services to receive messages. However while implementing certain JMS related scenarios you might run into some error conditions. The rest of this blog post focuses on a couple of such commonly encountered problems and how to get rid of them, once and for all.

CNF? NCDF?

If you ever run into any ClassNotFoundExceptions or NoClassDefFoundExceptions while using JMS support in WSO2 ESB, chances are you haven’t deployed all the required client libraries. Please inspect what you have copied into repository/components/lib directory and make sure the required class is available in one of the deployed jars.

Also note that WSO2 ESB ships with geronimo-jms library which contains the javax.jms packages. Therefore you do not have to deploy them again.

HTTP Header Conversion Problem

When forwarding HTTP traffic to a JMS queue using WSO2 ESB, you might at times run into an error similar to the one given below.

ERROR JMSSender Error creating a JMS message from the axis message context

javax.jms.MessageFormatException: MQJMS1058: Invalid message property name: Content-Type

This exception is thrown by the JMS client libraries used to connect with your JMS broker and the problem is specific to the JMS broker being used. I have encountered this issue mostly with IBM Websphere MQ, but I guess there might be other broker implementations that may display the same issue. So here’s what happens….

The incoming HTTP message contains a bunch of HTTP headers that contain the ‘-‘ character. Some notable examples are ‘Content-length’ header and the ‘Transfer-encoding’ header. When ESB attempts to forward the message over JMS it sets the headers of the incoming message to the outgoing JMS message as JMS properties. According to the JMS spec the ‘-‘ character is prohibited in JMS property names. Some JMS brokers like ActiveMQ do not strictly check for this and hence they won’t complain. But certain other brokers will throw exceptions similar to the one shown above.

Even though the problem sounds very complex and tricky the solution is almost trivial. We just need to get rid of the troublemaking HTTP headers from the message before it is delivered over JMS. You can make use of the property mediator as follows to achieve this:

<property action="remove" name="Content-Length" scope="transport">
<property action="remove" name="Accept-Encoding" scope="transport">
<property action="remove" name="User-Agent" scope="transport">
<property action="remove" name="Content-Type" scope="transport">

JMS Property Data Type Mismatch

Sometimes when the ESB attempts to forwards a message over JMS, the client libraries may complain, saying that the data type of a particular message property is invalid. You may often run into this problem if you are using the property mediator to manipulate property values set on the message. This is because certain implementations of JMS, have restrictions on data types of properties. The property mediator always sets property values as strings.

To overcome this problem you will have to revise your mediation sequences and avoid manipulating property values which should contain non-string values. If you must set a non-string property value you will have to write a simple custom mediator to get the job done. Starting from WSO2 ESB 3.0 (scheduled to go out this month), the property mediator is type aware. Therefore to get rid of this problem you can set properties with specific data types. For an example if you want to set a property named foo with the integer value 12345 use the property mediator as follows:

<property name="foo" value="12345" type="INTEGER" scope="transport/">

If you leave the type attribute out the property value will be set as a string.

Too Many Threads? Out of Memory?

With certain JMS brokers WSO2 ESB tends to spawn new worker threads indefinitely until it runs out of memory and crashes. I have observed this behavior with SwiftMQ. This problem is caused by a bug in the underlying Axis2 engine and there is a simple workaround to the problem. In your mediation sequence engage the property mediator as follows:

<property action="remove" name="transportNonBlocking" scope="axis2">

This will prevent the ESB from creating new worker threads unlimitedly and everything will be back to normal. Note that you can always use a JMX client like jconsole to monitor the active threads and memory consumption of the ESB.

"I See You"

I saw it and it’s a beauty!

Went to see Avatar last week with some friends, and I got only two words about the movie – “Frikkin Awesome”. It’s undoubtedly one of the best movies I’ve ever watched and definitely the best movie I’ve watched in last couple of years. Everything about Avatar is so fascinating. The plot, sceneries, special effects, music and just about everything else is near perfection. I also liked the message that movie attempts to give. Avatar is about people who are at the two ends of the spectrum. On one end we have people who are willing to engage in dirty act, commit any crime, kill anyone and destroy anything to earn a quick buck. On the other end of the spectrum there are people who are willing to sacrifice their lives to defend what is considered correct and uphold the moral values. The movie takes you through the conflict between these two types of people.

If you still didn’t watch it you are not even a person. So make haste. It’s science fiction and fantasy at their very best. Also please do yourselves a favor by watching it on the big screen in a real theatre, rather than watching it on your home theatre system.

Mr. James Cameron sir, you are a genius. Thank you loads to the actors and the behind screen team of Avatar for the great entertainment. And congratulations on breaking the record previously held by Return of the King for the second highest-grossing movie ever. With the way things have been going, may be even the first place (which is currently held by Titanic) is a possibility right now.