ApacheCon EU 2009....Finishing Touches

The ApacheCon EU 2009 (Amsterdam, Netherlands) came to an end today evening. We had a BarCamp on 23rd followed by a Hackathon on 24th. The conference commenced on 25th and continued until today evening. We had two keynotes and around 64 technical presentations during the last three days. I was fortunate enough to take part in the following presentations.

• Embrace OSGi (by Carsten Ziegeler, member of ASF)
• Tales from OSGi trenches (by Bertrand Delacretaz, member of ASF)
• OSGi as a framework for building a product line (by Ruwan Linton and Afkham Azeez, WSO2)
• HBasics: Hadoop's big database (by Michael Stack, Microsoft)
• Apache license as a business model (by Paul Fremantle, WSO2)
• Clustered Web services for high availability and scalability (by Ruwan Linton, WSO2)
• EDA with Apache Synapse (by Paul Fremantle, WSO2)
• Performance tuning Apache Tomcat (by Filip Hanik, SpringSource)
• How to become a project at ASF (by Martijn Dashorst)
• Introduction to NIO 2.0 (by Jeanfrancois Arcand)
• Using MINA 2.0 (by Emmanuel Lecharmy, IKTEK)
• Becoming a Tomcat super user (by mark Thomas, SpringSource)
• High availability != High cost (by Norman Maurer, HEAG MediaNet)
• Apache POI (by Nick Burch, Torchbox)
• Enterprise build & test in the cloud (Carlos Sanchez, G2iX)
• Shindig for blogs and wikis (Dave Johnson, VP Apache Roller)

I also got the opportunity to chair most of the above sessions thanks to the organizers and folks of the Apache TAC. Thanks guys! You rock!

Out of the two keynotes, I really enjoyed the keynote on 'open sourcing the analyst business' by James Governor. It was a real blast!

In addition to all the technical knowledge I gathered during the conference I also made quite a lot of new friends. I met some very interesting people working at various levels and projects of the ASF. All in all it was a very successful ApacheCon, not just to me but to everybody that participated. (I have a lot more to mention but unfortunately I have to cut it short. I have lot of packing left to do before I start the return journey tomorrow morning.)

Happy Birthday ASF

The Apache Software Foundation (ASF) celebrates its 10th anniversary today. Having finished the regular activities of the ApacheCon scheduled for the day, all the committers, PMC members and members got together to celebrate this great occasion. Jim Jagielski, the chairman of the ASF board of directors cut the giant birth day cake. It was indeed a fun filled and proud moment for everyone at Apache. Another important event that took place during the reception was the key signining party where around 20 folks including myself, got the opportunity to get signed PGP keys from ASF.

All in all it was a great birth day party with loads of fun and excitement. Happy 10th anniversary ASF!!!

ApacheCon EU 2009 Begins

After a 3 hour flight from Sri Lanka to India followed by an 8 hour flight from India to Netherlands, I finally arrived at beautiful Amsterdam city, to take part in the ApacheCon Europe 2009. The event is now being held at the Movenpick hotel at the Amsterdam city center. The main attraction of the first day was BarCamp Apache where a number of developers, committers and open source geeks like me presented on a variety of topics. I myself conducted a presentation on Apache Synapse and how it can be used to scale up Web Services deployments. Among the other presentations I really liked the presentation by Ricardo Varela on software for mobile platforms and the presentation by Ross Gardler on teaching open source. Sri Lankan student community was credited for their achievements in the last Google Summer of Code during the presentation by Ross :)

All in all it was a really awesome BarCamp with lots of great talks, t-shirts, sweets and coffee. Let's see how the things turn out tomorrow.

Web Services for Mission Critical Apps

While reading the “Beautiful Code” from O'Reilly Media I came across a chapter compiled by Ronald Mak. He was a senior scientist at the Research Institute for Advanced Computer Science and he has been contracted twice by the NASA to develop their enterprise applications to manage space missions. In the chapter 20 of “Beautiful Code” Ronald describes the internals of the Collaborative Information Portal (CIP) which he and his team developed for the NASA for the Mars Exploration Rover (MER) mission which commenced in year 2003. The development of the project has taken nearly two years and needless to say being a highly mission critical ERP solution the requirements of the system have been very demanding. Among the most fundamental features of the system were,

• Time management (Managing all time zones on Earth and two time zones on Mars)
  
• Personnel management (Managing all mission personnel) and,
  
• Data management (Managing all the data sent from the two automated Rovers on Mars)
  

So how did Ronald and his team tackled this mammoth task? They used a 3-tiered service oriented architecture (SOA). Most of the code has been written in Java and they have followed J2EE standards. Also the developers of the CIP have used commercial off-the-shelf (COTS) software as much as possible to avoid re-inventing the wheel thus saving loads of time and effort. The client tier of the system consists mainly of standalone GUI applications developed mostly with Swing. These client applications communicate with a set of Web Services hosted on a J2EE compliant application server. This forms the middleware tier of the CIP. Finally there is a data tier which consists of mission file servers, databases and other metadata repositories. Ronald explains that using an SOA based on J2EE, was one of the main driving forces behind their success with the CIP.

The Web Services in the middleware tier are perhaps the most important components of the system. These services have been implemented using Enterprise Java Beans (EJB). They have developed two types of beans, namely stateless session beans and stateful session beans. Stateless session beans act as the service providers by receiving all the client requests and routing them to relevant stateful session beans which implement the business logic of the system. Ronald further elaborates that use of Web Services made CIP modular, language independent and loosely coupled.

All-in-all Ronald and his team have manged to deliver the solution on schedule. The end system has recorded a 99.9% uptime, indicating how reliable the system was. In addition to using Web Services and SOA to make the system highly maintainable and scalable, they have employed a number of other techniques to shape up the CIP into a highly reliable ERP solution. According to Roland, logging was one such technique. They have used a system wide logging mechanism based on Apache Log4j to log all the important (and even not so important) activities that take place within the system. They have also developed services to specifically support the monitoring function of the system which means that they have considered monitoring as a fundamental requirement of the system. In addition, Ronald and the team made the CIP dynamically configurable and hot swappable to minimize the down time maximize the robustness.

I believe that CIP of NASA is one of many examples which effectively demonstrates the power of Web Services and SOA. These technologies can help even huge government agencies like NASA to develop large complex enterprise software systems on time while meeting all the strict reliability, availability and scalability requirements as desired. As the time goes on these state of the art technologies will prove to be even more powerful and useful. I'm just glad that I'm into further exercising SOA as a Software Engineer.

Social Networking: Bandwidth Eater?

Me and a group of my colleagues at University of Moratuwa, conducted a small research to analyze the impact of social networking on bandwidth usage in corporate networks. Under the guidance of Dr. Chandana Gamage, we studied the network resource utilization patterns triggered by browsing social networking sites such as Facebook and YouTube. We then put our observations and findings into a short research paper titled, “Network Resource Utilization and Social Networking”. Our research was rather small and probably not highly analytical but I think anybody who runs a corporate network should have a quick look at it. However if you are a sworn social networking fan, I would say it’s best not to read it since our findings might seriously piss you off ;)

On a finishing note let me just say that bandwidth on corporate networks is a valuable resource that is generally shared among many people. So use it carefully and use it responsibly. Having some fun with social networking is fine. But always know your limits and ethical responsibilities.

Pushing to the Limits

Just a couple of days back, me and the rest of the MOINC team conducted a large scale performance testing round for MOINC at the advanced computing lab of the Department of CSE, University of Moratuwa. We used over 20 PCs for our tests and the resulting numbers we obtained were pretty amazing. My friend and colleague Aravinda is already in the process of analyzing the statistics collected and we hope to compile a research paper based on the results. We will be publishing the details regarding the experiments as well as the collected data soon on our website in a more aesthetically pleasing manner. In the meantime I would like to say that we as the developers are very happy regarding the level of scalability the prototype MOINC platform has displayed during the tests.

Following are some of the pictures that were taken during the performance testing round. The sight of the MOINC screen saver coming up in about 20 PCs at the same time was truly a treat to the eye :)

I thank our department head, Mrs. Vishaka Nanayakkara and officer in charge of the advanced computing lab, Mr. Ananda Fransiscu for all the support in carrying out this round of testing.

Get Ready to Get MOINCed

Few weeks back me and my team managed to finalize developing the first prototype of our dream project, MOINC. If you are clueless as to what project MOINC is all about, it is an attempt to combine the Web Services paradigm with grid computing. The goal of the project is to be able to deploy enterprise Web Services for high availability and high scalability using commodity hardware. So far we have done three formal presentations describing each of the major components of the MOINC platform and we have written three research papers which we hope to publish very soon. In addition to the presentations we also conducted a formal demonstration of MOINC at University of Moratuwa, using 6 computers which effectively gave a preview of MOINC in action to a panel headed by our project supervisor Dr. Sanjiva Weerawarana and our senior lecturer Dr. Chathura De Silva. Presentations were attended by Dr. Sanath Jayasena, Dr. Chandana Gamage, Dr. Chathura De Silva and Mr. Shantha Fernando of the department of CSE, University of Moratuwa.

We got pretty good feedback from all those who attended the presentations and the demonstration, clearly indicating that we are on the right track to make MOINC into a useful software solution for business organizations worldwide. Some of the core features of the MOINC platform that we demonstrated are listed below.

1. Deploy, manage and undeploy service artifacts (Service artifacts are uploaded as Axis archives - *.aar files)
   
2. Track down idling computers in the local network and add them to the MOINC grid dynamically
   
3. Download service artifacts into idling computers from a centralized registry/repository
   
4. Action script based screen saver for client PCs
   
5. Load balance the incoming service requests among all the active nodes connected to the MOINC grid (Powered by Apache Synapse)
   
6. Monitor the grid via an AJAX based Web interface (Powered by WSO2 WSF/AJAX)
   
7. Collect statistics related to computers connected to the MOINC grid and use them in the intelligent load balance mode
   
8. MOINC community portal and forums (backed by WSO2 Registry and JForum)
   

That’s certainly a lot of features for a mere prototype. No wonder we got pretty good feedback. However there is certainly lot more work to be done. We need to improve the overall security of the platform. Currently there is a couple of security loop holes in MOINC SMM that we need to close off. Also Dr. Sanjiva suggested writing a Java security manager for MOINC client agent which ensures the security of PCs connected to the MOINC server. We need to start working on that soon as well. Also we have to finish our Maven2 integration stuff and get the release artifacts for MOINC 0.1-alpha out soon. We still haven’t cut the release artifacts for all the components of MOINC but the full source code of the prototype can be checked out from our SVN. Also checkout the developer resources section on our website for all the slides we used for the above mentioned three presentations.

I will also make arrangements for all of you humble readers to have a sneak peak at our research papers before we actually publish them. I promise. Meanwhile enjoy the presentations and other design documents :)

Carbonizing SOA

It's probably bit late to blog regarding this subject, but what the heck? After all, late is better than never.

WSO2 folks released their flagship product, the mother of all open source SOA solutions, WSO2 Carbon – Middleware a la carte. It is a componentized, customizable SOA platform built on Equinox and OSGi. The platform can be configured and tailored to suit the enterprise architecture of any business organization. Among the major components of the Carbon platform are;

1.WSO2 Web Services Application Server 3.0
2.WSO2 ESB 2.0
3.WSO2 Registry 2.0
4.WSO2 Business Process Server 1.0

All these high level components share the same set of core Carbon modules. It is really exciting to see how those high level middleware solutions combine with the core Carbon framework to deliver the results. Sounds interesting? Feel like learning more? Here you can find an introductory presentation on WSO2 Carbon. It gives a good insight into this revolutionary product and its architecture. Also don't forget to checkout other related articles and presentations on WSO2 Oxygen Tank.