GSoC 2008 is On

The all important initial phase of the Google Summer of Code 2008 (GSoC) came to an end last week. The selected project proposals were published on the GsoC website on 21st April. I too applied for this year's contest and my project proposal to implement type alternatives for Apache Xerces2-J was among the qualified proposals. The Apache Software Foundation has received around 30 slots this year and one of them is allocated to my proposal (yipeee!!!). I consider this as an great opportunity to learn and master XML and XML schema while contributing to a world renowned open source project.

Apache Xerces2-J is a high performance, standard compliant XML parser. It currently supports a number of XML related standards like XML 1.0, XML 1.1, DOM, SAX, JAXP and XML schema 1.0. A variety of open source and proprietary software projects make use of Apache Xerces2-J as the core XML parsing and processing mechanism. The reason for this immense popularity of Apache Xerces2-J is probably the high number of standards it supports and the way it supports them. Nowadays Apache Xerces2-J is even distributed along with popular Sun's JDK.

Xerces development team is currently involved in getting Xerces2-J to support the XML schema 1.1 standard which is the latest XML schema specification. XML schema 1.1 specification like its predecessor is comprised of three main parts namely the primer, structures and data types. Type alternatives is a feature that falls under the XML schema 1.1: structures spec. This is one of the most significant additions to the XML schema standard and it provides a well organized mechanism to implement conditional type assignment which has been in the XML schema feature wish lists for years.

With type alternatives XML elements can be assigned types based on one or more conditions (thus the name conditional type assignment). The conditions are specified as Xpath 2.0 expressions and the relationship between a condition and the corresponding type can be expressed using the 'alternative' element as in the following example.

The Xpath expression which specifies the condition is expressed as the value of the 'test' attribute and the corresponding type is expressed as the value of the 'type' attribute. Alternatively one could use 'simpleType' or 'complexType' child elements to specify the type instead of using the 'type' attribute. A complete example illustrating XML schema type alternatives would be as follows.
Here we have defined an element named 'value' which is of declared type 'valueType'. But based on the actual value of the 'kind' attribute the 'value' elements can have a different governing type. When XML schema validations are performed the elements will be validated against their governing types.

Type alternatives can add lot more flexibility to the way XML schema documents are used and it gives more freedom and power to the XML schema author. With type alternatives the XML elements having the same name can be of different governing types. In the above example different 'value' elements can take one of three types (integer, short or byte). Also the same element can have a governing type that is different of the declared type. This wouldn't have been possible if not for the type alternatives.

All in all type alternatives is a very interesting and useful feature for XML schema authors. That makes it very important for Xerces2-J to support it. I have worked with XML and XML parsers like DOM and SAX a lot in the past. I have used Apache Xerces2-J in a number of occasions too. But to be honest I haven't really worked with XML schema much. So this really is a big learning opportunity to me. I have been studying the XML schema specs for the last few weeks and I have already collected a whole bunch of stuff on XML schema to my knowledge base.

My heart is itching to start with the coding part but I know that there are lot of things to be studied, analyzed and clarified before I get to that point. Wish me luck!!

Say Hello to WSO2 Mashup Server

You have heard about it! You have read about it! You may even have dreamt about it! The WSO2 Mashup Server is a platform for creating, deploying, and consuming Web services Mashups in the simplest fashion possible. And when I say simple I mean like a piece of cake. In fact you don't have to know a damn thing about Web services to get started with the WSO2 Mashup Server. Some basic knowledge on JavaScript is all you need. Now how cool is that?

So today I'm going to provide some guidelines to those out there who are willing to get their hands dirty with this whole Web services Mashups thing. After reading this entry you will realize how easy (and not to mention the fun) it is to develop and deploy Web Services Mashups with the WSO2 Mashup Server. Provided that you have Java 5 or a higher version installed on your computer the first step is to download the Mashup server. Extract the downloaded archive and go to the bin directory. Simply execute the server startup script relevant to your operating system. Windows users should execute the startup.bat and Linux users should execute the startup.sh (you might need to set the execution permissions for this file). The server will start throwing out a bunch of text on your console and when the server has successfully started you will see something similar to the following.

INFO [2008-02-15 10:57:14,690]  HTTP port            : 7762
INFO [2008-02-15 10:57:14,691]  HTTPS port           : 7443
INFO [2008-02-15 10:57:14,691]
INFO [2008-02-15 10:57:14,691]  WSO2 Mashup Server started in 51597 ms

Now open up a Web browser instance and browse to the URL http://localhost:7762. You will be provided with a form to create a user account for yourself since this is the first time you are signing in to the Mashup server. Simply fill in the form and submit to create your account leaving the sign in check box checked. At this point you should be seeing your Mashup server home page.

And now begins the fun part. Click on the 'Create a new service' link under the management tasks. Give a name to the service and click create button. For this simple demonstration I'm going to create a service named 'helloworld'.

Then you will be provided with an editor to write your mashup. You may notice that a considerable amount of coding is already done for you but that is really not necessary at this point. Remove all that code and copy and paste the following code segment into the editor.

this.serviceName = “helloworld”;
this.documentation = “My first mashup”;

sayHello.documentation = “Greet”;
sayHello.inputTypes = { “name” : “String” };
sayHello.outputType = { “String” };
function sayHello(name)
{
return “Hi, and welcome “+name;
}

What we have done here is defining a simple service with one operation. The name of the operation is 'sayHello'. It takes in a string parameter called 'name' and returns a string value. More specifically when you invoke the operation sayHello with a name like 'Joe' it will return the string 'Hi, and welcome Joe'. Now click on the 'Save changes' button to save your workings. It will take some time to your service to get properly deployed and listed under the' My Mashups' list on the home page. If it seems to be taking too much time simply refresh the page.

Click on 'helloworld' link under 'My Mashups' to see the details of the helloworld mashup. Now you can go about and start playing with your mashup.
Do not forget to test your mashup with the 'Try It' feature.

Also you can view and edit the source of your mashup, give a rating to it, view the WSDL for the service and do a lot more. See how easy it is to get up and running with the WSO2 Mashup Server? Did you have to use any serious Web services stuff? No! Did you have to do any complex Java programming? No! All we did was typing a few lines of simple JavaScript code and making a few clicks here and there on a nice looking web interface. Now if you have signed up on http://mooshup.com you can directly upload your mashup to the mooshup community site by using the 'Share Mashup' feature so other mashup developers can have a look at your mashup. Sounds fun! Isn't it?

Let's Get Mashed Up!!!

Day by day Web Services are making the Web more dynamic, interactive and fun. Thanks to Web Services the World Wide Web which used to be a pile of content for reading and browsing has turned into the most dynamic and active source of information in the world. At the same time it has become a play field for those who love to play, a huge world wide dating service for those who wanna flirt, a medium for expressing viewpoints for those who want their opinion to be heard by others (I personally know a guy who used his blog to announce that he has started an affair with a girl) and a primary means of interactive communication for everybody in general. With Web Services the entire World Wide Web is restructuring itself into a pull-based model where the user pulls out exactly the content he wishes to see from the network. The traditional push-based model where the Web administrators simply push the content into their Websites that users has to browse and see is becoming more and more outdated. All the businesses are now rapidly adopting SOA and Web Services and they are organizing their business activities around Web services. Web services allow organizations to stay up-to-date on current socio-economic trends, competitors and the organization itself thus effectively reducing lot of overheads associated with management and control.

The best thing about Web services is the flexibility associated with them. One can decide to use a Web Service as it is or he can mix it up with other Web services to build his own customized Web Services. For an example take a Web service which brings you the latest up-to-date weather information in Asia continent in plain text format. If you are lazy, dull and a bit tardy you can consume the service as provided by the service provider. Or else if you are active, energetic and fun loving you can mix it up with other services to make it even more useful. In the example I have picked one might mix the weather data service up with an on-line mapping service like Google maps to create a dynamic weather map of Asia. Now how cool is that? As another example take a service which brings you the latest prices of cars. If we can mash it up with a service which brings you the architectural details of cars we can create a live feed which gives you all the details one would possibly want to know about cars.

Well it really sounds pretty exciting isn't it? But Web services are serious business. They are inherently complicated and difficult to handle. So how on earth one can just 'mash up' multiple services to form one service out of them when even handling one service requires lot of effort? Well believe it or not it's lot easier than you think, 'if' you are using the right tool. And what is this 'right tool'? It's called the WSO2 Mashup Server. All you need to know is a bit of JavaScript and you can get started with mashing up services in no time.

WSO2 Mashup server 1.0 was released recently and it's already making a lot of hype in the SOA world. WSO2 also put the mooshup.com (beta), the community site of mashup developers on-line couple of days back and there we already have a number of users sharing the mashups they have created. WSO2 Mashup server really makes mixing and mashing up services embarrassingly easy. I myself didn't know a damn thing about mashing services up few weeks back and now thanks to the WSO2 Mashup server I have already put a couple of mashups on-line at mooshup.com. 'YahooTunes', one of the mashups I developed recently takes the RSS feed from the Yahoo music (http://music.yahoo.com) which has a top chart of songs and mashes it up with the Youtube data API (http://gdata.youtube.com) to bring you the videos of the top ten songs. All I had to do was to write a few lines of JavaScript code. Most of the advanced tasks handled for you by the host objects provided by the Mashup server. 'YahooTunes' mashup is now on-line at https://mooshup.com/services/hiranya/yahootunes and is a good demonstration of the power of WSO2 Mashup server and what it has to offer. You will also realize that WSO2 Mashup server allows generation and displaying of WSDL and source code of mashups. Deploying, editing and managing mashups are also made easier tasks with the cool looking Web frontend. User management is provided using WSO2 User Manager and InfoCard authentication is facilitated using WSO2 Identity Solution. Sharing the mashups you create is also easy. Simply create and test the mashup on your local mashup server and hit share to upload it to the community site or some other mashup server instance.

WSO2 Mashup server is pretty stable and feature-rich for a 1.0 release. I didn't really find any major drawbacks and pitfalls so far. Full credit goes to all the developers that contributed to put this wonderful piece of software engineering together. However beware people. WSO2 Mashup server is known to be addictive!!!

FIXing Synapse!!!

Recently I got the opportunity to get involved in developing a new transport module for Apache Synapse to support the FIX protocol. Apache Synapse which is a light weight mediation framework for Web services, has had support for a number of application layer protocols like HTTP/S, SMTP, JMS and VFS. The development of the FIX transport module took nearly one and a half month and now this module is available in the Apache Synapse SVN trunk along with couple of samples and some documentation.

FIX protocol or the Financial Information eXchange is a messaging standard developed specifically to facilitate securities transactions. Strangely enough this protocol which is being used by hundreds of banks, stock exchanges and broker dealers all around the world is still not very popular in the Web services world (See here for a list of FIX users). The protocol has been in existence since 1992 and there are six major versions of the specification at the moment of writing (4.0, 4.1, 4.2, 4.3, 4.4, 5.0). The specs are owned by the FIX Protocol Limited (FPL) but it is essentially a free and open standard.

FIX specifications focus on two layers of the OSI reference model, namely the application layer and the session layer. Any application that wishes to communicate with another application using the FIX protocol must first establish a FIX session. A FIX session can exist among only two parties where one party is the acceptor and the other party is the initiator. Initiator is the one who starts the conversation by sending out the initial login request.

FIX messages are essentially a series of key-value pairs where each key-value pair is known as a field. Fields are separated by using the ASCII Start of Header (0001) character as the delimiter. The key of a field is simply a positive integer. But these integers have meanings and they are defined in the specifications. A typical FIX message might appear to be as follows.

8=FIX.4.09=10235=D34=1649=BANZAI
52=20080314-05:01:4756=SYNAPSE11=1205470907396
21=138=540=154=155=IBM59=0
10=078

A FIX message can be logically separated into a header, body and a trailer. The fields that should appear in each of these portions are clearly specified in the FIX specs. For an example the BeginString (8) field is a header field. The Checksum (10) is a trailer field. The content of a FIX message can vary greatly depending on the type of the message.

We used an open source FIX engine known as Quickfix/J to develop the FIX transport module for Apache Synapse. It currently supports five out of the six major versions of the FIX specification. Quickfix/J provides a very simple API to develop FIX based applications and applications developed on Quickfix/J are highly configurable. In addition to that Quickfix/J offers powerful message parsing, validation and logging.

Quickfix/J project is driven by a very active development team and a very enthusiastic user base. Quickfix/J uses Apache MINA and hence is based on Java NIO asynchronous network communications system.

All the transport modules of Apache Synapse are developed using the Apache Axis2 transport framework. Any transport module developed on the Axis2 transport framework must have two main elements, namely the transport listener and the transport receiver. The Axis2 transport framework provides the necessary interfaces and the base classes to implement these elements. The transport listener implementation is basically responsible for accepting in bound messages. For each accepted incoming message the transport listener should create an Axis2 message context, populate the message context accordingly and hand it over to the Axis2 kernel for further processing. The transport sender implementation is used by Axis2 kernel to send out messages. This implementation should be capable of processing Axis2 message contexts and converting the SOAP messages embedded in message contexts into messages that can be sent over the wire. Also depending on the nature of the protocol transport sender may also have to handle incoming response messages.

The implementations of the transport listener and the transport sender for the FIX transport module are named FIXTransportListener and FIXTransportSender respectively. The FIXTransportListener makes use of a FIXSessionFactory class which takes care of creating, storing and managing FIX sessions. The class FIXIncomingMessageHandler is where Apache Synapse binds with Quickfix/J. This class implements the quickfix.Application interface. For each accepted FIX message the transport module forks off a new thread from the thread pool associated with the transport listener implementation. This thread then converts the FIX message into XML using the Apache AXIOM API.

The FIX message converted into XML is then placed in a SOAP envelope. When converting FIX messages into XML, CDATA tags are used, basically as a precaution because theoretically FIX messages can have any kind of data in the fields. Fields can even contain XML or binary data. The transport do not change any of the field values while converting the FIX messages into XML. However if binary data is found in the message then necessary action will be taken to put the binary data in an Axis2 message context as a binary attachment. Finally the SOAP envelopes holding the FIX messages will be handed over to Axis2 kernel for further processing.

Another possible approach we could have taken here was to embed the FIX messages in SOAP envelopes without converting them into XML. But converting the FIX messages into XML has many advantages. Once converted into XML the Synapse user have more control over the FIX message content. Technologies like XPath and XQuery can be used to manipulate the FIX message content within the Synapse core.

One major problem we faced during the development of the new transport module was implementing in order message delivery. In order message delivery and processing is a characteristic feature of the FIX protocol. But since Synapse uses a separate thread to handle each incoming message we could notice that messages are not sent out in the order they were received. This is due to the thread switching that takes place while Synapse is performing the mediation. As a solution to this issue we introduced an application level sequence numbering mechanism to the transport module. Each and every incoming FIX message will be given a sequence number by the transport listener. A sequence number is unique for a given FIX session. A string that uniquely identifies the session is also associated with the messages. These information are specified in the SOAP envelope it self as attributes of the message element. The FIX transport sender implementation looks at these values and sends the messages in the exact order they were received. However having these attributes in the SOAP envelope is optional. They are required only if the user wants the FIX transport sender to send out messages in the order they were received.

While developing the FIX transport module we tried our best to provide all the options and choices Quickfix/J normally provides to the users. Apart from the logging features provided by Apache Synapse, users can enable logging at transport level (before messages are converted into XML) using Quickfix/J. Quickfix/J offers a number of options when it comes to message logging. You can either log messages on to the console, into a file or into a database. Same thing can be done with Synapse as well. Also all the message store implementations that come with Quickfix/J are available with Synapse too. By default Synapse will try to use the memory based message store implementation with acceptors and initiators. But the users can use other implementations (file, jdbc etc) if they want.

All in all developing the FIX transport module for Apache Synapse turned out to be a huge success and it indeed was a great learning opportunity for me. Not much ESBs in the world currently support the FIX protocol so it really improved the value and marketability of Apache Synapse. This module is still somewhat in its child state so we expect the contributions from the developers and FIX experts out there to improve it.

Hello World with Web Services

“Time for minor skirmishes is over!!! Now we do battle!!!”

Don’t get alarmed by reading the above. I’m still blogging about Web services and related technologies (not about ancient war craft). I just wanted to say that having discussed a lot of theoretical stuff in the previous blog entries (or rather going around and around the bush) now I’m fully armed and ready to start developing and deploying Web services of my own. There are loads of tested and proven methods, tools and IDEs to help Web services developers. So to start with I’m going to stick to a very simple and straight forward method which involves Apache Axis2 (well actually I will be using WSO2 WSAS, which is powered by Axis2), Eclipse IDE and Axis2 Eclipse plugins. It is pretty much a very painless and known-to-work kind of method and ideal for budding Web services developers (and even for pros of the industry). Let this blog entry be a tutorial to those out there who have just stepped into the world of Web services and willing to experiment with the technology (and may the best developer win :-)).

Here’s what you need…

• Java Development Kit (Download the latest version from http://java.sun.com/javase/downloads/index.jsp)

• Eclipse IDE (Download the latest version from http://www.eclipse.org/downloads)

• WSO2 WSAS (Download from http://wso2.com/products/wsas)

• Axis2 Eclipse Plugins (Download from http://ws.apache.org/axis2/tools/index.html)
  (You are going to need the 'Service archiver wizard – Eclipse plugin' at minimum)

Set them up...

Installing these software is fairly easy and straight forward. Remember to setup the JAVA_HOME environment variable after installing the Java development kit.

• For a tutorial on how to install and configure JDK on your system refer http://www.javacoffeebreak.com/tutorials/gettingstarted/part1.html

• For a tutorial on how to install Eclipse SDK visit http://www.cs.umd.edu/class/fall2004/cmsc131/EclipseTutorial/install.html

• For a tutorial on how to install Eclipse plugins visit http://open.ncsu.edu/se/tutorials/install_plugin/

• For a tutorial on how to install WSO2 WSAS refer http://www.wso2.org/project/wsas/java/2.1/docs/installation_guide.html

(A public WSO2 WSAS instance is available at http://wso2.org/tools from which you can test your Web services and get a feel for it.)

Once you have all the software tools you need properly installed and tested we can get on with developing out first Web service.

And here’s the recipe…

Run Eclipse IDE and create a new Java project. (File --> New --> Java Project)

Give the project any name you like.

Create the following class in the project. (Names of the class and the package do not matter. Just give any name of your choice.)

To add a new class to your project simply right-click on the node corresponding to your project on the Package explorer pane and click on ‘New’ and then select ‘Class’ from the submenu.

package testws;

public class MyFirstWS
{
   public String greet(String name)
   {
    return "Hi, " + name + ", welcome to Web services World!!!";
   }
}

As you can see it is a very simple Java class with just one method. It accepts a String parameter called ‘name’ and returns a String value.

Now save all the files and build the project. (Project --> Build Project)

Then run the Axis2 Service Archiver Wizard. (File --> New --> Other --> Axis2 Wizards --> Axis2 Service Archiver)

Browse and specify the class file location of your project. Generally this is the ‘bin’ folder in your top level project folder. Also check the ‘Include .class files only’ check box (“this is not really necessary, but then again why take any risks right?”) and click ‘Next’.

Next window allows you to specify a WSDL for the Web service. Since we don’t have a WSDL for our service simply check the ‘Skip WSDL’ check box and click ‘Next’.

You can add any additional Java libraries required to run the service from the next Window. Since we don’t need any additional libraries for our service simply click ‘Next’ to proceed. (However if you want to add any additional libraries just enter the full path of the library and click ‘Add’. You may add any number of libraries in this fashion.)

Next window deals with the service.xml file which is important when it comes to deploying the Web service. In this case we will get our service.xml file auto generated. So check the ‘Generate the service xml automatically’ and click ‘Next’ to continue.

Now you can specify the Java class that you want to expose as a Web service. Type the fully qualified name of the class (in my case it is testws.MyFirstWS) and click ‘Load’. The methods of the class should appear in the area below. Select the methods you want to be in your service. Also at the top most text box you can specify a name for your Web service. This name will be displayed to the external parties so best to pick a nice and cool name.

Finally specify a name for the service archive file that will be generated and also a location to save it. Click ‘Finish’ to exit the wizard. If everything went right Eclipse will display a message box saying ‘Service archive generated successfully!’

Now browse to the location you specified in the last step of the wizard. A file with the name you specified and an .aar extension will be there. This is the service archive file of your Web service and you are going to need this file to deploy the Web service. Service archive files are in a way similar to compressed .zip files. They are collections of other files. Mainly it consists of the class files and the service xml.

Now it’s time to put the service on-line. If you are using WSO2 WSAS simply start the server, log on to https://localhost:9443 using your web browser and sign in to the WSAS management console. The default username and password are ‘admin’ and ‘admin’. Click on ‘Services’ and Click on ‘Upload service artifact’ which is the top most link in the page. Browse to the location of your .aar file and click ‘Upload’. WSAS will display a message saying the service archive has been uploaded. (WSO2 WSAS supports hot deployment. So there is no need to restart the server.)

If by chance you are using Tomcat or Axis2 standalone server to deploy Web services copy the .aar file manually to the ‘webapps’ directory of your Tomcat or Axis2 installation. Then restart the server.

So that’s basically it. You have created and deployed a Web service. You can access service by logging into the WSAS management console and clicking on the ‘Services’. Your service will be listed their along with other Web services. Now you can do a lot of things with it. WSO2 WSAS gives you a whole bunch of features to play around with deployed Web services. You can view the WSDL of your service, generate clients for it and even try it out on-line using the WSAS 'Try It' feature.

You can directly invoke the service and test it if you want. Open a browser window and log on to http://localhost:9443/services//greet?name=

If you get a result similar to the following you have done right. It means your Web service is up and running and also it is working without any faults. So pat on your back and be proud of yourself.

Wow!!! Wasn't that easy? Well that's mainly because we used a very powerful IDE to develop the service and a feature rich application server to deploy it. If we had to do everything manually this would have been a pain in the butt. I think we really should appreciate these software tools and the people who have put in a lot of effort to create them.

Please note that I have done the above example in Linux. But the exact procedure can be safely carried out in the other environments like Windows and Mac-OS.

The Bee and the Eye

A new paradigm…

Information technology is probably not the only science that has gone through some drastic improvements during the last couple of decades. Many other sciences have lifted themselves from the ground level to a much higher level where humans now find it difficult to live without them. Some new sciences have popped up from nowhere and have managed to entirely change the way humans used to look at the nature and the universe at the beginning of the last century. Some sciences have combined together to create so powerful technologies giving rise to a variety of new application domains.

Business management, technology management and human resource management are few such sciences that have really become key aspects of any organization, society or nation today. Even though these sciences are not at all strange or peculiar to the present day human beings, I seriously doubt that people who lived about 100 years back had even heard the terms. Above sciences which are sometimes collectively referred to as ‘management’ have really become absolute necessities of organizations and people than just areas of study. So many new theories and principles have emerged in these sciences which in turns have affected the way people now carry out various tasks within an organizational environment.

In the meantime information technology has always been in its usual ever developing mode discovering more and more new application domains. It is now impossible to find an area of study or a science that hasn’t been touched by information technology. Management sciences have also been vastly reshaped and reworked with the introduction of information technology. Computer has become the most widely used business tool in the world (Paul Fremantle, one of the co-founders of WSO2 believes that a laptop computer is one of the basic things one needs to have, in order to start a business) and most of the software systems developed today are targeted towards organizations to help them with their management activities. Information technology is now so tightly entwined with management sciences and that has given birth to a new area of study formally known as ‘Business Intelligence’ or in short BI.

Explaining the Bee(B) and the Eye(I) …

BI is mainly concerned with applications and technologies used to gather, provide access to and analyze data related to organizational operations. A business intelligence system of an organization is one of the most important and key factors which directly affects the organization’s decision making process. Business intelligence systems generally have powerful data processing capabilities and effective presentation methods. A business intelligence system enables the administration of an organization to make more informed, timely and accurate decisions while contributing to the organization’s competitive advantage. With a BI system the administration no longer has to depend solely on the common sense and the experience. Also risky, ad-hoc methods of problem solving and decision making can be avoided.

Talking about BI systems there are hundreds of software tools, (for different types of platforms) each optimized for one or more subtasks associated with BI. Some of these subtasks are data gathering and storing, data security, data analysis, reporting and communication. BI systems are ideally suited for handling huge amounts of data. They can generally work well with multiple databases or data warehouses. BI systems are slightly smarter than usual software systems in that they can consider a large number of parameters while analyzing data and make accurate decisions. In the industry we can see BI systems being used in three modes.

• As a means of analyzing performance, projects and operations internal to organizations

• As a means of data storing and analyzing

• As a tool for managing the human side of businesses (marketing, HRM etc.)

BI has its own terminology with unique technical terms and buzzwords. Each of the above mentioned usages of BI are backed by a variety of BI systems and tools, both proprietary and open source.

OLAP Overlapped!!!

Since I have strongly emphasized on the usefulness and power of BI systems it is worth mentioning something about OLAP (On-Line Analytical Processing) as well which is a concept associated with BI. The term ‘OLAP’ is probably the most popular buzzword in the study of BI. Some common applications of OLAP are business reporting, management reporting, budgeting and forecasting. With OLAP the primary concern is to optimize databases and other related applications to support fast retrieval, analysis and reporting of data. In OLAP, databases are optimized for fast retrieval rather than efficient use of space. There are special APIs (eg: ODBO) and query languages (eg: MDX) to be used with OLAP. (In fact MDX is the de-facto standard being used with OLAP)

Here Comes Web Services…

Web services being the next ‘big thing’ in the world of IT has already begun to make an impact on BI. Web services can take BI to the next level by adding the notion of interoperability to BI systems. With Web services there is no limit to the number of data sources a BI system might use as Web services can enable the BI system to talk to remote APIs, databases and other data sources, even ones implemented using incompatible technologies. Also one can use Web services to expand the usability of a BI system by distributing the services offered by the system over the web. It is almost impossible imagine what a BI system can become when used along with Web services.

However there are some pitfalls that we should lookout for. When the services of a BI system are exposed to the external world as Web services some critical measures should be taken to enforce security and reliability of the system. Many BI system providers use various tools, protocols and technologies for this purpose (ranging from tools like LDAP to protocols like HTTPS). Also using Web services with BI systems has a significant impact on how metadata is handled by the BI system. Metadata management is one of the most influential features of a BI system. When used with Web services almost all the metadata management tasks including metadata modification and synchronization across applications should be exposed as Web services.

By combining the power of interoperability of Web services with the power of mega scale data analysis of BI systems we can create dynamic real time BI systems so that organizations can monitor events in real time and make very accurate decisions based on the most up-to-date information. All the users of such a system can be dynamically notified and kept in synchronism with the organizational activities by sending constant data feeds or periodic updates. Data gathering and reporting processes can be fully automated in this approach. Traditional BI systems are more or less batch processing systems that takes a collection of data and perform some operations on it as and when users make requests. But with Web services BI systems can constantly monitor data and related events in real time and take actions as and when a situation arises.

Huawie ETS 1000 on Linux

My CDMA dial-up Internet connection at home has been driving me insane for months. I have both Windows XP and Ubuntu Linux installed on my PC. From the two operating systems I prefer to use Linux for all my academic work, development work and documentation stuff. The purpose of having Windows is simply to play computer games. But unfortunately up to now I have never been able to connect to the Internet from Linux using the dial-up connection.

My CDMA dial-up connection is made using a Huawie ETS 1000 wireless terminal and the installation disk I got from my ISP had only Windows drivers. So ironically I had to use Windows to surf the Internet from home since Linux didn't seem to treat the wireless terminal nicely. Not to mention that this was rather silly and painful because every time I had to search the Internet for something during my day to day work I had to shutdown Linux and start Windows on my PC.

Recently I realized that I never really looked for an answer to this ridiculous problem. After searching for a solution on Google for a few minutes I understood that the answer has been lying right under my nose for about ten months. Apparently one my friends had already discovered a workaround for the problem and he had published the solution on his blog on last June.

Moral of the story – 'Discuss your technical issues openly with your friends. They may have already solved them.'

The solution my friend Mohanjith had found worked for me and some of the comments made on his blog suggest that it had worked for many others as well. So I thought of sharing it since I'm pretty sure there could be lot of others like me who have trouble getting connected to the Internet using CDMA dial-up connections on Linux. In Sri Lanka of course almost all the ISPs that offer CDMA based Internet connections provide their clients with Huawie ETS 1000 wireless terminals. Let this be a walkthrough to those who have trouble getting connected to the Internet using CDMA dial-up connections on Linux.

Let's Roll
Start by checking the Linux kernel version installed on your computer. It has to be above 2.6.*. You may check the kernel version by entering the command uname -a on your console.

$ uname -a
Linux hiranya-laptop 2.6.22-14-generic #1 SMP Sun Oct 14 23:05:12 GMT 2007 i686 GNU/Linux

Step 1:
If that condition is met simply plug in the USB cable coming from your Huawie terminal to your PC. Then type in dmesg -c on the console. You might have to become the super user for this command to execute. On Ubuntu of course you can make use of sudo. However executing this command will throw out a bunch of text on the console. Try to see if you can find the following couple of lines among the text.

ti_usb_3410_5052 1-1:2.0 : TI USB 3410 1 port adapter converter detected
usb 1-1: TI USB 3410 1 port adapter converter now attached to /dev/ttyUSB0

If you can find them you are lucky. Your wireless terminal is installed and ready to use. Simply proceed to step 3. But if you find the following couple of lines instead you got some work to do.

ti_usb_3410_5052 1-1:1.0: TI USB 3410 1 port adapter converter detected
ti_usb_3410_5052: probe of 1-1:1.0 failed with error -5

Problem here is with the connector cable that hooks the Huawie terminal up with your PC. No need to panic. Let's take care of this problem.

Step 2:
Create a new rules file called 026_ti_usb_3410.rules in your /etc/udev/rules.d directory. You will have to become the super user to create a new file in this directory. Using your favorite text editor add the following entries in the newly created file.

#TI USB 3410
SUBSYSTEM=="usb_device" ACTION=="add" SYSFS{idVendor}=="0451",SYSFS{idProduct}=="3410" \
SYSFS{bNumConfigurations}=="2" \
SYSFS{bConfigurationValue}=="1" \
RUN+="/bin/sh -c 'echo 2 > /sys%p/device/bConfigurationValue'"

Once done save and close the file. Now disconnect the USB cable from the Huawie terminal and plug it in again. Enter dmesg -c command in your console and inspect the output. If you have done everything right so far you should be able to locate the following lines in the output.

ti_usb_3410_5052 1-1:2.0: TI USB 3410 1 port adapter converter detected
usb 1-1: TI USB 3410 1 port adapter converter now attached to /dev/ttyUSB

That's it!! Your computer is now ready to go on-line with your CDMA dial-up connection.

Step 3:
Become the super user and edit the /etc/wvdial.conf file of your PC accordingly. My configuration is as follows.

[Dialer Defaults]
Modem = /dev/ttyUSB0
Baud = 230400
Phone = your-isp-phone-number
Init1 = ATZ
Stupid Mode = 1
Dial Command = ATDT
Username = your-username
Password = your-password
New PPPD = yes
PPPD Options = crtcts multilink usepeerdns lock defaultroute

Now try to get connected by entering the command wvdial pctl. You might have to execute this command as the super user.
There may be issues with name servers. In such a situation you will have to configure the name servers in the /etc/resolv.conf file. The name server IPs can be found from the console output when wvdial is trying to connect to your ISP.

WvDial<*1>: local IP address 122.255.2.26
WvDial<*1>: pppd: ��[06][08]
WvDial<*1>: remote IP address 2.2.2.2
WvDial<*1>: pppd: ��[06][08]
WvDial<*1>: primary DNS address 202.124.160.2
WvDial<*1>: pppd: ��[06][08]
WvDial<*1>: secondary DNS address 122.255.1.2
WvDial<*1>: pppd: ��[06][08]

First Step into The World of Web Services

Down the Memory Lane...

Information technology has grown in leaps and bounds over the last two or three decades. It has gradually developed into a state which was never anticipated at the inception of the computer technology. Today, computers are used as a powerful business tool, a medium of communication which connects people all around the globe and an unlimited source of information and entertainment. Computers play a major role in almost all kinds of businesses and have become a key ingredient in the areas like education, communication and resource management.

Probably the two most important driving forces behind this rapid and significant improvement of information technology are the developments in the technologies (hardware and software) itself and the introduction and rapid growth of the World Wide Web (WWW).

Hardware and software technologies have gone through some major improvements in the last few decades. Few years back having a computer running on a 400MHz Intel Pentium II was considered great. I used to admire some of my friends who had such computers. But nowadays having a computer of that nature will make others think either you are extremely poor or you have gone nuts. This is a simple indication of how hardware technologies have developed. Nowadays finding a computer running on a processor with a clock speed in excess of 3.0GHz and having physical memory in excess of 1.0GB is sometimes easier than finding a loaf of bread. Software technologies are not so far behind hardware in terms of performance and features. The days of small software applications with boring UIs and meager capabilities are long gone. Enterprise scale software applications with loads of features and dashing interfaces have come into play. Industry no longer talks about small applications. If a software does not have much features it is considered bad.

The WWW has given an entirely new set of capabilities to the existing software. Even though the WWW was initially used merely as a source of information and a medium to publish and retrieve information, today it is being used for things which were never thought possible few years back. With WWW the software residing in one physical machine can communicate and collaborate with some other software located in a distant remote machine to get some job done. This has become a possibility mainly because of the inception of the concept of Web services. According to the W3C a Web service is defined as follows.

"A software system designed to support interoperable machine to machine interaction over a network"

The keyword to be noted here is 'interoperability'. Web services allow machines and software systems to interoperate by exchanging data and information among them without relying on their underlying implementations. This means a software systems written using Microsoft .NET framework running on Microsoft Windows operating system may collaborate with a software system written using Java running on Linux operating system. As long as the two systems can understand the messages sent by the each other system they can work in collaboration like a newly married couple.

Web Services is the Way....

Web services can be very helpful (may be more than just 'helpful') in today's complex and growing network of businesses. Today, organizations around the globe often need to collaborate with many other organizations to get jobs done. Concepts like outsourcing, globalization and internationalization have become necessary ingredients of all types of business models. With this kind of dynamic communication and collaboration oriented businesses and organizations, even standalone enterprise software systems do not stand much of a chance. Different organizations use different kinds of technologies, different types of hardware platforms and different types of software systems. Trying to enable these diverse systems to collaborate could be difficult than finding kryptonite without Web services. As mentioned before Web services allow systems to leave their technological and environmental differences behind and interoperate to get a particular job done.

Also Web services allow valuable resources (eg: databases, software APIs) to be accessed and used remotely by a large number of people. Thanks to the Web services people no longer have to work on one dedicated PC anymore. They can decide to work at home, at a friend's place or at the other end of the world. Web services are very flexible in that we can develop different types of client applications with different capabilities to consume the same Web service. Web services are generally secured, reliable and easy to manage and maintain.

Many organizations around the world are fast switching to Web services from casual software systems because they are more reliable and cheaper. It gives a whole lot of confidence, flexibility and freedom to the end user and improves the profitability of the organization as a whole. Organizations that use Web services are more visible and reachable from the point of view of the external world. Web services are getting attention all around the world due to these reasons any even most of the software companies now openly support development and use of Web services. Some organizations like WSO2 and Apache Software Foundation have clearly undertaken the mission of popularizing the use of Web services at industry level and at day to day applications.

A Glimpse at Web Services Platform...

Web services are fairly complex software systems to understand. They are made of many different components. So a layered model (a stacked model) called the Web services stack is used to easily analyze and understand the principles behind Web services. Two key ingredients of the Web services platform are HTTP and XML. HTTP is an application layer protocol in the TCP/IP stack commonly used in WWW. XML is a simple but very effective language that can be used for message passing and data representation. In Web services XML is the most commonly used medium of message passing. Almost all the key protocols and standards are built around XML.

Adhering to the standards and specifications is very important in Web services. To enable two different systems to communicate and collaborate it is vital to have an agreed upon set of standards. Then only the messages sent by one party can be fully understood by the other parties. SOAP is one standard protocol used in Web services for message passing. It is based on XML and provides the basic messaging framework for Web services and forms the foundation layer of the Web services stack. Upon this foundation formed by SOAP many other abstract layers can be built to complete the Web services stack. Higher layers of the Web services stack provides more advanced features like Web services security, Web services policy and Web services manageability.

WSDL, another XML based language is used to describe Web services to external parties so they can properly communicate with the Web services and build up client applications to consume Web services.

The Dark Side of Web Services...? - Well it's Not That Dark...

One major point against the popularization of Web services is that they are very complex to understand and difficult to develop. Some people question the performance levels achieved by Web services due to the use of XML, SOAP and HTTP. Some argue that Web service principles tend to violate the normal model on which the Internet operates. According to the TCP/IP stack HTTP resides in the application layer and that is the top most layer. With Web services we mount another set of layers on top of application layer or to be more specific on HTTP.

However in my personal opinion Web services are in existence as a result of the inevitable improvements in the technology and changing user requirements. Therefore sooner or later they are going to dominate the world of information technology. This transformation is gradually taking place as you read this page. New development techniques, tools and frameworks have been developed for easy development of Web services and client applications. As a simple example, the WSO2 Web Services Application Server, a state of the art server for Web services (with lots of other features) comes with a plugin for the popular IDE Eclipse. With that the end user can easily develop Web services using Eclipse and directly deploy them using the WSO2 Web Services Application Server (you don't have to know a damn thing about Web services stack and stuff). Many popular IDEs today, including Eclipse and Microsoft Visual Studio support development and testing of Web services. Lot of support and guidance is also available for Web services developers.

Web services are indeed the next big thing! (Perhaps they already are) Because of that may be after few years people will start making fun of enterprise software systems like they do now about those old console based applications. (“Console applications...ha ha ha...what a retard!!!”)