Showing posts with label http. Show all posts
Showing posts with label http. Show all posts

Thursday, January 3, 2013

The Era of Webapps is Over - Say Hello to Web APIs


I remember a time when developing a web application (webapp) was considered one of the coolest and most exciting feats a software developer could perform. It was not so long ago when a software product with no web application component was considered uncool, unpresentable and unmarketable. A wide range of dynamic web programming technologies and standards were born in this era and they continued to thrive thanks to the unprecedented growth and evolution of the Internet. However today if we take a closer look at how the Internet is being used in our day-to-day lives and in the business world, one thing becomes hauntingly obvious. The era of the webapps has passed. Now is the era of the web APIs.
First of all lets take a closer look at webapps. Webapps are designed to be directly consumed by human users. The user is aware of the URL through which the application can be accessed which he/she enters into a web browser. The web browser then interacts with the remote URL by making HTTP GET requests to pull content and HTTP POST requests to submit content. HTML is used as the primary data exchange format and how the content appears on the user’s web browser (style and formatting) is actually a huge deal. All the important information that should be communicated to the user must be embedded in the HTML payload as that’s the only thing that’s going to get rendered on the screen by the web browser. Things like HTTP status codes and headers do not play a big role in webapps, except perhaps when reporting an error (404 Not Found, 500 Internal Server Error etc) and performing a redirect (302 Found + Location header). 
But webapps are increasingly becoming less interesting to both developers and users. Today it’s all about web APIs. Interestingly most of the technologies that are used to develop webapps can also be used to create and expose web APIs. In fact it’s not wrong to say that web APIs are the next generation webapps and webapp development frameworks mutated into web API development frameworks as a consequence of the natural evolution of the web.
So how do web APIs differ from webapps? And what makes them cooler? Unlike webapps, web APIs are not designed to be directly consumed by human users. They are APIs, meaning they are designed to be consumed by other applications. Developers can use web APIs to construct other high level APIs and end-user applications. The end-user may or may not know the exact location or the URL with which the application is interacting with. Web APIs may use any content exchange format but JSON and XML are the most popular choices. A properly designed web API would use most of the available HTTP methods (at very least GET, POST, PUT, DELETE and OPTIONS) combined with the proper use of HTTP status codes and headers to pass critical control information. Things like layout and formatting don’t mean anything in the web API world but effective use of URL patterns, intuitiveness and simplicity of the APIs mean everything.
The end-user applications that are developed using web APIs can include desktop applications and more interestingly mobile applications. In my opinion this is the last nail in the coffin of the webapps. Simply put people are no longer browsing the web. Rather they use mobile apps. Latest reports and Internet usage surveys show that the amount of mobile Internet traffic is starting to bypass the amount of desktop Internet traffic (see the references section). This means the webapps are increasingly becoming obsolete. To point out a real world example lets take GitHub. GitHub is a pretty cool webapp, one of the best in my personal opinion. We can consume this app in a desktop environment using a traditional web browser such as Firefox. We can do the same with a smart phone, using the mini web browser the device is equipped with (eg: Safari for iPhones). But that’s not good enough for us. We need a native GitHub app for our smart phones. And as a result we now have the GitHub app for iPhone and Android platforms. Soon these mobile apps will become dominant traffic sources of GitHub. This has already happened to many of the popular social networking service providers such as Twitter and Facebook. The web API is a more critical component in these systems compared to their webapp counterparts. 
This transition from webapps to web APIs is a crucial one for technology driven companies. They have to carefully assess this trend and adjust their game plans accordingly. Many organizations have already realized the shift towards the web APIs and started to expose their business functionality as web APIs rather than as webapps. Web APIs open up businesses towards a larger and more diverse clientele with ample future proofing and more room for change. This massive push towards APIs has also given birth to the field of API management, which has now become a business of its own and starting to produce many lucrative business opportunities around the world. The growing number of on-premise and cloud API management solutions (Layer7, Apigee, Mashery etc) available is a testament to this fact.
Perhaps the most impacted by this change are the developers. They need to take a whole different stance in the way they think about software solutions that run over the web. They should learn to implement systems for other developers rather than for end users. They should learn to optimize systems for machine-to-machine interaction rather than for human computer interaction. They need to pay a lot of attention to little things like using proper HTTP codes, using proper HTTP headers, and adhering to open standards. Problems they are used to be messing with such as improving the browser compatibility of webapps and session management are becoming less important by day. They will have to learn to pay less attention to things like form authentication and adopt more HTTP-friendly security mechanisms such as BasicAuth and OAuth. API management is going to become a mainstream technology and developers will have to start treating it as primary development tool just like version controlling and issue tracking. Technology platform providers will also have to take this shift seriously. Developers no longer need webapp development frameworks. They need web API development frameworks. This is why platforms like Google App Engine has become so successful. Their staying power resides in their ability to facilitate the development of powerful web APIs with very little amount of code.
So does all this mean traditional webapps are dead (as in dead for good)? Well, not really. The need for webapps will always be there, at least for the foreseeable future. But we will see more dominant deployment and adoption of web APIs compared to webapps. Webapps will become the Cobol of 21st century; Thousands of lines of code are written every year, but nobody gives a damn. Newly implemented webapps will sit on a layer of web APIs making the webapp just one of the many frontends of a larger system. 
All in all, if you’re a developer, some very exciting times are ahead of you. So buckle up!

References

Sunday, December 13, 2009

WSO2 ESB Tips & Tricks 03: Transport Switching

Back to WSO2 ESB Tips and Tricks!
Today we are going to see how to implement a transport switching scenario with WSO2 ESB. WSO2 ESB supports receiving and sending messages over following transport and application layer protocols.
  • HTTP/HTTPS
  • Mail protocols (POP3, IMAP, SMTP)
  • JMS / AMQP
  • VFS (Virtual File System)
  • FIX (Financial Information eXchange)
  • TCP
  • UDP
With WSO2 ESB having support for such a wide range of transports, it is easy to implement a scenario which involves receiving messages over one protocol and forwarding them over another. As an example I’m going to explain how to receive messages over HTTP and forward them over JMS, using Apache Active MQ as the JMS provider. Looking at how most system integrators use WSO2 ESB it seems this is one of the most common use cases for ESB products.
The first thing we have to do is enable the JMS transport sender in the ESB. By default only the HTTP and HTTPS transports are enabled in the ESB configuration. But configuring and enabling other transports is a trivial task. Transports can be enabled by uncommenting the corresponding entries in the axis2.xml file or by using the ESB management console.
First, shutdown the ESB if it is already running. Then we need to deploy the Active MQ JMS client libraries into the ESB runtime. Download and extract an Apache Active MQ binary distribution (I prefer v5.2). Copy the following jar files from the Active MQ lib directory to the ESB_HOME/repository/components/lib directory.
  • geronimo-j2ee_management.jar
  • activemq-core.jar
Also this would be a good time to start the Active MQ broker.
Now start the ESB. The underlying Carbon platform will detect the newly deployed jar files and automagically convert them into OSGi bundles on the fly. Once the server has started go to the ESB_HOME/repository/components/dropins directory and do a file listing. You will see a set of OSGi bundles created out of the jar files you deployed.
Now that the server is up and running login to the management console as an admin. Click on ‘Transports’ under ‘Manage’ on the left navigation bar. A list of available transport receivers and senders will be displayed.

Select the ‘Enable’ option for the JMS sender. Click on ‘Enable’ to start the JMS transport sender. Once started, you can further configure the transport by specifying various parameters. But for this demonstration we can go ahead with the default settings (ie no parameters).

If you prefer to enable the JMS sender the old school way open up the axis2.xml file in the conf directory and uncomment the following entry before starting the ESB.
<transportSender name="jms" class="org.apache.axis2.transport.jms.JMSSender"/>
Having enabled the JMS sender, we should now create a proxy service which will accept messages over HTTP and send them to a JMS endpoint. Click on ‘Add’ under ‘Proxy Services’ to start the proxy service creation wizard. On the first step simply specify a suitable name for the service (eg: JMSBridge) and click on ‘Next’.

On the second step of the wizard, create an anonymous in-sequence consisting of a property mediator. Configure the property mediator to set the ‘OUT_ONLY’ property to the value ‘true’. Save the in-sequence and create an anonymous endpoint for the proxy service. Specify the following JMS EPR as the target address. This EPR points to a dynamic queue named ‘SimpleStockQuoteService’ in Apache Active MQ broker.
jms:/SimpleStockQuoteService?transport.jms.ConnectionFactoryJNDIName=QueueConnectionFactory&java.naming.factory.initial=org.apache.activemq.jndi.ActiveMQInitialContextFactory&java.naming.provider.url=tcp://localhost:61616&transport.jms.DestinationType=queue
In this scenario we will be doing a one-way invocation using the sample client application. In other words, the client sends a message to the ESB but does not expect a response in return. So the ESB should not expect a response from the backend JMS service either. We indicate this to the ESB by setting the OUT_ONLY property in the in-sequence of the proxy service. ESB will not register any call backs for messages containing the OUT_ONLY property thus no response would be expected.
Having specified the in-sequence and the target endpoint click on ‘Next’ to proceed to the third step of the wizard. Leave the out-sequence and the fault-sequence settings unchanged and simply click on ‘Finish’ to save and exit the wizard. A new proxy service will be deployed on the ESB at this point.
If you prefer to create the above proxy using the ESB configuration language you can use the XML snippet given below.
<syn:proxy name="JMSBridge" transports="https http" startOnLoad="true" trace="disable">
<syn:target>
<syn:endpoint>
<syn:address uri="jms:/SimpleStockQuoteService?transport.jms.ConnectionFactoryJNDIName=QueueConnectionFactory&java.naming.factory.initial=org.apache.activemq.jndi.ActiveMQInitialContextFactory&java.naming.provider.url=tcp://localhost:61616&transport.jms.DestinationType=queue"/>
</syn:endpoint>
<syn:inSequence>
<syn:property name="OUT_ONLY" value="true" scope="default"/>
</syn:inSequence>
</syn:target>
</syn:proxy>
On the management console, click on the ‘Synapse’ link under ‘Configure’ and add the above XML as a child of the top level ‘definitions’ element. Click ‘Update’ to apply the changes.
Now we are all set to run the scenario. Deploy the SimpleStockQuoteService into the sample Axis2 server. Open up the axis2.xml file for the sample Axis2 server and enable the JMS transport receiver in it. This file can be found at samples/axis2Server/repository/conf directory. To enable the JMS listener uncomment the following XML fragment.
<transportReceiver name="jms" class="org.apache.axis2.transport.jms.JMSListener">
......
</transportReceiver>
Finally start the sample Axis2 server. Since we have enabled the JMS receiver for the sample server, the SimpleStockQuote service will be exposed over JMS and the server will start listening on a JMS queue named ‘SimpleStockQuoteService’. When you haven’t specified a queue name in the service configuration, the Axis2 JMS transport will by default listen on a queue with the same name as the service. To explicitly specify the queue name for a service one must set the transport.jms.Destination parameter as a service parameter (ie in the services.xml file).
Finally go to samples/axisClient and run the following command to send a one-way message to the ESB proxy service over HTTP. (Here we invoke the in-only placeOrder operation on the backend service)
ant stockquote –Daddurl=http://localhost:8280/services/JMSBridge –Dmode=placeorder
The ESB will take the message and forward it over JMS to the queue named ‘SimpleStockQuoteService’. The sample Axis2 server which is currently listening on this queue will pick up the message and process it. Note the following log entry in the Axis2 server log when it takes the message from the JMS queue.
Sun Dec 13 16:15:39 IST 2009 samples.services.SimpleStockQuoteService  :: Accepted order for : 5789 stocks of IBM at $ 89.61765128551305
So there you have it. The proxy service abstraction in WSO2 ESB makes it extremely simple to perform protocol conversions and route messages back and forth. I suggest you also have a look at the set of transport switching samples that come with WSO2 ESB distribution. We got samples covering all the major protocols and even some of the domain specific transports like FIX and AMQP.
In my next post, I will focus on common problems that most developers encounter while implementing transport switching scenarios involving JMS and possible solutions to such pitfalls.