Flexible Cloud Computing with AppScale

I recently started contributing to AppScale, an open source project aimed at developing a scalable Platform-as-a-Service (PaaS) solution. AppScale project was initiated by UC Santa Barbara with the intention of implementing an open cloud PaaS that would enable more research and studies in the area of cloud computing. But over the years AppScale has evolved rapidly gathering a wide range of features and now many enterprise users are finding it useful as a platform that facilitates private, public and hybrid cloud deployments. 

One of the most attractive features of AppScale is the flexibility it provides to the cloud administrators and cloud application developers. Cloud administrators can deploy AppScale on a variety of infrastructure setups. It can be deployed on virtualized computing clusters based on solutions such as Xen and KVM. AppScale also runs on Infrastructure-as-a-Service (IaaS) solutions such as Amazon EC2 and Eucalyptus (It is worth mentioning that Eucalyptus also started out as a research project in UC Santa Barbara). Also if needed AppScale can be deployed directly on physical hardware without the support of any virtualization service. AppScale also comes with a Ruby API and a set of command line tools that can be used to deploy AppScale clouds on any of the above infrastructure setups with minimal human intervention. A single shell command is all it takes to deploy even a 100-node AppScale cloud. To make this process even easier, I recently implemented a new web UI component which allows users to deploy AppScale without bothering about the infrastructure complexities at all (more on this in a future blog post).

As a PaaS offering, AppScale exports a wide range of services for the cloud application developers to use in their applications:

• Datastore - Persistent storage for application data. Generally operates as a replicated key-value store with support for range queries and transactions within entity groups.
• Namespace - Facilitates segmenting data into multiple partitions. Can be used in scenarios where certain data items need to be separated from each other (e.g: Production data vs Test data)
• Memcache - Distributed cache. Useful in developing stateful applications and improving application performance.
• Blobstore - Persistent storage for large data objects and files.
• XMPP - Provides instant messaging capabilities to AppScale applications.
• Channel - Allows pushing data into client's JavaScript code.
• Users - User account creation and profile management.
• Mail - Facilitates sending e-mails from applications.
• Images - Supports programmatic manipulation of images.
• URL Fetch - Facilitates consuming local and remote REST APIs.
• Task Queue - Facilitates asynchronous execution of long running jobs.

All these fundamental services are fully API compatible with Google App Engine (GAE). Therefore any GAE application can be deployed on AppScale with zero modifications. This has two very interesting outcomes for the users. First it makes it absolutely simple for the users to migrate from GAE to their own private or hybrid cloud offering based on AppScale. Second, it allows developing applications for AppScale quite straightforward as all GAE APIs are very well documented and comes with a powerful SDK. As a result AppScale has managed to gather a large number of sample applications and a very large developer community actively writing apps for AppScale in very quick time. Just like in GAE, applications can be developed in Java, Python or Go for AppScale. Another interesting aspect is that AppScale allows using a wide range of database systems underneath its Datastore API. Currently supported database systems include Cassandra, HBase, Hypertable, MongoDB, MemcacheDB, MySQL cluster, Voldemort and Redis. This is one area where the flexibility of the AppScale architecture can be observed clearly as it enables cloud administrators to setup AppScale with any one of these database solutions depending on their application requirements and organizational standards.

One thing that should be stressed is that AppScale is not just about running GAE applications. It facilitates deploying a wide range of other applications in the cloud too. This is mainly enabled by Neptune, a software overlay that runs on top of AppScale. It's comprised of a domain specific language that allows developers to execute any arbitrary program in the AppScale cloud PaaS. These programs may include standalone programs written using any arbitrary language, MapReduce jobs and high performance computing applications developed using technologies such as MPI, UPC, X10 and StochKit. The ability of AppScale and Neptune to execute high performance computing applications in the cloud has attracted a lot of attention from the scientific research community as it enables executing long running resource intensive tasks on the cloud using as many nodes as required thus greatly reducing the task completion time and eliminating the need to procure expensive server grade hardware.

On top of all this flexibility, AppScale also provides excellent fault-tolerance and autoscaling capabilities. All the critical services such as the database and the application server can be easily replicated for high availability. ZooKeeper is used to keep track of all the active services and nodes, and automatic failover is performed upon detecting failures. The AppScale autoscaler component keeps track of resource utilization and system performance related metrics, and spins out new nodes dynamically as the demand changes over time. Autoscaler is another very flexible component in the AppScale architecture, in that it allows cloud administrators to engage custom autoscaling policies depending on their application performance and scalability requirements. Some of the built-in autoscaling policies include HA aware autoscaling, QoS aware autoscaling and cost aware autoscaling. If needed more than one autoscaling policy can be engaged at once with an administrator defined priority arrangement. 

One of my personal favorite features of AppScale is its placement support. This is the ability of the PaaS to smartly place cloud services given a set of nodes. For an example if we start an AppScale instance  using three nodes (that is 3 physical or virtual machines), it will place an application server component and a database component in each of them. One of the database components would act as the master and the others would act as slaves. Automatic data replication will be enabled among all database components. One of the three nodes will be designated as the head node and the load balancer and ZooKeeper will be deployed in that node. Note that AppScale attempts to use the nodes in the most optimal manner possible by replicating all the critical services. The actual placement strategy however is also configurable in AppScale. But if the administrator does not explicitly state a placement strategy, we can rely on AppScale to figure out a suitable placement strategy on its own. 

If my introduction of AppScale has intrigued you to try it out, feel free to grab the latest stable source from our github repo. Detailed instructions on building the source and creating your own AppScale machine images can be found in the following wiki pages:

• Setting up an Ubuntu Lucid image for Xen or Eucalyptus: http://code.google.com/p/appscale/wiki/ImageCreationLucid
• Installing AppScale on a machine image: http://code.google.com/p/appscale/wiki/ImageSetupViaBzr

If you need a more ready-to-roll distribution of AppScale to take a quick look, check out our public EC2 image ami-52912a3b which is preloaded with AppScale 1.6.3. If you already have an EC2 account, you can simply setup AppScale command line tools on your computer and start an AppScale instance in EC2 using the tools and the above AMI.

I will roll out a couple of detailed blog posts on setting up AppScale in the near future, so stay tuned.

WSO2 API Manager: Designed for Scalability

Scalability is a tough nut to crack. When developing enterprise software and deploying them in mission critical environments, you need to think about the scalability aspects from day one. If you don’t, you may rest assured that a whole bunch of unpleasant surprises are heading your way. Some of the problems you may encounter are systems crashing inexplicably under heavy load, customers constantly rambling about the poor performance of the system and system administrators having to play watch dog to the deployed applications day in and day out. In addition to these possible mishaps, experience tells us that attempting to make a live production system scalable is hell of a lot more difficult and expensive. So it’s always wise to think about scalability before your solutions go live.

The crew at WSO2 have a firm grip on this reality. Therefore when designing and developing the WSO2 API Manager, we made scalability of the end product a top priority. We thought about how the overall solution is going to scale and how its individual components are going to scale. In general we thought about how the API Manager can scale under following circumstances.

• Growing number of API subscribers (growth of the user base)
• Growing number of APIs (growth of metadata and configurations)
• Growing number of API calls (growth of traffic)

Now let’s take a look at the architecture of WSO2 API Manager and how it can scale against the factors listed above. Following schematic provides a high level view of the major components of the product and their interactions.

When you download the WSO2 API Manager binary distribution, you get all the above components packaged as a single artifact. You can also run the entire thing in a single JVM. We call this the standalone or out-of-the-box setup. If you only have a few hundred users and a handful of APIs, then the standalone setup is probably sufficient to you. But if you have thousands and thousands of users and hundreds of APIs then you should start thinking about deploying the API Manager components in a distributed and scalable manner. Let’s go through each of the components in the above diagram and try to understand how we can make them scalable.

Databases

WSO2 API Manager uses 2 main databases - the registry database and the API management database. The registry database is used by the underlying registry components and governance components to store system and API related metadata. API management database is primarily used to store API subscriptions. In the standalone setup, these 2 databases are created in the embedded H2 server.

In a scalable setup, it will be necessary to create these databases elsewhere, ideally in a clustered and high available database engine. One may use a MySQL cluster, SQL Server cluster or an Oracle cluster for this purpose. As you may see in the next few sections of this post, in a scalable deployment we might cluster some of the internal components of the WSO2 API Manager. Therefore there will be more than one JVM involved. All these JVMs can share the same databases created in the same clustered database engine.

Settings for the registry database are configured in a file named registry.xml which resides in the repository/conf directory of the API Manager. API management database settings are configured in a file named api-manager.xml which also resides in the same directory. Additionally there’s also a master-datasources.xml file where all the different data sources can be defined and you have the option of reusing these data sources in registry.xml and api-manager.xml.

API Publisher and API Store

These 2 components are implemented as 2 web applications using Jaggery.js. However they require some of the underlying Carbon components to function – most notably the API management components, governance components and registry components. If your deployment has a large user base, then chances are both API Publisher and API Store will receive a large volume of web traffic. Therefore it’s advisable to scale these two web applications up.

One of the simplest ways to scale them up is by clustering the WSO2 API Manager. You can run multiple instances of the API Manager pointed at the same database. An external load balancer (a hardware load balancer, WSO2 Load Balancer or any HTTP load balancer) can distribute the incoming web traffic among the different API Manager nodes. Tomcat session replication can be enabled among the API Manager nodes so that the HTTP sessions established by the users are replicated across the entire cluster.

The default distribution of WSO2 API Manager has both API Publisher and API Store loaded into the same container. Therefore an out-of-the-box API Manager node plays a dual role. But you have the option of removing one of these components and making a node play a single role. That is a single node can act either as an API Publisher instance or as an API Store instance. Using this capability you can add a bit of traffic shaping into your clustered API Manager deployment. In a typical scenario there will be only a handful of people (less than 50) who create APIs but a large number of subscribers (thousands) who consume the published APIs. Therefore you can have a large cluster with many API Store nodes and a small cluster of API Publisher nodes (or even a single API Publisher node would do). Two clusters can be setup separately with their own load balancers.

Key Management

Key management component is responsible for generating and keeping track of API keys. It’s also in charge of validating API keys when APIs are invoked by subscribers. All the core functions of this component are exposed as web services.  The other components such as the API Store and API Gateway communicate with the key manager via web service calls. Therefore if your system has many consumers and if it receives a large number of API calls, then it’s definitely advisable to scale this component up.

Again the easiest way to scale this component is by clustering the API Manager deployment. That way we will get multiple key management service endpoints which can be put behind a load balancer. It’s also not a bad idea to have a separate dedicated cluster of Carbon servers that run as key management servers. An API Manager node can be stripped of its API Publisher, API Store and other unnecessary components to turn it into a dedicated key management server. 

User Management

This is the component against which all user authentication and permission checks are carried out. API Publisher and API Store frequently communicate with this component over a web service interface. In the standalone setup, a database in the embedded H2 server is used to store user profiles and roles. But in a real world deployment, this can be hooked up with a corporate LDAP or an Active Directory instance. To scale this component, we can again make use of simple clustering techniques. All the endpoints of the exposed user management services can be put behind a load balancer and exposed to the API Publisher and API Store.

API Gateway

This is the powerhouse where all the validating, throttling and routing of API calls take place. It mainly consists of WSO2 ESB components and hence can be easily clustered, just as how you would setup an ESB cluster. One of the gateway nodes will function as the primary node through which all API configuration changes are applied. API Publisher will communicate with the primary node via web service calls to deploy, update and undeploy APIs. Carbon’s deployment synchronizer can take care of propagating all the configuration changes from the primary node to rest of the nodes in the gateway cluster.

API Gateway also caches a lot of information related to API key validation in order to prevent having to query the key manager frequently. This information is stored in the built-in distributed cache of Carbon (based on Infinispan). Therefore in a clustered setup, information cached by a single gateway node becomes visible to other gateway nodes in the cluster. This further helps to reduce the load on the key manager and improves the response time of API invocations.

Usage Tracking

We use WSO2 BAM components to publish, analyze and display API statistics. BAM has its own scalability model. Thrift is used to publish statistics from API Gateway to a remote Cassandra cluster. Use of Thrift ensures that statistics can be published from API Gateway to the Cassandra store at a rapid rate. The BAM data publisher also employs its own queuing mechanism and thread pool so that data can be published asynchronously without having any impact on the messages routed through the API Gateway. Use of Cassandra enables fast read-write operations on enormous data sets. 

Once the data has been written to the Cassandra cluster, Hadoop and Hive are used to process the collected information. Analyzed data are then stored in a separate database from which API Manager (or any other monitoring application) can pull out the numbers and display in various forms of tables and charts.

Putting It All Together

As you can see WSO2 API Manager provides many options to scale up its individual components. However it doesn’t mean you should scale up each and every piece of it for the overall solution to be scalable. You should decide which components to scale up by looking at your requirements and the expected usage patterns of the solution. For instance, if you only have a handful of subscribers you don’t have to worry about scaling up API Store and API Publisher, regardless of how much traffic they are going to send. If you have thousands of subscribers, but only a handful of them are actually sending any traffic, then the scalability of API Store will be more important than scaling up the Gateway and statistics collection components.

MOINC and Scalability

My colleague Aravinda just published a cool post on his blog regarding the scalability testing round we carried out on MOINC a few weeks back. He has also published some charts illustrating the results of our tests. Have a look and see for yourself.