This presentation is available for download from: http://ciurana.

eu/epicenter2009

Mission-Critical Enterprise Cloud Applications

Eugene Ciurana Open-Source Evangelist CIME Software Labs http://ciurana.eu/contact

About Eugene...

15+ years building missioncritical, high-availability systems 13+ years Java work Open source evangelist Official adoption of open source/ Linux at Walmart worldwide State of the art tech for main line of business roll-outs

Largest companies in the world Retail Finance Oil Background: robotics to on-line retail

This presentation is about...

How to design, implement, and roll-out cloud/ enterprise hybrid applications Different cloud architectures

PaaS SaaS Infrastructure as a service (IaaS)

Technologies: ESB, clouds, mini-clouds, Java, App Engine, Chef!/Puppet, etc. How cloud technologies lower costs Understanding the advantages of:

Cloud deployments Hybrid deployments distributed in the cloud and enterprise

What Youll Learn

Identify the applications best suited for cloud deployments Identify the advantages of Platform-as-a-Service or Software-as-a-Service resources Learn the caveats of cross-platform and crosslanguage integration Learn about high-performance alternatives to XML serialization for data exchange Learn how to structure event-based, stateless apps for maximum scalability

What is the Cloud Anyway?

Ask 10 different people, get 10 different answers In general, you may use 4 types of cloud offerings

Platform as a Service Software as a Service Infrastructure as a Service Pure infrastructure

Some times you integrate pre-fabricated apps, some times platform, some times both

Cloud Services Features

Quick deployment of prepackaged components Uses commodity, virtualized hardware and network resources
Amazon Elastic Cloud 2 (EC2) and Simple Storage Service (S3) Google App Engine (Python, Java) Rackspace Cloud Services

The overall model is pay as you consume Horizontal scalability is achieved by adding or removing resources as needed May host full applications or only services

Cloud Services Features

They could replace the data centre Basic administration moves to the application owner

It may move away from the IT team - political fallout Tax advantages Turn on or off as needed In a tight economy, IT infrastructure ends up under the CFO - give the guy options

For the bean counters... its an operational expense!

Assuming sensible SLAs, the ROI is better than for co-located or company-owned data centres

Scalability and High Availability

What do scalability and high-availability mean? Scalability: the property of a system to handle bigger amounts of work or to be easily expanded in response to increased demand

Vertical Horizontal
Virtual Node 3
Virtual Node 2
Node

Load Balancer

Node

Node

Virtual Node 2 Virtual Node 1

Scales out

Scales up

Virtual Node 1

Virtual Node 0 Virtual Node 0 Dual Core Single Processor 16 GB RAM Dual Core Dual Processor 32 GB RAM

Load Balancer

Node

Node

Node

Node

Scalability and High Availability

Availability: how the system provides useful resources over a set period of time

Uptime != availability Many factors affect it: network, storage, processor, etc.
Availability 90% 99% 99.9% 99.99% 99.999% 99.9999% Downtime 36.5 days 4 days 9 hours 53 minutes 5.3 minutes 32 seconds you afford Can
this?

Amazon S3 Outage (summer 2008)

Major companies affected (LeapFrog, eBay, Apple, many others) Amazons SLA? Sorry - it wont happen again.

SLAs and Availability

Service Level Agreements drive the architectural and technological decisions Cloud systems scalability characteristics are often pitched What is the availability?

What is the impact on business? How do you perform disaster recovery?

What service level are the vendors offering?

The higher the availability (3-nines, 4-nines, 5-nines), the higher the cost Co-lo, data centre deployments still make more sense for 4nines availability and above

Platform and Software as a Service

Identify the type of implementation after establishing the availability requirements

From a vendor? In-house development?

Software as a Service (SaaS)

The vendor provides the full application and is responsible for scaling it and meeting SLAs Integration with the vendor via web services or EDI Examples: GSI Commerce, Salesforce.com

Platform as a Service (PaaS)

The vendor provides the operating system and/or application stack The vendor provides a well-defined API for interacting with the system Examples: Amazon Web Services, Google App Engine, Nirvanix

PaaS Systems

Amazon EC2 and S3

Virtual images with various Intel-based operating systems, app servers, databases, etc. Billed hourly + bandwidth Java, PHP, Ruby, other technologies supported Data centre in the sky Lower cost than Rackspace or Nirvanix

Weaker SLAs?

Google App Engine

The apps are written as callbacks from a virtualized service Datastore ~= database; Memcache ~= caching; most interaction via HTTP Billed on usage You run on the same infrastructure as Google Low cost, still evolving, minimal SLAs

A Hybrid Cloud Architecture

Many mission-critical systems will live behind the corporate firewall The cloud is used for high-load applications and services The clouds applications work independently of the data center applications, and vice versa
Loose coupling over web services Assume that the data in the cloud is throwaway - its either consolidated or sourced in the data centre The cloud exists mostly for distribution and scalability

A Hybrid Cloud Architecture

Amazon S3
http://media.company.com

Google App Engine

http://www.company.com
Datastore Datastore Node Node

PNG PNG

PNG PNG

PNG PNG

PNG PNG

Node Node

Node Node

User

Firewall

services.company.com services.company.com

Enterprise Service Bus

Node

Node

Node

Node

Legacy Back-end

Enterprise Database

A Hybrid Cloud Architecture

End-User System (Mac, Windows)

USB

Connected App

Web Browser

Third-party Partner Site

S3 Content Repository

Internet

www.leapfrog.com

connected products

LearningPath

Firewall
Mule ESB backbone
HTTP, SOAP (CXF), REST, etc. routing, ltering, and dispatching; ActiveMQ JMS broker; dedicated services

Mule ESB
Connected products SOAP, REST web services

Mule ESB
Device log upload, processing, servlet container

Content Management System REST, JCR

Crowd SSO

Customer Data

Game play Data

Servlets App Logic

Device Logs

Content Authoring

User Credentials

Which Parts Go To The Cloud?

Internet
Load Balancer

Application Server Tomcat 6

Services Proxy

Application Server Tomcat 6

Load Balancer - Backbone

Backbone - message ltering, routing, dispatching, queuing, events

Mule ESB Mule ESB Mule ESB Mule ESB

Load Balancer - Tailbone

Load Balancer - Funnybone

Load Balancer - Message Broker

Mule ESB SOAP, REST

Mule ESB SOAP, REST

Mule ESB servlet, MTOM

Mule ESB servlet, MTOM

ActiveMQ

ActiveMQ

Database

NFS share

NFS share

Which Parts Go To The Cloud?

Depending on the cloud configuration, load balancing may not be necessary

Amazon provides Elastic Load Balancers and Elastic IP Google App Engine provides stateless calls only

The client itself or Memcache keep state instead

Rackspace provides either elastic addresses or explicit load balancers

Enterprise Service Bus and other integration may exist in both the cloud and the data centre
Message routing is OK is if latency is below the SLA requirements Strategic partner, application, and services integration may occur 100% on the cloud, with data consolidation behind the firewall

Case Study

Large consumer and services company .Net / Windows servers infrastructure

ASP.Net SQL Server The system just grew and became a business

Two-tier architecture

Server pages, services, media 6-month scalability project began in April

Implementation complete 25.Sep of the same year

Case Study - Objectives

Stable architecture defined by July Low cost Build scalability wherever possible Optimal data transfer rate for all properties

Web systems Media properties

Meet business requirements and SLAs

Hard dates set by the business, no input from the technology team Milestones: 1.July, 10.Oct

Case Study - Initial Configuration

System grew as needed - no planning Combines end-user and internal applications in the same server It only scales vertically -- and not very well Applications and database are tightly coupled Application servers and services tightly coupled Single environment: from development to production without passing Go nor collecting $200 Disaster recovery?

4 hours minimum From (stale?) backups

Case Study - Initial Configuration

Client Client Client
Client

Zone Server

UDP UDP Services UDP UDP Services Windows Services Services Windows Windows Windows

rsync rsync Linux Linux

Internet

Master, IRC Linux Services .Net Applications .Net

SQL Server

Case Study - Phase I Scalability

Case Study - Phase I Scalability

Introduces a CDN for asset delivery

Amazon S3 (optional Cloud Front) for asset delivery Reduces load on company servers and bandwidth costs

Introduces database replication for production environments Establishes a continuous integration environment

Set tools for hybrid UNIX/Windows environment

UNIX tools take precedence; Cygwin everywhere

Improved build/release process

Production
.Net Server services HTTP other proxy .Net Admin App Server .Net App Server .Net Server services HTTP other proxy

QA
.Net Admin Server .Net App Server

CDN CDN CDN

Services

Load balancer

Services

Deployment

Continuous Integration Server Version Control System, Automatic Builds, Unit Tests

Deployment

CDN CDN CDN

Services

IRC Databases

IRC

Databases

Databases

Case Study - Phase I Scalability

Development
.Net Server

Staging
.Net Server services HTTP other proxy .Net Admin App Server .Net App Server

services HTTP other proxy

.Net Admin Server

.Net App Server

CDN

Services

CDN CDN CDN

Services

Load balancer

Deployment
IRC Databases

Services

Deployment

IRC

Databases Developer 0 Developer 1 Developer 2 Developer 3

Case Study - Phase I Scalability

Fail-over with traditional database replication techniques Primary Cluster

Node 0 Node 1

ESB as app services provider

Partition 0

Partition 1

DB 0

DB 1

DB 0b

DB 1b

Case Study - Phase II Cloud Deployment

Case Study - Phase II Cloud Deployment

Web applications move to a uniform technology (.Net) The database and stored procedures are normalized and optimized Applications use common resources via Mule ESB and services

No more direct calls from apps to databases Business logic is implemented as stateless POJOs Windows, other servers driven by business requirements Open source software wherever possible

Software stack is best of breed

Case Study - Phase II Cloud Deployment

Web and other RPC services must coexist

Different partners use different protocols Mule transformers take care of all the interfaces so that development may scale / continue independently of what happens in other layers

Bandwidth can be expen$ive! Data exchange protocols

Clients: custom, XML, JSON Images: HTTP, S3 Cloud-to-HQ: custom, XML/SOAP, protocol buffers HQ data centre: XML/SOAP, protocol buffers The cloud isnt trustworthy yet

Replication strategy: data centre

Case Study - Phase II Cloud Deployment

174.129.238.145 174.129.238.126

balance

World

balance

web

174.129.238.110

174.129.238.129

balance

los01

los02

balance

ep01 FRONT END

ep02 FRONT END

web01

web02

losgamesvr01

losgamesvr02

174.129.238.159

services

balance 174.129.238.184 174.129.238.188 master02

174.129.238.189

balance services02 Web services Proxy Routing Queuing

syncserver

downloads S3

services01 Web services Proxy IMUService Routing Queuing

master01 IRC game server list

syncserver01 IRC game server list rsync repository

syncserver02
174.129.239.42

rsync repository

174.129.239.45 174.129.239.57 game server 174.129.239.68 game server 174.129.239.81 game server

174.129.239.11

game server app-x server

legacyservices
174.129.239.18

database

174.129.239.41

balance

balance

balance

legacyservices01 Web services EvolverSP Dimenxian IMUService EvolverMP DimensionMP

legacyservices02 Web services EvolverSP Dimenxian IMUService EvolverMP DimensionMP database master replicate ep01 SERVICES ep02 SERVICES

174.129.15.199

QA DNS

database slave

Case Study - Phase II Cloud Deployment Traditional Hosted Environments

Setup Production Staging QA Dev $1,300 $1,300 $ 600 $ 600 Monthly $5,5580 $5,5580 $1,450 $1,450 Bandwidth Total / year $2,000 $0 $0 $0 Total $70,260 $68,260 $18,000 $18,000 $174,520

Case Study - Phase II Cloud Deployment Amazon EC2 Configuration

Setup Production Staging QA Dev $0 $0 $0 $0 Monthly $4,320 $4,320 $1,080 $0 Bandwidth Total / year $1,200 $0 $0 $0 Total $66,240 $51,840 $12,960 $0 $131,040

$43,000 cheaper

Case Study - Phase III Final Deployment

174.129.238.145

World

174.129.238.126

balance

balance

web

174.129.238.110

174.129.238.129

balance

los01

los02

balance

ep01 FRONT END

ep02 FRONT END

web01

web02

losgamesvr01

losgamesvr02

174.129.238.189

balance 174.129.238.184 master01

174.129.238.159

syncserver

174.129.238.188 master02 syncserver01 syncserver02 IRC game server list

downloads S3

services

balance

IRC game server list

174.129.239.42

rsync repository

rsync repository

174.129.239.45 174.129.239.57 game server 174.129.239.68 game server 174.129.239.81 game server

game server app-x server

services01 Web services Proxy Routing Queuing services01 Web services Proxy Routing Queuing services01 Web services Proxy Routing Queuing services01 Web services Proxy Routing Queuing
174.129.239.18

database
174.129.15.199

balance

QA DNS
database master replicate database slave

Thanks for Coming!

Wanna know more about real life cloud, scalable systems? Subscribe to the newsletter!

http://ciurana.eu/scalablesystems

Questions?
Eugene Ciurana
Open source evangelist irishdev@ciurana.eu +41 44 586 8462