Cloud Aneka

Aneka
A Platform for Enterprise Grid/Cloud Computing
Christian Vecchiola Ph.D.

GRIDSLab, Computer Science Software Engineering
The University of Melbourne
csve@unimelb.edu.au
Master in Engineering Distributed Computng - Subject
678
Outline
Introduction
− Cloud Computing
Aneka
− Features
− Architecture
− Programming Models
Hands on Aneka
Projects
Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 2
678
Introduction

678
Cloud Computing
Q oS
Am
az o
nE
Amaz
Virt SaaS C2
uali I aaS
z atio
n
on S3
PaaS
Storage n g
Billing Prici
ices
Res r v
are our
ce M Se
W l eb
Hy VM ve eter
ing W
e
L nt
pe e e
r vi c
vi eem
so r Utility
rs Se gr Provision
ing
A Computing
on Deman
d y
ur i t Man
Pub S ec jr asof
lic Clou t An
d eka
Web 2.0
Privacy Uhm, I am not quite
Private Clo clear…Yet another
ud d buzzword..?
l ou Go o
r i s eC gl e
A pp
er p Eng
Ent om i ne
.c
or ce
s F
le Mosso
Sa
678
Cloud Computing
About the Cloud
Cloud Computing
The “Cloud” is the default The broader term of “Computing”

symbol of the internet in encompasses:
diagrams. - Computation
- Coordination logic
- Storage
Cloud Computing is about moving computing from

the single desktop pc/data centers to the internet.

678
Cloud Computing
More formally….
− Fox et al. (Berkeley)
Cloud Computing refers to both the applications delivered as
services over the Internet and the hardware and systems software
in the datacenters that provide those services. The services
themselves have long been referred to as Software as a Service
(SaaS), so we use that term. The datacenter hardware and software
is what we will call a Cloud.
− Buyya et al.
A Cloud is a type of parallel and distributed system consisting of a
collection of interconnected and virtualized computers that are
dynamically provisioned and presented as one or more unified
computing resources based on service-level agreements
established through negotiation between the service provider and
consumers.

678
Cloud Computing
So what ...?
− Internet-based (the Internet is central..)
− The entire stack (hw & sw) is leveraged to the internet
− HaaS  Amazon S3
− IaaS  VMWare, Amazon EC2
− PaaS  Google AppEngine, Mosso, Aneka
− SaaS  SalesForce
− Computing becomes a real utility used by all the masses
− researchers
− end users
− companies
− service providers (web hosting)
− The economic aspect become more important (billing, pricing,
QoS, SLA)
These are the two main differences
from Grid Computing.
678
Anatomy of a Cloud
Pricing and Billing

Accounting
Google Apps (Gmail, Docs,…)
Salesforce.com
Software as a Service
Financial Applications
Web 2.0 Interface

uting
Programming API
c Comp
rks
t
emen
two
Admission Control
Scripting & Programming
e
Amus
Languages
lN
n tifi
cia
Ns
ie
CD
So
Sc
Platform as a Service
QoS Metering
Google AppEngine
Microsoft Azure
Manjrasoft Aneka
Infrastructure as a Service
SLA Negotiation
Virtualization
Private Cloud
VM Management & Deployment
Amazon S3, EC2

Public Cloud
OpenNebula, Eucalyptus
Other
Mosso Grids/Clouds
Clusters
Data Centers
Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise
StorageGrid/Cloud 8
678
Embracing the Cloud

Benefits
− Infinite compute resource available on demand
− Hardware virtualization
− Accessibility anytime and anywhere
− Internet (web based) access
− Elimination of the upfront commitment of users
− Reduced costs due to dynamic hardware provisioning
− Pay per use basis (and also other models)
− No need to plan for peak load in advance
− Easy management
− Software versioning and upgrading

678
Embracing the Cloud

Migration Risks
− Privacy
− Who access your data?
− Security
− How much you trust your provider?
− What about recovery, tracing, and data integrity?
− Political and legal issues Will it be
fitting
− Who owns the data? for me?
− Who uses your personal data?

− Government
− Where is your data?
− Amazon Availability Zones

678
References
Web pointers
− cloud-computing@googlegroups.com
− http://twitter.com/cloudcomp_group
− http://sites.google.com/site/cloudcomputingwiki/
− http://wiki.cloudcommunity.org/
− http://en.wikipedia.com/cloud_computing
− http://cloudslam09.com/

678
Aneka

678
Features
Characterization
− Middleware for Enterprise Grids/Clouds
− Service oriented architecture
− .NET/Mono based environment
− Languages:
− C#, C++, VB, Delphi, Java/IKVM…
− … and 20 more languages
− Platforms:
− Windows XP/2000/2003
− Linux & Mac OS X

678
Features
Why Aneka?
− Aneka (sanskrit): many, in many ways, many in one…
− This means:
− Multiple programming/deployment models
− Multiple scheduling strategies
− Multiple authentication models
− Multiple persistence backends
− Multiple platform and OSs
Designed to be a configurable middleware with the aim of

supporting an open ended set of abstractions for distributed
computing and deployment scenarios

678
Features
Aneka & Clouds
Software as a Service Private Cloud
Public Cloud
Platform as a Service Aneka
Infrastructure as a Service
Aneka fits into the cloud architecture at the platform layer. This means that it
provides a programming based interface for developing distributed application
and a virtual execution environment in which the applications developed
according to the published APIs can run.
678
Features
Current Applications
− Scientific
− Distributed evolutionary computation
− Proteine structure prediction
− Commercial
− Engineering: Go Front (China): Train models rendering
− Media and games: platform for on-line gaming
− Financial: risk analysis
− Office automation: Excel integration
− Educational
− Image filtering
− Image rendering
− Distributed systems teaching

678
Architecture
System Overview Aneka enterprise Cloud
Aneka
Container
work units
Executor Executor
Manager
Executor
internet
work units
Scheduler
Workers
internet Manager(s)
Executor
Manager
Client
Applications
678
Architecture
Anatomy of a Container Persistence
Scheduling
Monitoring
Execution
Directory
Storage
Characterization
Service Negotiation
Resource Allocation
Core
Security
Communication Layer

678
Architecture
Services
− Scheduling Services:
− Basic scheduling services
− Support for…
− … independent bag of task scheduling
− … different and pluggable scheduling algorithms
− … advanced reservation aware schedulers
− … automatic/custom task resubmission on failure
− … node quarantine/black listing
− Execution Services:
− Basic sandboxing
− Support for legacy code execution (PSM)
− Storage:
− Multi-protocol based storage service (FTP)

678
Programming Models
Development
− Aneka is Platform as a Service cloud
middleware
− This means:
− It exposes an API for development
− It provides access to the cloud at programming
level
− More precisely…
− It provides different programming models

678
Aneka
Scenario
Aneka enterprise Cloud
public DumbTask: ITask

{
…
public void Execute() Executor Executor
{
…… i=0; i<n; i++)
for(int
{ }
}…
DumbTask task = new DumbTask();
app.SubmitExecution(task);
} Executor
internet
work units
Scheduler
internet
Executor
Manager
Programming / Deployment Model

678
Programming Models
Overview
Task MapReduce
infrastructur scheduling Model Model
e
execution TaskScheduler MapReduceScheduler
coordination TaskExecutor MapReduceExecutor
client component TaskManager MapReduceManager
end users
abstractions Task Mapper
units of execution Reducer

678
Programming Models
Currently supported:
− Task Programming Model
− Thread Programming Model
− MapReduce Programming Model
− Parameter Sweeping Model
− .. Implement your own..

678
Programming Models
Task Programming Model
− Used to model Independent Bag of Tasks (BoT)
applications
− The application is a collection of execution unit
− Each execution unit is not related to the others
− There is no order in the execution of the units
− If there is any ordering or sequencing it is performed by the
client application and NOT by the middleware

678
Programming Models
− Aneka Support
− User API:
− Interface ITask
− Only one operation: Execute
− GridTask class
− ITask wrapping
− Support for remote execution
− Middleware:
− Task Scheduling Service
− Coordinates and sequences the execution of GridTask instances
− Task Execution Service
− Provides the execution environment for the GridTask instance
− Task Application Manager
− Task specific submission component

678
Programming Models
− Developing Task based Applications
− Implement the ITask interface
− Create a GridApplication instance
− Create and submit tasks [Serializable]
public class MyTask: ITask
public − Collect
class results.
TaskSample
{
public void Execute()
{
{
…
public class TaskSample // implement your logic here
public void Setup()
{ ………
{
… }
public class TaskSample
Configuration conf = new Configuration();
public void Execute() }
{ conf.SchedulerUri = “tcp://localhost:9090/”;
{
… this.app = new GridApplication<GridTask,TaskManager>(conf);
for(int i=0; i<100; i++)
public void OnWorkUnitFailed(object
this.app.WorkUnitFailed sender, WorkUnitEventArgs args)
+= new EventHandler<WorkUnitEventArgs>(this.OnWorkUnitFailed);
{
{ this.app.WorkUnitFinished += new EventHandler<WorkUnitEventArgs>(this.OnWorkUnitFinished);
MyTask task = new MyTask();
Console.WriteLine(“Task
……… {0} failed!”, args.WorkUnit.Id);
GridTask gt = new GridTask(task);
} ………
this.app.ExecuteWorkUnit(gt);
} }
}
public void OnWorkUnitFinished(object sender, WorkUnitEventArgs args)
………
{
}
Console.WriteLine(“Task {0} finished!”, args.WorkUnit.Id);
}
………
Dr. Christian
} Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 26
678
Programming Models
− Task Model Samples
− Convolution
− PovRay
− Gauss (TaskDemo)
− ExcelGrid
DEMO

678
Programming Models
Thread Programming Model
− Based on the concept of distributed thread
− Like a local thread but executed remotely
− Implements a subset of the common operations on thread
− Start
− Stop
− State Query
− Join
− Provides a quick way for porting on a distributed
middleware, multi-threaded applications

678
Programming Models
− Aneka Support
− User API:
− GridThread class
− Distributed version of System.Threading.Thread
− Support for remote execution
− Middleware:
− Thread Scheduling Service
− Coordinates and sequences the execution of GridThread
instances
− Thread Execution Service
− Provides the execution environment for the GridThread instance
− Thread Application Manager
− Thread specific submission component (client component)

678
Programming Models
− Developing Thread based Applications
− Define a worker method
− Create a GridApplication instance
− Create and start GridThread[Serializable]
instances
public class MyThreadWorker
public − Use
class the common thread APIs
ThreadSample
{
public void DoWork()
{
{
…
public class ThreadSample // implement your logic here
public void Setup()
{ ………
{
private
public IList<GridThread>
class ThreadSample workers = new List<GridThread>();
}
Configuration conf = new Configuration();
{ public void Start()
conf.SchedulerUri = “tcp://localhost:9090/”;
}
{
… this.app = new GridApplication<GridThread,ThreadManager>(conf);
for(int
public i=0; i<100; i++)
……… void Join()
{ {
}
MyThreadWorker worker
foreach(GridThread thread=in
new MyThreadWorker();
this.workers)
}
{ GridThread thread = new GridThread(new ThreadStart(worker.DoWork), this.app);
thread.Start();
thread.Join();
} this.workers.Add(thread);
}
………
} ………
} }
Dr. Christian
} Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 30
678
Programming Models
− Thread Model Samples
− Mandelbrot
− Warholizer (ThreadDemo)
DEMO

678
Programming Models
MapReduce Programming Model
− Based on the MapReduce framework from Google
− Functional-style like primitives:
map :: (key1 , value1 ) ⇒ list (key2 , value2 )
reduce :: (key2 , list (value2 )) ⇒ list (value3 )
− A distributed application becomes a collection of map

and reduce operations.
678
Programming Models
− Aneka Support
− User API:
− Mapper<key1,value1> (base class)
− Defines the map function
− Reduder<key2,value2> (base class)
− Defines the reduce function
− Middleware:
− MapReduce Scheduling Service
− Coordinates and sequences the execution of Mapper and Reducer
instances
− MapReduce Execution Service
− Provides the execution environment for the Mapper and Reducer
instances
− MapReduce Application Manager
− MapReduce specific submission component (client component)

678
Programming Models
− Developing MapReduce based Applications
− Define map and reduce operations
− Provide the data
− Run the MapReduce engine
Execution:
-File staging
-Task scheduling
-Failed task resubmission
Input data -Replication and Fault tolerance
-Collection of result
MapReduce engine
Map & Reduce network
map & reduce

678
Programming Models
MapReduce Programming Model
− MapReduce Model Samples
− Estimation of Pi
− Word Counter
DEMO

678
Programming Models
Parameter Sweeping Model
− Based on the Task Programming Model
− Provides a set of facilities to run applications where
− ..all the tasks are homogeneous (same task)
− ..the specific instance of tasks is specialized by parameters
− ..all the possible combination of parameters are explored by
generating a task instance for each of the combinations
− These are also called:
[B,2,#]
− Embarrassingly parallel applications.
− Ideal for running legacy applications
[A,1,#]
[A,2,#]
678
Programming Models
− Aneka Support
− User APIs
− Additional APIs that customize the task model for Parameter Sweeping
application
− IParameterizedTask, RangeParameter, RandomParameter
− …..
− A set of ready to use micro-tasks to compose the PSM task
− CopyTask, ExecuteTask, DeleteTask
− …
− Middleware:
− (The same as Task Model)
− PSMManager (client submission component for PSM application)

678
Programming Models
− Developing Parameter Sweeping based Applications
− Use the APIs:
− Implement IParameterizedTask
− Compose your task with micro-tasks
− Use the PSMManager to submit the execution
− Use the Design Explorer
− Wizard based interaction for task templates
− Execution monitoring and statistics
DEMO

678
Programming Models
Observations
− Task Programming Model
− Simple, Intuitive, and General Model
− Can be used to run legacy tasks or C# code
− “Submit and Forget” approach
− Rapid prototyping with the PSM
− Thread Programming Model
− Low level programming model that allows for more control
− Suitable for porting local multi-thread apps to a distributed
middleware

678
Programming Models
Observations
− MapReduce Programming Model
− Particularly suitable for data intensive application
− Task and Thread models are task-based:
− the user defines the tasks that will be executed
− the user submits the task to the middleware
− MapReduce model is function-based:
− the user defines the functions operating on the data
− the user configures the middleware with these functions
− the user provides the data
− the middleware automatically generates the tasks required to
execute the functions on the data

678
Programming Models
Extensibility
− Implementing a new Programming Model
− Defines the user abstractions
− Provide an implementation for..
− Execution Service (middleware)
− Scheduling Service (middleware)
− Application Manager (client component)
− Where to start:
− Aneka.Entity.*, Aneka.Data.Entity*;
− Aneka.Execution.*;
− Aneka.Runtime.Scheudling.*;

678
Hands on Aneka

678
Hands on Aneka
Work In Progress
− Developing application with Aneka
− Choosing the programming Model
− Defining the application logic
− Implementing the code
− Setup and running..

678
Projects

678
Projects
Contributing to Aneka RESERVATION
RESERVATION (ALL
(ALL MODELS)
MODELS)
− Infrastructure: MARKET
MARKET ORIENTED
ORIENTED SHEDULING
SHEDULING
− services
VIRTUAL
VIRTUAL MACHINES
MACHINES
− scheduling algorithms
− resource allocation policies INDIPENDENT
INDIPENDENT BAG
BAG OF
OF TASKS
TASKS
− security
DISTRIBUTED
DISTRIBUTED THREADS
THREADS
− programming/deployment models
MAPREDUCE
MAPREDUCE
− End users: WORKFLOW/DATAFLOW?

WORKFLOW/DATAFLOW?
− Aneka-based application ACTORS?

ACTORS?
− integration

678
Projects
Research
− Cooperative Scheduling
− Virtual Execution Environment
− Advanced Quality of Service
− Resource Pricing
− Cloud shift (happens)
Development
− Programming and Deployment Models
− Dataflow
− MPI
− Workflow implementation
− Platform Porting

678
Projects
Opportunities…
− Looking for a student project?
− Contact us:
− A/Prof. Rajkumar Buyya – raj@csse.unimelb.edu.au
− Dr.Christian Vecchiola – csve@unimelb.edu.au
− Mr. Xingchen Chu – xchu@csse.unimelb.edu.au
Uhm…..

678
Projects
References
− Manual and tutorials
(Manjrasoft web: http://www.manjrasoft.com)
− C. Vecchiola and X. Chu – Aneka Tutorial Series: Developing Task Model Applications
− C. Vecchiola and X. Chu – Aneka Tutorial Series: Developing Thread Model
Applications
− C. Jin and C. Vecchiola – Aneka Tutorial Series: Developing MapReduce Applications
− X. Chu and C. Vecchiola – Aneka Installation Guide
− Papers
− C. Vecchiola, M. Kirley, and R. Buyya, Multiobjective Problem Solving with Offspring on
Enterprise Clouds, Proc. of HPC ASIA 2009, Mar. 2-5, 2009, Kaohsiung, Taiwan.
− C. Jin, C. Vecchiola, and R. Buyya, MRPGA: An Extension of MapReduce for Parallelizing
Genetic Algorithms, Proc. of the 4th IEEE Int. Conf. on e-Science, (e-Science 2008, IEEE CS Press,
Los Alamitos, CA, USA), Dec. 10-12, 2008, Indianapolis, Indiana, USA.
− X. Chu, K. Nadiminti, C. Jin, S. Venugopal, and R. Buyya, Aneka: Next-Generation Enterprise Grid
Platform for e-Science and e-Business Applications, Proc. 3rd IEEE Int. Conf. on e-Science and
Grid Computing (e-Science 2007, IEEE CS Press, Los Alamitos, CA, USA), Dec. 10-13, 2007,
Bangalore, India.

678
? ?
? ?
The End.. …questions?
? ?
?

Cloud Aneka

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Cloud Aneka

Uploaded by

Copyright:

Available Formats

Aneka

A Platform for Enterprise Grid/Cloud Computing

Christian Vecchiola Ph.D.

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 3

The “Cloud” is the default The broader term of “Computing”

Cloud Computing is about moving computing from

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 5

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 6

Pricing and Billing

Web 2.0 Interface

Amazon S3, EC2

Embracing the Cloud

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 9

Embracing the Cloud

− Who uses your personal data?

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 10

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 11

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 12

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 13

Designed to be a configurable middleware with the aim of

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 14

Platform as a Service Aneka

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 16

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 18

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 19

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 20

public DumbTask: ITask

Programming / Deployment Model

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 21

coordination TaskExecutor MapReduceExecutor

client component TaskManager MapReduceManager

units of execution Reducer

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 22

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 23

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 24

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 25

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 27

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 28

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 29

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 31

map :: (key1 , value1 ) ⇒ list (key2 , value2 )

reduce :: (key2 , list (value2 )) ⇒ list (value3 )

− A distributed application becomes a collection of map

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 33

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 34

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 35

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 37

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 38

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 39

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 40

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 41

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 42

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 43

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 44

− End users: WORKFLOW/DATAFLOW?

− Aneka-based application ACTORS?

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 45

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 46

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 47

Dr. Christian Vecchiola Ph.D. Aneka a Platform for Enterprise Grid/Cloud 48

You might also like