Integration of Cloud Computing and Cloud Storage Overview and Introduction

Patrick Dreher
Chief Scientist ,Renaissance Computing Institute Adjunct Prof. Computer Science, NC State University
IEEE Mass Storage Conference Tutorial May 3, 2010

Level of General Interest in Cloud Computing

Where Would One Look To Find Cloud Computing?

CloudTest/Soasta

Plus Numerous Others.

So many choices

Picking a Cloud Option is Only The First Step

Data are not always located near the computation and analysis infrastructure
There is data available .large quantities of data In many geographically different places With millions of files holding that data

Goal is to extract useful information from data These large data collections need computational systems for analysis One option is to move the data to the compute location

Observation of Current and Historical ESnet Traffic Patterns

Courtesy of William Johnston, LBL
Projected volume for Jun 2010: 8.6 Petabytes/month Actual volume for Jun 2009: 4.3 Petabytes/month

ESnet Traffic Increases by 10X Every 47 Months, on Average

Terabytes / month

Aug 1990 100 GBy/m o

Oct 1993 1 TBy/mo

Jul 1998 10 TBy/mo

Nov 2001 100 TBy/mo

Apr 2006 1 PBy/m o

Log Plot of ESnet Monthly Accepted Traffic, January 1990 June 2009

Network Traffic, Science Data, and Network Capacity Long-term trends

Courtesy of William Johnston, LBL
All Four Data Series are Normalized to 1 at Jan. 1990
100000000

10000000

1000000

2010 value -ESnet traffic 2010 value 40 PBy HEP exp. data xx -- 40 Pby ESnet capacity xx Climate modeling data 4 PBy4 Pby
Expon. (ESnet traffic) Expon. (HEP exp. data)

Historical

Projection
y = 0.8699e
0.5714x 0.6704x

y = 2.3747e

y = 0.4511e0.5244x

100000

Expon. (ESnet capacity) Expon. (Climate modeling data)

10000 y = 0.1349e0.4119x

1000

100

10

Jan, 90

Jan, 91

Jan, 92

Jan, 93

Jan, 94

Jan, 95

Jan, 96

Jan, 97

Jan, 98

Jan, 99

Jan, 00

Jan, 01

Jan, 02

Jan, 03

Jan, 04

Jan, 05

Jan, 06

Jan, 07

Jan, 08

Jan, 09

Jan, 10

Jan, 11

Jan, 12

Jan, 13

Jan, 14

0 (HEP data courtesy of Harvey Newman, Caltech, and Richard Mount, SLAC. Climate data courtesy Dean Williams, LLNL, and the Earth Systems Grid Development Team.)

Jan, 15

In The Tutorial This Afternoon

What are the types of options for cloud computing How does one select a cloud computing option from among the numerous choices What are the important design questions to ask when constructing a coherent cyberinfrastructure of computing and data

Data Grids
Policy-based Data Management
Remote locations Aggregate sensor data in cache Event Detection Cloud Compute

Message Bus
Sensors

Cloud Storage Cache

Multiple Protocols Clients Remote Users

Data Grid

SuperComputer Simulations

External Repositories Archive Digital Library

Questions