Bluesky: A Cloud-Backed File System For The Enterprise: Michael Vrable Stefan Savage Geoffrey M. Voelker

BlueSky: A Cloud-Backed File System for the Enterprise
Michael Vrable Stefan Savage Georey M. Voelker

University of California, San Diego
Computer Science and Engineering Department
February 16, 2012
Vrable, Savage, Voelker (UCSD) BlueSky February 16, 2012 1 / 16
Computing Services for the Enterprise
Our work is focused primarily on small/medium-sized organizations
These organizations run a number of computing services, such as

e-mail and shared le systems
Often brings signicant cost:
Purchasing hardware
Operating hardware
Managing services
Outsourcing these services to the cloud oers the possibility to lower

costs
. . . Migrated to the Cloud
Some services are already migrating to the cloud. . .
Network le systems have not yet migrated, but still have potential
benets:
File system size entirely elastic: simpler provisioning
Cloud provides durability for le system data
Hardware reliability less important
Integration with cloud backup

We build and analyze a prototype system, BlueSky, to investigate how to
do so
. . . Migrated to the Cloud
Some services are already migrating to the cloud. . .
Network le systems have not yet migrated, but still have potential
benets:
File system size entirely elastic: simpler provisioning
Cloud provides durability for le system data
Hardware reliability less important
Integration with cloud backup

We build and analyze a prototype system, BlueSky, to investigate how to
do so
Cloud Computing Oerings
Spectrum of service models:
Software-as-a-Service: Complete integrated service from a provider
Platform/Infrastructure-as-a-Service: Building blocks for custom

applications
In both cases:
Infrastructure moved within network
Reduce/eliminate need for hardware maintenance
Reduce need for ahead-of-time capacity planning

SaaS: Easy to set up
PaaS/IaaS: More choice among service providers, potentially lower cost

applications
In both cases:


applications
In both cases:


applications
In both cases:

Challenges
Cloud storage (e.g., Amazon S3) acts much like another level in the
storage hierarchy but brings new design constraints:
New interface
Only supports writing complete objects
Does support random read access
Performance
High latency from network round trips
Random access adds little penalty
Security
Data privacy is a concern
Cost
Cost is very explicit
Unlimited capacity, but need to delete to save money

BlueSky: Approach
For ease of deployment, do not change

software stack on clients
Clients simply pointed at a new server,

continue to speak NFS/CIFS
Deploy a local proxy to translate requests

before sending to the cloud
Provides lower-latency responses to

clients when possible by caching data
Implements write-back caching
Encrypts data before storage to cloud

for condentiality
BlueSky: Approach
BlueSky adopts a log-structured design
Each log segment uploaded all at once
Random access allowed for downloads
Log cleaner can be run in the cloud (e.g.,

on Amazon EC2) for faster, cheaper
access to storage
Log cleaner can run concurrently with

active proxy
Cleaner not given full access to le

system data
File System Design
Checkpoint
Last segments seen:
cleaner: 3
proxy: 12
Inode maps:
[0, 4095]
[4096, 8191]
Inode map [0, 4095]
2
3
5
6
11
200
Inode 6
Type: regular file
Owner: root
Size: 48 KB
Data blocks:
0
1
Data Block
Inode number: 6
Length: 32 KB
Data Block
Inode number: 6
Length: 16 KB
Unencrypted Objects Encrypted Objects
Cloud Log Directories:
Proxy:
Cleaner:
Segment #11 #12
Segment #2 #3 #4
Architecture
Segment
Writes
Range
Reads
Disk J ournal
Writes
Disk Cache
Reads
Client
Requests
Client
Responses
NFS
CIFS
S3
WAS E
n
c
r
y
p
t
i
o
n
Disk
Network
Memory
Front
Ends
Back
Ends
Resource
Managers
Proxy
Proxy internally buers updates briey in memory
File system updates are serialized and journaled to local disk
File system is periodically checkpointed: log items are aggregated into

segments and stored to cloud
On cache miss, log items fetched back from cloud and stored on local
disk
Cloud Storage Performance
We are assuming that users will have fast connectivity to cloud

providers (if not now, then in the near future)
Latency is a fundamental problem (unless cloud data centers built

near to customers)
0.0001
0.001
0.01
0.1
1
10
100
1000
1 100 10000 1e+06 1e+08
E
f
f
e
c
t
i
v
e

U
p
l
o
a
d

B
a
n
d
w
i
d
t
h

(
M
b
p
s
)
Object Size (bytes)
1
2
4
8
16
32
Network RTT: 30 ms to
standard (US-East) S3
region, 12 ms to US-West
region
Proxy can fully utilize

bandwidth to cloud
Results argue for larger

objects, parallel uploads
Application Performance
Simple benchmark: unpack Linux kernel sources, checksum kernel sources,
compile a kernel
Unpack Check Compile
(write) (read) (R/W)
Local NFS server 10:50 0:26 4:23
NFS server in EC2
65:39 26:26 74:11
BlueSky/S3-West
warm proxy cache
5:10 0:33 5:50
cold proxy cache
26:12 7:10
full segment prefetch
1:49 6:45
BlueSky/S3-East
warm proxy
5:08 0:35 5:53
cold proxy cache
57:26 8:35
3:50 8:07
compile a kernel
NFS server in EC2 65:39 26:26 74:11
BlueSky/S3-West
warm proxy cache
5:10 0:33 5:50
cold proxy cache
26:12 7:10
1:49 6:45
BlueSky/S3-East
warm proxy
5:08 0:35 5:53
cold proxy cache
57:26 8:35
3:50 8:07
compile a kernel
NFS server in EC2 65:39 26:26 74:11
BlueSky/S3-West
warm proxy cache 5:10 0:33 5:50
cold proxy cache
26:12 7:10
1:49 6:45
BlueSky/S3-East
warm proxy
5:08 0:35 5:53
cold proxy cache
57:26 8:35
3:50 8:07
compile a kernel
NFS server in EC2 65:39 26:26 74:11
BlueSky/S3-West
cold proxy cache 26:12 7:10
1:49 6:45
BlueSky/S3-East
warm proxy
5:08 0:35 5:53
cold proxy cache
57:26 8:35
3:50 8:07
compile a kernel
NFS server in EC2 65:39 26:26 74:11
BlueSky/S3-West
full segment prefetch 1:49 6:45
BlueSky/S3-East
warm proxy
5:08 0:35 5:53
cold proxy cache
57:26 8:35
3:50 8:07
compile a kernel
NFS server in EC2 65:39 26:26 74:11
BlueSky/S3-West
BlueSky/S3-East
warm proxy 5:08 0:35 5:53
Read Performance Microbenchmark
0
50
100
150
200
250
300
350
400
0 20 40 60 80 100
R
e
a
d
L
a
t
e
n
c
y
(
m
s
)
Proxy Cache Size (% Working Set)
Single-Client Request Stream
32 KB
128 KB
1024 KB
Read performance depends on working set/cache size ratio
At 100% hit rate, comparable to local NFS server
Even at 50% hit rate, latency within about 2 to 3 of local case

Write Performance Microbenchmark
0
20
40
60
80
100
120
140
0 5 10 15 20 25 30 35
A
v
e
r
a
g
e

W
r
i
t
e

L
a
t
e
n
c
y

(
m
s
/
1

M
B

w
r
i
t
e
)
Client Write Rate (MB/s): 2-Minute Burst
Latency vs. Write Rate with Constrained Upload
128 MB Write Buffer
1 GB Write Buffer
Congure network to constrain bandwidth to cloud at 100 Mbps
Write performance: similar to local disk, unless write rate exceeds

cloud bandwidth and write-back cache lls
Aggregate Performance: SPECsfs2008
100
200
300
400
500
600
700
800
900
1000
0 200 400 600 800 1000 1200 1400 1600
0 10 20 30 40 50
A
c
h
i
e
v
e
d
O
p
e
r
a
t
i
o
n
s
p
e
r
S
e
c
o
n
d
Requested Operations per Second
Working Set Size (GB)
Local NFS
Models a richer workload mix
BlueSky is comparable to local NFS (as before, slight advantage on

writes from log-structured design)
Performance is less predictable with a constrained network link
Fetching full segments is a big loss with mostly random access

100
200
300
400
500
600
700
800
900
1000
0 200 400 600 800 1000 1200 1400 1600
0 10 20 30 40 50
A
c
h
i
e
v
e
d
O
p
e
r
a
t
i
o
n
s
p
e
r
S
e
c
o
n
d
Local NFS
BlueSky


100
200
300
400
500
600
700
800
900
1000
0 200 400 600 800 1000 1200 1400 1600
0 10 20 30 40 50
A
c
h
i
e
v
e
d
O
p
e
r
a
t
i
o
n
s
p
e
r
S
e
c
o
n
d
Local NFS
BlueSky
BlueSky (100 Mbps)


100
200
300
400
500
600
700
800
900
1000
0 200 400 600 800 1000 1200 1400 1600
0 10 20 30 40 50
A
c
h
i
e
v
e
d
O
p
e
r
a
t
i
o
n
s
p
e
r
S
e
c
o
n
d
Local NFS
BlueSky
BlueSky (100 Mbps)
BlueSky (norange)


Monetary Cost: SPECsfs2008
Normalized cost: cost per million SPECsfs operations
(for S3 prices: $0.12/GB download, $0.01/100010000 ops)
Down Op Total (Up)
Log-structured baseline $0.18 $0.09 $0.27 $0.56
No aggregation 0.17 2.91 3.08 0.56
Full segment downloads 25.11 0.09 25.20 1.00
Log-structured design minimizes cost for cloud storage operations
Support for random access on reads (byte-range request) needed for

low cost
Storage cost also an important consideration, but less sensitive to

system design
Conclusions
BlueSky is a prototype le server backed by cloud storage
Prototype supports multiple client protocols (NFS, CIFS) and storage

backends (Amazon S3, Windows Azure)
Allows clients to transparently move to cloud-backed storage
Performance comparable to local storage when most access hits in

cache
Design is informed by cost models of current cloud providers


Bluesky: A Cloud-Backed File System For The Enterprise: Michael Vrable Stefan Savage Geoffrey M. Voelker

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Bluesky: A Cloud-Backed File System For The Enterprise: Michael Vrable Stefan Savage Geoffrey M. Voelker

Uploaded by

Copyright:

Available Formats

BlueSky: A Cloud-Backed File System for the Enterprise

Michael Vrable Stefan Savage Georey M. Voelker

Our work is focused primarily on small/medium-sized organizations

These organizations run a number of computing services, such as

Often brings signicant cost:

Outsourcing these services to the cloud oers the possibility to lower

File system size entirely elastic: simpler provisioning

Cloud provides durability for le system data

Hardware reliability less important

Integration with cloud backup

File system size entirely elastic: simpler provisioning

Cloud provides durability for le system data

Hardware reliability less important

Integration with cloud backup

Software-as-a-Service: Complete integrated service from a provider

Platform/Infrastructure-as-a-Service: Building blocks for custom

Infrastructure moved within network

Reduce/eliminate need for hardware maintenance

Reduce need for ahead-of-time capacity planning

Software-as-a-Service: Complete integrated service from a provider

Platform/Infrastructure-as-a-Service: Building blocks for custom

Infrastructure moved within network

Reduce/eliminate need for hardware maintenance

Reduce need for ahead-of-time capacity planning

Software-as-a-Service: Complete integrated service from a provider

Platform/Infrastructure-as-a-Service: Building blocks for custom

Infrastructure moved within network

Reduce/eliminate need for hardware maintenance

Reduce need for ahead-of-time capacity planning

Software-as-a-Service: Complete integrated service from a provider

Platform/Infrastructure-as-a-Service: Building blocks for custom

Infrastructure moved within network

Reduce/eliminate need for hardware maintenance

Reduce need for ahead-of-time capacity planning

Only supports writing complete objects

Does support random read access

High latency from network round trips

Random access adds little penalty

Data privacy is a concern

Cost is very explicit

Unlimited capacity, but need to delete to save money

For ease of deployment, do not change

Clients simply pointed at a new server,

Deploy a local proxy to translate requests

Provides lower-latency responses to

Implements write-back caching

Encrypts data before storage to cloud

BlueSky adopts a log-structured design

Each log segment uploaded all at once

Random access allowed for downloads

Log cleaner can be run in the cloud (e.g.,

Log cleaner can run concurrently with

Cleaner not given full access to le

Proxy internally buers updates briey in memory

File system updates are serialized and journaled to local disk

File system is periodically checkpointed: log items are aggregated into

We are assuming that users will have fast connectivity to cloud

Latency is a fundamental problem (unless cloud data centers built

Proxy can fully utilize

Results argue for larger

Read performance depends on working set/cache size ratio

At 100% hit rate, comparable to local NFS server

Even at 50% hit rate, latency within about 2 to 3 of local case