Chapter 4b SNIA-SSM-slides-2003-04-13

Obligatory rubric
• Copyright © 2000,2003 Storage Networking

Industry Association (SNIA).
• SNIA-TC members may use this material freely; all
others require permission from the SNIA.
• See the speaker notes for this slide for important

usage information.
• See SNIA-SSM-colors.ppt for printing and
color/graphics conventions.
1 Copyright © 2000,2003, Storage Networking Industry Association

SNIA shared-storage model
A work in progress …
An architectural overview
This revision:
• 2001-06-05 last content update
• 2003-04-13 last graphics update
Copyright © 2000,2003, Storage Networking Industry Association

Contents
• Purpose
• The SNIA storage model
Layers, functions, and services
Networks and interfaces
• Applying the SNIA storage model
Common storage architectures
• Conclusions

Purpose
• Present a simple model for shared storage

architectures
• Use it to describe common examples
graphically
• Expose, for each one:
What services are provided, where
Where interoperability is required
[future] Pros and cons of the architecture

Benefits
• A common “architecture vocabulary”

• Reference comparisons between common
solutions
• Help to align the industry
Customers can better structure their choices
Vendors can better explain their differences

What the model is and is not
• It is not:
A specification, an architecture, a design, a product,
a recommendation, or an installation
• It is:
A framework that captures the functional layers and
properties of a storage system

Classic storage model
Application
Storage domain:
“anything goes!”
NAS?
Appliance?
Network?
Data
Array?
mover?
JBOD?

The SNIA shared storage model
Application
File/record layer
Storage domain
Database File system

(dbms) (FS)
Services
Host
Network
Block
aggregation Device
Storage
Storage devices
devices (disks,
(disks, …)
…)
Block layer
The SNIA storage model:
File/record layer
File/record layer

(dbms) (FS)

File/record layer — functions
• Aka “access methods”

File system, database
• Primary responsibility: packing many
smaller things into a few larger ones
Fine-grain naming, space allocation
• Secondary responsibilities
Caching for performance
Coherency in distributed systems

Block layer
Block
Block aggregation
aggregation
Storage
Storage devices
devices (disks,
(disks, …)
…)
Block layer
Block layer — functions
• Storage devices – storing data

disk drives, tape drives, solid-state disks, …
• Block aggregation – address mapping
in-SN aggregation, or “virtualization”
slicing & concatenation, striping
local & remote mirroring, RAID-n
• Examples
volume managers
disk array LUs
• Secondary responsibilities
caching

Access path examples
Application
File/record layer Note:
Note: all
all 88
possible
possible paths
paths
Database File system can
can be
be used!
used!
(dbms) (FS)
Block
Block aggregation
aggregation
Storage
Storage devices
devices (disks,
(disks, …)
…)
Block layer
Block layer
• Block-mapping functions:
what can be done Block
Block
aggregation
aggregation
• Functional decomposition:
Storage
Storage devices
devices (disks,
(disks, …)
…)
Block
Block layer
layer
where it can be done
• Sample architectures

Block layer
What can be done
• Space management
making a large store from many small ones
packing many small stores into one large one
• Striping
for performance (load balancing, throughput)
• Redundancy
full(local & remote mirroring, RAID-1, -10, …)
partial (RAID-3, -4, -5, …)
point-in-time copy

Block layer
Where it can be done
• Host-side
logical volume managers Host
Network
device drivers, HBAs Block
Block
aggregation
aggregation Device
• SN-based Storage
Storage devices
devices
HBAs, specialized SN appliances

• Device-based
array controllers (e.g., RAID)
disk controllers (e.g., sparing)

Block layer
How it is done
…
• Building blocks
Block
Block
input: vector of blocks aggregation
aggregation
output: vector of blocks

• Result: building blocks … …
can be stacked
enables the 3 layer model for the
block layer
layers can be nested on one another
could be extended to more layers

Block layer
Sample architectures
1. Direct-attach
Application 2. SN-attach
3. SN aggregation
File/record
(+ software RAID?)
layer
Host. with LVM
no LVM
Host,
with LVM
Host
Host block-aggregation
Aggregation
appliance Network block-aggregation
SN
Block
layer
Device block-aggregation
Disk array
Storage devices

File/record layer
File/record
File/record layer
layer
• Byte-mapping functions:
(dbms) (fs)
what can be done

• Functional decomposition:
where it can be done
• Sample architectures

File/record layer
What can be done
• Database management systems

tuples → tables
tables → table-spaces
table-spaces → volume
• File systems
files → volume
• New types
http caches: a kind of distributed file system?

File/record layer
Where it can be done
FS/dbms
Host with local
NFS/CIFS
Host with
• Host-side
client
file systems and databases
NFS, CIFS, etc. are client-server
splits inside the file system
• SN-based
NAS
NAS head
• Device-based
NAS functions in array box
device

File/record layer
1. Direct-attach
3. NAS head
File/record
Host Host
4. NAS server
and software RAID
layer
Host. with LVM
Host. with LVM

LAN
NAS
head Host block-aggregation
NAS
Network block-aggregation
SN server
Block
layer
Disk array

The SNIA storage model
A layered view
IV. Application
IV Application
III. File/record layer File/record
File/record layer
layer
IIIa. Database III Database
(dbms)
File system
(fs)
IIIb. File system

IIc Host
II. Block aggregation IIb

Block
Block
Network
IIa. Host IIa aggregation

aggregation Device
IIb. Network I Storage

Storage devices
devices
IIc. Device
I. Storage devices

Storage domain
(dbms)
Database
Storage
Block
Storage devices

Application
Services subsystem
Block layer
Block aggregation
aggregation
devices (disks,
File/record layer
(fs)
(disks, …)
…)
File system
The SNIA storage model
Services
Discovery, monitoring
Resource
Resource mgmt, configuration
mgmt, configuration
Security, billing
Security, billing
Redundancy
Redundancy mgmt
mgmt (backup, …)
(backup, …)
High
High availability
availability (fail-over, …)
(fail-over, …)
Capacity planning
Capacity planning
Services
• Operations off the critical path

naming, discovery, monitoring, configuration, security, billing,
redundancy management (backup, …), high availability
management (fail-over, …), capacity planning, …
strong ties into system-wide management services
• Vital for successful operation

and a major opportunity for SNIA …
… but not discussed further in this presentation

Caching
… can be added to almost any layer
Application Ideally,
Ideally, caching
caching only
only
affects
affects performance.
performance.
File/record
Host Host But:

But: coherency
coherency
and software RAID
Cache implications
implications dodo affect
affect
layer
Host. with LVM
Host. with LVM

LAN management
management
(protocols,
(protocols, interfaces)
interfaces)
NAS NAS
head head Host block-aggregation
Cache Cache
Cache Network block-aggregation

SN appliance
Block
layer
Disk array
Cache
Cache

Clustering
Inter-box aggregation
Application Purposes:
Purposes:
•• load
load spreading
spreading across
across
peers
peers (scalability)
File/record
Host Host (scalability)

•• alternate
alternate paths
paths (high
and software RAID
(high
layer
Host. with LVM
Host. with LVM

availability,
availability, scalability)
scalability)
LAN
NAS Cluster NAS

FS
head head Host block-aggregation
Multi-node LVM
SN
Block
layer
Disk array

Q: Data versus storage?
A: Putting information into containers
user: data (“learning my preferences”)
application: container(“user keystroke history”)
application: data (“user keystroke history file”)

file system: container (“byte vector”)
file system: data (“a named file”)

volume system: container (“blocks in volume”)
volume system: data (“replicated, striped layout”)

disk array: container (“blocks in LU”)

Sharing
Content sharing or resource sharing?
• Content sharing (“logical”, “data”)

contents accessed and understood by multiple
clients
• e.g., file system, Oracle Parallel Server dbms
some of the hard issues:
• coherency
• heterogeneous data formats
• Resource sharing (“container”, “physical”)

e.g., disk array where hosts access disjoint LUs

Sharing
Content sharing and resource sharing
Application
Data
Data sharing
sharing
File/record
Host Host NAS

NAS system
system isis
shared
shared –– andand so
so
layer
LAN are
are the
the files
files
with LVM
with LVM
Host
Host
NAS system Host block-aggregation
SN
Resource
Resource
Block
layer
sharing
sharing
Array
Array is
is
shared,
shared,
but
but LUs
LUs are
are
disjoint
disjoint

are pervasive in the model API
Bus
Application
Network
Full benefits
Operating System come only from
common, open
interfaces
File layer
Block layer
Network or interface
Storage device

Composition and scaling API
Bus
API
Network or interface Network
API
Open interfaces
allow:
1. vertical
Network
composition
Network or interface Bus 2. horizontal scaling
3. supplier
independence
Open interfaces require … API
Bus
• Well defined:
functions (what they do) Network
interface protocols (data formats)

access protocols
(system call, RPC, flow control, …)
• That are:
published
supported by multiple products
=> standards (which is where SNIA comes in)

Q: “SAN” versus “NAS”?
A: a poorly-formed question
Files Blocks
“SAN” iSCSI
• Q: hardware: FibreChannel vs
Ethernet vs InfiniBand? ? NAS
• Q: API: blocks vs files (aka “NAS”) FC E’net
vs objects (OSD)?
• Q: protocol: FCP vs TCP/IP vs … ?
Files Blocks
• A: (to all the above) it depends …
FS
/b
lo
ck
co
m
• Storage network (SN):
bo
FC E’net
any (mostly) dedicated network,
installed (mostly) for storage traffic
whatever the hardware, API, or protocol
Some common storage
architectures
Mapping the SNIA model onto some

current implementations

Direct-attach block storage
Application •• Direct-attach
Direct-attach
•• Multi-attach
Multi-attach box
box
File/record
and software RAID E.g.,

E.g., SCSI
SCSI
layer
Host. with LVM
no LVM
Host,
with LVM
Host
Block
layer
Disk array

SN-attached block storage
Application “SN”
“SN” == any
any network
network
used for storage
used for storage
access.
access.
File/record
E.g.,
E.g., Fibre
Fibre Channel,
Channel,
layer
Ethernet, …
no LVM
Host, Ethernet, …
Host with
LVM
SN
Block
layer
Disk array

SN-attached block storage
with metadata server
Application Aggregation
Aggregation map
map
(metadata) is
(metadata) is
supplied
supplied by
by an
an
File/record
external
external server
server
layer
no LVM
Host,
Host with
LVM
Block-aggregation Network block-aggregation

SN metadata server
Block
layer
Disk array

Block storage aggregation
in a storage network appliance
Application aka
aka “SN
“SN
virtualization”
virtualization”
File/record
Functions:
Functions: LVM,
LVM, caching
caching
layer
no LVM
Host,
with LVM
Host
Aggregation
SN
Block
layer
Disk array

Multi-site
block storage Functions:
Functions: point-in-time
point-in-time
copy, caching,
copy, caching,
local
local &
& remote
remote mirroring,
mirroring, …
…
Application Application
Host-to-host
Host-to-host
File/record
Host. with LVM

Host. with LVM
SN
SN appliance
appliance
layer
Device-to-device
Device-to-device
WAN Host block-aggregation
Aggregation
WAN
Aggregation
appliance
appliance
SN
Block
layer
Disk array Disk array
WAN

File server
Application LAN-attached
LAN-attached
“NAS”
“NAS” system
system
File/record
Host Host May

May dodo SN/block
SN/block
aggregation,
aggregation, etc.
etc. inside
inside
layer
in
in the
the NAS
NAS system
system boxbox
LAN
NAS system Host block-aggregation
private Network block-aggregation

SN?
Block
layer

File server controller
(“NAS head”)
Application No
No storage
storage in
in the
the
file
file server
server
controller
controller box
File/record
Host Host box

layer
LAN
NAS
SN
Block
layer
Disk array

NAS/file server metadata
manager (“asymmetric”)
Application Hosts
Hosts get
get file
file meta-
meta-
data from FS/NAS
data from FS/NAS
controller,
controller, then
then
File/record
Host. with LVM Host

access
LAN
access the
the data
data
layer
directly
directly
FS
FS controller
controller can
can
File system also be NAS server
also be NAS server
metadata
File Host block-aggregation
metadata
Block Network block-aggregation

SN
accesses
Block
layer
Disk array

Object-based Storage Device
(OSD), CMU NASD
Application
File/record
Host Host
File
layer
metadata
LAN
Object
metadata
OSD
Block
layer
device Security
metadata Device block-aggregation

Summary & conclusions

Storage domain
(dbms)
Database
Block
Host
aggregation
Network

Block layer
Application
File/record layer
Device
(FS)
Storage devices (disks, …)

File system
Services
Resource
Resource mgmt, configuration
mgmt, configuration
Security, billing
Security, billing
Redundancy
Redundancy mgmt
mgmt (backup, …)
(backup, …)
High
High availability
availability (fail-over, …)
(fail-over, …)
The SNIA shared storage model
Capacity planning
Capacity planning
Block layer
1. Direct-attach
3. SN aggregation
File/record
and software RAID

layer
Host. with LVM
Host. with LVM
no LVM
Host,
Aggregation
SN
Block
layer
Disk array

File/record layer
1. Direct-attach
3. NAS head
File/record
Host Host
4. NAS server
and software RAID
layer
Host. with LVM
Host. with LVM

LAN
NAS
NAS
SN server
Block
layer
Disk array

Uses for the model
• Vendors
place products in the space of architectures
clarify product differences
• Customers
understand vendor offerings better
• The industry
basis for common definitions, communication,
understanding, interoperability

Conclusions
• The SNIA shared storage model is both simple and

useful
to highlight similarities and differences
as a basis for comparisons
• Still a work in progress
data movers, tape drives, …
better comparisons …
suggestions?
• The SNIA-TC welcomes input:
<snia-tc@snia.org>

Authors
• John Wilkes, Hewlett-Packard (project lead)

• Harald Skardal, NetApps
• David Black, EMC
• Wayne Rickard (SNIA TC chairperson), Gadzoox
• Co-conspirators: the rest of the SNIA Technical
Council
Dave Anderson, Seagate Technology
Jim Carlson, IBM
Garth Gibson, CMU/Panasas
Kevin Phaup, HighGround Systems
David Thiel, Compaq

Chapter 4b SNIA-SSM-slides-2003-04-13

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Chapter 4b SNIA-SSM-slides-2003-04-13

Uploaded by

Copyright:

Available Formats

Obligatory rubric

• Copyright © 2000,2003 Storage Networking

• See the speaker notes for this slide for important

1 Copyright © 2000,2003, Storage Networking Industry Association

Copyright © 2000,2003, Storage Networking Industry Association

3 Copyright © 2000,2003, Storage Networking Industry Association

• Present a simple model for shared storage

4 Copyright © 2000,2003, Storage Networking Industry Association

• A common “architecture vocabulary”

5 Copyright © 2000,2003, Storage Networking Industry Association

6 Copyright © 2000,2003, Storage Networking Industry Association

7 Copyright © 2000,2003, Storage Networking Industry Association

Database File system

Database File system

9 Copyright © 2000,2003, Storage Networking Industry Association

• Aka “access methods”

10 Copyright © 2000,2003, Storage Networking Industry Association

• Storage devices – storing data

12 Copyright © 2000,2003, Storage Networking Industry Association

14 Copyright © 2000,2003, Storage Networking Industry Association

15 Copyright © 2000,2003, Storage Networking Industry Association

 HBAs, specialized SN appliances

16 Copyright © 2000,2003, Storage Networking Industry Association

 output: vector of blocks

17 Copyright © 2000,2003, Storage Networking Industry Association

Host. with LVM

18 Copyright © 2000,2003, Storage Networking Industry Association

what can be done

19 Copyright © 2000,2003, Storage Networking Industry Association

• Database management systems

20 Copyright © 2000,2003, Storage Networking Industry Association

21 Copyright © 2000,2003, Storage Networking Industry Association

Host. with LVM

Host. with LVM

22 Copyright © 2000,2003, Storage Networking Industry Association

IIIb. File system

II. Block aggregation IIb

IIa. Host IIa aggregation

IIb. Network I Storage

23 Copyright © 2000,2003, Storage Networking Industry Association

24 Copyright © 2000,2003, Storage Networking Industry Association

• Operations off the critical path

• Vital for successful operation

25 Copyright © 2000,2003, Storage Networking Industry Association

Host Host But:

Host. with LVM

Host. with LVM

Cache Network block-aggregation

26 Copyright © 2000,2003, Storage Networking Industry Association

Host Host (scalability)

Host. with LVM

Host. with LVM

NAS Cluster NAS

27 Copyright © 2000,2003, Storage Networking Industry Association

 application: data (“user keystroke history file”)

 file system: data (“a named file”)

 volume system: data (“replicated, striped layout”)

28 Copyright © 2000,2003, Storage Networking Industry Association

• Content sharing (“logical”, “data”)

• Resource sharing (“container”, “physical”)

29 Copyright © 2000,2003, Storage Networking Industry Association

Host Host NAS

NAS system Host block-aggregation

HBAs, specialized SN appliances

output: vector of blocks

application: data (“user keystroke history file”)

file system: data (“a named file”)

volume system: data (“replicated, striped layout”)

interface protocols (data formats)