Fibre Channel Basics

Cisco Systems CAE.

 Small Computer System Interface - SCSI

  SCSI is a standard that defines an interface between an initiator

(usually a computer) and a target (usually
a storage device such as a hard disk)
  Logical Unit Number (LUN): A 64-bit field within SCSI that
identifies the Logically addressable Unit within a target SCSI
device

Host (Initiator) Disk (Target)

SCSI READ OPERATION
DATA STATUS
DATA DATA READ

SCSI I/O Channel

Host (Initiator) Disk (Target)

SCSI WRITE OPERATION
READY
STATUS DATA WRITE
DATA DATA

SCSI I/O Channel

Cisco Systems CAE.

The SCSI I/O Channel
SCSI I/O Channel provides half-
duplex pipe for SCSI Applications
Half-Duplex
command, data, and status File System I/O Channel
SCSI I/O channel can be Block Device
internal or external to host
SCSI Generic
Multiple SCSI I/O channels can
exist within host TCP/IP Stack

A network approach can scale

SCSI
NIC Driver Adapter Driver
the I/O channel in many
areas (length, devices, NIC Adapter SCSI Adapter

throughput)

SCSI
Initiator
SCSI
Target

Cisco Systems CAE.

Fibre Channel I/O Networking
Host System
Fibre Channel provides high speed transport
for SCSI payload via Host Buss Adapter Initiator

(HBA)
SCSI
Fibre Channel overcomes many
shortcomings of Parallel I/O including:
Addressing for up to 16 million nodes Fibre Channel
Loop (shared) and fabric (switched) transport HBA

Host speeds of 100 to 400 MBps (1–4 Gbps)

Segments of up to 10km (without extenders) Fibre Channel
Fabric
Support for multiple protocols
Combines best attributes of a channel
and a network together

Target
Cisco Systems CAE.
Consolidated I/O Networking
Traditional Ethernet with Fibre FCoE
Host
Channel over Ethernet (Initiator)
FCoE
Consolidated Network Adapter Ethernet
Network
– (CAN)
FCoE CNA
Native Ethernet NIC FCoE
(Target)
Native FC HBA
Relies on Lossless Ethernet
PAUSE per IEEE 802.1p
Able to build Ethernet-based SCSI

SANs using FCoE arrays FC

Fabric FC HBA
Attached
Host
(Initiator)
Storage
(Target)

Cisco Systems CAE.

Fibre Channel Communications
Point-to-point oriented
  Facilitated through device login
N_Port-to-N_Port connection
  Logical node connection point
Flow controlled
  Buffer-to-buffer credits and end-to-end basis
Acknowledged
  For certain classes of traffic, none for others
Multiple connections allowed per device

Node Node
Transmitter Transmitter
N_Port N_Port
Receiver Receiver

Link

Cisco Systems CAE.

Fibre Channel Port Types
‘N’ port: Node ports used to connect devices to N N

switched fabric or point to point configurations

‘F’ port: Fabric ports residing on switches
connecting ‘N’ port devices
N F
‘L’ port: Loop ports are used in arbitrated loop
configurations to build networks without FC
switches; these ports often also have ‘N’ port
capabilities and are called ‘NL’ ports NL FL

‘E’ port: Expansion ports are essentially

trunk ports used to connect two Fibre L L
Channel switches
‘GL’ port: A generic port capable of operating
as either an ‘E’ or ‘F’ port; it’s also capable of
E E
acting in an ‘L’ port capacity; Auto Discovery

Cisco Systems CAE.

Fibre Channel Port Types
Fibre Channel Switch
Input Fabric Output
Port X Port

Fabric
Node NL_Port FL_Port E_Port E_Port Switch

G_Port F_Port N_Port Node

Node NL_Port

G_Port F_Port N_Port Node

Node NL_Port

G_Port F_Port N_Port Node

Cisco Systems CAE.

Inter-Switch Link (ISL)

EISL

The interconnection between switches is called the ISL

  E_Port to E_Port (‘Expansion port)
Supports all classes of service
  Class 1, 2, 3, and a special Class F (switch-to-switch)
FC-PH permits consecutive frames of a sequence to be routed over
different ISL links for maximum throughput
Cisco’s implementation is to dedicate an FC_ID pair and/or a given
exchange to an ISL bundle member to guarantee in-order delivery
for exchange/sequence frames
Cisco Extended ISL (EISL, TE port)

Cisco Systems CAE.

Worldwide Names

Each switch element is assigned a WWN at time

of manufacture
Each switch port is assigned a WWN at the
time of manufacture
During FLOGI the switch identifies the WWN
in the service parameters of the accept frame
and assigns a Fibre Channel ID (FCID)
These address assignments can then correlate
each fabric port with the switch element

Cisco Systems CAE.

 Fabric Addressing
FC_ID Address Model

FC_ID address models help speed up routing

Switches assign FC_ID addresses to N_Ports
Some addresses are reserved for fabric services
Private loop devices only understand 8-bit address (0x0000xx)
FL_Port can provide proxy service for public address translation
Maximum switch domains = 239 (based on standard)

8 Bits 8 Bits 8 Bits

Switch Topology Switch

Area Device
Model Domain

Private Loop Device Arbitrated Loop

Address Model 00 00 Physical Address
(AL_PA)

Public Loop Device Arbitrated Loop

Switch
Address Model Area Physical Address
Domain
(AL_PA)
Cisco Systems CAE.
Fabric Routing: FSPF

FSPF routes traffic based on destination domain ID

For FSPF a domain ID identifies a single switch
This limits the max number of switches that can support
in the Fabric to 239 when FSPF is supported
FSPF performs hop-by-hop routing
FSPF uses total cost as the metric to determine most
efficient path
FSPF supports hierarchical path selection
Provides the scalable routing tables in large topologies
Static routes can be applied

Cisco Systems CAE.

Buffer to Buffer Credit Flow Control
BB_Credits and Distance
~ 2 km per Frame
1 Gbps FC

~ 1 km per Frame
2 Gbps FC

~ ½ km per Frame
4 Gbps FC

16 Km

BB_Credits are used to ensure enough FC frames in flight

A full (2112 byte) FC frame is approx 2 km long at 1 Gbps,
1 km long at 2 Gbps and ½ km long at 4 Gbps
As distance increases, the number of available BB_Credits
need to increase as well
Insufficient BB_Credits will throttle performance—no data will be
transmitted until R_RDY is returned
Cisco Systems CAE.
Fibre Channel Fabric Zoning

Zoning operation
Zone members “see” only other members of the zone
Zones can be configured dynamically based on WWN
Devices can be members of more than one zone
FC-AL zoning allows the creation of private loops on a single hub
Switched fabric zoning can take place at the port or device level
Based on physical switch port
Based on device WWN SAN
Based on LUN ID Disk2 Disk3
Disk1
ZoneA Host1
Benefits ZoneC

Secured device access Host2

Disk4
Allows operating system co-existence ZoneB

Cisco Systems CAE.

Virtual SANs (VSANs)

Cisco MDS 9000

A Virtual SAN (VSAN) Provides a Family with VSAN Service
Method to Allocate Ports within a
Physical Fabric to Create Virtual
Fabrics
Analogous to VLANs in Ethernet
Physical SAN
Virtual fabrics created from larger cost- Islands Are
effective redundant physical fabric Virtualized onto
Common SAN
Reduces wasted ports of island approach Infrastructure
Fabric events are isolated per VSAN—
maintains isolation for HA (i.e. RSCNs)
Hardware-based isolation—traffic is explicitly
tagged across interswitch links with
VSAN membership info
Statistics can be gathered per VSAN

Cisco Systems CAE.

Cisco Systems CAE.