Toronto, Canada

May 30, 2013

Ciscos Cloud Ready

Infrastructure

VMDC Virtualized Multi-service Data

Center
Sunil Cherukuri (sunilc@cisco.com)
Technical Lead, Systems Development Unit

2012
2011 Cisco and/or its affiliates. All rights reserved.

Cisco Connect

Hotels and Data Centers

Question: What do they have in common?

Hotels and Data Centers

Answer: Both are Shared Infrastructures

Hotel Rooms are expected to have

Secure Separation

Data Centers without Secure Separation look like

hotel rooms with military sleeping quarters

Data Centers Deployed Following VMDC Guidelines Ensures Critical Workloads are Protected
2012 Cisco and/or its affiliates. All rights reserved.

Cisco Connect

Acronyms
VMDC Virtualized Multi-Service Data Center
ICS Integrated Compute Stack
Vblock An ICS based on Cisco, EMC & VMWare components

FlexPod An ICS based on Cisco, NetApp & VMWare components

HCS Hosted Collaboration Solution. A Cisco software solution to provide rich voice &

collaboration software in a cloud environment

DCI Data Center Interconnect.
VPDC Virtual Private Data Center
HVD Hosted Virtual Desktop

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Connect

Agenda
Cloud Drivers and Introduction
VMDC System Overview
VMDC Physical Infrastructure
VMDC Logical Infrastructure
VMDC Infrastructure Updates
VMDC Management and Automation
VMDC Summary

Cloud Drivers &

Introduction

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Connect

Global Data Center Traffic Growth

Data Center Traffic Nearly Quadruples from 2011 to 2016
31% CAGR 20112016

7.0

6.6 ZB

Zettabytes / Year

6.0
5.0

5.2 ZB

4.0

4.1 ZB

3.0

3.3 ZB
2.6 ZB

2.0
1.8 ZB

1.0
0.0

2011

2012

2013

2014

2015

2016

Workload Shift: Cloud vs. Traditional

Nearly Two-thirds of all Workloads Will Be Cloud-based by 2016
20% CAGR 20112016

Installed Workloads in
Millions

200
180

Cloud Data Center

160

Traditional Data Center

140
62%

120
100

52%

80
60

30%

40
20

38%

48%

70%

2011

2012

2013

2014

2015

2016

Global Data Center Traffic by Destination

Most Data Center Traffic Consistently Stays Within the Data Center
A

Data Centerto-Data Center

7%

Within Data Center (76%)

Storage, production
and development data,
authentication

Data Centerto-User
17%

Within
Data Center
76%

Data Center-to- Data Center (7%)

Replication,
inter-database
links
C

Data Center-to-User (17%)

Web, email,
internal VoD,
WebEx, et al.

2012 Cisco and/or its affiliates. All rights reserved.

Cloud Adoption Drivers

The Challenge

Business Goals

1.

Grow revenue

1.

2.

Lower overall
operating costs

Improve IT
efficiency

2.

Acquire/retain
customers

Streamline/improve
business processes

3.

Drive new market

offering or business
practices

Increase IT
resources to drive
business innovation

4.

Improve customer
management
capabilities

3.

4.

5.

Improve workforce
productivity

IT Initiatives
Raising the
productivity of
employees whose jobs
can't be automated is
the next great
performance
challenge.
McKinsey & Company, The 21st
Century Organization, 2005

Ciscos Cloud Strategy

Enabling Cloud Applications/Services by Uniquely Combining
the Unified Data Center and Cloud Intelligent Network
Tailored Solutions for
Building Clouds

Rich Ecosystem of
Integrated Solutions
Research In Motion

Enable customers to
build and operate private,
public or hybrid clouds

Innovative
Cloud Services

SAMSUNG

Enable customers to
deploy tested, best of
breed solutions

Enable cloud services

including people-centric
collaboration and other
applications

Cisco IT Case Study CITEIS

Cisco IT Elastic Infrastructure Services (Internal Private Cloud)
Average
TCO

-37%
-31%
Speed of delivery

6-8 Weeks

Average
TCO

IT Maintenance /
IT Innovation

Speed of Delivery

2-3 Weeks

15 Minutes

IT Maintenance /
IT Innovation

70/30

100% Physical,
Legacy Computer Platform

Speed of Delivery

Average
TCO

60/40
40% Physical, 60% Virtual,
Legacy Computer Platform

Virtualization

IT Maintenance /
IT Innovation

40/60
35% Physical, 65% Virtual,
Unified Computing Platform,
100% Automated

Unified Infrastructure
and Automation

What Is Cloud Computing?

Taxonomy Check
Visual Model of NISTs Working Definition of Cloud Computing
Essential
Characteristics

Service
Models

Deployment
Models

Measured Service

Rapid Elasticity

On-Demand
Self Service

Broad Network
Access

Resource
Pooling

Software as a
Service (SaaS)

Platform as a
Service (PaaS)

Infrastructure as a
Service (IaaS)

Public

Private

Hybrid

Community

http://www.csrc.nist.gov/groups/SNS/cloud-computing/index.html

A Style of Computing Where Massively Scalable IT-Enabled Capabilities Are

Delivered as a Service to Multiple External Customers Using Internet Technologies
Source: Gartner 2008

IT Resources and Services that Are Abstracted from the

Underlying Infrastructure and Are Provided On-Demand and At Scale

Cloud Deployment Models

Private, Public, Hybrid
Private Cloud

Private Cloud

Open Cloud
Private Cloud

Virtual
Private Cloud

Private Cloud

Inter-Cloud

Stand-Alone
Data Centers
Public Cloud

PRESENT

14

2009 Cisco Systems, Inc. All rights reserved.

Public Cloud

Enterprise
Extension
Hybrid Cloud

Public Cloud
#1
Public Cloud
#2

The Journey to Cloud

Evolution of IT + Business Agility
Consolidation
(Reduce Costs)

Virtualization
(Improve Agility)

Automation
(Transform IT)

Platinum
Gold

IT Infrastructure

Business Applications

IT-as-a-Service

VMDC System
Overview

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Connect

16

Virtualized Multiservice Data Center

Inter-Data
Center Networking

WAN

Multi-Site Connectivity

Unified Fabric and

Data Center
Networking

Services

Networking Fabric

Providing Network
and Services
Virtualization

Cloud Service
Management

Business
Support

Network
Fabric

Unified Computing and

Integrated
Systems

Unified Computing

Access

Storage

Providing Server
and Application
Virtualization

Compute

NAS

Compute

SAN

All Specifications Subject to Change without Notice

Portability/
Interoperability

VMDC

Provisioning
Configuration

Cisco Virtualized Multi-Service Data Center (VMDC)

Cisco VMDC: A validated reference architecture
CVD Design & Implementation Guides
Validated Orchestration & Assurance tools

Reducing time to deployment

Reducing risk
Increasing flexibility
Improving operational efficiency

A flexible, modular, scalable and validated architecture that

combines integrated computing stacks, unified data center, and data
center interconnects into an end-to-end architecture Blueprint for
Cloud
Multi-service & Multi-tenant
One-cloud solution for any layer infrastructure, platform, and software
as a service (IaaS and SaaS) and any deployment
(Private, Public, or Hybrid Cloud)
Design the Cloud with Confidence

Cisco Validated Design Process

Innovation and Quality Through System Level Design and Validation
Key Customer Engagements

Consider end-to-end view

Product Development

Thought Leadership

System level innovations

System
Development
Fundamentals

Cross platform collaboration

System Delivery

Tested and validated designs

System Development Guidelines

Documentation

Customer

System

Integration

End-To-End Validation

Feature

Design

Unit

Planning

VMDC Solution Validation Scope

- Synopsis
Systems Level End to end feature/integration testing to enable service delivery, and multi-tenancy / isolation
Multi-dimensional Scalability (i.e., Tenants, VMs, VLANs, MAC, HSRP, Routes, Contexts, etc.)
3rd party components, including BMC CLM, Zenoss CSA, Vmware vSphere, vCloud Director, EMC , NetApp, Citrix Netscaler, Microsoft,
RedHat

Service Differentiation Validation of Service Tier offering (network, compute, storage) and DC Services
(VPN, FW, IPS, SLB, GSS etc). Stress/Load tests to validate end-end Service Flows, QoS, reliability, monitor
cpu/memory.
Real-World Simulations Baseline Steady State Traffic and background traffic injection (N-S, E-W, Stateful

+ Stateless)
HA Focus Failover/Negative tests to validate redundancy designs and technologies end to end (Routing,

vPC/MEC, ECMP, VSS, HSRP, Active-Active service modules, Clustering, SAN, Fabric, UCS blades,
Storage controllers). Analysis and characterization of end to end service restoral.
Manageability Validation of statistics and monitoring capabilities SNMP, Sylog, Netflow, I/O statistics, etc.

Validation of Orchestration & Management tools

Cloud Megatest (LightReading / EANTC)

http://www.cisco.com/en/US/solutions/ns341/eantc_cloud.html

Cross Architecture Validation Cloud/DC, Mobility, IPv6, Video, Collaboration

VMDC 2.2 based architecture each test overlaid as tenant in multi-tenant cloud
70+ 10G IXIA ports, 75+ VRFs/tenants, 600+ VLANs, 1500+ VMs
80 Gbps of north-to-south (next-generation network [NGN]
to cloud) traffic: 1 million clients to 50,000 servers
300 Gbps of east to west (within data center ) traffic: switched & routed - with Cisco
FabricPath 2-tier design showing 15,000 MAC addresses and 256 VLANs
67 million NAT64 sessions simulated, at 80Gbps, 4 million/sec
1 million residential gateways shown for 6rd, at 80Gbps
40Gbps of video streaming using Cisco CDS Internet streaming appliance, and on a
Cisco ASR 9000 Series Cisco Integrated Services Module (ISM)
PCRF for throttling mobile subscribers in real time
Any video format, any device, any where: iPad, Android, PC, etc.
Cisco VM-FEX in VMDirectPath performance demonstrated:
VM-FEX compared to software DVS in 4 ways:
iSCSI read-write, L3 IMIX traffic, HTTP traffic, and video encoding
20 to 30% performance improvement in throughput, CPU, and IOPS with Cisco Data
Center VM-FEX DirectPath I/O

Challenges in Building and Maintaining Cloud

You need to have a plan!
Predictably grow the data center

Provision the DC

Scale the data center

Manage and Monitor the DC

Secure the data center from external and

Cloud Automation

internal threats
Protect the DC from HW and SW failures

Establish virtual containers to be assigned

to consumers with pre-defined service

policy profiles
Securely separate these virtual containers
Connect the DC to other DCs

Provision virtual containers and

assign to consumers
Manage virtual containers
Provide differentiated services for

consumers of virtual containers

Cloud resource management and

assurance

VMDC Physical
Infrastructure

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Connect

23

Building a Multi-Service Infrastructure

Architecture Pillars

Modularity

High Availability

Pod based design

Scalability framework for manageable increments
Predictable physical and cost characteristics
Streamline Turn-up of New Services

Carrier Class Availability

Platform/Network/Hardware/Software Resiliency
Minimize the probability and duration of incidents
Focus on your business, not fighting fires

Secure Multi-tenancy
Shared Physical Infrastructure
Tenant Specific Resources
Use Cases
Comply with business policies

Differentiated Service Support

Service Orchestration

Design logical models around use cases

Services-oriented framework
Combines compute/storage/network
Resources are applied and tuned to meet needs

Dynamic application and reuse of resources

Automated service orchestration and fulfillment
Integration with Network Containers
Rapid Self Service IT

The Challenge:
How do I scale my data center?

Point of Delivery (PoD)

Architectural consistency
through a modular approach

PoD
Service
Appliances

Data
Center
Services
Node

Integrated Compute Stack

Storage

The Solution

Point of Delivery
(POD)
2012 Cisco and/or its affiliates. All rights reserved.

Network

Compute

Integrated Compute Stack

Storage

Network

Compute

Modular, tiered construct consisting of

groupings of integrated compute stacks
plus storage and networking
infrastructure
A single Pod can be deployed and
operated by itself or connected together
to other Pods to achieve scale
VMDC validates 2 styles of Pods:
Compact and Large

Benefits
Simplified capacity planning
Ease of new technology adoption
Consistent and efficient operation

Cisco Virtualized Multi-Service Data Center

Scale the Compute with UCS and Integrated Compute Stacks
Data Center

VMDC

Enhanced Data
Center Interconnect

Core

Aggregation

Business
Support
VMDC

Unified
Data Center
Networking

Cloud Service
Management

Services

Access

Portability/
Interoperability
VMDC

Integrated
Compute Stacks

NAS

FlexPodCompute

SAN

VMDC

Provisioning
Configuration

Scalable Compute: VMDC Supported ICS

VCES Vblock Family of Cloud Infrastructure Packages
Pre-Integrated and Supported
Cloud Infrastructure
Focus teams on using infrastructure vs.
assembling and supporting the individual
components

Cloud Service Provider

Operational Model
Provisioning, service delivery, chargeback, etc.

Vblock Series 700

Storage: EMC Symmetrix Vmax

Compute: Cisco UCS

Virtualization: VMware
Orchestration:
Unified Infrastructure
Manager (UIM)
Vblock Series 700
model MX

Vblock Series 300

Storage: EMC VNX

Accelerates the Shift to a Private

Cloud Model
Less time debating, more time using

Compute: Cisco UCS

Virtualization: VMware
Orchestration: Unified
Infrastructure Manager (UIM)
Four Models

Scalable Compute: VMDC Supported ICS

Cisco and NetApps FlexPod Reference Architecture
Standard, pre-validated, best-in-class

infrastructure building blocks

Flexible: One platform scales to

Cisco UCS B-Series

Blade Servers and
UCS Manager

fit many environments and

mixed workloads
Add applications and workload
Scale up and out

Cisco Nexus 5000

Family Switches

Simplified management and

repeatable deployments
Design and sizing guides
Services: Facilitate deployment of

different environments

NetApp FAS
10GE and FCoE

Scaling the Infrastructure

Add PoDs to Meet Business Demands
Key Factors to Consider

PoD
Service
Appliances

Service
Appliances

Data
Center
Services
Node

Data
Center
Services
Node

Integrated Compute
Stack
Storage

Network

Compute

The Solution

PoD replication

2012 Cisco and/or its affiliates. All rights reserved.

Integrated Compute
Stack
Storage

Network

Compute

PoD

Integrated Compute
Stack
Storage

Network

Compute

Integrated Compute
Stack
Storage

Network

Compute

L2 Scale - Virtual Machine Density,

VMNics per VM, MAC Address
Capacity,
Cluster Scale, ARP Table Size,
VLAN scale, Port Capacity, Logical
Failure Domains L2 Control Plane
L3 Scale BGP Peering, HRSP
Interfaces, VRF Instances, Routing
Tables and Convergence, Services
Resource Oversubscription
Network Compute, and Storage
Oversubscription, Bandwidth per VM

Benefits
Optimize CAPEX savings while maintaining SLAs
Predicable performance and scale based on building blocks
Effective way to add separate application environments
Pods are interconnected at Core or WAN layer
29

VMDC 2.2 Topology

VMDC 2.2 Scale

N7k Core

PoD1

Service
Appliances

N7k AGG

DataCenter
Services
Node

Integrated Compute Stack

Integrated Compute Stack

Nexus 7k access
Storage

Network

Compute

Storage

Network

Compute

Pod
max. of 512 servers, 12,000 VMs
200 tenants

DC
max. of 3072 servers, 72,000 VMs
200 tenants
Scale can be increased through tweaking design
- removing Core layer
- reducing cross-connects (ladder)
- reducing BGP peering
- using Static routes instead of BGP
- using different Core or Agg devices
- using MPLS instead of VRF-Lite

Network Scale Considerations

PoD Scalability
What Determines the Host Scale in a PoD?
AggregationNumber of ICS, Blades
Work-load domain
Number of MAC address and VLANs
Failure Domain
Features to facilitate L2/L3 Boundary

Compute

Network

VM Density

VM Sizing

MAC per VM

STP logical ports

Logical Ports

Virtual Switch

Storage

Number of vFiler IP
Space

Number of VLANs
supported

Number of VLANs,
Tenants

Number of 10 Gig
NICs

Routing Peers,
Adjacencies

Number of LUNs,
Ports, Zones, WWNs

Total number of MAC

Addresses & ARP
entries

PoD Sizing Considerations

For Reference

Network

SAN/NAS ports/links

Traffic throughput

Storage throughput, oversubscription, IOPS

Number of Tenants (VRFs), VLANs

Number of VSAN, Zones

Oversubscription factors

Storage Array Density (disks, ports)

High Availability (redundant links)

Distributed or Centralized Storage

Port and Line-card/Chassis density

Platform scalability (VRFs, VLANs, Interfaces)
Ratio of Service Tiers (Gold:Silver:Bronze)

Security & L4-7 Services

Service Modules or Appliances
Type and Number of Services
Number of Virtual Contexts (Modules)
Number of VLANs tied to service modules
Application throughput

Storage

Compute
Number of VMs per tenant, per VLAN
VM to Core ratio, Memory size per VM

Number of links, oversubscription factors

Ratio of Service Tiers (Gold:Silver:Bronze)
Number of blades in a UCS cluster
Number of blades in a ESX cluster

Number of VMs per blade, per cluster, per Pod

VCenter limits on VMs, Servers, DataStores, Ports

The Challenge:
How do I ensure high availability?

Redundant links, nodes and

paths, end to end plus:
L2 Redundancy vPCs,
MEC, and MACpinning
L3 Redundancy - HSRP,
Non-stop forwarding, nonstop routing, LDP sync,
MPLS graceful restart
Compute Redundancy -
UCS end host mode, others
(N1KV and MAC-pinning,
Active/Standby Redundancy,
Intra-Cluster HA)

End to end HA
architecture
2012 Cisco and/or its affiliates. All rights reserved.

Access

NAS

The Solution

Resilient Fabric
& Services

Storage Redundancy FC
port channeling, multi-pathing
software from VMware or
SAN vendor
Services Redundancy
ASA, ACE redundancy
Routing Protocol
Redundancy - BGP, OSPF

Compute

SAN

Benefits
Maximize infrastructure uptime
Comprehensive end to end architecture
Focus on your business, not fighting fires
34

The Challenge:
How do I enable QoS for SLA Compliance?

Define low latency traffic

classes in this new
multimedia service tier (i.e.,
VoIP bearer and video
conference) are
characterized by three
metrics - bandwidth, delay,
and availability.

Data Center

Core

Aggregation

Services

Support QoS across hybrid

public/private domains

Access

NAS

The Solution

Quality of Service

2012 Cisco and/or its affiliates. All rights reserved.

Compute

Traffic Classification and

Marking - It is a general
best practice to mark traffic at
the source-end system or as
close to the traffic source as
possible in order to simplify
the network design.
Hierarchical QoS for MultiTenancy
Queuing, Scheduling, and
Dropping accounts for
differences in queuing
structures
Shaping and Policing

SAN

Benefits
Supports applications with differing latency requirements
Provides end to end QoS
Supports QoS across hybrid public/private domains
35

VMDC Logical
Infrastructure

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Connect

36

Cloud Consumer
Defining the Role
Cloud Consumer
Customers

Business Units

Applications

Departments

Shared IT Infrastructure

Cloud consumers use shared infrastructure resources

Each organization must choose its approach to resource allocation and separation
These policies define a form of multi-consumer or multi-tenancy

VMDC Consumer Models

Deployed as Containers
Consumer models are logical constructs

instantiated on the VMDC infrastructure

Consumer models

are customizable
Consumer models must address application

requirements
Consumer models should be unaware of

underlying technology
Models will evolve as new technologies are

introduced

Cloud Consumer X

VMDC Consumer Models

Addressing Application Requirements
Network Requirements

Session persistence

High Availability

Scalability

Latency Mitigation

Reliable transport

Cloud Consumer X

Security Requirements

Secure sessions with encryption

may be required
Each layer of the application stack
authenticates data transport

Cloud Consumer Y

Cloud Consumer Z

The Challenge:
How do I create tenant (consumer) containers?
Physical Data Center
Components

Resilient
Fabric

Services
Firewall

Load
Balancer

Compute

Virtual
Firewall

UCS

Virtualized
Contexts
Zone 1
Tenant 1

The Solution

Pre-defined and
Automated
Virtualized
Containers

2012 Cisco and/or its affiliates. All rights reserved.

Tenant 10

Zone 2
Tenant 20

Tenant 30

Benefits
Quickly and securely onboard similar consumers
Covers different levels of network services for a variety of needs
Addresses varying security, QoS, and other requirements
Solutions available to automate the process
40

VMDC Container Model

Tiered Security in VMDC 2.2

Private
(Tenant VRF)

Public/Shared

Less Trusted Zones

VRF
ASA Context
(per tenant)
Protected VRF
(control point)

vPath

Nexus
1000v

Public Zone (DMZ)

Protected FE

Front-end Zones

VSG

Zone 1

Zone 2

Back-end Zones

Sub-Zone
W

Sub-Zone
X

Sub-Zone
Y

Sub-Zone
Z

Zone 3

VMDC Sample Network Containers

Differentiated Services Common Platform
Bronze

Silver

L3

Gold

L3

Palladium

Expanded Gold

L3
LB

FW
LB

Public Zone

LB

LB

FW

vFW

Private Zone

L3
L2

L3
L2

FW

Protected
Front-End

L3
L2

L3
LB
L2

FW

Protected
Back-End

L3
vFW

vFW

vFW

LB
L2
vFW

Predefined containers provide examples for different types of deployments

Automated provisioning and management logic for each container type is pre-defined in the Management
and Orchestration software
Customers can choose from existing models or define their own customized models Flexible framework
allows variations
2012 Cisco and/or its affiliates. All rights reserved.

The Challenge:
How do I secure my containers?

Defense in Depth per

consumer (front end ASA,
back end VSG)
VRF-lite implemented at core
and aggregation layers
provides per tenant isolation
at L3
Separate dedicated pertenant routing and forwarding
tables insuring that no intertenant (server to server)
traffic within the data center
will be allowed, unless
explicitly configured

The Solution

Container service
abstraction and right
sizing
2012 Cisco and/or its affiliates. All rights reserved.

WAN
L3

WAN Edge

Si

Si

Layer 2 Trunks

Core

L3

Layer 2 Trunks
Aggregation

L3
L2

HSRP/L3
Gateway

Web

Web

Web

App
Data
base

App
Data
base

App
Data
base

VLAN IDs and the 802.1q tag

provide isolation and
identification of tenant traffic
across the L2 domain
Compute Separation
(vNICs, VLANs, Port Profiles)
Storage Separation (Cluster
File System Mgmt, VSAN and
FC Zoning, LUN Masking,
vFilers)
Application Tier (Network
Centric, Logical and Physical
segmentation with L2/L3
firewalling and security
zoning)

Benefits
End to end secure separation across the data center
Overlapping IP addresses are allowed
Automation tools to simplify deployment
43

The Challenge:
How do I achieve efficient Business Resilience and Disaster Recovery?
Cisco Catalyst
6500 Series

Ciscos Data Center

Interconnect
LAN Extentions:
OTV, VPLS, EoMPLS,
Path Optimization
LISP, GSS
SAN Extensions
NetApps FlexCache,
MetroCluster
EMCs VPLEX

The Solution:

Cisco DCI Solution

enables new operational
models for BR, DR and
Multi-site operations
2012 Cisco and/or its affiliates. All rights reserved.

Cisco Nexus 7000

Series

Cisco Nexus 7000

Series

Benefits
Workload balancing across data centers and clouds
Proactive response to disruptions mitigates risks of Approaching disasters, viz. hurricanes,
floods, etc., Power grid maintenance, Data center maintenance and migrations
Planned events scheduled over a period of time
Backup and Disaster Recovery aaS

44

Data Center Interconnect

Ent DC1

Enterprise DCI back door

PE

Interconnection Models:
Enterprise to Enterprise (E2E)
Enterprise to Service Provider (E2SP)
Service Provider to Service Provider (SP2SP)

Ent DC2

Overlay-Based Techniques
OTV, LISP, VXLAN

PE

SP NGN
DCPE

CE

CE

DCPE
DCE
DCE

SP DC1

SP DC2

Standalone DCI network

Ethernet

(e)TRILL / 802.1ad

MPLS
IP

VPLS, A-VPLS, EVPN, EoMPLS

OTV, LISP, VXLAN

Suitable for intra-Ent DC interconnect

NGN-Based DCI Solution:
Addresses E2SP for workload migration
Addresses SP2SP for regional or distributed data centres

Standalone DCI Network

Provides interconnection between main SP DCs
Owned by SP DC team
Addresses SP2SP only
Very high bandwidthpacket/optical solution likely the
most cost effective

VMDC
Infrastructure
Updates

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Connect

46

VMDC 2.3
Lower Cost, Higher Tenancy Scale VMDC design
Based on VMDC 2.2, with some optimizations in the architecture to achieve higher tenancy scale, with

lower cost.
Increased tenant scale - can support upto 500 mixed tenants in a Pod, and 2000 in a DC
Lower VM scale per can support upto 6000 VMs in a Pod, and 24000 in a DC.
VRF-Lite and vPC based architecture - VMDC 2.x aligned
Collapsed Core/Aggregation layer uses smaller Nexus 7004 platform with F2 modules to reduce

cost. Can use other Nexus 7000 form-factors and modules for higher port-density.
ASA/ACE appliances directly connected to Nexus 7004 no 6500-DSN layer

ASR 1006 as DC-PE. Can use other ASR form factors for higher port-density.
Architecture works with Vblock, FlexPod or any other integrated compute stacks. For validation, a

FlexPod-aligned topology was used

VMDC 2.3 Physical Topology

VMDC 2.3 Expanded Gold Container

VMDC 2.3 Silver Container

VMDC 2.3 Bronze Container

VMDC 2.3 Copper Container

VMDC 2.3 Scaled Pod

VMDC 2.3 Scaled DC

Tenant Scale with VMDC 2.3

(as of NX-OS 6.1) ***
Tenancy Model

Scale per POD

Scale in DC
(4 POD)

All Expanded
Gold

125

500

All Gold

200

800

All Silver

300*

1200*

All Bronze

300

1200

All SMB

500

2000

Mixed *

500

2000

Mixed = 10 Ext.Gold, 20 Silver, 220 Bronze, 250 SMB per POD

* Needs multiple pairs of ASA and/or ACE appliances per POD
*** These numbers will increase 2-3x with NXOS 6.2

CLOS based model as a new L2 option for the Data Center

Data Center Perimeter

DC Core
SPINE
FP

SPINE
FP

SPINE
FP

SPINE
FP

FP
LEAF

FP
LEAF

FP
LEAF

L3 Boundary
L2

L3 Boundary
L2

DC Aggregation/Access

FP
LEAF

FP
LEAF

FP
LEAF

DC Access

Multi-Layer L3 Hierarchical Design

L2 CLOS Design

VMDC 2.x releases validated topology variants (i.e., collapsed core/aggregation, as the L2/L3

boundary)

VMDC 3.x releases validated with FabricPath based architecture for intra-POD or inter-POD VM

Mobility

If needing more Scale..

Scale Factor

Change in design

Benefit

More than VM per Pod

Use M2 linecards instead of F2 on Nexus 7004

Agg

128k MAC on M2, so can get to 50k VMs

More ICS and UCS blades in

Pod

Use N7009 instead of N7004

Can get more port-density for connecting

more servers, or for more throughput

More Pods in DC

Use ASR1013 or ASR9010 instead of ASR1006

Get higher port density and throughput to

connect more Pods.
With ASR9000 get more VRF and BGP scale to
support more Pods and Tenants

Higher Tenant Scale

Use MPLS in the DC.

Inter-AS option B or Intra-AS MPLS between
ASR and Nexus7000

Eliminates per-VRF BGP and mitigates control

plane scale limits. Can scale to 1000-2000
tenants per Pod (* with NxOS 6.2)

More Gold/Silver tenants

Use multiple ASA/ACE appliances through

N2000 FEX to N7000. Or, use 6500 DSN with
service modules

Support more FW/SLB contexts.

For even lower scale, cost,

footprint

Use Nexus 5500 or Nexus 6000 as Aggregation

layer

1 or 2 RU with 50-100 tenants per Pod.

Numbers depend on tenancy type and distribution. Will increase with NXOS 6.2

Not validated as CVD

VMDC Cloud Ready Infrastructure Kits

Pre-Defined Unified Data Center Networking Scalable Bill of Materials
Securely share common Cloud infrastructure
Based on VMDC Cisco Validated Designs
Secure separation between workloads
Shared network services
Pre-Built Bill of Materials available as CCW templates
Enhanced DC
Interconnect

Data Center

Core

Unified
Data Center
Networking

Aggregation

Services

Access

Integrated
Compute Stacks
VMDC 2.3
NAS

Compute

SAN

VMDC Cloud Ready Infrastructure Kit

Pre-defined and Pre-Validated BOMs Sized to Meet Customer Requirements
http://www.cisco.com/en/US/docs/solutions/Enterprise/Data_Center/VMDC/CRI/VMDC_CRI_Kits.html

Ideal for initial Cloud deployments

Starts small and provides incremental growth
Space Optimized
Pay as You Grow

Use BoM as Baseline

Pre-Built Bill of Materials available as CCW
templates
Tweak design, platforms, quantity etc based on
deployment requirements

CRI-SO-S

CRI-PO-S

CRI-PO-M

CRI-PO-L

Aggregation Switch

Nexus 5548UP

Nexus 7004

Nexus 7009

Nexus 7018

Network Services

ASA5555-X
ASA5555-X

ASA5555-X
ASA5585-X

DSN + ASA5555-X

DSN + ASA5585-X

Starting List Price

Does not include
service

$300,000

$800,000

$ 1,400,000

$ 1,700,000

VMDC 2.3

Does not include ICS

Based on Compute & Storage requirements,
specific Vblock, FlexPod or other ICS types need
to be included

CCW configurations are available from the

Sales Acceleration Center
sac-support@cisco.com

Scalability with Price Performance

VMDC 3.0 with FabricPath

New Infrastructure Designs
Simplified Network, Reducing Operating Expenses

Switch addresses are assigned automatically

A single control protocol

Easily expanded in a plug and play manner

Non-FabricPath switches can still be without STP

Reliability Based on Proven Technology

Cisco FabricPath is built on top of IS-IS, an industry

Loop prevention and mitigation is available in the data plane

Efficiency and High Performance

2.56 terabits per second (Tbps) of bandwidth between

switches
(16-way ECMP combined with 16-port 10-Gbps
PortChannels)

Lower Latency than Spanning Tree based solution

Cisco FabricPath enables massive scalability of the L2

domain

VMDC 3.0 with FabricPath

Design Options and Criteria
Design criteria included:

Available FabricPath modules:

M1/F1 mixed VDC

M1/F2 split VDC

VLAN scale: constrained by HSRP, GLBP

MAC scale

ARP learning rate

Conversational MAC address learning

Port Density

Forwarding Paths

Port-channel vs. single links

VPC, VPC+ options

QoS

Distance (intra-PoD)

VMDC 3.0 Tenant Container

VMDC 3.0
Palladium
Public Zone

LB

vFW

Private Zone 1

L3

L3
L2

FW Private Zone 32

FW

LB
vFW

L2

LB
vFW

Trend towards Virtualised Services

Insertion of services {load-balancing, firewalling, tenant routing} within the tenant

container fundamentally drives the logical design (both L2 and L3) within the data center
Services are typically L3

Industry transition underway from network-based services to virtualised services

VMDC 4.x focus
Address transition to virtual services
Address tenancy scalability constraints of current solution
Routing as a Service (RaaS) for Cloud providers
Highlight service chaining considerations/issues
Highlight new scalability considerations (virtual appliances in compute tier)

CSR1000V as Virtual Router (vCE) inside DC

Virtual CE

Alleviates need to extend L3 VPN natively into

L3 VPN

WAN Edge
(ASR9k)

the data centre

L3 VPN Edge
(NGN-PE)

E/W L3 via vCE

E/W L2 extension via L2 overlay (VXLAN)

PAYG solution virtual CE per tenant follows

Aggregation
/ core
(N7k)

same model for tenant routing as for other tenant

services, i.e.. RaaS
Could be multiple vCEs per tenant

Mirrors branch CE model, i.e. can support same

features and management models

Access
(61x0, N5k)

Allows for end-to-end services with enterprise sites (WaaS,

LISP, IPSEC, etc)

No cross-tenant dependencies, simplifies

L2 or L3 Fabric
Compute
& Storage

management and orchestration

Tenant L3
Edge (vCE)
Services

VMs

Cisco working through dynamic PE VRF provisioning models

Requires scalable DC WAN gateway and PE-

vCE segmentation technology

VMDC with Virtual Services - Physical Topology

Tenant Scale:
2000 per Pod or DC
* Depends on Design

VMDC 4.x: Virtual Private Cloud Container

Possible Components:
L3 VPN

CSR1000V XE 3.9 (IOS FW,

Routing, VPN, ZBF, AppNav
Controller, NBAR2)
Netscaler VPX 10.1
Nexus1000V 2.2
VXLAN on N1kV
VSG
ASA1000V
vWaaS 5.2 (vPath and AppNav
redirection)
vNAM 6.0
IPv6 Dual Stack (TBC)
Hyper-V (TBC)

Internet

ASA1000v
VPN

CSR1000v (vCE)

vWaaS
vWaaS

VPX

VPX

vNAM

VPX
VSG

SubZone W

SubZone Y

Public Zone (DMZ)

Protected FE

Front-end Zones

VXLANs used to extend segmentation scale within tenant containers

Mapping of VXLAN to VLAN occurs on N1kv

Also, mixed Physical + Virtual Containers

Zone 1

Zone 2

Back-end Zones

Nexus 1000v + VPATH

SubZone X

SubZone Z

Zone 3

Sample Container with CSR1000V, VSG, VPX

VMDC
Management &
Automation

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Connect

68

The Challenge:
How do I manage, automate, and monitor service performance?

VMDC offers an open

management framework
through a documented set of
component APIs
The VMDC open
management solution
provides flexibility
Cisco offers domain element
management and network
specific offerings such as:
Cisco Network Services
Manager (NSM)
Cisco Data Center Network
Manager
Cisco UCS Manager

The Solution

Extensible, Open
Management Framework
Pre-Integrated Automation
and Orchestration
2012 Cisco and/or its affiliates. All rights reserved.

WAN Edge/
DCI
Core

API

Aggregation/
Access
Services

API

MEC
vPC

Compute

vPC

API

vPC

API

4x10GE

Storage

MEC

Storage solutions vary by

vendor
An open framework
expedites VMDC integration
into existing management
solutions and the
development of new ones

4x10GE

API
API

Benefits
Simplifies and accelerates the USE of cloud services by providing capabilities
to rapidly offer users cloud services
This new management integrates with the existing IT environment

VMDC Cloud Management Solutions

High Scale & Multi-tenant Apps

Significant Complexity
Established Market Position
Complex Cloud Target/SP

Automation of IT processes
Integration of apps to the business process
Private Cloud/ Large Enterprises

Others like OpenStack

Leveraging partner company assets

CIAC, Prime, Cloupia

OpenStack/
Others
CISCO COMMON TECHNOLOGY

VNMC, UCSM, etc

OnePK, OneController (Future)

Automating the Service Deployments

BMCs CLM Workload Automation and Lifecycle Management

Provides an end-to-end

automated lifecycle
management

Completes a
Service Request

User
Request

Translate to
Business Offerings
VXI, HCS, IaaS

2nd Generation Cloud

Lifecycle Management
Platform

Service
Catalog

Design Services
and Options

Service Catalog
Service
Blueprint

Network
Blueprint

Integrated full-stack cloud

Storage
Map to
CMDB

Service Governor

services
Intelligent placement of

Service
Blueprint

Operational
Repository

Resource Management

provisioned services
Service-Catalog Platform for

Automation, Orchestration,
and Management
Day 2 management

performance, compliance,
security

Virtualized Multi-Service Data Center

Secure Network
Containers

Networks

Network Services

Integrated
Compute Stack

Storage

Servers

FlexPod, Vblock

BMC Cloud Lifecycle Management (CLM)

System Components

MoM
Cloud
Portals

Service
Catalog

Service
Governor

Cloud API

Compute

BSA

Network

BNA

Virtual Compute
VMware vSphere, Citrix
XenServer, Microsoft HyperV

VMDC 2.0/2.1/2.2/2.3

Compute
UCS B & C
Series/UCSM

Storage
NetApp/EMC

Network
Nexus/VSS/ASA/ACE/AS
R/CRS/VSG

BMC CLM Validated Network Containers (to-date)

VMDC 2.0 + CLM 2.1

CRS, N7k, C6k, FWSM, ACE20

VMDC 2.1 + CLM 2.1

C6k, N7k, FWSM, ACE20

VMDC 2.2 + CLM 3.1

(With VSG & Expanded Gold
container)
A9k, N7k, C6k, ASA, ACE30

VMDC 2.2 + CLM 3.0

4-Zone Flexible Container
(No VSG)
A9k, N7k, C6k, ASA, ACE30

Flexible Container

Cloud Service Assurance

Key Objectives & Functions
Automate
service
enablement &
lifecycle

Auto-provisioning of service
assurance system through autodiscovery and integration with
orchestration

Consolidate
monitoring

Single pane of glass for fault and

performance monitoring of
compute, storage, network and OS

Reduce MTTR
(Reduces OPEX,
Increases
Availability)

Reduce cost of
OSS/BSS
integration

Root-cause analysis and tenant

based service impact analysis

Abstracted and normalized

interfaces to simplify integration and
maintenance of northbound systems

VMDC Cloud Assurance

-With Zenoss CSA

Delivers unified fault & performance management of

compute, storage, network & applications infrastructure
Validated design for cloud operations automation
Single view for compute, virtualization, DC network and storage fault &
performance
Complete VMDC/Flexpod/Vblock component coverage
Multi-tenant service availability & performance reporting

Providing service visibility to tenants through tenant portals

Orchestration integration or stand alone operations
Advanced Services deployment services
Zenoss Cloud Service Assurance Solutions plus on Cisco

GPL in UCS plus software category

Zenoss CSA 4.2 validated for VMDC 2.3

Recently Updated
for VMDC 2.2, 2.3
and 3.0 Systems

Cloud Service Assurance

Simplified Service Impact & Root Cause Analysis
Tenant Service

Application

Service
Impact

VM
Data Store
Host
UCS Service
Profile

UCS Blade

Root
Cause

1
Ranked probable root-cause events

2
Service Impact Events

UCS Chassis

VMDC Summary

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Connect

77

Summary
VMDC is the Cisco validated reference architecture for Cloud infrastructure - Public/Private/Hybrid
Cisco CVDs for cloud infrastructure, orchestration and assurance enable quicker adoption and

deployment of complex technologies for end-end solutions

Multi-tenancy, service differentiation, tiered security services, virtualization and automation are key

for cloud deployments

Defined System for today, evolving for tomorrow
Reduced complexity as system is characterized via validation efforts with supporting collateral
Facilitates Modularity, Scalability, Pay-as-you-Grow models
Speed, Risk mitigation for Deployments

Multiple VMDC phases and tenancy models evolving with new technologies/platforms and customer needs

Validated Automation & Management

Resource pools encompassing network, compute and storage

Out-of-Box Orchestration & Assurance support for different VMDC designs and tenancy models

Validated Onboarding of services

Hosted Collaboration (HCS), Hosted Virtual Desktop (HVD), DRasS, Hybrid

VMDC based Cloud Ready Infrastructure kit provides pre-packaged BOMs in CCW

Resources

www.cisco.com/go/vmdc

ask-vmdc-external@cisco.com
http://www.cisco.com/go/dci

http://www.cisco.com/go/cloudverse

http://www.cisco.com/en/US/products/ps11104/serv_home.html

http://www.cisco.com/en/US/solutions/ns341/eantc_cloud.html
http://www.cisco.com/en/US/partner/docs/solutions/Enterprise/Data_Center/VMDC/2.3/implementation_guide/VMDC_2.3_IG.html

http://www.cisco.com/en/US/partner/solutions/ns340/ns414/ns742/cloud_orchestration_bmc_clm.html
http://www.cisco.com/en/US/partner/docs/solutions/Enterprise/Data_Center/VMDC/CLSA/Partner_DIG/CLSA_VMDC_DIG.html
http://www.cisco.com/en/US/docs/solutions/Enterprise/Data_Center/VMDC/CRI/VMDC_CRI_Kits.html

http://www.cisco.com/go/vblock

http://www.cisco.com/go/flexpod
http://www.cisco.com/go/bmc

Complete Your Paper

Session Evaluation
Give us your feedback and you could win
1 of 2 fabulous prizes in a random draw.
Complete and return your paper
evaluation form to the room attendant
as you leave this session.
Winners will be announced today.
You must be present to win!

..visit them at BOOTH# 100

Thank you.

2012 Cisco and/or its affiliates. All rights reserved.

Cisco Connect

81