Professional Documents
Culture Documents
LJ Wobker
Technical Marketing Engineer
High End Routing & Optical Group
Level Setting
Apologies for the accent – throw things when I talk too fast.
Impossible to give a truly thorough talk on all these topics ;-)
Most of the output/show command slides are for reference
Infinite time to answer questions...
but not in this room within this specific session...
Meet the engineer and/or World of Solutions is your friend
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
Cisco ASR9000 – Next-Gen Edge Routing Platform
Key Design Goals & System Benefits
ASR9922
Architectural Design for Longevity
ASR9001
Product Portfolio with significant HW and SW
commonality
ASR9000v
Highly integrated network processors
Cisco IOS XR based ASR9010
– Truly modular, full distributed OS
– Enhanced for the Edge (L2 and L3)
ASR9006
nV (Network Virtualisation) for Operational
Simplicity
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 4
Agenda
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
ASR 9000 Hardware Overview
ASR 9010 and ASR 9006 Chassis (circa 2008)
Front-to-back
Integrated cable airflow
management
with cover
System fan trays Side-to-back
airflow
RSP (0-1)
Line Card (switch fabric)
(0-3)
Line Card
RSP (0-1) (0-3, 4-7)
(switch fabric)
System fan trays
Air draw
cable
management 2 power shelves
V1 power shelf: 3 Modular V1 PS 6 or 8 PS
V2 power shelf: 4 Modular V2 PS
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
ASR 9001 Compact Chassis
Shipping Since XR 4.2.1, May 2012
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
ASR 9001-S Compact Chassis
Shipping Since XR4.3.1, May 2013
Supported MPAs:
Feature Description
I/O Slots 2 I/O slots
Rack size 6RU
Fan Side to Side Airflow
1 Fan Tray, FRU
RSPs RSP440, 1+1
Power 1 Power Shelf, 4 Power Modules
2.1 KW DC / 3.0 KW AC supplies
Fabric Phase 1: 770G/slot
Bandwidth Future capability: 1.7 Tb per Slot
SW XR 5.1.0 – August 2013
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
ASR 9912 Chassis
Shipping Since XR4.3.2 & 5.1.0, Sep 2013
Features Description
Fan 2 Fan Trays
Front to back airflow
I/O Slots 10 I/O slots
Rack Size 30 RU
RP 1+1 RP redundancy
Fabric 6+1 fabric redundancy
Power 3 Power Shelves, 12 Power Modules
2.1 KW DC / 3.0 KW AC supplies
N+N AC supply redundancy
N:1 DC supply redundancy
Bandwidth Phase 1: 550Gb per Slot
Future: 2+Tb per Slot
SW XR 4.3.2 & 5.1.0
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
ASR 9922 Chassis
8+8 ACs or Front-to-back
11+1 DCs airflow
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
System Components
Power and Cooling
DC Supplies
A
B 1.5 kW Single power zone
A
B 2.1 kW All power supplies run in active mode
Power draw shared evenly
Power Supply AC Supplies 50 Amp DC Input or 16 Amp AC
for Easy CO Install
A 3 kW
B 3 kW
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
Control Processors (RP and RSP)
RSP used in ASR9006/ASR9010, RP used in ASR9922
RSP RSP440 9922-RP
Cores
Alarms
BITS/DTI/J.211 1588
Console & Aux Status LEDs
CF card HDD
or USB
Mgt Eth CPU
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
ASR 9000 Ethernet Line Card Overview
First-generation LC
(Trident)
-TR, -
SE A9K-MOD160
A9K-MOD80
Second-gen LC A9K-24x10GE A9K-2x100GE
MPAs
(Typhoon) 20x1GE
2x10GE
4x10GE
8x10GE
1x40GE
2x40GE
A9K-36x10GE
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
ASR 9000 ISM (Integrated Service Module)
ASR 9000 ISM
CDS Streaming: Feature
Capabilities
TV and internet streaming Ultra-Dense VoD, TV,
Error repair Applications Internet Streaming, Error Repair,
CGv6
30-40 Gbps streaming capacity
Bandwidth
CGN (carrier grade NAT): ~3 Gbps cache fill rate
Compatibility Works with all CDS appliances
NAT44, DS-Lite
Concurrent
Up to 8,000 SD equivalent
NAT64 Streams
QSFP
All 10G and 40G Ports do
support G.709/OTN/FEC CFP 1) Using Optical Shelf
(ONS15454 M2/M6)
For latest Transceiver Support Information
http://www.cisco.com/en/US/prod/collateral/routers/ps9853/data_sheet_c78-624747.html
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
Fabric Architecture
Cisco ASR9000 System Architecture
Line Card
RSP
CPU
CPU BITS/DTI
FIA
FIC
Switch Fabric
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 22
ASR9000 Switch Fabric Overview
Stage 1 Stage 2 Stage 3 Fabric load balancing:
Unicast is per-packet
Multicast is per-flow
Egress Linecard
Ingress Linecard 8x55Gbp
fabric
s
fabric fabric
Arbiter FIA
FIA FIA
FIA RSP0 FIA
FIA
2nd gen Line Card
2nd gen Line Card
fabric
8x55Gbp
s
Fabric bandwidth:
Arbiter
8x55Gbps =440Gbps/slot with dual RSP
RSP1 4x55Gbps =220Gbps/slot with single RSP
2nd gen Fabric (RSP440)
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
1st/2nd Generation Switch Fabric Compatibility
Ingress Linecard 8x23G
bi-directional
= 184Gbps 2nd Generation
FIA0 Fabric (RSP440)
8x55G
bi-directional
Egress Linecard
FIA1 = 440Gbps
fabric
fabric
Dual-FIA
8xNPs 1st gen Arbiter FIA
Linecard FIA
RSP0 FIA
FIA 2nd gen Line Card
fabric
Single-FIA
4xNPs 1st gen 4x23G
Linecard bi-directional
Arbiter
= 92Gbps
RSP1
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
ASR 9912/9922 Fabric Architecture : 5-plane System
5x2x55G
bi-
directional
= 550Gbps
fabric
FIA
FIA FIA
FIA FIA
fabr FIA
ic
2nd gen Line Card 2nd gen Line Card
5x2x55G
bi-
directional Fabric cards
= 550Gbps
770Gbps/LC or
660Gbps/LC with fabric
redundancy 7x2x55G
bi-
directional
= 770Gbps
fabric
FIA
FIA FIA
FIA FIA
fabric FIA
2nd gen Line Card 2nd gen Line Card
7x2x55G
bi-
directional
= 770Gbps Fabric cards
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
Linecard Architecture
Generic Linecard Architecture – Components
Pluggable physical interfaces
• speeds: GE, 10GE, 40GE, 100GE
PHY CPU CPU
• Distributed Control planes
• form factors: SFP, SFP+, XFP, QSFP, CFP • SW switched packets
• media/reach: T, SR, LR, ZR, LR4, SR10 • Inline Netflow
• colours: gray, CWDM, DWDM, Tunable • Program HW forwarding tables
Network Processor
NP
• forwarding and feature engine for the LC
• scales bandwidth via multiple NPs
– up to 8 NPs/LC for performance vs. density options
• highly integrated silicon as opposed to multiple discrete components
– shorter connections, faster communication channels
– higher performance, density with lower power draw
3x10GE
NP
FIA
SFP +
3x10GE
SFP +
NP
3x10GE
NP
FIA Switch
Fabric
SFP +
RSP0
3x10GE
SFP +
NP Switch
3x10GE FIA Fabric
SFP +
NP
3x10GE Switch
SFP +
NP Fabric
3x10GE FIA RSP1
SFP +
NP
6x10GE
PHY (SFP+) NP FIA
6x10GE
PHY (SFP+) NP FIA Switch
Fabric
6x10GE RSP0
PHY (SFP+) NP FIA
6x10GE Switch
PHY (SFP+) NP FIA Fabric
Switch
6x10GE
PHY (SFP+) NP FIA Fabric
RSP1
6x10GE
PHY (SFP+) NP FIA
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
2port 100GE Linecard Architecture
CPU
100G
Ingress NP FIA
100GE
MAC/PHY
100G
Egress NP FIA Switch
Fabric
RSP0
Ingress Switch
100G
NP FIA Fabric
100GE
Switch
MAC/PHY Fabric
Egress
100G
NP FIA RSP1
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
Module Cards – MOD80
CPU
NP FIA
MPA Bay 0
Switch
Fabric
RSP0
Switch
NP FIA Fabric
MPA Bay 1
Switch
Fabric
RSP1
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
Module Cards – MOD160
CPU
NP FIA
MPA Bay 0
NP FIA Switch
Fabric
RSP0
Switch
NP FIA Fabric
MPA Bay 1
Switch
Fabric
NP FIA RSP1
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
ASR9001 Architecture
Same Hardware Components as the Modular Systems
MPAs
2,4x10GE
20xGE
1x40GE
NP FIA
SFP+ 10GE
SFP+ 10GE
On-board Internal LC RP
4x10 SFP+ ports EOBC
SFP+ 10GE CPU CPU
SFP+ 10GE
MPAs NP Switch
Fabric
2,4x10GE
20xGE
FIA
1x40GE
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
Cisco IOS-XR Overview
Industry Hardened IOS XR
Modular, Fully Distributed
Device File
Driver Micro-Kernel
System
10+ years in the most Process Mgmt
demanding internet Memory Mgmt
…
Scheduler
deployments TCP/IP HW Abstraction
microkernel
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
Cisco IOS-XR – High Level Benefits
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
Cisco IOS-XR Software Modularity
Line card
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
Distributed In-Memory Database
improves performance
and Scale
Reliable Multicast and Unicast IPC
Fabric ASIC
Switch
SFP +
3x10G FIA HSRP/VRRP, etc
E NP
SFP +
3x10G
E
SFP +
NP LC CPU: ARP, ICMP, BFD, Netflow,
3x10G FIA
E NP OAM, etc
SFP +
3x10G
E NP
SFP +
3x10G FIA
E NP
SFP +
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 41
Local Packet Transport Services (LPTS)
on RP
Application1
Forwarding
Information
LPTS on RP
Internal FIB (IFIB)
Base (FIB)
Bad packets
Local Stack
on LC
Non-Stop Forwarding
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 44
Control Plane and Packet Forwarding
Infrastructure
Layer 3 Control Plane Overview
BGP OSPF
LDP RSVP-TE
Static
ISIS EIGRP RP CPU
LSD RIB RP
Selective VRF
Over internal download
EOBC
ARP
LC CPU
FIB Adjacency
SW FIB
AIB LC NPU
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
Hierarchical Layer 3 Forwarding Data Structure
Enabling Prefix Independent Convergence for TE FRR, BGP, LAG
0 1 2 3 . . . … n
VRF table PIC: Prefix independent convergence
LAG: Link aggregation group
LDI: Load information
Search Tree
OutI/F
adj
LAG OutI/F
Recursive
Leaf Table (FIB NR Prefix adj
Prefix Leaf
table) Table Leaf Table
Protected
LAG OutI/F
TE adj
Recursive NR LDI 0
LDI/ADJ table LDI NR LDI 1
… Backup
OutI/F
NR LDI N TE adj
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
IOS-XR Two-Stage Forwarding Overview
Scalable and Predictable
3x10GE
SFP +
NP
3x10GE
FIA
NP
SFP + 1 Ingress 100G
FIA
NP
3x10GE 100GE
SFP +
NP
MAC/PHY
Switch Fabric
3x10GE FIA Switch
Egress
NP Fabric
Switch Fabric
100G
SFP + FIA NP
2
ASIC
3x10GE
ASIC
SFP +
NP
3x10GE
FIA FIA
Ingress
100G
NP NP
SFP +
100GE
MAC/PH
3x10GE
NP Y
SFP +
Egress
FIA Switch FIA NP 100G
3x10GE 1 Fabric
SFP +
Typhoon
2
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
MAC Learning and Sync
Hardware based MAC learning:
1 NP learn MAC address in hardware (around
~4Mpps/NP
4M pps)
RP
2 NP flood MAC notification (data plane) Punt
CPU
message to all other NPs in the system to sync FPGA FIA
up the MAC address system-wide. MAC
notification and MAC sync are all done in
hardware
Switch Fabric Switch Fabric
CPU LC1
LC2
CPU
3x10GE
SFP + 1NP 2 3x10GE
3x10GE FIA NP
NP SFP +
SFP + FIA
3x10GE
3x10GE
SFP +
NP 2
Fabric ASIC
Switch
SFP + NP
FIA 3x10GE
3x10GE NP
Fabric ASIC
Switch
NP SFP +
SFP + FIA
3x10GE
3x10GE
SFP +
NP
SFP + NP
FIA 3x10GE
3x10GE
SFP +
NP
SFP + NP FIA
3x10GE
3x10GE
SFP +
NP
SFP + NP
3x10GE FIA 3x10GE
NP
NP SFP +
SFP + FIA
3x10GE
SFP +
NP
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
Multicast Replication Overview
1 Fabric to LC Replication 3 FIA to NP Replication
Efficient: replicate only where required
2 LC fabric to FIA Replication 4 NP to egress port Replication
Simple: clean architecture with common codepaths
Consistent: predictable performance
MFIB Line rate for fully loaded chassis even w/ failures
3x10GE Typhoo MGID
SFP + 4n 3
3x10GE
FIA
SFP + NP
4
FIA Ingress
NP
3x10GE FGID/ 100GE
NP
ASIC
Switch Fabric
SFP + FPOE
MGID/ MAC/PHY
MGID/
FIA 1
Switch FPOE MFIB
ASIC
Switch Fabric
3x10GE FPOE MGID Egress
NP Fabric FIA
SFP +
2 2 3 4
NP
3x10GE
SFP + NP
FIA FIA
Ingress
3x10GE
NP NP
SFP +
100GE
3x10GE MAC/PHY
SFP + NP 3
Egress
FIA Switch FIA NP
3x10GE 4 Fabric
SFP + NP
CPU CPU 4
NP PHY
3
2 NP PHY
FIA FIA
PHY NP Switch
Fabric
PHY NP1
1 2 3 4
Ingress Port Egress Port
4 VOQ per each virtual port in 4 Egress Queues per each
QoS QoS
the entire system virtual port, aggregated
Up to 4K VOQs per FIA rate per NP
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
QOS Architecture
52
System QoS Refresh
End-to-End priority (P1,P2, Best-effort) propagation
Guarantee bandwidth, low latency for high priority traffic One Queue set (4 queues)
at any congestion point per each NP on the LC
3 strict priority level across all internal HW components
CPU CPU 4
PHY NP0 NP0 PHY
3
PHY NP1 2 NP1 PHY
FIA FIA
PHY NP2 Switch NP2 PHY
Fabric
PHY NP31 NP3 PHY
2 3
1 Virtual Egress FIA 4
Ingress (sub-)interface Output Queues Egress (sub-)interface
QoS Queues Queues QoS Queues
Implicit Configuration
Two strict high priority + Configure with
Configure with Normal priority Egress MQC
Ingress MQC 4-layer hierarchy 4-layer hierarchy
Two strict high priority + Normal priority Two strict high priority +
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public Normal priority 53
Arbitration & Fabric QoS
Arbitration is being performed by a central high speed arbitration ASIC on
the RSP
At any time a single arbiter is responsible for arbitration (active/active
“APS like” protection)
The Arbitration algorithm is QOS aware
– ensurs that P1 classes have preference over P2 classes, both of which have
preference over non-priority classes
Arbitration is performed relative to a given the egress VQI
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 54
System QoS Refresh (3) –
Backpressure and VoQ Mechanism
Egress NP congestion backpressure to ingress FIA
Packet is en-queued in the dedicated VoQ
No impact of the packet going to different egress NP
One VoQ set (4 queues) No head-of-line-block issue
per each NP in the system
Backpressure: egress NP egress FIA
fabric Arbiter ingress FIA VoQ
Ingress side of LC1 Egress side of LC2
Packet going to different egress NP put into different VoQ set Congestion on
BRKSPG-2904 one NP©won’t block
2014 Cisco the
and/or its packet
affiliates. going
All rights to different NP Cisco Public
reserved. 55
MQC to System QOS Mapping
ASR 9000 supports traffic differentiation at all relevant points within the system
• Priority awareness at all interfaces: P1 > P2 > Low/Normal
• Classification into these priorities is based on input MQC classification on the
ingress linecard into P1, P2, Other
• a packet classified into a P1 class on ingress is mapped to PQ1 system queue
• a packet classified into a P2 class on ingress is mapped to PQ2 system queue
• a packet classified into a non-PQ1/2 class on ingress will get mapped to LP
queue along the system qos path
• Note: The marking is implicit once you assign a packet into a given queue on
ingress; its sets the fabric header priority bits onto the packet.
• no specific “set” action is required – the priority level is taken from the MQC
class configuration
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 56
ASR 9000 QOS Implicit Trust
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 57
4 Layer Hierarchy Overview
L1 L2 L3 L4 Note: We count hierarchies as
Port Subscriber Subscriber Class follows:
Level group Level Level Level 4L hierarchy = 3 Level nested p-
map
PQ1 VoIP – Bearer + Control
3L hierarchy = 2 level nested p-
Customer1 - egress
Business Critical
1
EVC
BW
BW Internet – Best Effort
map
2
EVC
PQ2 Telepresence
Hierarchy levels used are
BW Internet – Best Effort
determined by how many nested
levels a policy-map is configured
VoIP – Bearer + Control
for and applied to a given
PQ1
Customer2 - egress
Business Critical subinterface
EVC3
BW
BW Internet – Best Effort
Max 8 classes (L4) per
subscriber level (L3) are
supported
PQ1 VoIP – Bearer + Control
4
EVC
PQ2 Telepresence
BW Internet – Best Effort
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 58
3 Layer Hierarchy Example
policy parent
•Objective: Apply a SLA to an EFP with parent class-default
shape/bandwidth/BRR and child class based queuing
shape average 100 mbps
bandwidth 50 mbps
bandwidth-remaining-ratio 50
service-policy child
policy child
COS5 VoIP
PQ
BW
COS1 Internet class-voip {classify on cos=5}
priority level 1
police 20 mbps
class-internet {classify on cos=1}
EFP int GigE 0/1/2/3.4 l2transport bandwidth 10
VLAN service-policy output parent
101 int GigE 0/1/2/3.5 l2transport
service-policy output parent
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 59
ASR9K QoS Classification Criteria
Very flexible L2/L3 field classification on L2 interfaces
Inner/outer cos
Inner/Outer vlan *
DEI*
Outer EXP
Dscp/Tos
TTL, TCP flags, source/destination L4 ports
Protocol
Source/Destination IPv4
Source/Destination MAC address*
Discard-class
Qos-group
match all/match any
Note:
– Not all fields are supported on L3 interfaces*
– Some fields don’t make sense on ingress (e.g. dicard-class, qos-group)
– MPLS classification is based on EXP only
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 60
ASR9K QoS - Classification Formats
Per Policy-map a given classification format is chosen by SW, i.e a given
policy-map can only classify based on a single format
Format 0 Format 1 Format 2 Format 3
Fields supported -IPV4 source address -Outer VLAN/COS/DEI -Outer VLAN/COS/DEI -Outer VLAN/COS/DEI
(Specific/Range)[1] -Inner VLAN/COS -Inner VLAN/COS -Inner VLAN/COS
-IPV4 Destination address -IPV4 Source address -IPV4 Destination address -MAC Destination address
(Specific/Range) (Specific/Range) (Specific/Range) -MAC source address
-IPV4 protocol -IP DSCP / TOS / Precedence -IP DSCP / TOS / Precedence -QOS-group (output policy
-IP DSCP / TOS / Precedence -QOS-group (output policy only) -QOS-group (output policy only) only)
-IPV4 TTL -Discard-class (output policy only) -Discard-class (output policy only) -Discard-class (output
-IPV4 Source port (Specific/Range) policy only)
-IPV4 Destination port
(Specific/Range)
-TCP Flags
-QOS-group (output policy only)
-Discard-class (output-policy only)
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 61
ASR9K QoS - Packet Marking Details
“settable” packet fields:
dscp/precedence
EXP imposition
EXP topmost
cos inner/outer
qos-group
discard-class
Remember that mpls encapped packets can’t match on L3 criteria (same for ACL)
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 62
ASR9K QoS - Policing Details
RFC 2698 supported (2r3c) and 1r2c • transmit
Ingress & egress policing supported • drop
• set (implicitly behaves like set and
General Rule: Policing required on transmit)
priority queues. • each colour can have two set actions:
– Priority level 2 classes can also accept shaping
instead of policing.
Granularity of 8Kbps supported
(typhoon, 64k on trident)
2-level nested policy maps supported Policy-map parent
– Note: policers at parent and child work Class class-default
independently Police rate 10 Mbps peak-rate 20 mbps
conform-action set dscp af12
64k policers per NP (shared for conform-action set cos 2
ingress/egress) on extended linecards exceed-action set dscp af13
Policer actions supported: exceed-action set cos 3
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 63
Normal Hierarchical Policer
policy-map child
class class1
police rate 20 mbps peak-rate 50 mbps
class class2
police rate 30 mbps peak-rate 60 mbps
At parent level, if it’s over the CIR,
policy-map parent packet will be dropped randomly.
class class-default There is no awareness which packet
police rate 60 mbps to be dropped
service-policy child
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 64
Conform Aware Policer
policy-map child
class class1
police rate 20 mbps peak-rate 50 mbps
class class2
police rate 30 mbps peak-rate 60 mbps
Parent CIR must > aggregated child CIR
policy-map parent Parent police only support 1R2C, child police
class class-default support all: 1R2C, 2R3C, or 1R3C
service-policy child
If drop happen at parent level, it will drop
police rate 60 mbps child out-of-profile packet, but guarantee the
child-conform-aware child in-profile packet
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 65
ASR 9000 QoS - Queue Scheduling
“shape” for a shaped PIR for a graceful enforcement of a maximum bandwidth“
• shaping at all configurable levels
• Min. granularity: 64kbps (L3, L4, 256kbps for L2)
priority levels: priority level 1, priority 2, minBw/CIR and Bw remaining
“bandwidth” (minBw) for a CIR guarantee relative to the parent hierarchy level
Min. RATE: 64kbps (8k granularity)
bandwidth remaining ratio/percent” for the redistribution of excess bandwidth
that is available after PQ classes have been scheduled
configurable ratio values 1-1020
Two parameter scheduler support at class level and subscriber group level (L4,
L2):
– Shape & BwR (ratio / percent)
– Shape & MinBw (absolute / percent)
– Not supported: BwR & MinBw on the same class
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 66
Show/debug QOS Commands (Reference Only)
show running-config
show policy-map interface <interface> [iNPt | output] Policy-map statistics on a particular non-bundle interface
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 67
Show/debug QoS Commands (Reference Only)
show qoshal fcu <limits|status|profile> Displays all Traffic Manager (TM) Flow control related info
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 68
What Consumes a Queue
Class name
Bandwidth, Priority and Shaping will consume a queue
On ingress, priority setting will not consume a queue
RP/0/RSP0/CPU0:A9K-BNG#show qos int g 0/0/0/0 out | i "QueueID|Level|Class"
Queuing
level
Thu Mar 28 13:48:56.683 EDT
QueueID
Level: 0 Policy: SHAPE Class: class-default And priority
class
QueueID: N/A
Bandwidth: 0 kbps, BW sum for Level 0: 0 kbps, Excess Ratio: 1
Level: 1 Policy: child Class: class1
Child class
Parent Policy: SHAPE Class: class-default belonging to
QueueID: 136 (Priority 1) parent class
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 69
What is Programmed in HW?
COMMAND: show qos interface gigE 0/0/0/0 out
----------------------------------------------------------------------
Level: 0 Policy: xtp Class: class-default
QueueID: N/A
Shape CIR : NONE policy-map xtp
Shape PIR Profile : 0/4(S) Scale: 195 PIR: 199680 kbps PBS: 2496000 bytes class class-default
WFQ Profile: 0/9 Committed Weight: 10 Excess Weight: 10 service-policy xt
Bandwidth: 0 kbps, BW sum for Level 0: 0 kbps, Excess Ratio: 1 shape average 200 mbps
---------------------------------------------------------------------- !
Rate is rounded to the nearest 8k or 64k value end-policy-map
Shape sets PIR
PBS is default rate of 100msec of configured shape rate
BW is zero or 64k, only applicable in oversubscription at sum of parent levels
Note that all hardware parameters (WRED, queue, burst sizes, etc exist whether configured or not!
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 70
Egress Line
ASR9K Default QOS card
IPP=5 DSCP=44
Ingress Line
card 802.1p = 1 *
IPP=5 DSCP=44
802.1p = 1*
IPP=5 DSCP=44
L2 (VPWS,
802.1p = 1 VPLS,
internal
IPP=5 DSCP=44 802.1p= 1*
Bridging) cos = 1 EXP= 1*
IPP=5 DSCP=44
EXP= 1*
Carried in IPP=5 DSCP=44
internal
buffer
header
IPP=5 DSCP=44
Note: VPWS will be treated like a L2 operation on ingress - Applies for all tags/labels in the stack that get imposed.
Not for VLAN translation. Bridging on egress without adding an vlan header is an hypothetical case – in case we have a need.
IPP = IP Precedence, showing IPP & DSCP seperately since policymap can treat precedence and dscp separately as required.
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 71
ASR9K Default QOS
IPP=5 DSCP=44
L2 (VPWS,
Untagged VPLS,
internal 802.1p = 0 *
IPP=5 DSCP=44 IPP=5 DSCP=44
Bridging) cos = 0
802.1p = 0*
IPP=5 DSCP=44
802.1p= 0*
EXP= 0*
IPP=5 DSCP=44
EXP= 0*
IPP=5 DSCP=44
IPP=5 DSCP=44
Note: Trust cos in case of bridged interfaces in ingress. For untagged packets use cos = 0.
* - Applies for all tags/labels in the stack that get imposed.
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 72
ASR9K Default QOS
IPP=5 DSCP=44
Untagged
L2 (VPWS,
EXP=3 VPLS,
internal 802.1p = 0 *
IPP=5 DSCP=44 IPP=5 DSCP=44
Bridging) cos = 0
802.1p = 0*
IPP=5 DSCP=44
802.1p= 0*
EXP= 0*
IPP=5 DSCP=44
EXP= 0*
IPP=5 DSCP=44
IPP=5 DSCP=44
Note: Trust cos in case of bridged interfaces in ingress. For untagged packets use cos = 0.
•- Applies for all tags/labels in the stack that get imposed.
•- Explicit NULL EXP is treated the same as
BRKSPG-2904 anCisco
© 2014 topmost EXP of
and/or its affiliates. nonreserved.
All rights NULL labels. Cisco Public 73
ASR9K Default QOS
IPP=5 DSCP=44
Routed
Interface
internal 802.1p = 5 *
IPP=5 DSCP=44 IPP=5 DSCP=44
(IP/MPLS) cos = 5
802.1p = 5 *
IPP=5 DSCP=44
802.1p= 5 *
EXP= 5 *
IPP=5 DSCP=44
EXP= 5*
IPP=5 DSCP=44
IPP=5 DSCP=44
Note: Trust dscp in case of routed interfaces in ingress. For Non IP packets use cos = 0
* - Applies for all tags/labels in the stack that get imposed.
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 74
ASR9K Default QOS
IPP=5 DSCP=44
Routed
802.1p = 1 Interface
internal 802.1p = 5 *
IPP=5 DSCP=44 IPP=5 DSCP=44
(IP/MPLS) cos = 5
802.1p = 5 *
IPP=5 DSCP=44
802.1p= 5 *
EXP= 5 *
IPP=5 DSCP=44
EXP= 5*
IPP=5 DSCP=44
IPP=5 DSCP=44
Note: Trust dscp in case of routed interfaces in ingress. For Non IP packets use internal dscp= 0
* - Applies for all tags/labels in the stack that get imposed.
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 75
ASR9K Default QOS
IPP=5 DSCP=44
802.1p = 1
Routed
EXP = 3 Interface
internal 802.1p = 3 *
IPP=5 DSCP=44 IPP=5 DSCP=44
(IP/MPLS) cos = 3
802.1p = 3 *
IPP=5 DSCP=44
802.1p= 3 *
EXP= 3 *
IPP=5 DSCP=44
EXP= 3 *
IPP=5 DSCP=44
IPP=5 DSCP=44
Note: Trust EXP/dscp in case of routed interfaces in ingress. For Non IP packets use internal dscp= 0. Do not
overwrite DSCP fields exposed during disposition – to support pipe mode by default.
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
* - Applies for all tags/labels in the stack that get imposed. 76
ASR9000 nV – Network Virtualisation
ASR 9000 nV Technology Overview
SP Services/ Third-Party
Content Services/
Content
Before: nV Technology After: nV Technology
Cisco
Prime IP NGN
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
nV Satellite
ASR 9000 nV Satellite Overview
Plug and Play, Zero Touch Satellite Access Device
Satellite Protocol
Satellite ASR9K
access local ports
ports nv fabric links
Satellite
Host
One ASR 9000 nV System
Satellite and ASR 9000 Host run satellite protocol for auto-discovery, provisioning and management
Satellite and Host may be in different locations. There is no distance limitation between satellite and Host
The satellite<->host connection is called “nv fabric link”, which may be L1 or L2.
Satellite access port have feature parity with ASR9K local ports
it works/feels just as local port
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
Satellite Hardware – ASR 9000v Overview
Power Feeds
Field Replaceable Fan Tray
• Redundant -48V DC power feeds
• Redundant Fans
• Single 110-240VAC power feed
• ToD/PSS Output
1 RU ANSI & ETSI
• Max Power 210W
Compliant • Bits Out
4x GE 4x GE
2x DC Feeds (-24 or -48 1) (SFP)
BITS (RJ45)
VDC)
Console
2)
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
Satellite Hardware – ASR903 Overview
Route Switch Processor
• Currently only 1x RSP
supported
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 83
Satellite – Host Control Plane
Satellite Discovery and Control Protocol
CPU MAC-DA MAC-SA Control VID Payload/FCS CPU
Control Phase
– Used for Inter-Process Communication between Host and Satellite
– Cisco proprietary protocol over TCP socket, it could get standardised in the future
– Get/Set style messages to provision the satellites and also to retrieve notifications from the
satellite
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
Satellite – Host Data Plane Encapsulation
MAC-DA MAC-SA VLANs (OPT) Payload MAC-DA MAC-SA VLANs (OPT) Payload
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
Initial Satellite Configuration
Satellite Protocol
Satellite ASR9K
access local ports
ports
Satellite nv fabric links
101 Host
One ASR 9000 nV System
nv
satellite 101 define satellite
type asr9000v
ipv4 address 10.0.0.101
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
Satellite Port Configuration
Comparison to Local Port Configuration
Satellite Protocol
Satellite ASR9K
access local ports
ports
Satellite nv fabric links
101 Host
Satellite access port One ASR 9000 nV System Local port
configuration examples configuration examples
interface GigabitEthernet 101/0/0/1 interface GigabitEthernet 0/0/0/1
ipv4 address 1.2.2.2 255.255.255.0 ipv4 address 2.2.2.2 255.255.255.0
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
Satellite Deployment Models
ASR9000v Example
Access ports are mapped to a single Fabric links are forming a Link-Bundle
Fabric Link
Access port traffic is “hashed” across
Fabric Link failure does bring Access Bundle Members
Port down
Fabric link failure keeps all Access
Ports up, re-hashing of Traffic
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 88
Satellite Monitoring and Troubleshooting
Normal operation, like show CLIs are done on the Host directly, for example
– Satellite inventory reporting, environmental monitoring
– Interface counts, stats
– SNMP MIB
– NMS support, including ACT, ANA/ PRIME
Low level debug could still be done directly on the satellite device
– User can telnet into satellite via out-of-band management console, or in-band from
Host, and run regular show/debug CLIs
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
Satellite Software Management
Everything Controlled from the asr9k Host
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 90
Satellite Plug and Play
Configure, Install and Ready-to-Go
Initial satellite
configuration (on
ASR9K Host
Rack & Plug Go
Fabric
Linecard Links
chassis Leverage existing IOS-XR
Inter-
CRS multi-chassis SW Ctrl Chassis
Ctrl
Links
infrastructure Links Links
Linecard Fabric Simplified/Enhanced for
chassis chassis ASR 9000 nV Edge
CRS Multi-Chassis ASR 9000 nV Edge
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
nV Edge Architecture Details
Control Plane EOBC Extension RSP440 nV EOBC ports fo
control plane connection
0 1
Active Secondary Standby Secondary Internal
RSP RSP RSP RSP
EOBC
LC LC LC LC LC LC LC LC
One Virtual ASR 9000 nV System Inter-chassis data link (L1 connection) Regular 10G data ports
10G bundle (up to 16 ports)
Control plane connection: Active RSP and standby RSP are on the different chassis, they
communicate via external EOBC links
Data plane connection: bundle regular data links into special “nV fabric link” to simulate switch fabric
function between two physical chassis for data packet
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 94
nV Edge Configuration
NO need to configure the inter-chassis control plane EOBC ports. It’s plug-and-play
After this configuration, rack 1 will reload and then join cluster after it boot up
Now you successfully convert two standalone ASR 9000 into one ASR 9000 nV Edge
As simple as this !!!
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 95
nV Edge Interface Numbering
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 96
nV Edge System Monitoring
RP/0/RSP0/CPU0:ASR4-Rack0(admin)#show dsc
Thu Apr 12 03:01:12.225 UTC
---------------------------------------------------------
Node ( Seq#) Role Serial# State
---------------------------------------------------------
0/RSP0/CPU0 ( 0) ACTIVE FOX1545GRM1 PRIMARY-DSC
0/RSP1/CPU0 ( 31785) STANDBY FOX1545GRM1 NON-DSC
1/RSP0/CPU0 ( 31763) STANDBY FOX1325G77H NON-DSC
1/RSP1/CPU0 ( 32001) ACTIVE FOX1325G77H BACKUP-DSC
RP/0/RSP0/CPU0:ASR4-Rack0#show platform
Thu Apr 12 03:00:32.799 UTC
Node Type State Config State
-----------------------------------------------------------------------------
0/RSP0/CPU0 A9K-RSP440-SE(Active) IOS XR RUN PWR,NSHUT,NMON
0/RSP1/CPU0 A9K-RSP440-SE(Standby) IOS XR RUN PWR,NSHUT,NMON
0/0/CPU0 A9K-2x100GE-TR IOS XR RUN PWR,NSHUT,MON
0/1/CPU0 A9K-MOD160-TR IOS XR RUN PWR,NSHUT,NMON
0/1/0 A9K-MPA-2X40GE DISABLED PWR,SHUT,MON
0/1/1 A9K-MPA-20X1GE OK PWR,NSHUT,MON
0/3/CPU0 A9K-SIP-700 IOS XR RUN PWR,NSHUT,MON
0/3/0 SPA-8XOC12-POS OK PWR,NSHUT,MON
0/3/1 SPA-2XCHOC12/DS0 OK PWR,NSHUT,MON
0/3/2 SPA-2XOC48POS/RPR OK PWR,NSHUT,MON
1/RSP0/CPU0 A9K-RSP440-SE(Standby) IOS XR RUN PWR,NSHUT,MON
1/RSP1/CPU0 A9K-RSP440-SE(Active) IOS XR RUN PWR,NSHUT,MON
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 97
nV Topologies
Satellite
Satellite
Satellite
Satellite
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 98
EVC: Ethernet Virtual Circuits
ASR 9000 Flexible Ethernet Infrastructure
(“EVC” SW Infrastructure)
L3
EFP (Ethernet Routing
Flow Point) or EoMPLS PW
sub-interface Bridging
VPLS
EoMPLS PW
Flexible VLAN
tag classification P2P VPWS
EoMPLS PW
Flexible VLAN
tag rewrite
X Bridging Multipoint bridging
Flexible P2P
Ethertype (.1Q, local
QinQ, .1ad) connect
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
Flexible Service – L2VPN Multi-Point
L2VPN MP service configuration example
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public
Multiple Services on the Same Physical Port
Access port core interface, L2 trunk or L3 MPLS
l2vpn
xconnect group LocalConnect Local
p2p someone
interface GigabitEthernet0/0/0/1.5 connect
Ethernet Flow Point interface GigabitEthernet0/0/0/1.6
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public 106
Q&A
Complete Your Online Session Evaluation
Give us your feedback and receive a
Cisco Live 2014 Polo Shirt!
Complete your Overall Event Survey and 5 Session
Evaluations.
Directly from your mobile device on the Cisco Live
Mobile App
By visiting the Cisco Live Mobile Site
www.ciscoliveaustralia.com/mobile
Visit any Cisco Live Internet Station located
throughout the venue
Learn online with Cisco Live!
Polo Shirts can be collected in the World of Solutions
on Friday 21 March 12:00pm - 2:00pm
Visit us online after the conference for full access
to session videos and presentations.
www.CiscoLiveAPAC.com
BRKSPG-2904 © 2014 Cisco and/or its affiliates. All rights reserved. Cisco Public