ACE 4710 Design Guide

Design Guide
All contents are Copyright 19922006 Cisco Systems, Inc. All rights reserved. This document is Cisco Public Information. Page 1 of 13
Interconnecting the Cisco Application Control Engine (ACE)
4710 Appliance

The Cisco ACE 4710 appliance provides maximized application availability to help ensure
business continuity and the best service to end users by taking advantage of availability
through highly scalable Layer 4 load balancing and Layer 7 content switching, and
minimizes effects of application, device, or network failure. This configuration overview,
targeted at enterprise and service provider customers, helps ensure the optimized and
secure delivery of mission-critical application traffic in a highly available environment.
This document provides a best practice example of how to configure the Cisco ACE 4710
appliance in a high available environment.

ACE 4710 Physical Characteristics
The ACE appliance provides four physical Ethernet ports for processing traffic. The four Layer 2
Ethernet ports can be configured to provide an interface for connecting to 10-Mbps, 100-Mbps, or
1000-Mbps networks. Each Ethernet port supports auto-negotiate, full-duplex, or half-duplex
operation on an Ethernet LAN and can carry traffic within one or more designated VLANs.

Design Guide
The ACE appliance does not have additional ports used specifically for management traffic. The
four Ethernet ports are used to handle all data and management traffic in and out of ACE. They are
also used for ACE appliances deployed in a redundant fashion utilizing a fault tolerant VLAN to
maintain high availability.
Figure 1. ACE 4710 Appliance font and rear chassis views

Figure 1 shows the LED link indicators for the Ethernet port and the pin number assignments for
the RJ-45 port. As shown in Figure 1 for Ethernet port 4, the link LED in the lower right below each
Ethernet port serves as the indicator for the associated port. The ports are numbers from right to
left.
Figure 2. The ACE 4710 Appliance physical interfaces are labeled from right to left.

The states of each Ethernet port link LED are as follows:
1. Off when the 10-Mbps Ethernet link is connected or when there is no link.
2. Glows steady green when the 100-Mbps Ethernet link is connected.
3. Glows steady orange when the 1000-Mbps GigabitEthernet link is connected.
The second LED flashes yellow when there is activity.

C A N
To maximize application and infrastructure availability, the Cisco ACE 4710 appliance takes
advantage of all four gigabit Ethernet interfaces and ACE virtualization. These interfaces can be

Design Guide
configured in a port-channel to create a single logical link between the Cisco ACE 4710 Appliance
and Cisco Catalyst Series Switches. Trunked VLANs can be used to carry all client/server
messaging, management traffic and fault tolerance (FT) communication.
Figure 3. Port Channel carries traffic for all VLANs

Connecting the ACE 4710 to a Catalyst Switch in this manner has several obvious advantages:
1. It allows for the creation of a single very high-bandwidth logical link ensuring the highest
level (4 Gbps) of throughput possible on the ACE 4710 appliance. Gracefully handles
asymmetric traffic profiles typical of web architectures.
2. It simplifies the interface configuration since the single port-channel and 802.1q trunk
need only be configured once and applied to each physical interface.
3. Future upgrades, for example from 1 Gbps to 4 Gbps, can be accomplished in real time by
installing a license for increased throughput without needing to physically re-cable the
appliance interfaces.
4. Individual ACE contexts are not limited by the throughput of a single 1 Gbps interface.
Traffic can be shaped according to the available throughput at the context, VIP, or real
server level rather than at the interface level.
5. Allows the ACE to reach throughput license limits including throughput additionally
reserved management traffic. By default, the entry-level ACE appliance has a 1-Gbps
through-traffic bandwidth limit and an additional guarantee of 1-Gbps management-traffic
bandwidth resulting in a maximum bandwidth of 2 Gbps. Similarly, with the 2-Gbps
license, the ACE has a 2-Gbps through-traffic bandwidth and a 1-Gbps management-
traffic bandwidth for a total maximum bandwidth of 3 Gbps.
6. The port-channel provides redundancy should any one of the 4 physical interfaces fail.
The single logical link can support all the common deployment modes including routed, bridged,
one-arm and asymmetric server return while also addressing high availability and stateful
connection replication with out issue.

Design Guide
Figure 4. Example network topology incorporating ACE 4710.

How bandwidth is calculated on ACE
Each gigabit link on ACE has the potential for simultaneously transferring 1Gbps input and 1Gbps
output. Since the wires for input and output are physically separate (two each) the input doesn't
affect the output. When the 4 inks are aggregated with etherchannel, ACE can provide a maximum
of:
4Gbps input - *from* clients and servers
4Gbps output - *to* clients and servers

As you can see, that is 4Gbps throughput - in from one side, out the other. The traffic flow is
illustrated in Figure 5 below.
Figure 5. Example of the throughput calculation on the ACE 4710.

The 4Gbps is the theoretical maximum of client-to-server + server-to-client + FT sync traffic +
probes, assuming traffic equally spread across those ports. In some environments the default
4Gbps input to ACE
+
4Gbps output from ACE
4 Gbps Full-duplex
3.5Gbps
server response
0.5Gbps
client requests

Design Guide
ether channel hash many not provide the optimal balance between the 4 aggregated 1Gbps links.
The ACE 4710 supports various port-channel hashes to optimally distribute a vast variety of traffic.

Layer 2 Configuration of the Cisco Catalyst Switch
Once the 4 physical interfaces on the Cisco ACE 4710 Appliance have been physically connected
to the Catalyst switch ports, the first step is to configure the port channel and switch ports on the
Catalyst switch.
Switch Port Channel Configuration
In the following example a Cisco Catalyst 6500 is configured with a port-channel utilizing an 802.1q
trunk allowing the associated VLANs. The native VLAN of the trunk is VLAN 10, it is recommended
not to use the default VLAN 1 for the native VLAN since this VLAN is used internally on the ACE
4710 Appliance.
Port Channel load balancing is used to distribute the traffic load across each of the links in the port-
channel ensuring efficient utilization of each link. Port-channel load balancing on the Cisco Catalyst
6500 can use MAC addresses or IP addresses, Layer 4 port numbers, source addresses,
destination addresses, or both source and destination addresses. By default the ACE uses src-dst-
mac to make a load balancing decision. The recommended best practice is to use source and
destination L4 port for the load balancing decision.

switch/Admin(config)#port-channel load-balance src-dst-port
switch/Admin(config)# interface Port-channel1
switch/Admin(config-if)# description ACE 4710
switch/Admin(config-if)# switchport
switch/Admin(config-if)# switchport mode trunk
switch/Admin(config-if)# switchport trunk encapsulation dot1q
switch/Admin(config-if)# switchport trunk native vlan 10
switch/Admin(config-if)# switchport trunk allowed vlan 10,20,30,31,
40,50
switch/Admin(config-if)# switchport nonegotiate
switch/Admin(config-if)# mls qos trust cos

switch/Admin(config-if)# do sho run | begin Port
interface Port-channel1
description to ACE 4710
switchport
switchport trunk encapsulation dot1q
switchport trunk native vlan 10
switchport trunk allowed vlan 10,20,30,31,40,50
switchport mode trunk
switchport nonegotiate
mls qos trust cos
no ip address
!

Design Guide
Once the port channel is configured on the switch, it can then added to the configuration of the four
interfaces.
Note: The Cisco ACE 4710 Appliance does not support Port Aggregation Protocol (PAgP) or
Link Aggregate Control Protocol (LACP) so the port-channel is configured using mode on.

Switch Interface Configuration
On ACE 4710 Appliance you can configure the Ethernet port speed for a setting of 10, 100, or 1000
Mbps by using the speed command in interface configuration mode. The default for the ACE 4710
appliance is auto-negotiate interface speed. It is recommended to avoid relying on auto negotiation
of interface speed by explicitly configuring the speed to 1000 on both the switch and the appliance.
This will avoid the possibility of the interface operating below the expected Gigabit speed and
ensure the port-channel can reach the maximum 4 Gbps throughput.
The ACE 4710 does not implement Spanning-Tree protocol and therefore does not take part in
Spanning-Tree root bridge election process. PortFast is configured on the switch to reduce the time
required for spanning tree to allow traffic on the port connected to the ACE interface by immediately
moving to forwarding state, bypassing block, listening, and learning states. The average time for
switch port moving into a forward state is approximately 30 seconds. Using PortFast reduces this
time to approximately 5 seconds.
Note: In virtual partitions operating in bridge mode, the ACE offers an option to bridge Spanning-
Tree BPDUs between two VLANs in order to prevent the possibility of a loop. Such a loop may
occur when two partitions end up actively forwarding traffic. While this should not happen during
normal operation, the option to bridge BPDUs provides a safeguard against this condition. Upon
seeing BPDUs circling around, the switch connected to the ACE 4710 will immediately block the
port/VLAN the loop originated from. The following ethertype ACL should be configured on ACE and
applied to Layer 2 interfaces in bridgemode: access-list BPDU ethertype permit bpdu
For more information on PortFast, see the following URL:
http://www.cisco.com/warp/public/473/12.html#bkg.

The following commands are used to configure the switch ports:

Router(config-if)# int range Gig 3/9 - 12
Router(config-if-range)# channel-group 1 mode on
Router(config-if-range)# speed 1000
Router(config-if-range)# spanning-tree portfast trunk
Router(config-if-range)# no shut

The port channel configuration is then added to each of the interfaces resulting in the following
configuration:

Router(config-if)# do sho run | beg GigabitEthernet3/9
Building configuration...

Design Guide
interface GigabitEthernet3/9
description ACE 4710 int1
switchport
speed 1000
no ip address
spanning-tree portfast trunk
channel-group 1 mode on
!
switchport
speed 1000
no ip address
!
switchport
speed 1000
no ip address
!
switchport
speed 1000
no ip address

Design Guide
Configuring the Cisco ACE 4710 Appliance
Once the switch is configured, the next task is to configure the Cisco ACE 4710 Ethernet interfaces
and Port Channel. In this design we configure the four Ethernet ports as 1000-Mbps full-duplex and
associate each of the four ports as a member of an Layer 2 Port Channel. The Port Channel
bundles the individual physical Ethernet ports into a single logical link associated as an 802.1Q
trunk.
ACE Port Channel Configuration
In the following example the ACE 4710 is configured with a port-channel utilizing an 802.1q trunk
allowing the associated VLANs. Similar to the Catalyst switch configuration the native VLAN of the
trunk is VLAN 10, it is recommended not to use the default VLAN 1 for the native VLAN since this
VLAN is used internally on the ACE 4710 Appliance.
Note that the port-channel number on ACE can be different from that of the switch. For example in
an HA configuration it would be possible for the distribution switch port-channel 1 defined for
primary Cisco ACE 4710 Appliance and port-channel 2 defined for the back-up. During HA
replication the port channel is replicated to the backup device. This means one of the Cisco ACE
4710 Appliances will always have a different port-channel number than that of the switch. Since
the port-channel numbers are not required to be consistent between devices there will be no issue.

switch/Admin(config)# interface port-channel 3
switch/Admin(config-if)# switchport trunk native vlan 10
switch/Admin(config-if)# switchport trunk allowed vlan
10,20,30,31,40,50
switch/Admin(config-if)# port-channel load-balance src-dst-port
switch/Admin(config-if)# no shutdown

ACE Ethernet Interface Configuration
In the following example the ACE 4710 is configured similarly to the Catalyst switch configuration.
The interface speed on ACE is set to 1000M Full Duplex and each of the four interfaces is
associated with the Port Channel using the channel-group command. It is recommended to
configure a carrier delay of 30 seconds for deployments in which ACE is configured with fault
tolerance and preemption.
Note: Refer to the section HA with preemption and Carrier Delay at the end of the document
for more information regarding carrier-delay.
Additionally ACE appliance is configured to prioritize incoming HA heartbeat traffic (CoS value of 7
by default) on each of the ports.
Note: Refer to the section Enabling Quality of Service for High Availability at the end of the
document for more information regarding QOS and HA traffic.

switch/Admin(config)# interface gigabitEthernet 1/1
switch/Admin(config-if)# speed 1000M
switch/Admin(config-if)# duplex FULL
switch/Admin(config-if)# channel-group 3

Design Guide
switch/Admin(config-if)# carrier-delay 30
switch/Admin(config-if)# qos trust cos




The port channel configuration is then added to each of the interfaces resulting in the following
configuration:

switch/Admin(config)# do show run int
Generating configuration....
interface gigabitEthernet 1/1
speed 1000M
duplex FULL
channel-group 3
carrier-delay 30
no shutdown
speed 1000M
duplex FULL
channel-group 3
carrier-delay 30
no shutdown
speed 1000M

Design Guide
duplex FULL
channel-group 3
carrier-delay 30
no shutdown
speed 1000M
duplex FULL
channel-group 3
carrier-delay 30
no shutdown

Verify Layer 2 Network Connectivity
At this time the port-channel and trunk should be up on both the switch and the Cisco ACE 4710
Appliance. There are several show commands that can verify the port-channel and trunk status.
For example, to view the configuration status for port-channel interface 3, enter:

switch/Admin# show int port-channel 3
PortChannel 3:
----------------------------
Description:
mode: Trunk native vlan: 10
status: (UP), load-balance scheme: unknown
PortChannel 3 mapped phyport:1/1 1/2 1/3 1/4
PortChannel 3 mapped active phyport:1/1 1/2 1/3 1/4
PortChannel 3 allow vlan:
vlan<10> vlan<20> vlan<30>-<31> vlan<40> vlan<50>
11606094 packets input, 970121368 bytes, 0 dropped
Received 694844 broadcasts (10877868 multicasts)
0 runts , 0 giants
0 FCS/Align errors , 0 runt FCS, 0 giant FCS
85431 packets output, 12278955 bytes
22334 broadcast, 0 multicast, 0 control output packets
0 underflow, 0 single collision, 0 multiple collision output
packets
0 excessive collision and dropped, 0 Excessive Deferral and
dropped

It is important to note that the status should indicate UP and that the all four of the interfaces
appear in the mapped output. Also verify that the mode is Trunk with the correct VLANs
associated. Similarly the status of each physical interface can be verified using the show
interface command:

switch/Admin# show interface gigabitEthernet 1/4
GigabitEthernet Port 1/4 is UP, line protocol is UP
Hardware is ACE Appliance 1000Mb 802.3, address is 00.00.00.00.20.62

Design Guide
MTU 0 bytes
Full-duplex, 1000Mb/s
0 packets input, 0 bytes, 0 dropped
Received 0 broadcasts (0 multicasts)
0 runts , 0 giants
0 FCS/Align errors , 0 runt FCS, 0 giant FCS
0 packets output, 0 bytes
0 broadcast, 0 multicast, 0 control output packets
0 underflow, 0 single collision, 0 multiple collision output packets
0 excessive collision and dropped, 0 Excessive Deferral and dropped

You can also inspect the interface counters on ACE using the following command:
switch/Admin# show interface gigabitEthernet 1/1 counters

On the Catalyst 6500 Switch the following show commands can be used to verify the Port Channel
and interface configuration:

Router(config)#do sho int port 1 ether
Age of the Port-channel = 5d:20h:33m:48s
Logical slot/port = 14/1 Number of ports = 4
GC = 0x00000000 HotStandBy port = null
Port state = Port-channel Ag-Inuse
Protocol = -

Ports in the Port-channel:

Index Load Port EC state No of bits
------+------+------+------------------+-----------
0 11 Gi3/9 On 2
1 22 Gi3/10 On 2
2 44 Gi3/11 On 2
3 88 Gi3/12 On 2

Time since last port bundled: 0d:01h:40m:54s Gi3/12
Time since last port Un-bundled: 0d:01h:40m:54s Gi3/12

Router(config)#do sho int trunk

Port Mode Encapsulation Status Native vlan
Po1 on 802.1q trunking 10

Port Vlans allowed on trunk
Po1 10,20,30-31,40,50

Port Vlans allowed and active in management domain

Design Guide
Po1 10,20,30-31,40,50

Port Vlans in spanning tree forwarding state and not pruned
Po1 10,20,30-31,40,50

Router# show interfaces counters etherchannel
Port InOctets InUcastPkts InMcastPkts InBcastPkts
GPo200 165864149 2368526 0 9245
GFa1/11 165859783 2368463 0 9244
GFa1/12 82 0 0 1
GFa1/13 4502 65 0 1
GPo201 453730 31 0 6636
GFa1/21 453940 31 0 6639
Port OutOctets OutUcastPkts OutMcastPkts OutBcastPkts
GPo200 21834046550 11134457 238140029 35757886
GFa1/11 21834038356 11134457 238140022 35757833
GFa1/12 4215 0 5 12
GFa1/13 4507 0 5 45
GPo201 21625744426 10796354 238141120 35761258
GFa1/21 21625744788 10796354 238141122 35761259

8 k M 1
By default, the entry-level ACE has a 1-Gbps through-traffic bandwidth and a 1-Gbps management-
traffic bandwidth for a total maximum bandwidth of 2 Gbps. However when the 4-Gbps throughput
license is applied, the ACE cannot reserve additional bandwidth beyond the four 1-Gbps port limit.
Therefore some fraction of the total available bandwidth must be reserved at the context level for
management traffic sent to the ACE appliance using the limit-resource command. In the example
below a resource class is created and 2% of the total 4-Gbps of bandwidth is reserved for
management traffic in the ACE Admin context:

switch/Admin(config)# resource-class GLOBAL
switch/Admin(config-resource)# limit-resource rate mgmt-traffic
minimum 2 maximum equal-to-min
switch/Admin(config)# context Admin
switch/Admin(config)# resource-class GLOBAL

When you allocate a minimum percentage of bandwidth to management traffic, the ACE subtracts
that value from the maximum available management traffic bandwidth for all contexts in the ACE.
By default, management traffic is guaranteed a minimum bandwidth rate of 0 and a maximum
bandwidth rate of 1 Gbps, regardless of the bandwidth license that you install in the ACE. The best
practice recommendation is to reserve roughly 100 Mbps for management traffic per context.

nA C D
The carrier-delay command was introduced in the ACE 4710 1.8 software release. This command
was added to handle a very specific scenario involving fault tolerant configurations and preemption.
In this scenario two ACE 4710 appliances are connected to each other through a common LAN

Design Guide
switch such as a Catalyst 6500. ACE A is Active while ACE B is Standby. Suppose ACE B takes
over because of a failure of ACE A. Moments later, ACE A comes back and wishes to reclaim its
active role (it is configured to preempt). When the ACE 4710 comes back up, it brings up its
Ethernet interfaces and assumes shortly thereafter that the switch is ready to accept and process
traffic. This might not be the case due to timing differences. For example the Spanning-Tree
process could still be determining whether the port can safely be put in the forwarding state on the
switch side. In the meantime, the ACE 4710 has already sent gratuitous ARPs to refresh the switch
fabrics MAC addresses. To prevent this timing discrepancy, it is recommended you configure a
carrier-delay of 30 seconds on the ACE 4710 that is configured to preempt.
L S n A
By default, Quality of Service (QoS) is disabled for each physical Ethernet port on the ACE. You
can enable QoS for a configured physical Ethernet port that is based on VLAN Class of Service
(CoS) bits (priority bits that segment the traffic in eight different classes of service). If a VLAN
header is present, the CoS bits are used by the ACE to map frames into class queues for ingress
only. If the frame is untagged, it falls back to a default port QoS level for mapping.
When you enable QoS on a port (a trusted port), ingress traffic is mapped into different ingress
queues based on their VLAN CoS bits. If there are no VLAN CoS bits, or QoS is not enabled on the
port (untrusted port), the traffic is then mapped into the lowest priority queue.
You can enable QoS for an Ethernet port configured for fault tolerance. In this case, heartbeat
packets are always tagged with CoS bits set to 7 (a weight of High). We recommend that you
enable QoS on all ports utilizing the FT VLAN to provide a higher priority for incoming FT
heartbeats.
FOR MORE INFORMATION
For more information about the Cisco ACE product family, visit
http://www.cisco.com/go/ace
For more information about Application Networking Services, go to:
http://www.cisco.com/en/US/products/hw/contnetw/index.html
or contact your local account representative.

Printed in USA C78-331727-01 10/06 Printed in USA C78-331727-01 10/06

ACE 4710 Design Guide

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

ACE 4710 Design Guide

Uploaded by

Copyright:

Available Formats

Design Guide

You might also like