International Telecommunication Union

IP Performance Metrics:
Definitions and Implementation Examples

Al Morton
AT&T Labs
Workshop on End-to-End Quality of Service.What is it? How do we get it?
Geneva, 1-3 October 2003

Outline
ITU-T
o

Performance Management Framework

o
o
o

Relationship to the E2E QoS goal

IP Parameters/Metrics Summary
In-progress Metric Development
Implementations
Service Providers
2. Customers
3. 3rd Parties
1.

1.

Performance for MPLS-enabled IP Nets

ITU-T

Network Performance
Management Framework
o Fault Monitoring -- failure detection
o Passive Info Collection (single point)
Read MIB counters or control data

Sample Traffic
o Active Measurements
Synthetic Traffic Dedicated to meas.
o Customer Measurements
Live or Synthetic traffic

ITU-T

Relationship to E2E QoS:

Provide answers to ...
o Network Provider
Is the design meeting requirements for
various traffic classes or applications?
How can I demonstrate the superior
performance of my service offering?
o Customer
Is Network Performance Agreement?
o 3rd Parties
What does the net look like? Hot spots?
What Network Provider is best?

ingress MP

egress MP

IPRE1

Packet Perf. Parameters

ITU-T

tTmax

IPRE2
Valid header and
error-free payload

Successful
IP packet outcome

IPRE2
Corrupted header or
errored payload

Errored
IP packet outcome

IPRE2
(Note)

Spurious
IP packet outcome

IPRE1

tTmax

IPRE1
Never delivered or
delivered to an unpermitted
egress MP

Lost
IP packet outcome

IPRE1

t > Tmax

NOTE Outcome occurs independent of IP packet contents

IPRE2
(Note)

Lost
IP packet outcome

ITU-T

Metric/Parameter Definition
Summary

Framework
Sampling
Loss
Delay
Delay Variation

IETF IPPM RFCs

2330
2330 Poisson
3432 Periodic
2680
2679 (1-way)
2681 (Round Trip)
3393

Availability
2678
Bulk Transfer Cap 3148
Loss Patterns
3357

ITU-T Recs.
Y.1540 cl 1 thru 5
(future work in
SG4 ?)
Y.1540 cl 5.5.6
Y.1540 cl 6.2
Y.1540 cl 6.2.2
Y.1540 cl 7
Possibly in G.IPP

ITU-T

Comparison of IETF and ITU-T

Delay Variation Metrics
IETF IPDV is a measure of transfer
delay variation w.r.t. previous packet.
For Packet n,
IPDV(n) = Delay(n) - Delay(n-1)
or = R(n) - R(n-1) - T(n) - T(n-1)
If the nominal transfer time is
=10msec, and packet 2 is delayed in
transit for an additional 5 msec, then
two IPDV values will be affected.
IPDV(2) = 15 - 10 = 5 msec
IPDV(3) = 10 - 15 = -5 msec
IPDV(4) = 10 - 10 = 0 msec

Tx

Rcv

Playout

2
1
t

2
4
3

ITU-T SG 13 PDV is delay w.r.t. a

reference, usually minimum delay.
PDV(n) = Delay(n) - Min[Delay(*)]
PDV(1,3,4)=0 PDV(2)=5
Time spent in:

Transit

Rcv Buffer

Inter packet
arrival time,
longer than
send interval

Transient Delay Variation

caused by burst traffic

ITU-T

100
90

Delay or Jitter (ms)

80
70
60
50
40
30
20
10
0
-10
-20
120

220

320

420

ITU PDV

520

delta ms

620

720

820

920 1020 1120

Time in ms

IETF

IPDV ms

RT-delay ms

ITU-T

Packet Metrics for VoIP and

other voiceband applications
o *new* metrics in G.IPP
Consecutive Packet Loss

Degraded Seconds
Short-term Delay Variation
Overall VoIP Parameters
o Alan Clarks Presentation

What is Packet Reordering?

ITU-T
Packets arrive at Dst, but not in send order.
1, 2, 3, 7, 8, 9, 10, 11,... Loss,no reordering
1, 2, 3, 7, 8, 9, 4, 5, 6, 10, 11,...reordering
In the world of order all these packets are of
interest.
1, 2, 3, 7, 8, 9, 4, 5, 6, 10, 11,...
| Early | Late |
No reordering until Late Packets Arrive
# of Early Packets => Reordering Extent

ITU-T

Affect of Reordered Packets on

most applications
o Receivers must perform work to restore order

1, 2, 3,

7, 8, 9,10, 4, 5, 6, 11, 12,...

| Buffered ||Reordered|

Dst Time axis

1

2
1

3
2

7
3

8 9 10

5
4

6
5

11

Higher
layers

6, 11
(& 7 to 10)

Definition of Reordered Packet

its sequence number is less than the
Next Expected threshold (set by the
arrival of a previous packet).
12
11

Next Expected
10

10
9

Send Order (s)

ITU-T

o Packet n is designated reordered when

8
7

6
5

4
3

2
1

0
1

Arrival Order (i)

10

11

Failure Recovery Time

ITU-T
o When recovery was a simple outage,

characterization was simple, too.

o IETF Benchmarking Methodology WG has
identified 5 possible recovery scenarios:
Lost packets
7

Induced delay
2

Out-of-order packets

Duplicate packets
6 5 4 4 3

4 3

7 5 6 3 4

Errored packets
2

ITU-T

Implementations: Customers,
Service Providers & 3rd Parties
$ ping R2 (or R3)
R1
or

R2

R3
or
R4

o Select Ping Target - make Round-trip

connectivity and RTT measurement

o Accuracy Issues include path through
router, path through net (asymmetries),
response time at target, sampling rates
o Compare to current perf. to normal

ITU-T

Beyond ping: ICMP Timestamp

or Timestamp Reply Message
0
1
2
3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
Type
|
Code
|
Checksum
|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
Identifier
|
Sequence Number
|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
Originate Timestamp
|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
Receive Timestamp
|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
Transmit Timestamp
|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

R3

Originate
Code=13

R1

R2

Receive
R4
Transmit
Code=14

o Time spent processing packet at target can be

removed, for more accurate RTT.

ITU-T

Implementations:
AT&T Global IP Measurements
AT&T GLOBAL IP BACKBONE INFRASTRUCTURE

BR

BR

(City 1)

BR
(City n)

(City 2)
MEASUREMENT
COLLECTION
SERVER (MCS)

Measurement Probes

Measurement
probe
AGGREGATED
MEASUREMENT
DATA
MEASUREMENT
AGGREGATION
& REPORTING
SERVER (MRS)

http://www.att.com/ipnetwork

WEB
CLIENTS
(forreport
viewing)

AT&Ts IP Measurement
Design
24 hours
...

ITU-T

15 minutes

o Poisson Sequence (RFC2330)

15 minute duration

= 0.3 pkts/sec
Type UDP, IPv4
278 bytes total
~300 packets sent
unbiased sample

o Periodic Sequence (RFC3432)

1 minute duration

Random Start Time

20 ms packet spacing
Type UDP, IPv4
60 bytes total
~3000 packets sent

Technical Collaborators at AT&T

ITU-T
o Len Ciavattone
o George Holubec
o Madhukar Kshirsagar
o Ron Kulper
o Arvind Ramarajan
o Gomathi Ramachandran

ITU-T

New Measurement Challenges

for MPLS-enabled IP Networks
o Most (all?) IP/Packet Network challenges
o Two main categories of MPLS Domains:
LDP-based, connection-less

Traffic Engineering, connection oriented

o Label Switched Paths are Unidirectional
o point to point and multi-point to point
o Many options for Failure Recovery
o LSP identity optionally removed (PHP)
o Work in progress in SG 13 = Y.MPLSperf

ITU-T

New Measurement Challenges

for MPLS-enabled IP Networks
MPLS Domain

Scope of OA&M Measurements:

single Network Section or MPLS Domain

MPLS Domain

Network
section

Network
section

Exchange
link
Exchange
link

Exchange
link

Network
section

Exchange
link

Exchange
link
MPLS Edge Node, or
MPLS Ingress Node, or
LSR if both IP and
MPLS are enabled

MPLS
Node

Label Switched Paths

Network Section Ensemble (NSE)

MPLS Network

ITU-T

New Measurement Challenges

for MPLS-enabled IP Networks
o New Protocols = New Opportunities to

Blackhole Traffic
o Detect this new class of failures with
Y.1711 MPLS OA&M Connectivity Verific.
First version approved, adding fast failure
detection

LSP-Ping, Like ICMP Echo Request, plus

One-way Delay measurement possible
LSP Traceroute possible

ITU-T

New Measurement Challenges

for MPLS-enabled IP Networks
o New Availability Definition? Crossroad:
Connection-Oriented Transport has used
a 10 second sliding window
Connection-Less Packet Transport has
used a 5 minute fixed window
o MPLS Networks => both transport types
When Connection-oriented Services use a
Connection-less transport, which
precedent should the Availability
Definition follow?

Summary
ITU-T
o

Performance Management Framework

o
o
o

Summary of existing Parameters/Metrics

In-progress Metric Development
Active Measurement Implementations

Measurement Systems are a key step

toward the goal of E2E QoS

Ping for connectivity and ...

Dedicated Measurement Systems

Parameter Framework for MPLS has new

challenges

Resources and References

ITU-T
o

L. Ciavattone, A. Morton and G.

Ramachandran, "Standardized Active
Measurements on a Tier 1 IP Backbone,"
IEEE Communications Magazine, June
2003.
Geoff Huston, Measuring IP Network
Performance, The Internet Protocol
Journal, vol 6, no.1, March 2003
http://www.cisco.com/ipj
X.Xiao, et al., A Practical Approach for
Providing QoS in the Internet Backbone,
IEEE Communications Magazine,
December 2002.
D. Meyer, et al., Trends in Measurement
and Monitoring of Internet Backbones,
Panel at NANOG 26, slides etc. at
http://www.nanog.org/mtg0210/measurement.html

ITU-T Rec. Y.1540, Internet Protocol Data

Communication Service IP Packet
Transfer and Availability Performance
Parameters, 2003.
IETF IP Performance Metrics Working
Group, links to RFC 2330, other IPPM
RFCs and Internet Draft on Reordering:
http://www.ietf.cnri.reston.va.us/html.charte
rs/ippm-charter.html
Draft New Recommendation Y.MPLSperf,
Performance and Availability Parameters
for MPLS Networks
Draft New Recommendation G.IPP,
Performance Parameter Definitions for
Quality of Speech and other Voiceband
Applications Utilising IP Networks
RFC 792, Internet Control Message
Protocol, J. Postel, September 1981.