JDSU RFC6349 Testing Methodologies

RFC6349 & Testing Methodologies
for Ethernet presented by JDSU

Tahmina J Hoque
Solutions Specialist Organization
April 2014
Testing Methodologies
RFC2544
Y.1564
RFC6349
© 2012 JDS Uniphase Corporation | JDSU CONFIDENTIAL AND PROPRIETARY INFORMATION 2

Network Configuration Problems in Ethernet/IP Circuits
7 - Failed -
revisit
91 - passed
120 - after work
Passed - wth carrier
1st test 45% of all tests fail the first
time
Failed – requires retest 1
Passed – after work with the carrier 2
Passed 3
Jitter Fail-Over failures
Top problems:
VLAN
• Auto-Negotiation set incorrectly Config
CIR Issues
• 802.3 versus DIX framing
Auto-
• Misconfigured CIR Ethernet Negotiation
Framing
• VLAN configuration problems
• Jitter
• Fail-Over Failures
Ethernet Frame Format
The Datalink layer relies on layer 2 overhead for physical

Addressing and Error Detection.
Same frame format regardless of rate
Frames are sent from the Source MAC Address (SA) to
Destination MAC Address (DA)
Length or Type (L/T) Field indicates the Frame Type
At the end of each frame is Frame Check Sequence (FCS)
field used for data integrity
6 Bytes 6 Bytes 2 Bytes 46 - 1500 Bytes 4 Bytes
Destination Source
Length
MAC MAC Type
Data FCS
Address Address

Ethernet Frame with VLAN Tag
VLAN ID
• Specifies which VLAN group
• Which customer is this traffic
VLAN Priority
• Prioritize traffic, gold, silver, bronze
• If both voice and data on the link, prioritizing voice traffic
4 Bytes
Destination Source
VLAN Length
MAC MAC Type
Data FCS
Tag
Address Address
TPI Priority CFI VLAN ID

2 Bytes 3 Bits 1 Bit 12 Bits
QinQ - VLAN Stacking
Customers use VLAN tags for end to end circuit transparency

Providers use VLAN tags in their network for traffic
prioritization/engineering
Customer Perspective
Stacked VLANs
Dot1Q Trunk
802.1Q Service Provider Tag CE CE
(S-Tag) Switch Switch
Provider Perspective
TPID CoS D
VLAN PE PE
Priority E ID CE
Switch Switch
Switch
Service Provider
802.1Q Customer Tag Network w/
(C-Tag) 802.1Q
CE
VLAN
Rx Monitor
ID
TPID CoS C VLAN Customer VLANs Switch

F
Priority I ID VLAN
ID
VLAN
ID
Stacked VLANs

L1, L2, L3 and L4 Rates
L1 = L2 + 20 in Bytes
L1
L2
Pre SFD DA SA T/L IPv4 UDP Data FCS IFGmin

(7) (1) (6) (6) (2) (20) (8) (M) (4) (12)
Pre SFD DA SA T/L IPv4 TCP Data FCS IFGmin

(7) (1) (6) (6) (2) (20) (20) (N) (4) (12)
L4
L3
L2 = L3 + 18 in Bytes

SLAs Verification
MSC
Y.1564?
Core
Cell Site Network
Ethernet Backhaul
Network
Data
MSPP
Network
How do we test SLA compliance?

SLA verification should include testing the link for Throughput,
Delay, Frame loss, and Jitter using various frame sizes
End result is a Pass/Fail assessment on the overall quality of
the link
Multiple standards exist that address SLA verification - RFC
2544 or Y.1564 – which one is most applicable?
Test application - Single service
- RFC 2544
RFC2544
RFC2544 RFC2544
Test set Test set
The goal is to prove that it is possible to obtain a constant throughput of

1.5 Mbps at all frame lengths .
(64 octets, 128 octets, 256 octets and so on up to 1518 octets)
LAN WAN LAN

Page 10 | April 2012 | RFC6349
© 2012 JDS Uniphase Corporation http://www.ietf.org/rfc/rfc2544

| 10
JDSU CONFIDENTIAL AND PROPRIETARY INFORMATION
RFC2544 Throughput test bandwidth discovery
10M
5M
2.5M
1.25M … 1.5M Discovered bandwidth = 1.5M
0 0
0 0
0 0
0 0
The goal is to prove that it is possible to obtain a constant throughput of

1.5 Mbps at all frame lengths .
(64 octets, 128 octets, 256 octets and so on up to 1518 octets)
LAN WAN LAN


Automated turn-up test – RFC 2544
For Ethernet and Fibre Channel
Goal
• Run a sequence of tests to
verify
the general performance of a
circuit.
Test method
• Packet based, in loop or
end-to-end
Test application
– Throughput rate in F/s or %
– Frame loss absolute or %
– Delay/Latency in ms or us
– Back-to-Back in frames or time
– Packet Jitter (no part of the RFC)
Test parameter
– Packet size: 64, 128, 256, 512, 1024, 1280, 1518 bytes
– Packet rate: 10, 20, 30, 40, 50, 60, 70, 80, 90, 100% of maximum rate
– Burst: Time or number of packets
RFC-2544 - Frame loss rate and throughput
What is possible in constant terms and what is lost during overloading?
Frame Loss Rate

• What is lost during constant overload.
Throughput
– Maximum throughput at which no packet is lost.
This goes in This comes out

DUT/SUT
Switch or network
This goes through constantly

RFC-2544: Delay/Latency
Time through the DUT
Delay
First bit in First bit out
Store and forward
DUT
Test packets Device under test
with time-stamp
Ethernet tester inserts in each frame at time stamp.

RFC 2544: Back-to-Back
Which burst (time or number of frames) is forwarded without frame loss

This is not a CBS test as discussed by Y.1564
TX switch = 20.000 f/s

TX test load = 100.000 f/s TX switch = 40.000 f/s
TX TX
100 %
Frames from buffer
40 %
20 %
t
1s 2s Buffer: 20.000 frames 1s 2s 3s t
The higher the transport rate with same buffer size, the larger the burst.
Test is only useful (important), if the actual transmission rate is less than the
access rate. (Ex.:100 Mbps access rate and 45 Mbps transmission rate.)
If the constant load equals the transmission rate, a small buffer creates no
problem. Whereby an only small burst load can cause already substantial packet
loss.

RFC 2544 History
Ratified by IETF in March 1999

Recommended field service turn-up and installation test to meet customer SLAs
Automated end-to-end single Ethernet/IP service test using Loopback on far end
Verifies Throughput, Delay (Latency), Frame Loss, and Back-to-Back
Problems Solved
Network Mis-Configurations - VLAN, Max Throughput (policers)
Poor Quality of Service - Too much Latency, Jitter, or Loss
T-BERD/MTS-6000A
MSAM
T-BERD 5800
T-BERD 6000A
MSAM
HST-3000

JDSU RFC 2544 Enhancements 2000 - 2011
Configurable test times

5 min – 6 hours
Configurable frame sizes
Including Jumbo
Added Packet Jitter
Real-time services – voice and video
Concurrent tests to reduce test time
by half
Measure Throughput, Delay, and
Jitter simultaneously
Pictorial Results
Easy to understand and provides
real-time training
Import/Export Saved Test
Configurations
Enable consistent test methods
J-QuickCheck
Fast configuration verification
J-QuickCheck
The Problems The Solution
• It takes too long to get the test set-up • Saves valuable time by performing quick end-
correctly before actually starting to-end connectivity and configuration test
• Users mis-configure the test set auto- • Verifies test set auto-negotiation settings and
negotiation and loopback settings connectivity to far end with proper loopback
• It’s a waste of time to run the full test if • Quickly verifies end-to-end throughput
throughput is way off from expected

Test application - Multiple Services
- Y.1564 Service Activation Methodolgies
Y.1564 Overview
Ratified by ITU-T in March 2011

Multiple services field turn-up and installation test to meet customer SLAs
Automated end-to-end multi-Ethernet/IP service test using Loopback on far end
Ideal for LTE/4G IP services and Triple Play testing
Problems Solved
Network Mis-Configurations – VLAN ID & Priority, IP TOS, Max Throughput
Poor Quality of Service - Too much Latency, Jitter, or Loss
How well do all services play together on same network under load conditions?
CPE 7609 7609 CPE
3550 7609 7609 3550
Access Edge Edge Access

10/100 10/100
Gigabit 7609 7609 Gigabit
Ethernet Ethernet
Ethernet Core Ethernet

High-Level View of ITU-T Y.1564
Objectives
• To validate the KPIs of Carrier Ethernet-based services
- Validate the configuration of each stream/service
- Validate QoS as defined by the Service Level Agreement (SLA)
Two Phase Methodology

• Network Configuration Test (Phase 1)
- Each defined stream/service validated to meet the SLA at CIR
- Each stream/service tested individually
• Ethernet Services Test (Phase 2)
- Multi-service test is compared to a pass/fail criteria
- All services tested concurrently at CIR (Max. error free throughput)

Y.1564 Overview
Mainly an Ethernet Installation Tool

• Verifies multi-service SLA compliance
Aligns itself with MEF terminology
• Bandwidth profiles
- CIR, CBS, EIR, EBS
• Key Performance Indicators (KPIs) or Parameters
- Frame Delay (FD), Jitter (FDV), Frame Loss Rate (FLR)
• MEF Color Modes (Green, Yellow & Red)

Y.1564 New Terminology – Bandwidth Profiles
CIR – Committed Information Rate

• Throughput committed to conform to Key
Performance Indicators (KPIs)
EIR – Exceeded Information Rate

• Best effort throughput above CIR - may or
may not be dropped entirely
MIR – Maximum Information Rate

• Calculated Maximum throughput used to test
policers - any traffic above EIR up to MIR is
guaranteed to be dropped

Y.1564 New Terminology – Key Performance Indicators
Frame Delay (FD) • Voice: over-talk, echo, dropped calls

– Latency • Video : choppiness and delays
• Data: long download times
Frame Delay • Voice: clicking and popping noises

Variation (FDV) – • Video: pixelization or blue screens
Jitter • Data: minimal affect
Frame Loss Rate • Voice: clicks/fuzziness, dropped calls

(FLR) • Video: pixelization or blue screens
• Data: long download times

Y.1564 Phase 1: Network Configuration Test
Phase 1 validates the network configuration of each defined services (rate limiting,
traffic shaping, QoS)
First stage, X steps to CIR, 1 to 60 seconds each
• Verifies SLA parameters are met for different rates lower and equal
to CIR
• SLA parameters: Throughput, Delay (FD), Jitter (FDV) and Frame
Loss (FL)
Then Step to EIR and EIR + x% line rate
• Verifies throughput with error in excess of CIR
• Verifies Max Throughput does not go over the maximum allowed
EIR + x%
Maximum
EIR Throughput
Threshold
CIR
Level below
Throughput as which
seen at output SLA parameters
of test instrument are verified
time
1 to 60 sec © 2012 JDS Uniphase Corporation | JDSU CONFIDENTIAL AND PROPRIETARY INFORMATION 25
Y.1564 Phase 2: Ethernet Services Test
Phase 2 validates the quality of service of each defined service and

proves SLA conformance
All services are generated simultaneously to their CIR and all KPIs are
measured for all services
This phase is a single measurement done over a mid to long-term time period
This procedure allows the characterization of each service and its influence on
others and ensures that they all comply to their respective SLA
Mbps Transmitted traffic Mbps Measured traffic
Service 1
CIR
Service 2
CIR
Service 3
CIR
t t

How does the Y.1564 compare to the RFC 2544
Y.1564 is needed in the following scenarios

• When EIR is specified as part of the SLA
• When multiple priorities for a customer are specified at the same
drop location
• If neither of these conditions are met then there isn’t a strong push
for using Y.1564 (jitter testing has been added to RFC 2544)
RFC 2544 is the current standard for validating Ethernet links as it tests
common SLA parameters
With Y.1564 becoming a standard the industry may move towards a new
method of testing but the test is not fixing a strong unmet need
The RFC-2544 test will test ALL frame sizes whereas the Y.1564 – if set
to random may or may not test the desired frame size
Options:
Test with RFC-2544 if a single service is deployed.
Test with Y.1564 if multiple services are deployed – with CIR
configuration
Test with RFC-6349 if Traffic Shaping is implemented at turn-up or
troubleshooting
Y.1564 – undefined & unaddressed
Burst Testing is Optional

• CBS & EBS
Does not define a One Way Delay Test
Not a good Network Shaping Test

• Service Performance Test verifies all services/streams at CIR + an
over value
• Service Performance Test DOES NOT test one service/stream
ramping or bursting
• What is the attainable MIR when all streams are running?
- Answers buffer allocation question in a multi-service/multi-stream environment
- How is EIR/MIR allocated between streams?
No Zeroing in Algorithm like RFC 2544

• Ramp test will need to be repeated to find true IR on failures
- True CIR/MIR is needed for troubleshooting
Was the rate set incorrectly – what was it set to?
Was the effect of traffic encapsulation - VLAN, Q-in-Q, MPLS/VPLS tag(s) - factored in?
- Good for establishing cause of failure and for rejecting 3rd party circuits

Key JDSU Advantages
Integrated J-QuickCheck
• Pre-test before starting
• Saves time on end-to-end setup
Time Saving Throughput Test Time Saving

• Start at CIR • Steps to CIR X Time
per Step
• If CIR fails start at -0- and step up
• Ex: 3 steps x 60
seconds = 3 Minutes
Easy Guided Workflow
• 1. Configure test
• 2. Run Test
• 3. Create Report
Time Saving Troubleshooting

• RFC 2544 Zeroing-In Algorithm
Triple Play Stream Configuration

• Configure # of TV or VoIP Streams
• Same as Triple Play Test

Time is Money – Save at least 12 minutes per job
1 Minute 4 Minutes 19 Minutes

10 Minutes 16 Minutes 31 Minutes
# Jobs Time saved
5 60 min
A Competitive Solution
8 1h 36min x (hour rate) = OPEX savings
10 2h
100 20h

Test application - Stateful TCP
- RFC 6349
TCP Testing Applications
Customer Site Ethernet Provider Customer Aggregation
T-BERD/MTS 5800 T-BERD/MTS 8000
T-BERD
Backhaul
Network
T-BERD/MTS 8000
T-BERD/MTS 6000 T-BERD/MTS 6000
1. Service providers offering Ethernet Business services
2. Service providers offering Ethernet Backhaul services
3. Mobility service providers deploying 3G or 4G/LTE

Why is TCP Testing Important?
Running RFC2544, Y.1564 or other Layer 2/3 installation tests

is always the first step
But even when these Layer 2/3 tests “pass”, end-customers

can still complain that the “network is slow” and the cause of
poor application performance (i.e. FTP, web browsing, etc.)
Lack of TCP testing is a turn-up gap because end-customer

applications are transported using TCP in Layer 4
Save up to 30% OPEX costs by eliminating or quickly

resolving painful end-customer finger pointing scenarios

What Applications use TCP?
HTTP, FTP, E-mail, Sharepoint,

7 Communicator, Facebook, YouTube, etc
6
5
TCP RFC 6349 TrueSpeedTM

4
3 IP Y.1564
2 &
RFC 2544
Ethernet
1

Lack of TCP Testing is a Turn-up Gap!
• Typical network problems addressed by RFC2544, Y.1564 and RFC 6349.

Case Study: Customer Issue
100 Mbps Service

~20 msec RTT
Layer 2 traffic testing was done twice, Test results = “Pass”

End-customer performance dipped to 50-60 Mbps at various times of the day (end-customer
running “iperf” based speed tests)
TrueSpeedTM test was run from network provider demarc in Tampa to Atlanta
– TCP throughput was rock solid at nearly 100 Mbps (95 Mbps layer 4)
– Finger was clearly pointed at customer data center network; investigation proceeded in the customer
arena (provider was “cleared of guilt”)

Case Study: Network Provider Issue
300 Mbps Service

~75 msec RTT
Layer 2 traffic testing was done twice, Test results = “Pass”

End-customer file transfer performance was low (150 Mbits/sec)
TrueSpeedTM (30-40 TCP sessions with 64 KB window) stressed network buffers
– Problem was reproduced
– Network provider isolated to mis-configured device at California site

TCP Fundamentals
38
TCP Behavior
• TCP tries to use all of the available bandwidth

while avoiding congestion.
– Obtain a high throughput while maintaining
stability.
• TCP performance is influenced by its

congestion avoidance algorithms.
– Segment loss and/or RTT increases lead to
throughput reduction.

TCP (3 Way Handshake)
SYN
SYN - ACK
ACK
Client Servers
• The connection initiator advertises its TCP RWND, MSS and optional parameters.
• The receiving host also advertises its TCP RWND, MSS and optional parameters.
• The connection initiator completes the 3 way handshake.
• The TCP connection is now established.

TCP (RWND & CWND)
TCP Sequence number
TCP Sequence number

TCP ACK + TCP RWND
Client Servers
• Usually the connection initiator makes requests to the server.
• Then the server becomes the transmitter.
• The receiver calculates and sends its TCP RWND within each of its TCP ACK.
• The server or transmitter constantly adjust its TCP CWND.
Important note :
The minimum value between the transmitter TCP CWND and the TCP
RWND advertised by the opposite end determines how many Bytes can be
sent
Page by the2012transmitting
41 | April | RFC6349 side at a given time.
TCP (The 3 phases of the Congestion Window)
cwnd = TCP estimation of the available throughput capacity
High TCP CWND RTO Timer

ssthresh halving upon loss expiration
cwnd
cwnd=1 Adjusted
Congestion ssthresh
2 Avoidance upon
Slow “timeout”
1 Start
3 Loss Recovery
(with Fast Retransmits or Fast Recovery) Slow
Start
Retransmissions
Time
© 2012 JDS Uniphase Corporation | Reference page

42 6
JDSU CONFIDENTIAL AND PROPRIETARY INFORMATION
Overview of RFC 6349
43
RFC 6349: TCP Throughput Test Methodology
• IETF RFC 6349 specifies the methodology for measuring end-to-end

TCP Throughput in a managed IP network.
• JDSU and two (2) Network Providers co-authored this RFC
“0”. Run traditional RFC2544 to verify the integrity of the network at Layers
2 and 3 before conducting TCP testing.
1. Path MTU Detection (per RFC4821)

– Verify network MTU with active TCP segment size testing to ensure
TCP payload does not get fragmented
2. Baseline Round-trip Delay and Bandwidth

– Predict optimum TCP window size to automatically calculate the
TCP Bandwidth Delay Product (BDP)
3. Single and Multiple TCP Connection Throughput Tests

– Verify TCP window size predictions to enable automated “full pipe”
TCP testing
RFC 6349 Metrics: TCP Efficiency
TCP retransmissions are a normal behavior in network communications.
• But what is the “efficiency” of a network transfer?
• Time spent transmitting “good” payload versus retransmitting it.
The TCP Efficiency metric is the percentage of Bytes that did not have
to be retransmitted and is defined as:
Transmitted Bytes - Retransmitted Bytes x 100

Transmitted Bytes
As an example, if 100,000 Bytes were sent and 1,000 had to be

retransmitted, the TCP Efficiency would be calculated as:
101,000 - 1,000 x 100 = 99%

101,000

RFC 6349 Metrics: Buffer Delay
TCP throughput is also affected by increase in RTT, which can be
caused by network congestion or buffer delay.
The Buffer Delay Percentage is defined as:
Average RTT during Transfer - Baseline RTT x 100

Baseline RTT
Example: If the baseline RTT for a network path is 2 ms and the

average RTT increases to 3 ms during the test. The Buffer Delay
Percentage would be calculated as:
3 – 2 x 100 = 50%
2

JDSU TrueSpeed = RFC 6349
Reduce operating expenses up to 30% by minimizing

truck rolls
Complete fast, repeatable, automated TCP tests in less

than 5 minutes
Verify results with an intuitive graphical user interface

that is easy to use by technicians at all skill levels
Closes the testing gap! Solve the tough, costly problems

that you miss with RFC 2544 and Y.1564sam

JDSU RFC6349 Testing Methodologies

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

JDSU RFC6349 Testing Methodologies

Uploaded by

Copyright:

Available Formats

RFC6349 & Testing Methodologies

for Ethernet presented by JDSU

© 2012 JDS Uniphase Corporation | JDSU CONFIDENTIAL AND PROPRIETARY INFORMATION 2

The Datalink layer relies on layer 2 overhead for physical

6 Bytes 6 Bytes 2 Bytes 46 - 1500 Bytes 4 Bytes

© 2012 JDS Uniphase Corporation | JDSU CONFIDENTIAL AND PROPRIETARY INFORMATION 4

TPI Priority CFI VLAN ID

Customers use VLAN tags for end to end circuit transparency

TPID CoS C VLAN Customer VLANs Switch

© 2012 JDS Uniphase Corporation | JDSU CONFIDENTIAL AND PROPRIETARY INFORMATION 6

Pre SFD DA SA T/L IPv4 UDP Data FCS IFGmin

Pre SFD DA SA T/L IPv4 TCP Data FCS IFGmin

© 2012 JDS Uniphase Corporation | JDSU CONFIDENTIAL AND PROPRIETARY INFORMATION 7

How do we test SLA compliance?

The goal is to prove that it is possible to obtain a constant throughput of

LAN WAN LAN

© 2012 JDS Uniphase Corporation http://www.ietf.org/rfc/rfc2544

The goal is to prove that it is possible to obtain a constant throughput of

LAN WAN LAN

© 2012 JDS Uniphase Corporation | JDSU CONFIDENTIAL AND PROPRIETARY INFORMATION 11

Frame Loss Rate

This goes in This comes out

This goes through constantly

© 2012 JDS Uniphase Corporation | JDSU CONFIDENTIAL AND PROPRIETARY INFORMATION 13

Store and forward

Ethernet tester inserts in each frame at time stamp.

© 2012 JDS Uniphase Corporation | JDSU CONFIDENTIAL AND PROPRIETARY INFORMATION 14

Which burst (time or number of frames) is forwarded without frame loss

TX switch = 20.000 f/s

© 2012 JDS Uniphase Corporation | JDSU CONFIDENTIAL AND PROPRIETARY INFORMATION 15

Ratified by IETF in March 1999

© 2012 JDS Uniphase Corporation | JDSU CONFIDENTIAL AND PROPRIETARY INFORMATION 16

Configurable test times

The Problems The Solution

© 2012 JDS Uniphase Corporation | JDSU CONFIDENTIAL AND PROPRIETARY INFORMATION 18

Ratified by ITU-T in March 2011

CPE 7609 7609 CPE

3550 7609 7609 3550

Access Edge Edge Access

© 2012 JDS Uniphase Corporation | JDSU CONFIDENTIAL AND PROPRIETARY INFORMATION 20

Two Phase Methodology

© 2012 JDS Uniphase Corporation | JDSU CONFIDENTIAL AND PROPRIETARY INFORMATION 21

Mainly an Ethernet Installation Tool

© 2012 JDS Uniphase Corporation | JDSU CONFIDENTIAL AND PROPRIETARY INFORMATION 22

CIR – Committed Information Rate

EIR – Exceeded Information Rate

MIR – Maximum Information Rate

© 2012 JDS Uniphase Corporation | JDSU CONFIDENTIAL AND PROPRIETARY INFORMATION 23

Frame Delay (FD) • Voice: over-talk, echo, dropped calls

Frame Delay • Voice: clicking and popping noises

Frame Loss Rate • Voice: clicks/fuzziness, dropped calls

© 2012 JDS Uniphase Corporation | JDSU CONFIDENTIAL AND PROPRIETARY INFORMATION 24

Phase 2 validates the quality of service of each defined service and

Mbps Transmitted traffic Mbps Measured traffic

© 2012 JDS Uniphase Corporation | JDSU CONFIDENTIAL AND PROPRIETARY INFORMATION 26

Y.1564 is needed in the following scenarios

Burst Testing is Optional

Does not define a One Way Delay Test

Not a good Network Shaping Test

No Zeroing in Algorithm like RFC 2544

© 2012 JDS Uniphase Corporation | JDSU CONFIDENTIAL AND PROPRIETARY INFORMATION 28

Time Saving Throughput Test Time Saving

Time Saving Troubleshooting