Ethernet Oam Tutorial Srinath Beldona

Ethernet OAM Tutorial
Srinath Beldona sbeldona@cisco.com

Technical Marketing Engineering, Edge Routing Business Unit
Yogesh Jiandani yogeshj@cisco.com
Consulting Systems Engineer, India and SAARC Region
Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential 1
HOUSEKEEPING
We value your feedback,

Please don’t forget to complete your online
session evaluations after each session
Please don’t forget to complete the Overall
Conference Evaluation which will be available
online
Please switch off your mobile phones!
Ethernet OAM Tutorial © 2006 Cisco Systems, Inc. All rights reserved. 2
Agenda
Review of Metro Ethernet technologies & Ethernet OAM
Manageability to the Customer Premises
Ethernet OAM Overview
Ethernet OAM Deployment Overview
802.1ag Connectivity Fault Management for Services
Management
802.3ah for Ethernet in the First Mile for Connectivity
Verification
Metro Ethernet Forum MEF-16 E-LMI for Rapid Service
Deployment
Ethernet OAM Interworking
Summary
Metro Ethernet – Deployment scenarios
Integrated System
Full Service Efficient Large Scale Intelligent Multi-service Intelligent Efficient Full Service
Customer Access Aggregation Edge Core Edge Access Customer
Equipment Equipment
Si
User Facing Provider Edge (U-PE) Metro C

Metro A U-PE
PE-AGG
Hub & 10/100/
10/100/ GE Ring Si
Spoke 1000 Mbps
1000 Mbps U-PE
P P
N-PE
MPLS/IP/TDM
10/100/
Metro B
1000 Mbps
N-PE
P P SONET/SDH
10 0 M
DWDM/
10
Ring
/1 bp
0
00 s
CDWM
/
N-PE
U-PE U-PE Metro D
Internet
Network Facing Provider Edge (N-PE)
Metro Ethernet—Service Descriptions
C
Ethernet Wire Service—EWS
SP
(Like a Leased Line) A Network
C
Ethernet Relay Service—ERS SP
(Like Frame Relay) A Network
Ethernet Multipoint Service—EMS SP

(Enhanced with VPLS, Port Mapping) Network
Ethernet Relay Multipoint Service—ERMS SP

(New service where the SP cloud acts like Network
a LAN, VLAN Mapping)
IETF Terminology for Metro Ethernet
Core
Core Device
Device (P)
(P)
Frame
Frame Forwarding,
Forwarding, Congestion
Congestion Management
Management
P
Emulated
VC Emulated
Emulated VC
VC Endpoint
Endpoint (Network
(Network PE,
PE, N -PE) **
N-PE) specific for
(Pseudowire) MPLS/L2TPv3
MPLS/L2TPv3 pseudowire
pseudowire encapsulation,
encapsulation, L2 Interworking, Ethernet:
L2 Interworking,
IP
IP Service
Service Integration,
Integration, Congestion
Congestion Management,
Management, L3VPN
L3VPN VPLS
N-PE Interconnection
Interconnection (for
(for PEs
PEs supporting
supporting that
that function)
function) Bridging
Replaces
Replaces PE-POP
PE-POP
Extension Aggregation
Aggregation Device
Device (PE -AGG)
(PE-AGG)
Traffic
Traffic Aggregation
Aggregation and
and Congestion
Congestion Management
Management
VC PE-AGG
(Note:
(Note: PE-AGGs
PE-AGGs and
and Extension
Extension VC’s
VC’s are
are formally
formally identified
identified in
in Switched
Switched Ethernet
Ethernet
Access
Access Domains,
Domains, although
although the
the functions
functions exist
exist in
in FR
FR and
and ATM
ATM networks
networks as
as well.)
well.)
U-PE Attachment
Attachment VC
VC UNI
UNI Endpoint
Endpoint (User
(User PE,
PE, U -PE) **
U-PE)
Admission
Admission Control,
Control, Policy
Policy Enforcement,
Enforcement, Classification,
Classification, specific for
Attachment Policing
Policing && Marking,
Marking, Congestion
Congestion management,
management, SLA
SLA Ethernet:
VC monitoring
monitoring and
and reporting,
reporting, VC
VC Mapping
Mapping to
to L1
L1 Channel,
Channel, Bridging
CE VC-ID translation, Replaces U-PE
VC-ID translation, Replaces U-PE
Note: These different roles can be collapsed within a single box
* For More Information refer to http://www.ietf.org/internet-drafts/draft-ietf-ppvpn-l2-framework-03.txt
Review of OAM
OAM &P: The Concept
Operations, Administration, Maintenance & Provisioning:
− fault indication − performance monitoring
− security management − diagnostic functions
− configuration & service provisioning
OAM covers both NÅÆ S and WÅÆE interfaces
Management Plane
N ÅÆ S
(NMS / EMS)
Network Plane
(NEs)
W ÅÆ E
OAM &P: The Concept
Operations, Administration, Maintenance & Provisioning:
− fault indication − performance monitoring
− security management − diagnostic functions
− configuration & service provisioning
OAM covers both NÅÆ S and WÅÆE interfaces
Management Plane
N ÅÆ S
(NMS / EMS)
Primary Focus of Ethernet OAM protocols is on W ÅÆ E interactions

(across NEs)
Network Plane
(NEs)
W ÅÆ E
OAM Layering
Operator A Operator B
Edge Router
Edge Bridge
ATM Switch
ATM Switch
Customer
Router
Router
Router
Bridge
Bridge
Bridge
switch
switch
Customer
Bridge
Bridge
Ethernet Ethernet 10G ATM
over SONET over MPLS Ethernet RFC1483
Service Layer Service OAM
Network Layer Network Network

OAM OAM
Ethernet
Transport Layer SONET MPLS Link ATM
OAM OAM OAM OAM
Model is iterative (layer stacking) & relative (service layer for

Operator is transport layer for SP)
Each Layer supports its own OAM mechanisms
Inter-working across and within OAM layers is possible
Manageability to the
Customer Premise
Carrier Ethernet Attributes
Platform and Technology Convergence
Cisco Carrier Ethernet Final Attribute to
Enable Carrier
Rich Ethernet Transport Grade Ethernet
Services Flexibility
Scalability H-VPLS Services
QoS MPLS
Network OAM
Resiliency Service Level
Assurance
Accelerate Migration to Carrier Ethernet
Ethernet OAM Provides Manageability to Customer Premise

Increased Business Confidence in Ethernet Service Level Agreements
Granular Bandwidth, Increased Transport Efficiency, Lower Costs
What Is Service Assurance?
New New
Detection Service Availability
• IEEE 802.1ag CFM for Ethernet Local Management
End-to-end service Interface (E-LMI)
management over any Ethernet OAM
• Rapid Start service
infrastructure turn-up
• IEEE 802.3ah for • Manageability to
Ethernet First Mile Customer Premise
Link Monitoring MPLS OAM with CPE
System and Control Infrastructure
Resiliency
• Non-Stop Forwarding (NSF)
• Stateful Switchover (SSO)
• Control Plane Failure with Zero Packet Loss
• Pseudowire Redundancy for L2 Failover
• MPLS Link Protection
Ethernet OAM: Service Availability
Rapid Start Service Turn Up—Reduce Truck Rolls
Rapid Start -- Service Turn-Up
Technician installs CPE Router
CPE communicates to UPE “I’m Alive”
UPE responds with available Ethernet connection services (EVCs)
CPE confirms connection to available EVCs
Dramatic time-to-install reduction
Router
Ethernet Switch
Standards-based E-LMI (MEF16)
U-PE
CPE
Service Assurance Detection
IEEE 802.3ah Link Layer OAM
802.3ah 802.3ah
Customer MPLS SONET/SDH Customer

Native
Equipment Ethernet Equipment
PW
MPLS
Standardized as IEEE 802.3ah Ethernet in the First Mile (EFM)

Maintain consistency of an Ethernet transport connection
(per link, or “physical” OAM)
Addresses three key operational issues when deploying Ethernet
across geographically disparate locations
Link monitoring, fault signaling, remote loopback, and OAM discovery
Operates on a single point-to-point link between two devices
Slow protocol using packets called OAMPDUs which are never
forwarded
Service Assurance: Detection
IEEE 802.1ag Connectivity Fault Management (CFM)
CPE
PE-Agg NPE NPE PE-Agg
CPE
Core
CPE
UPE UPE CPE
E-LMI
L2
E-LMI
Service
802.3ah 802.3ah 802.3ah 802.3ah 802.3ah 802.3ah
Link
IEEE 802.1ag “L1”
Service
IEEE 802.1ag (CFM) uses Ethernet frames, not MPLS, ATM, or SONET
frames, cells, or sub-Ethernet control information
All devices must support these Ethernet frames to allow service
provider to verify end-to-end service connectivity over any IP, MPLS,
or VPLS infrastructure
End-to-End Manageability
Customer Loyalty: Service Level Assurance
CPE
NPE
PE-Agg NPE PE-Agg
CPE
CPE
Core
U-PE U-PE CPE
E-LMI E-LMI
Service
802.3ah 802.3ah 802.3ah 802.3ah 802.3ah 802.3ah
Link
802.1ag
Service
Carrier Ethernet Capabilities Benefits

IEEE 802.1ag: Fault Management per End-to-end Service Manageability
Service/VLAN
IEEE 802.3ah Link Protection and First Mile Physical Connectivity
Monitoring Verification
MEF 16 E-LMI for Customer Premise Pro-active Service Status and
Equipment
Ethernet OAM Tutorial Availability
© 2006 Cisco Systems, Inc. All rights reserved. 17
Standards Landscape
Standard Organization Ethernet OAM
[WÅÆE, NÅ ÆS]
Link OAM [E<>W]
IEEE 802.3ah
Fault Mgmt [E<>W]
IEEE 802.1ag
MIBS
IEEE 802.3 + 802.1
EMS/NMS [N<>S]
EMS-NMS Info Model
Service OAM [E<>W]
Performance Mgmt
Ethernet OAM [E<>W]
Y.1731
Ethernet Protection
G.8031
FCAPS
SG4
L2VPN OAM Req. & Framework
[E<>W]
L2VPN WG
Carrier Ethernet
Ethernet OAM Standards Summary
Standards
Ethernet Services Architecture/Control Ethernet OAM* Ethernet
Body
802.3: MAC
802.3ar: Congestion Management
IEEE 802.1D/Q: Bridges/VLAN
802.17: RPR 802.3ah: EFM
802.3: PHYs
802.1ad: Provider Bridges 802.1ag: CFM
802.3as:
.1ah: Provider Backbone Bridges 802.1AB: Discovery Frame
Expansion
.1ak: Multiple Registration Protocol 802.1ap: VLAN MIB
.1aj: Two Port MAC Relay
.1AE/af: MAC/Key Security
.1aq: Shortest Path Bridging
MEF 10: Service Attributes

MEF 3: Circuit Emulation MEF 4: Generic Architecture
MEF MEF 13: UNI
MEF 6: Service Definition MEF 2: Protection Req and Framework Type 1
MEF 16: E-LMI
MEF 8: PDH Emulation MEF 11: UNI Req and Framework E-NNI
MEF 9: Service Certification MEF 12: Layer Architecture
MEF 14: Traffic Profiles
G.8011: Services Framework G.8010: Layer Architecture Y.1730: Ethernet OAM Req
ITU
G.8011.1: EPL Service G.8021: Equipment Model Y.1731: OAM Mechanisms G.8012:
UNI/NNI
G.8011.2: EVPL Service G.8010v2: Layer Architecture G.8031: Protection
G.8012v2:
G.asm: Service Mgmt Arch G.8021v2: Equipment Model Y.17ethqos: QoS UNI/NNI
G.smc: Service Mgmt Chnl Y.17ethmpls: ETH-MPLS Interwork Y.ethperf: Performance
Overview of
Ethernet OAM
Drivers for Ethernet OAM
Management Complexity
Large Span Networks
Multiple constituent networks belong to disparate
organizations/companies
IP overlay or plug-n-play no longer valid options!
Operational Efficiency
Reduce Opex, avoid truck-roll
Downtime cost
OAM benchmarks set by TDM and existing WAN
technologies
The Promise of Ethernet OAM
Simplify deployment of Ethernet WAN services
particularly important as Ethernet moves to a volume market with less
technically sophisticated customers
Enable monitoring & verification of SLAs

Not all customers and all services are equal.
SLAs will be a key differentiator for carriers and service providers.
Provide End-to-End OAM

Not only within the franchise of a single carrier
Eliminate finger-pointing between carriers
Facilitate deployment of Ethernet services over

multiple access technologies
Service independence from access and transport
Deliver SP’s customers packaged plug and play

solutions
Ethernet OAM: the Protocol Building Blocks
31
Se
I Y.17
E-LM
rvi
ce
La
yer
Ne ag
two
rk
La
802.1
yer
ah
Tra
ns 802.3
po
rt L
aye
r
IEEE 802.1ag: Connectivity Fault Management (CFM)

ITU-T Y.1731: OAM functions and mechanisms for
Ethernet based networks
IEEE 802.3ah: Ethernet Link OAM (EFM OAM)
MEF E-LMI: Ethernet Local Management Interface
802.1ag Connectivity Fault Management (CFM)
Customer Service Provider
Ethernet MPLS Core MPLS

Access Access
CE 1 uPE B PE-Agg B nPE B nPE A uPE A CE 2

Customer
Domain
Service Provider
Domain
Operator Domain
End to End per EVC OAM
Hierarchical Maintenance Domains
MEPs/MIPs
Standard Ethernet Frames (in-band)
Continuity Check
Loopback
Link Trace
Y.1731
CFM plus…
ETH-LCK (out of service diagnostics)
Multicast Loopback
AIS
TEST
Maintenance Communication Channel
Experimental OAM
Performance Measurement (Delay, Packet loss …)
802.3ah

Access Access
CE 1 CE 2
Link Layer
OAM
802.3ah 802.3ah 802.3ah
Link Level OAM

Operates on point-to-point link, not propagated beyond a single hop.
Slow Protocol (Max rate of 10 frames per second)
Functions:
OAM discovery – Discover OAM capabilities on peer device
Link monitoring – Event notification when error thresholds exceeded
Remote MIB Variable Retrieval – Polling and response (but not writing) of 802.3ah MIB
Remote Failure indication – Inform peer that receive path is down.
Remote Loopback – Puts peer in (near-end) intrusive loopback state. Statistics can be
collected while testing link.
E-LMI

Access Access
CE 1 uPE B uPE A CE 2
Service
Layer OAM
E-LMI E-LMI
Asymmetric protocol, applicable on UNI only (uPE to CE)
Specifies procedures & message formats exchanged and NOT how
uPE collects OAM data – relies on Service/Network OAM running
uPE to uPE
Allows uPE to communicate to CE:
EVC Status
Remote UNI Status
CE-VLAN to EVC Map
BW Profiles
802.1ag
Connectivity
Fault Management
Why do Service Providers want
Connectivity Fault Management?
Debugging Layer 2 networks is not easy in
the Enterprise space.
Debugging networks of Layer 2 networks
promises to be even harder.
Especially when the component networks
belong to different organizations and/or
different companies.
Experience in providing circuit-based
services provides a very useful model for
managing and debugging nets of nets.
What is Ethernet
Connectivity Fault Management?
Connectivity Fault Management (CFM) is an
important part of “End-to-End OAM”.
CFM is Ethernet frames, not MPLS, ATM, or
SONET frames, cells, or sub-Ethernet control
information.
Only bridges see CFM because bridges are
the only active relay functions that exist.
MPLS, ATM, and SONET OAM are important
for debugging Ethernet “wires”, but are not
always end-to-end.
(Terminology issue) There is no such
thing as a “Metro Ethernet Edge Router”
If interoperability is to be achieved among the various
technologies used for Metro Ethernet, the IEEE 802
LAN architecture must be observed.
There are only two kinds of active relay elements in
IEEE 802: Bridges and Hubs (Repeaters).
A “Metro Ethernet Edge Router” is a Bridge that does
not need to run spanning tree, has only two ports per
VLAN enabled (one trunk and one local Ethernet), and
uses Pseudowires for trunks.
(This notion saves a lot of verbiage in this document.
Whether a box is a router or a bridge depends on
which function you are looking at.)
Maintenance and other OAM issues not
discussed
Minimal discussion of Provider-to-Customer
(single-link) IEEE 802.3ah OAM.
No explanation of MPLS, ATM, or other OAM.
No explanation of other techniques such as
periodic confirmation of network topology
and configuration, SNMP-based “traceroute”,
or Layer 3 functions such as Ping.
No explanation of other protocols such as
Ethernet Line Management Interface, BPDUs,
etc.
Two standards bodies are defining CFM
or End-to-End OAM
IEEE 802.1ad defines Q-in-Q “Provider Bridges”. IEEE
802.1ag defines “Connectivity Fault Management” for all
IEEE 802 Bridges.
ITU-T Question 3, Study Group 13, is defining end-to-end
Ethernet OAM for both circuit-switched equipment (e.g.
Ether-over-SONET) and packet-switched equipment
(802.1ad Bridges).
Both 802.1 and Q.3/13 share common membership and
are cooperating fully.
802.1 should define low-level aspects tied closely to bridging
technology.
Q.3/13 should define high-level aspects tied to the service models.
Domains,
Maintenance Levels,
and
Flow Points
The OAM Environment
Customer Service Provider Customer
Ethernet MPLS Ethernet

Access Core Access
CE CE
Customer
Domain
Provider
Domain
Operator Operator Operator
Domain Domain Domain
Customer contracts with Provider for end-to-end service.

Provider contracts with Operator(s) to provide equipment and
networks.
Provider and Operator(s) may or may not be the same company
or same division.
Flow Points and Domains
Domains are defined in terms of “flow points”, which are
“MACs” to IEEE 802, and “interfaces” or “ports” to others.
A flow point (FP) at the edge of a Domain is called a
“Maintenance Point” or MP.
System administrators use MPs to initiate and monitor CFM
activity and report the results.
A flow point inside a Domain, and visible to an MP, is
called a “Loopback Point”.
Loopback Points (LPs) passively receive and respond to CFM
packets initiated by MPs.
Domains
OK
NO
Domains may nest or touch, but must never intersect.
Domains
Maintenance Points Loopback Points

Maintenance Points are always at the edges of Domains.
Loopback Points are always within Domains.
Maintenance Levels
Customer Operator A Operator B Customer
Equipment Bridges Bridges Equipment
Customer Level
Provider Level
Operator Level
Physical (or Layer 3 carrier) Level
At (at least) Operator and Physical levels, there may be

multiple Domains.
There could be multiple Domains at any level.
Lower CFM Levels
(Expanding red link in the previous slide)
Operator A Operator C Operator D Operator B
Bridge Routers Routers Bridge
“Physical” level of Connectivity Fault Management

MPLS OAM Level
Layer 3 Operator Level
Physical Level
There are levels below CFM.

This is where MPLS OAM, ATM OAM, etc., live.
These OAM functions ought to, but do not necessarily, follow the
MP/LP/FP model.
Loopback Points, Maintenance Points
M L L M
M L L M
M L L L L M M L L L L L L M
M MM MM MM MM MM MM MM M
Each Level’s Maintenance Points are the next-higher-level’s

Maintenance or Loopback Points.
Loopback Points, Maintenance Points
M L L M
M L L M
M L L L L M M L L L L L L M
M MM MM MM MM MM MM MM M
Each Level’s Loopback Points are invisible to all higher

levels.
Loopback Points and Maintenance
Points: Operator Levels (x3)

Each Level’s Maintenance Points are the next-higher-
level’s Maintenance or Loopback Points.
levels.
Loopback Points and Maintenance
Points: Provider Level
Each Level’s Maintenance Points are the next-higher-

level’s Maintenance or Loopback Points.
levels.
Connectivity Fault Management
Overview
CFM Technology Primer
CFM implementation
Standards-Based Ethernet OAM —
Comprehensive Approach
MPLS OAM: VCCV, LSP Ping/Traceroute
Ethernet MPLS Ethernet

Access Core Access
CE CE
Customer 802.3ah Eth in First Mile:

Domain
When applicable, physical
E-LMI: Automated Provider connectivity mgmt
config of CE based on Domain between devices. Most
EVCs and bw profiles; Operator Operator Operator applicable to “first mile”
L2 connectivity mgmt
Domain Domain Domain
Standards:
ITU-T SG 13 and SG 15:

802.1ag Connectivity Fault Management: • Ethernet Layer Ntwk Arch
• Uses Domains to contain OAM flows and bound OAM responsibilities (G.8010 SG 15)
• Ethernet OAM Functionality
• Provides per EVC connectivity mgmt and fault isolation
(Y.17ethoam SG 13)
• Three types of packets: Continuity Check, L2 Ping, L2 Traceroute • Reqmts for OAM in Ethernet
based twk (Y.1730 – SG 13)
Connectivity and Performance Mgmt: IEEE:
• Per EVC measurement of availability/connectivity, frame loss ratio, • 802.3ah – Ethernet in First Mile
(Physical OAM);
latency, and jitter • 802.1ad – Provider Bridges
• Ethernet OAM functionality integrated with MPLS OAM functionality • 802.1ag – Connectivity Mgmt
(Per VLAN OAM)
MEF:
• E-LMI
The Basics
CFM uses standard Ethernet Frames.
CFM frames are distinguishable by Ether-Type (and MAC Address for
multicast messages).
CFM frames are sourced, terminated, processed and relayed by
Provider Bridges – only Bridges see CFM.
The Players
Customer contracts with Metro-E Provider for end-to-end Ethernet
service. Provider sub-contracts with Operator(s) to provide equipment
and networks.
Maintenance Domains & Levels
A “Management Space” owned and operated by a single organization (department).
CFM message exchanges and operations are per-domain.
A Domain has a Maintenance Level associated with it (value 0 to 7). Two (or more) disjoint domains may be at the same Level, even
on the same device.
Domains have a hierarchical relationship that parallels the structure of the “Players”.
Maintenance Domains & Levels (continued…)

Domains may nest but cannot intersect.
OK
NO
As of this writing, higher Maintenance Levels are more “Phyward”, lower

Maintenance Level are more “Custward”.
Maintenance Points
A Maintenance Point (MP) is a demarcation point on an interface (port) that
participates in CFM within a Maintenance Domain.
Two classes of MPs:
Maintenance End Points (MEPs): At the edge of a Domain. Actively
source CFM messages. Are directional (inward or outward facing).
Maintenance Intermediate Points (MIPs): Internal to a Domain. Passive
points, only respond when triggered by certain CFM messages.
MPs should be explicitly provisioned on Cisco IOS devices.
The level of a MIP (or level 7 MEP) defines the port level.
MEP MIP MIP MEP
MEP MIP MIP MEP Customer

Level
Outward
Facing
MEP MIPMIP MEP MEP MIP MIP MIP MIP MEP
MIP MIP MIP MIP

Inward Provider
Facing Level
Operator Operator
Level Level
CFM Messages
All CFM Messages are per Maintenance Domain and per S-VLAN (PE-VLAN or
Provider-VLAN).
Regular Ethernet Frames, distinguishable by Destination MAC and/or
Ethertype.
Provider Bridges that cannot interpret CFM Messages must forward them as
normal data frames.
Four types of Messages:
Continuity Check (CC)
Loopback
Traceroute
Alarm Indication Signal (AIS)
Continuity Check (CC)
Per-Domain & per-VLAN Multicast “heart-beat” messages. Destination address
format:
well-known Maintenance Level (0 to 7)
45 bits
3 bits
Transmitted at a configurable periodic interval by MEPs (default is 30 seconds,
range from 10 sec to 65535 sec.)
Catalogued by MIPs at the same Maintenance Level.
Terminated by remote MEPs at the same Maintenance Level.
Uni-directional and do not solicit a response
Continuity Check (continued …)

catalogue
catalogue catalogue & terminate
MEP MIP MIP MEP
Loopback
Unicast frame, source must be a MEP, destination may be a MEP or a MIP.
Reply is unicast too.
Generated on-demand via MIB or CLI.
Loopback is specific to a Maintenance Domain and a VLAN.
Timestamps embedded in Loopback Message can be used to measure Round-
Trip Delay and One-Way Jitter.
Loopback (continued…)
Customer src Operator A dest Operator B Customer

MEP MIP MIP MEP
Traceroute
Multicast message, source must be a MEP, destination must be a MEP too.
Reply is a unicast.
Generated on-demand via MIB or CLI.
Traceroute is specific to a Maintenance Domain & a VLAN.
Allows the discovery of all MIPs belonging to the same Maintenance Domain
along the path to destination MEP.
On each visible hop indicate: Ingress Action, Relay Action, Egress Action.
Traceroute (continued …)
Customer src Operator A Operator B dest Customer

MEP MIP MIP MEP
SNMP Support (cont’d)
No standardization yet
Traps
Generated by MEPs
Two classes of traps:
CC
MEP Up/Down, Remote Port Status Change, CC Loop,
Configuration Error, Cross-connected Service
Crosscheck (provisioned state against running state)
Missing MEP, Unknown MEP, Service Operational
Sample Topology
sandiego-c3845 newyork-c2851
Gi0/1 Gi0/1
101.1.1.1 101.1.1.2
dallas-c7606 110.1.1.2
110.1.1.1
denver-3750me
4 4 sanjose-3750me
7 7
4 4 Gi3/23 4 Gi3/23 4 4
Fa1/0/1 Fa1/0/23 Gi3/48 Fa1/0/24 Fa1/0/1
4 Level 4 MIP Gi3/48

4 7
7 Level 7 MIP
miami-c1841
4
Gi3/2 Fa0/1
4 Level 4 MEP detroit-c7606
101.1.1.3
Configuring CFM
Network Provisioning
Done once at network bring-up or when devices are added or
removed.
Service Provisioning
Done every time a service is enabled on the network.
Configuring CFM
1. Configure Global parameters/attributes

Router (config)# ethernet cfm enable
2. Configure Global parameters/attributes

(Optional)
Router (config)# ethernet cfm traceroute cache [size entries | max-hold-time
minutes]
3. Configure Domains
Router (config)# ethernet cfm domain domain-name level level-id
Configuring CFM
4. Configure Operator Level MIPs

Router(config-if)# ethernet cfm mip level level-id
5. Configure per-Domain parameters

(optional)
Router (config-ether-cfm)# mep archive-hold-time minutes
Router (config)# ethernet cfm cc {level level-id | any} vlan any [interval
seconds] [loss-threshold num_msgs]
Sample Topology
ethernet
ethernet cfm cfm domain
domain CUSTOMER_DOMAIN
CUSTOMER_DOMAIN level level 77
ethernet
domain PROVIDER_DOMAIN
PROVIDER_DOMAIN level level 44
Gi0/1 Gi0/1
mep
mep archive-hold-time
archive-hold-time 60 60
101.1.1.1 101.1.1.2
ethernet cfm
dallas-c7606
ethernet cfm enable enable 110.1.1.2
110.1.1.1
ethernet cfm traceroute
ethernet cfm traceroute cache cache
denver-3750me
7 !! 4 4 sanjose-3750me
interface 7
4 Gi3/23
interface 4 FastEthernet1/0/1
Gi3/23
FastEthernet1/0/1 4
description connected 4
Fa1/0/1 Fa1/0/23 description
Gi3/48 connected to to sandiego-c3845
sandiego-c3845
Fa1/0/24
switchport
switchport trunk trunk encapsulation
encapsulation dot1q dot1q
Fa1/0/1
switchport
switchport mode mode trunktrunk
ethernet
ethernet cfm mip level
cfm mip level 77
!!
interface FastEthernet1/0/23
description 4 7
7 Level 7 MIP description connected connected to to dallas-c7606
dallas-c7606
switchport trunk
switchport trunk encapsulation encapsulation
4 dot1q
dot1q miami-c1841
switchport Gi3/2 Fa0/1
switchport mode mode trunktrunk
ethernet
cfm mip level 44
detroit-c7606
101.1.1.3
Configuring CFM
1. Configure Service IDs

Router (config-ether-cfm)# service csi-id vlan vlan-id
2. Configure Customer Level MIPs (optional)

Configuring CFM
3. Configure Service Provider Level MIPs

4. Configure MEPs for all Maintenance Levels

Router(config-if)# ethernet cfm mep level {level-id} [inward]
mpid id vlan {vlan-id | any | vlan-id-vlan-id [, vlan-id-vlan-id]}
Configuring CFM
5. Configure per-service parameters (optional)

Router (config)# ethernet cfm cc {level level-id | any} vlan {vlan-id | vlan-
id-vlan-id [, vlan-id-vlan-id]} [interval seconds] [loss-threshold
num_msgs]
6. Enable Traps (optional)

Router (config)# snmp-server enable traps ethernet cfm cc [mep-up |
mep-down | config | loop | cross-connect]
Router (config)# snmp-server
enable traps ethernet cfm crosscheck [
mep-unknown | mep-missing | service-up]
Configuring CFM
7. Enable CC (all enabled by default)

Router (config)# ethernet cfm cc enable level {any | level-id | level-id-
level-id [, level-id-level-id]} vlan {vlan-id | any | vlan-id-vlan-id [,
vlan-id-vlan-id]}
8. Configure and Enable Crosscheck (optional)

Router (config)# ethernet cfm mep crosscheck start-delay delay
Router (config-ether-cfm)# mep crosscheck mpid id vlan vlan-id [mac mac-
address]
Router (# ethernet cfm mep crosscheck {enable | disable} level {level-id |
level-id-level-id [,level-id-level-id]}vlan {vlan-id | any | vlan-id-vlan-id [,vlan-
id-vlan-id]}
Sample Topology
ethernet
CUSTOMER_DOMAIN level level 77
ethernet
Gi0/1 Gi0/1
mep
101.1.1.1 101.1.1.2
service
service customer_101_provider vlan
customer_101_provider
dallas-c7606
vlan 101 101 110.1.1.2
110.1.1.1
service
service customer_110_provider
customer_110_provider vlan
vlan 110 110
denver-3750me
4
ethernet cfm 4
enable sanjose-3750me
7 ethernet cfm enable 7
4 4 ethernet
Gi3/23
ethernet 4 cfm traceroute
traceroute cache
cfmGi3/23 4
cache
!! 4
Fa1/0/1 Fa1/0/23 Gi3/48 Fa1/0/24
interface
FastEthernet1/0/1
Fa1/0/1
description
description connected connected to to sandiego-c3845
sandiego-c3845
switchport
switchport trunk encapsulation dot1q
trunk encapsulation dot1q
switchport mode
switchport modeGi3/48 trunk
trunk
4 Level 4 MIP
ethernet
cfm mip level 77
ethernet 4 7
7 Level 7 MIP ethernet cfm cfm mepmep level
level 44 mpid mpid 1110
1110 vlanvlan 110110
ethernet cfm mep
ethernet cfm mep level4 4Gi3/2level 4 mpid 1101
mpid 1101 vlan 101
miami-c1841
vlan 101
!! Fa0/1
4 Level 4 MEP interface
detroit-c7606
FastEthernet1/0/23
description
description connected connected to to dallas-c7606
dallas-c7606101.1.1.3
switchport
switchport trunk trunk encapsulation
encapsulation dot1q dot1q
switchport mode
switchport mode trunk trunk
ethernet
ethernet cfm cfm mipmip level
level 44
!!
ethernet
ethernet cfm
cfm cc
cc enable
enable level
level 0-7
0-7 vlan
vlan 1-4095
1-4095
Sample Topology
ethernet cfm domain CUSTOMER_DOMAIN level 7
sandiego-c3845 ethernet cfm domain CUSTOMER_DOMAIN level 7
ethernet cfm domain PROVIDER_DOMAIN newyork-c2851
level 4
ethernet cfm domain PROVIDER_DOMAIN level 4
service customer_101_provider vlan 101
Gi0/1 service customer_110_provider vlan 110
service customer_110_providerGi0/1 vlan 110
ethernet cfm enable
ethernet cfm enable 101.1.1.2
101.1.1.1 !
dallas-c7606 ! 110.1.1.2
110.1.1.1 interface GigabitEthernet3/23
interface GigabitEthernet3/23
description connected to denver-c3750me
denver-3750me description connected to denver-c3750me
4 4 switchport sanjose-3750me
7 switchport
switchport mode trunk 7
switchport mode trunk
4 4 Gi3/23 4 Gi3/23 ethernet 4 cfm mip level 4
ethernet cfm mip level 44
!
Fa1/0/1 Fa1/0/23 Gi3/48 !
interfaceFa1/0/24
GigabitEthernet3/24
Fa1/0/1
description connected to sanjose-c3750me
description connected to sanjose-c3750me
switchport
switchport
!
4 Level 4 MIP Gi3/48!
4description
7
description
connected to detroit-c7606
connected to detroit-c7606
7 Level 7 MIP switchport
switchport miami-c1841
4
switchport
switchport
mode trunk
mode trunk
ethernet Gi3/2 Fa0/1
cfm mip level 4
4 Level 4 MEP !
detroit-c7606
!
ethernet cfm cc enable level 0-7 vlan 1-4095
ethernet cfm cc enable level 0-7 vlan 1-4095
101.1.1.3
Sample Topology
ethernet
ethernet cfm
sandiego-c3845cfm domain
CUSTOMER_DOMAIN level
level 77 newyork-c2851
ethernet
cfm domain PROVIDER_DOMAIN level
mep
mep archive-hold-time 60
archive-hold-time
Gi0/1 60
service Gi0/1
customer_101_provider vlan vlan 101
101
service 101.1.1.2
101.1.1.1 customer_110_provider vlan vlan 110
110 dallas-c7606 110.1.1.2
110.1.1.1
ethernet
ethernet cfm cfm enable
enable
ethernet
ethernet cfm cfm traceroute
denver-3750mecache
traceroute cache 4 4 sanjose-3750me
!! 7 7
interface
4
FastEthernet1/0/1 4 Gi3/23 4 Gi3/23 4
description connected to 4
description Fa1/0/1
to newyork-c3851
connected Fa1/0/23newyork-c3851 Gi3/48
switchport Fa1/0/24
switchport trunktrunk encapsulation
encapsulation dot1q
dot1q Fa1/0/1
ethernet
level 77
ethernet
ethernet cfm cfm mepmep level
level 44 mpid
mpid 3101
3101 vlan
vlan 101
101
ethernet cfm mep level 4 mpid 3110 vlan 110
4 Levelcfm
ethernet 4 MIP mep level 4 mpid 3110 vlan 110 Gi3/48
!!
interface FastEthernet1/0/24 4 7
interface
7 Level FastEthernet1/0/24
7 MIP
description
description connected
connected to to dallas-c7606
dallas-c7606 4 miami-c1841
switchport trunk encapsulation dot1q Gi3/2 Fa0/1
switchport trunk encapsulation dot1q
4 Level 4mode
switchport
switchport MEPtrunk
mode trunk detroit-c7606
ethernet
level 44 101.1.1.3
!!
ethernet
ethernet cfm cfm cccc enable
enable level
level 0-7
0-7 vlan
vlan 1-4095
1-4095
Sample Topology
ethernet
ethernet cfm
sandiego-c3845cfm domain
CUSTOMER_DOMAIN level
level 77 newyork-c2851
ethernet
cfm domain PROVIDER_DOMAIN level
mep
mep archive-hold-time 60
archive-hold-time
Gi0/1 60
service Gi0/1
customer_101_provider vlan vlan 101
101
service 101.1.1.2
101.1.1.1 customer_110_provider vlan vlan 110
110 dallas-c7606 110.1.1.2
110.1.1.1
ethernet
ethernet cfm cfm enable
enable
ethernet
ethernet cfm cfm traceroute
denver-3750mecache
traceroute cache 4 4 sanjose-3750me
!! 7 7
interface
4
GigabitEthernet3/2 4 Gi3/23 4 Gi3/23 4
description connected to 4
description Fa1/0/1
to miami-c1841
connected Fa1/0/23miami-c1841 Gi3/48
switchport Fa1/0/24 Fa1/0/1
switchport
ethernet
level 77
ethernet
ethernet cfm cfm mepmep level
level 44 mpid
mpid 4101
4101 vlan
vlan 101
101
!!
interface GigabitEthernet3/48 4
description
description connected
connected to
to dallas-c7606
dallas-c7606
7
7 Level
switchport 7 MIP
miami-c1841
switchport
switchport mode trunk 4
switchport mode trunk Gi3/2 Fa0/1
4 Levelcfm
ethernet
ethernet 4 MEP
cfm mip
mip level
level 44 detroit-c7606
!!
ethernet 101.1.1.3
enable level
level 0-7
0-7 vlan
vlan 1-4095
1-4095
Show Commands
Show Local Maintenance Points
denver-c3750me#show ethernet cfm maintenance-points local
MPID DomainName Level Type VLAN Port CC-Status MAC
1101 PROVIDER_DOMAIN 4 MEP 101 Fa1/0/1 Enabled
0013.c33c.5680
1110 PROVIDER_DOMAIN 4 MEP 110 Fa1/0/1 Enabled
0013.c33c.5680
Level Type Port MAC
7 MIP Fa1/0/1 0013.c33c.5680
4 MIP Fa1/0/23 0013.c33c.5680
dallas-c7606#show ethernet cfm maintenance-points local

Level Type Port MAC
4 MIP Gi3/48 0015.6215.3e85
4 MIP Gi3/24 0015.6215.3e85
4 MIP Gi3/23 0015.6215.3e85
Show Commands
Show Local Maintenance Points: CC-Status
Possible values for Local MEP CC-Status:

Enabled
Disabled
Inactive
1101 PROVIDER_DOMAIN 4 MEP 101 Fa1/0/1 Enabled 0013.c33c.5680
Level Type Port MAC

7 MIP Fa1/0/1 0013.c33c.5680
4 MIP Fa1/0/23 0013.c33c.5680
Show Commands
Show Local Maintenance Points: CC-Status
denver-c3750me(config)#ethernet cfm cc enable level 4 vlan 101

1110 PROVIDER_DOMAIN 4 MEP 110 Fa1/0/1 Disabled 0013.c33c.5680
Level Type Port MAC

7 MIP Fa1/0/1 0013.c33c.5680
4 MIP Fa1/0/23 0013.c33c.5680
detroit-c7606(config)#interface GigabitEthernet3/2
detroit-c7606(config-if)#switchport trunk allowed vlan 200
detroit-c7606#show ethernet cfm maintenance-points local
4101 PROVIDER_DOMAIN 4 MEP 101 Gi3/2 Inactive 0015.6215.46c5
Level Type Port MAC
7 MIP Gi3/2 0015.6215.46c5
4 MIP Gi3/48 0015.6215.46c5
Show Commands
Show Domains
denver-c3750me#show ethernet cfm domain
Domain Name : CUSTOMER_DOMAIN

Index : 1
Level : 7
Total Services : 0
Domain Name : PROVIDER_DOMAIN
Index : 2
Level : 4
Total Services : 2
Services:
VLAN Crosscheck ServiceID
101 Disabled customer_101_provider
110 Disabled customer_110_provider
Show Commands
Show Remote Maintenance Points
denver-c3750me#show ethernet cfm maintenance-points remote

MPID Level Mac Address Vlan PortState InGressPort Age(sec) Service ID
3101 4 0015.637b.4e00 101 UP Fa1/0/23 19 customer_101_provider
4101 4 0015.6215.46c5 101 UP Fa1/0/23 6 customer_101_provider
3110 4 0015.637b.4e00 110 UP Fa1/0/23 6 customer_110_provider
dallas-c7606#show ethernet cfm maintenance-points remote

MPID Level Mac Address Vlan PortState InGressPort Age(sec) Service ID
1101 4 0013.c33c.5680 101 UP Gi3/23 12 customer_101_provider
3101 4 0015.637b.4e00 101 UP Gi3/24 25 customer_101_provider
4101 4 0015.6215.46c5 101 UP Gi3/48 2 customer_101_provider
1110 4 0013.c33c.5680 110 UP Gi3/23 3 customer_110_provider
3110 4 0015.637b.4e00 110 UP Gi3/24 12 customer_110_provider
Show Commands
Other CFM Show Commands
denver-c3750me#show ethernet cfm traceroute-cache

Traceroute to 0015.637b.4e00 on Domain PROVIDER_DOMAIN, Level 4, vlan 101 issued at 1d04h
--------------------------------------------------------------------------------
MAC Ingress Ingress Action Relay Action
Hops Host Forwarded Egress Egress Action Next Hop
--------------------------------------------------------------------------------
B 1 dallas-c7606 0015.6215.3e85 Gi3/23 IngOk RlyCCDB
Forwarded Gi3/24 EgrOK sanjose-c3750me
! 2 sanjose-c3750me 0015.637b.4e00 Fa1/0/2 IngOk RlyNone
Not Forwarded
Clear Commands
denver-c3750me#clear ethernet cfm ?

errors CFM errors
maintenance-points CFM Maintenance Points
traceroute-cache Clear Contents of traceroute cache
CFM Crosscheck
Allows configuring static list of expected remote MEPs

per service, and then crosschecks this list against what
is learnt dynamically from CC messages. Generates
appropriate alarms when errors are detected.
Configure and Enable Crosscheck:
ethernet cfm mep crosscheck start-delay delay
mep crosscheck mpid id vlan vlan-id [mac mac-
address]
ethernet cfm mep crosscheck {enable | disable} level
{level-id | level-id-level-id [,level-id-level-id]}vlan {vlan-id
| any | vlan-id-vlan-id [,vlan-id-vlan-id]}
CFM Debugs
Ethernet CFM Debugs:

debug ethernet cfm all [domain domain-name | level level-id]
[vlan vlan-id]
debug ethernet cfm diagnostic [events | packets]
debug ethernet cfm errors
debug ethernet cfm events [domain domain-name | level
level-id] [vlan vlan-id]
debug ethernet cfm packets [domain domain-name | level
level-id] [vlan vlan-id]
CFM Debugs
21:56:15: CFM-PKT: Received a cc packet from interface FastEthernet1/0/24
21:56:15: CFM-PKT: cfm packet dump - 104 bytes, interface FastEthernet1/0/24, vlan 110
21:56:15: CFM-PKT: ethernet CFM message dump,
dest: 0100.0ccc.ccc4
src: 0013.c33c.5680
Version: 0
Maintenance Level: 4
MsgType: CC(0)
Transaction Id: 3334414845
21:56:15: 01 00 0C CC CC C4 00 13 C3 3C 56 80 00 5A AA AA
21:56:15: 03 00 00 0C 01 26 00 04 00 08 C6 BF 21 FD 01 00
21:56:15: 26 00 63 75 73 74 6F 6D 65 72 5F 31 31 30 5F 70
21:56:15: 72 6F 76 69 64 65 72 3A 50 52 4F 56 49 44 45 52
21:56:15: 5F 44 4F 4D 41 49 4E 02 00 02 04 56 03 00 02 00
21:56:15: 4B 04 00 02 00 01 05 00 0E 64 65 6E 76 65 72 2D
21:56:15: 63 33 37 35 30 6D 65 00
21:56:15:
21:56:15: CFM-PKT: TLV Service ID 'customer_110_provider:PROVIDER_DOMAIN'
21:56:15: CFM-PKT: TLV MPID 1110
21:56:15: CFM-PKT: TLV LIFETIME 75
21:56:15: CFM-PKT: TLV PORT_STATE 0x1 (UP)
21:56:15: CFM-PKT: Received a CC packet with MPID 1110, level 4, vlan 110 from interface
FastEthernet1/0/24
Ethernet
21:56:15:OAM Tutorial
CFM-EVT: Found remote mep for level 4 svlan 110, mpid 1110
Sample Topology
Crosscheck
sandiego-c3845 ethernet newyork-c2851
mep
Gi0/1 service
service customer_101_provider vlan
customer_101_provider vlan 101
101 Gi0/1
101.1.1.1 service customer_110_provider vlan
dallas-c7606 vlan 110
110 101.1.1.2
110.1.1.1 mep crosscheck mpid 3101 vlan 101 mac 0015.637b.4e00110.1.1.2
mep crosscheck mpid 3101 vlan 101 mac 0015.637b.4e00
mep crosscheck mpid
mpid 4101
4101 vlan 101
denver-3750me mep crosscheck
4 crosscheck 4 mpid vlan 101 mac
mac 0015.6215.46c5
0015.6215.46c5
sanjose-3750me
7 mep 3110 vlan 110 mac
mep crosscheck mpid 3110 vlan 110 mac 0015.637b.4e00 0015.637b.4e00
!!Gi3/23 7
4 4 4 Gi3/23 4
ethernet
enable level
level 0-7
0-7 vlan
vlan 1-40954
1-4095
Fa1/0/1 Fa1/0/23 ethernet Gi3/48
cfm mep crosscheck start-delay
Fa1/0/24 60
ethernet cfm mep crosscheck start-delay 60 Fa1/0/1

4 7
7 Level 7 MIP
miami-c1841
4
Gi3/2 Fa0/1
101.1.1.3
CFM Crosscheck
denver-c3750me#ethernet cfm mep crosscheck enable level any vlan any
denver-c3750me#show ethernet cfm domain
Domain Name : CUSTOMER_DOMAIN
Index : 1
Level : 7
Total Services : 0
Domain Name : PROVIDER_DOMAIN

Index : 2
Level : 4
Total Services : 2
Services:
VLAN Crosscheck ServiceID
101 Enabled customer_101_provider
110 Enabled customer_110_provider
Crosscheck:
MPID Vlan Remote-Mac Mep-Up
3101 101 0015.637b.4e00 Yes
4101 101 0015.6215.46c5 Yes
3110 110 0015.637b.4e00 Yes
CFM Crosscheck
denver-c3750me#show ethernet cfm maintenance-points remote crosscheck

MPID Level VLAN Mep-Up Remote Mac
3101 4 101 Yes 0015.637b.4e00
4101 4 101 Yes 0015.6215.46c5
3110 4 110 Yes 0015.637b.4e00
Troubleshooting with CFM
Proactive
Traps give asynchronous fault notification
Use Loopback and Traceroute to verify
and isolate faults
Reactive
Use Loopback and Traceroute to verify
and isolate faults
Start with Highest-Level
Maintenance Domain
Troubleshooting Workflow
Jump to Lower
Check Bridge
Maintenance
Error Status
Domain
Shows Error
No
Confirms Problem Traceroute
Loopback Test to Failure Isolated?
Destination
Yes
Take Corrective
Fault Verification Fault Isolation Action
Stop
Commands
Loopback
ping ethernet mac-address {domain domain-name | level level-id} vlan vlan-id
Traceroute
traceroute ethernet {mac-address}{domain domain-name | level level-id} vlan
vlan-id
Gi0/1 Gi0/1
101.1.1.1 101.1.1.2
dallas-c7606 110.1.1.2
110.1.1.1
denver-3750me
4 4 sanjose-3750me
7 7
4 4 Gi3/23 4 Gi3/23 4 4
Fa1/0/1 Fa1/0/23 Gi3/48 Fa1/0/24 Fa1/0/1

4 7
7 Level 7 MIP
miami-c1841
4
Gi3/2 Fa0/1
101.1.1.3
SNMP Traps are generated when a remote MEP is not
heard from.
Received SNMPv2c Trap:

Community: TEST
From: 127.0.0.1
sysUpTimeInstance = 46593
snmpTrapOID.0 = cEtherCfmCcMepDown
cEtherCfmEventServiceId.1.3.4 = customer_110_provider <===
cEtherCfmEventLclMacAddress.1.3.4 = 00 15 63 7b 4e 00 <===
cEtherCfmEventLclMepCount.1.3.4 = 1 <==== this is the total # of local MEPs in this vlan
cEtherCfmEventLclIfCount.1.3.4 = 1 <=== total number of distinct interfaces that host the local MEPs
cEtherCfmEventRmtMepid.1.3.4 = 110 <====
cEtherCfmEventRmtMacAddress.1.3.4 = 00 15 63 7b 4e 00 <====
cEtherCfmEventCode.1.3.4 = 5
Show ethernet cfm errors

denver-c3750me#show ethernet cfm errors
Level Vlan MPID Remote MAC Reason Service ID
4 110 3110 0015.637b.4e00 Lifetime Timer Expir customer_110_provider
Traceroute shows location and reason for failure.

denver-c3750me#traceroute ethernet 0015.637b.4e00 level 4 vlan 101
Type escape sequence to abort. TTL 255. Per-Hop Timeout is 10 seconds
Tracing the route to 0015.637b.4e00 on Domain PROVIDER_DOMAIN, Level 4, vlan 101
Traceroute sent via FastEthernet1/0/23
--------------------------------------------------------------------------------
MAC Ingress Ingress Action Relay Action
Hops Host Forwarded Egress Egress Action Next Hop
--------------------------------------------------------------------------------
B 1 dallas-c7606 0015.6215.3e85 Gi3/23 IngOk RlyCCDB
Not Forwarded Gi3/24 EgrDown sanjose-c3750me
Other Errors Reported by CFM:
CC Lifetime Expiration
CC Lifetime Zero
(Remote MEP removed or disabled)
Loop Error
(Receiving a CC with the same source MAC and MPID as a local MEP)
Crossconnect Error
(Receiving a CC who’s CSI does not match the local configuration)
Configuration Error
(Receiving a CC with the same MPID as a local MEP)
MEP Missing
(Crosscheck: Did not receive a CC from a statically configured remote MEP)
Unknown MEP
(Crosscheck: Received a CC from a remote MEP that was not statically
configured)
Ethernet LMI
Ethernet Link Management Interface (E-LMI)
Ethernet Services Model & MEF

Terminology
Ethernet LMI Overview
Ethernet Services Model
User Network Interface User Network Interface

(UNI) (UNI)
Metro Ethernet Network

(MEN)
Customer Customer
Edge (CE) Edge (CE)
UNI: Point of Demarcation between CE and SP’s MEN
MEF view: MEN provides service to CE
MEF Terminology

(UNI) (UNI)
(MEN) CE-B
CE-A
Ethernet Virtual Ethernet Virtual

Circuit Circuit
(EVC) – PTP (EVC) – MPTMP
EVC - An association of two or more UNIs
- Connection between two or more devices
Note:
MEF defines services in terms of EVC,UNI CE-C 1. Unlike ATM VC, EVC is unique and spans across MEN
2. EVC = Service Instance in CFM(802.1ag) World
attributes
End-to-End Ethernet OAM
(UNI)
(UNI)
MEN
(Provider/Operator)
CE PE PE CE
802.3ah 802.3ah
MEF 802.1ag MEF

E-LMI CFM E-LMI
E-LMI : Ethernet Provisioning and Management entity across UNI (CE-PE)

802.1ag : Ethernet Service Layer Monitoring entity across MEN (PE-PE)
802.3ah : Link Layer Monitoring
Ether LMI Requirement
• Report Connectivity Status to CE

Status of EVC across MEN
Remote UNI state
• Auto Configure CE
Creation, deletion of EVCs, VLANs, Bandwidth
profile changes…
Ethernet Local Management Interface –
Deployment with Ethernet Core

(UNI) (UNI)

(MEN)
CE PE PE CE
Ethernet LMI Ethernet LMI
Ethernet LMI: Ethernet Provisioning and

Management entity across UNI (CE-PE)
Ethernet Local Management Interface –
Deployment with MPLS core

(UNI) (UNI)
MPLS
CE PE PE CE
Ethernet LMI Ethernet LMI
Ethernet LMI: Ethernet Provisioning and Management entity across

UNI (CE-PE)
Information Carried by Ethernet LMI
Configuration Data
CE-VLANs
Bandwidth Profile per EVC
Bandwidth Profile per UNI
Provisioned Data
EVC service Identifier
UNI Identifiers for EVC
UNI Type
Remote UNI Identifier for EVC
Dynamic Data (Learned via MEN OAM)

EVC status
Remote UNI state
Ethernet LMI Protocol
Based on Polling procedure invoked by CE
- Heartbeat messages every T391 seconds

- Will get all data from PE when change is notified /
CE
- needs data (startup/ N391)
- MEN may asynchronously update EVC state
Typical Flow
CE MEN
•UNI Id
Startup STATUS ENQ (Full Status Req)
•EVC Id
•CE-VLAN
Auto STATUS (Full Status Resp) •Bandwidth Profile
Config- •EVC Status
•……..
T391 STATUS ENQ (Ethernet LMI Check)

timeout STATUS (Ethernet LMI Check) Link Ok !!
Link OK
!!
STATUS (ASYNC) Only EVC Status
E-LMI Configurations
Ethernet LMI Configuration Commands
PE : More config involved
CE : Less config
Ethernet LMI Configuration - PE
1. EVC Configuration (Global)
EVC Id
UNI Count
EVC MEN OAM Protocol
2. UNI Configuration (Per Interface)

UNI Id
Service Instance (CE Vlans Config)
3. Enable Ethernet LMI (Global/Per Interface)

Enable Ethernet LMI
Modify Ethernet LMI default parameters (Optional)
Objective:
Gi0/1 Gi0/1
101.1.1.1 101.1.1.2
dallas-c7606 110.1.1.2
110.1.1.1
denver-3750me sanjose-3750me
Gi3/23 Gi3/23
Fa1/0/1 Fa1/0/23 Gi3/48 Fa1/0/24 Fa1/0/1
Multi-Point EVC: EVC_MP2MP_101

Point-to-Point EVC: EVC_P2P_110 Gi3/48
miami-c1841
Gi3/2 Fa0/1
detroit-c7606
101.1.1.3
•We will configure above EVCs on denver.

•This config needs to be repeated on detroit, sanjose
with appropriate parameters
Step 1: Configure EVC
Global Config a. EVC NAME
Multi Point EVC: b. Num of UNIs associated

with EVC
denver-c3750me(config)# ethernet evc EVC_MP2MP_101
denver-c3750me(config-evc)# uni count 3
denver-c3750me(config-evc)# oam protocol cfm svlan 101 domain PROVIDER_DOMAIN
denver-c3750me(config-evc)# end
c. MEN OAM CFM and its
params.
If MPLS-xconnect is used,
Point-Point EVC: use “oam protocol ldp”
denver-c3750me(config)# ethernet evc EVC_P2P_110
denver-c3750me(config-evc)# oam protocol cfm svlan 110 domain PROVIDER_DOMAIN
denver-c3750me(config-evc)# end
Note: UNI count is 2 by default. Hence no need for explicit configuration of this for P2P
EVC.
Step 2: Configure UNI
a. UNI service
UNI Config
denver-c3750me(config)#int e0/0
denver-c3750me(config-if)# ethernet uni bundle b. Uni Id
denver-c3750me(config-if)# ethernet uni id cisco_sandiego
denver-c3750me(config-if)# service instance 101 ethernet EVC_MP2MP_101
denver-c3750me(config-if-srv)# ethernet lmi ce-vlan map 101
denver-c3750me(config-if)# service instance 110 ethernet EVC_P2P_110
denver-c3750me(config-if-srv)# ethernet lmi ce-vlan map 110
denver-c3750me(config-if)#end
c. Map Vlans
used by UNI
for this EVC
Step 2: Contd..
Other possible options for UNI type
denver-c3750me(config-if)#ethernet uni ?
bundle bundling service without multiplexing
multiplex multiplexing service without bundling
Note: Default is - bundle-multiplex
Step 3: Enable Ethernet LMI
Global: Enables Ethernet LMI per Box

denver-c3750me(config)# ethernet lmi global
Interface: Enables Ethernet LMI per interface/UNI

denver-c3750me(config-if)# ethernet lmi interface
Step 3. (Cont…): Ethernet LMI Config – Modify
default params
PE:
denver-c3750me(config)#in f1/0/1
denver-c3750me(config-if)#ethernet lmi ?
interface Enable Ethernet LMI on this interface
n393 Error counter
t392 Polling verification timer
CE:
sandiego-c3845(config-if)#ethernet lmi ?
interface Enable Ethernet LMI on this interface
n391 Polling counter
n393 Error counter
t391 Polling timer
Ethernet LMI Configuration - CE
• Enable Ethernet LMI (Global/Per Interface)

• Enable Ethernet LMI
• Modify Ethernet LMI default parameters
(Optional)
Complete E-OAM configuration - PE
!
ethernet lmi global
ethernet cfm domain CUSTOMER_DOMAIN level 7
ethernet cfm enable
!
!
ethernet evc EVC_P2P_110
oam protocol cfm svlan 110 domain PROVIDER_DOMAIN
!
NOTE: Required DATA plane
ethernet evc EVC_MP2MP_101
uni count 3
configuration should be
oam protocol cfm svlan 101 domain PROVIDER_DOMAIN done as done today.
!
!
description connected to sandiego-c3845
service instance 101 ethernet EVC_MP2MP_101
ethernet lmi ce-vlan map 101
service instance 110 ethernet EVC_P2P_110
ethernet lmi ce-vlan map 110
ethernet uni id cisco_sandiego
!
Complete E-OAM configuration on CE
!
no ip address
logging event subif-link-status
negotiation auto
ethernet lmi interface NOTE: Required DATA plane
end configuration should be
done as done today.
EVC, Ethernet LMI show commands
Start with root words “show ethernet service”

“ether service” commands
denver-c3750me#show ethernet service ?
reflect
evc Ethernet EVC
what is configured.
instance Ethernet Service Instance
interface Ethernet Service Interface
Start with root words “show ethernet lmi”

pe#show ethernet lmi ? “ether lmi” commands
evc Ethernet Virtual Connection on PE reflect
parameters Ethernet LMI Parameters what is sent to CE
statistics Ethernet LMI statistics
uni UNI information
show ethernet service evc
denver-c3750me#show ethernet service evc

Identifier Type Act-UNI-cnt Status
EVC_P2P_110 P-P 2 Active
EVC_MP2MP_101 MP-MP 3 Active
show ethernet service evc detail
denver-c3750me#show ethernet service evc detail
EVC ID: EVC_P2P_110
EVC Type: P-P
UNI Count: Configured = 2, Active = 2
EVC Status: Active
Associated Local Interfaces:
Interface CE-Vlans
FastEthernet1/0/1 110
Associated UNIs:
UNI-Identifier Status Location
cisco_sandiego Up Fa1/0/1
cisco_newyork Up Remote
EVC ID: EVC_MP2MP_101

EVC Type: MP-MP
UNI Count: Configured = 3, Active = 3
EVC Status: Active
Associated Local Interfaces:
Interface CE-Vlans
FastEthernet1/0/1 101
Associated UNIs:
UNI-Identifier Status Location
cisco_sandiego Up Fa1/0/1
cisco_newyork Up Remote
802.3ah
Ethernet in the First Mile
Connectivity Verification
Agenda
Overview
Ethernet OAM Technology
Ethernet OAM Configuration
Ethernet OAM Commands
Ethernet OAM Implementation
802.3ah Overview
MPLS Core
Eth Access Eth Access
CE CE
802.3ah
Ethernet in
the First Mile
N-PE
CE U-PE
802.3ah OAMPDUs
802.3ah OAMPDUs
Operates on a single point-to-point link between 2 devices

Slow protocol using packets called OAMPDUs which are never forwarded
Provides 5 functions:
Remote failure indication – Inform peer device that receive path is down. Requires unidirectional operation
support.
Remote loopback – Puts device in a state such that all non-OAM packets are looped back. Normal forwarding is
suspended. Statistics can be collected while testing link.
Link monitoring – Event notification and link information. Also provides polling and response (but not writing) of
802.3ah MIB
OAM discovery – Discovers OAM support and capabilities on peer device
Mib Variable Retrieval
Overview
What is 802.3ah Ethernet OAM?

Point-to-point link-level OAM
Why it is needed?
Monitor the health of a link
Determine the location of failing links or fault
conditions
Complement applications that reside in higher layers
802.3ah Technology
802.3ah Technology
What does Ethernet OAM support?
Discovery :- Discovery is the first phase of Ethernet OAM. It identifies
the devices in the network along with their OAM capabilities.
Remote failure indication :- Ethernet OAM provides a mechanism
for an OAM entity to convey slow degradation of Ethernet link to its peer
via specific flags in the OAMPDU.
Link monitoring :- Link monitoring in Ethernet OAM serves for

detecting and indicating link faults under a variety of conditions.
Remote loopback :- An OAM entity can put its remote peer into
loopback mode using the Loopback control OAMPDU. This helps
the administrator ensure the quality of links during installation or
when troubleshooting.
Polling of MIB variables :- Ethernet OAM provides a read-
only access to remote MIB variables limited to a specific
MIB branch and leaf.
802.3ah Technology
The Basics
Ethernet OAM uses standard Ethernet frames,
called OAMPDUs.
OAMPDU uses IEEE Slow Protocol’s MAC
address (0180.c200.0002) and Ether-Type (8809)
but has a unique subtype (03).
OAMPDU is sourced and terminated at both ends
of a link. It is not relayed by bridges.
802.3ah Technology
OAMPDU format
Destination Source Ether- Sub- Data/Pad FCS
Address Type
Address type
6 octets 6 octets 2 1 45 ~ 1499 4 octets
octets octet octets
Destination Address 01-80-C2-00-00-02

(slow protocol address)
Ether-Type 88-09
Subtype 03
Ethernet 802.3ah Technology
OAMPDU types
Information OAMPDU
Event Notification OAMPDU
Variable Request OAMPDU
Variable Response OAMPDU
Loopback Control OAMPDU
Organization Specific OAMPDU
802.3ah Technology
OAM Discovery
Ethernet OAM and it identifies the devices in
network along with their capabilities.
1. Send Information OAMPDU in a periodic fashion
(once a second, by default).
2. Discover OAM configurations (capabilities, eg:-
LB), state (Active/Passive), OUI (Organizational
unique identifier), OAMPDU configuration (max.
oampdu size and rate) of remote OAM client.
3. Decide whether OAM clients can be fully
operational on the link.
802.3ah Technology
OAM Remote Failure Indication

Three types of remote failures
Link Fault – hardware detected fault that occurred in the
receive direction of the local equipment
Dying Gasp – an unrecoverable failure (eg:- power
failure).
Oam client administrative turned off
1. Interface error disabled.
2. reload
Critical Event
The definition of the specific faults is implementation
specific.
802.3ah Technology
OAM Link Monitoring

Four types of errors
Errored Symbol Period :- Number of symbol errors
(coding error) that occurred during a specified period
exceeded the threshold.
Errored Frame :- Number of frame errors detected during
a specified period exceeded a threshold.
Errored Frame Period :- Number of frame errors within
the last N frames has exceeded a threshold
Errored Frame Seconds :- Number of errored seconds
(one second intervals with at least one frame error)
within the last M seconds has exceeded a threshold.
Event Notification OAMPDU is sent to remote OAM client if
any of the above errors is detected locally.
802.3ah Technology
OAM Remote Loopback

Fault localization and link performance testing
Loopback Control OAMPDU is used to control a remote OAM
client.
Traffic sent from master loopback port is loopback by slave port,
except Pause and OAMPDU.
X
OAM OAM
X
MAC MAC
PHY PHY
Master OAM
Slave OAM
802.3ah Technology
OAM MIB variable retrieval

Variable Request OAMPDU and Variable
Response OAMPDU are used to fetch a MIB
variable.
Query variables can be implementation specific.
802.3ah Configuration
Ethernet OAM Interface Configuration
device(config-if)# Ethernet oam ?

link-monitor Ethernet OAM Link Monitor
max-rate Maximum transmission rate (number of OAMPDUs
per second)
min-rate Minimum transmission rate(one OAMPDU per
number of seconds)
mode Ethernet OAM client mode
remote-failure Ethernet OAM remote failure indication
remote-loopback Ethernet OAM Remote Loopback
timeout Ethernet OAM client link timeout setting
802.3ag Template configuration
template oam
ethernet oam link-monitor symbol-period threshold low 10
ethernet oam link-monitor symbol-period threshold high 100
ethernet oam link-monitor frame window 100
ethernet oam link-monitor frame threshold low 10
Ceasar# conf t
ethernet oam link-monitor frame threshold high 100
ethernet oam link-monitor frame-period window 100 Enter configuration commands, one per
line. End with CNTL/Z.
ethernet oam link-monitor frame-period threshold low 10
ethernet oam link-monitor frame-period threshold high 100 Ceasar(config)# int fas 1/25
ethernet oam link-monitor frame-seconds window 1000
Ceasar(config-if)# source template oam
ethernet oam link-monitor frame-seconds threshold low 10
ethernet oam link-monitor frame-seconds threshold high 100
ethernet oam link-monitor receive-crc window 100
ethernet oam link-monitor receive-crc threshold high 100
ethernet oam link-monitor transmit-crc window 100
ethernet oam link-monitor transmit-crc threshold high 100
ethernet oam remote-failure link-fault action error-disable-interface
ethernet oam remote-failure dying-gasp action error-disable-interface
ethernet oam remote-failure critical-event action error-disable-interface
!
802.3ah Configuration -Topology
Fa1/1 Fa4/1
Ceaser-7606 Fa1/2 Fa4/2
Fa4/3
fulliautomatix-7606
Fa1/3
Fa1/4 Fa4/4
!
! interface FastEthernet4/1
no ip address
interface FastEthernet1/1 ethernet oam
no ip address !
interface FastEthernet4/2
ethernet oam no ip address
! source template oam
interface FastEthernet1/2 ethernet oam
!
no ip address interface FastEthernet4/3
ethernet oam ip address 6.6.6.7 255.255.255.0
ethernet oam remote-loopback supported
! ethernet oam
interface FastEthernet1/3 !
interface FastEthernet4/4
ip address 6.6.6.6 no ip address
255.255.255.0 ethernet oam max-rate 5
ethernet oam ethernet oam min-rate 2
ethernet oam mode passive
! ethernet oam timeout 30
interface FastEthernet1/4 ethernet oam remote-failure link-fault action error-disable-interface
ethernet oam remote-failure dying-gasp action error-disable-interface
no ip address ethernet oam remote-failure critical-event action error-disable-interface
ethernet oam ethernet oam
! Ethernet OAM Tutorial !
Configuration Verification
show ethernet oam status [interface <interface name>]
fulliautomatix#show ethernet oam status int fas 4/4

FastEthernet4/4 Link Monitoring
---------------
General Status: supported (on)
-------
Symbol Period Error
Admin state: enabled Window: 100 x 1000000 symbols
Mode: passive Low threshold: 1 error symbol(s)
PDU max rate: 5 packets per second High threshold: none
PDU min rate: 1 packet per 2 seconds Frame Error
Link timeout: 30 seconds Window: 10 x 100 milliseconds
Low threshold: 1 error frame(s)
High threshold action: no action
High threshold: none
Link fault action: error disable interface …..
Dying gaps action: error disable interface …..
…..
Critical event action: error disable interface fulliautomatix#
Ethernet OAM Commands:- Discovery
show ethernet oam summary
show ethernet oam discovery [interface <interface name>]
Ceasar#show ethernet oam summary

Symbols: * - Master Loopback State, # - Slave Loopback
State
Capability codes: L - Link Monitor, R - Remote Loopback
U - Unidirection, V - Variable Retrieval
Ceasar#
fulliautomatix# show ethernet oam summary
Local Remote Symbols: * - Master Loopback State, # -
Slave Loopback State
Interface MAC Address OUI Mode Capability Capability codes: L - Link Monitor, R - Remote
Loopback
Fa1/1 0011.9321.1640 00000C active L U - Unidirection, V - Variable
Retrieval
Fa1/2 0011.9321.1640 00000C active L
Fa1/3 0011.9321.1640 00000C active L R Local Remote
Interface MAC Address OUI Mode
Fa1/4 0011.9321.1640 00000C passive L Capability
Fa4/1 000b.45b5.3140 00000C active L

Fa4/2 000b.45b5.3140 00000C active L
Fa4/3 000b.45b5.3140 00000C active L
Fa4/4 000b.45b5.3140 00000C active L
fulliautomatix#
Ethernet OAM Commands:- Discovery
Ceasar#show ethernet oam discovery interface
fas 1/1
FastEthernet1/1
Local client
------------
Remote client
Administrative configurations: -------------
Mode: active MAC address: 0011.9321.1640
Vendor(oui): 00000C(cisco)
Unidirection: not supported
Link monitor: supported (on) Administrative configurations:
Remote loopback: not supported PDU revision: 1
Mode: active
MIB retrieval: not supported Unidirection: not supported
Mtu size: 1500 Link monitor: supported
Remote loopback: not supported
MIB retrieval: not supported
Operational status: Mtu size: 1500
Port status: operational
Loopback status: no loopback Ceasar#
PDU revision: 0
Ethernet OAM Commands:- Link monitoring
Show ethernet oam statistics [interface Local Faults:
<interface name>] -------------
0 Link Fault records
FastEthernet4/2 0 Dying Gasp records
Counters: 0 Critical Event records
---------
--------------
Information OAMPDU Tx :3
0 Link Fault records
Information OAMPDU Rx :3 0 Dying Gasp records
Unique Event Notification OAMPDU Tx :0 0 Critical Event records
Unique Event Notification OAMPDU Rx :0
Local event logs:
Duplicate Event Notification OAMPDU TX :0 -----------------
Duplicate Event Notification OAMPDU RX :0 0 Errored Symbol Period records
Loopback Control OAMPDU Tx :0 0 Errored Frame records
0 Errored Frame Period records
Loopback Control OAMPDU Rx :0
0 Errored Frame Second records
Variable Request OAMPDU Tx :0
Variable Request OAMPDU Rx :0 Remote event logs:
Variable Response OAMPDU Tx :0
------------------
0 Errored Symbol Period records
Variable Response OAMPDU Rx :0 0 Errored Frame records
Cisco OAMPDU Tx :0 0 Errored Frame Period records
Cisco OAMPDU Rx :0 0 Errored Frame Second records
Unsupported OAMPDU Tx :0 fulliautomatix#
Unsupported OAMPDU Rx :0
Frames Lost due to OAM
:0
Ethernet OAM Commands:- Remote Fault
Indication
Ceasar(config)#int fas 1/4
Ceasar(config-if)#sh
fulliautomatix#
1w0d: %ETHERNET_OAM-SP-6-RFI: The client on interface Fa4/4 has
received a remote failure indication from its remote peer(failure
reason = remote client administratively turned off)
1w0d: %PM-SP-4-ERR_DISABLE: oam-remote-failure error detected on
Fa4/4, putting Fa4/4 in err-disable state
01:18:54: %ETHERNET_OAM-SP-STDBY-6-EXIT_SESSION: The client
on interface Fa4/4 has left the OAM session.
1w0d: %ETHERNET_OAM-SP-6-EXIT_SESSION: The client on interface
Fa4/4 has left the OAM session.
01:18:55: %PM-SP-STDBY-4-ERR_DISABLE: oam-remote-failure error
detected on Fa4/4, putting Fa4/4 in err-disable state
fulliautomatix#
fulliautomatix#show int fas 4/4
FastEthernet4/4 is down, line protocol is down (err-disabled)
Ethernet OAM Commands:- Remote Fault
Indication
Ceasar(config)#int fas 1/4

Ceasar(config-if)#no ethernet oam
fullyautomatix#
1w0d: %ETHERNET_OAM-SP-6-RFI: The client on interface Fa4/4 has received
a remote failure indication from its remote peer(failure reason = remote client
administratively turned off)
1w0d: %PM-SP-4-ERR_DISABLE: oam-remote-failure error detected on Fa4/4,
putting Fa4/4 in err-disable state
01:22:28: %ETHERNET_OAM-SP-STDBY-6-EXIT_SESSION: The client on
interface Fa4/4 has left the OAM session.
1w0d: %ETHERNET_OAM-SP-6-EXIT_SESSION: The client on interface Fa4/4
has left the OAM session.
01:22:29: %PM-SP-STDBY-4-ERR_DISABLE: oam-remote-failure error detected
on Fa4/4, putting Fa4/4 in err-disable state
Ethernet OAM Commands:- Remote loopback
ethernet oam remote-loopback start interface <interface

name> puts the ports in loopback state.
Ceasar#ethernet oam remote-loopback start interface

fas1/3
Ceasar#
6d22h: %ETHERNET_OAM-SP-6-LOOPBACK: Interface Fa1/3 has entered the master
loopback mode.
fulliautomatix#
1w0d: %ETHERNET_OAM-SP-6-LOOPBACK: Interface Fa4/3 has entered the slave loopback
mode.
1w0d: SP: EOAM LB(Fa4/3): new status = 2
fulliautomatix#
Ethernet OAM Commands:- Remote loopback
Ceasar#show ethernet oam summary
Symbols: * - Master Loopback State, # - Slave Loopback State
Local Remote
Interface MAC Address OUI Mode Capability
Fa1/1 0011.9321.1640 00000C active L

Fa1/2 0011.9321.1640 00000C active L
* Fa1/3 0011.9321.1640 00000C active L R
Fa1/4 0011.9321.1640 00000C passive L
fulliautomatix#show ethernet oam summary

Symbols: * - Master Loopback State, # - Slave Loopback State
Local Remote
Interface MAC Address OUI Mode Capability
Fa4/1 000b.45b5.3140 00000C active L
Fa4/2 000b.45b5.3140 00000C active L
# Fa4/3 000b.45b5.3140 00000C active L
Fa4/4
Ethernet OAM Tutorial 000b.45b5.3140 00000C active
© 2006 Cisco L All rights reserved.
Systems, Inc. 142
Ethernet OAM Commands :- Remote loopback
show interface counters are used to verify that traffic is looped back.
Ceasar#show interface counters | inc Fa1/3
Port InOctets InUcastPkts InMcastPkts

InBcastPkts
Fa1/3 42271953 374 606773 6 ÅInput packets
Fa1/3 42184101 392 606934 1 ÅOutput packets
Ceasar#ping 5.5.5.5
Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 5.5.5.5, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
Ceasar#show interface counters | inc Fa1/3
Fa1/3 42282215 379 606917 8 ÅInput packets
Fa1/3 42187315 397 606975 1 ÅOutput packets
Ethernet OAM debug commands
Ceasar#debug ethernet oam ?
all All Ethernet OAM debugging messages
config Ethernet OAM configurations
ha Ethernet OAM High Availability events
link-monitor Ethernet OAM link monitor
loopback Ethernet OAM loopback
packet Ethernet OAMPDU
sm Ethernet OAM state machine
OAM Inter-working
What is OAM Inter-working?
This is NOT OAM inter-working!

Operator A Operator B
Edge Router
Edge Bridge
ATM Switch
ATM Switch
Customer
Router
Router
Router
Bridge
Bridge
Bridge
switch
switch
Customer
Bridge
Bridge
Ethernet Ethernet 10G ATM
Transport OAM message with embedded MAC address carried from

bridge to bridge, visible to ETH layer (when present), and translated to
new transport’s OAM format when crossing physical media boundaries.
Creates dependency on Physical layer and inter-operability issues.
What is OAM Inter-working?
Operator should
• Strict OAM layering A Operator
be honored: messages B
should not cross layers
Edge Router
Edge Bridge
ATM Switch
ATM Switch
• OAM Messages should not leak outside domain boundaries within a layer
Customer
Router
Router
Router
Bridge
Bridge
Bridge
switch
switch
Customer
• Inter-working is event translations & not necessarily 1:1 message mapping
Bridge
Bridge
• Inter-working may be inter-layer and intra-layer
Event Translation
Ethernet Ethernet 10G
inter-domainsATM
intra-
layer
Service OAM
Server Layers feed
events into Client
Layers
Network Network
OAM OAM
Transport Transport Transport Transport

OAM OAM OAM OAM
Why OAM Inter-working?
Enable management of ubiquitous Ethernet service
offerings on:
mixed media
heterogeneous networks/transports
legacy equipment
Leverage the benefits of server layer OAM mechanisms

when applicable
Potentially faster failure detection times
Redundant Alarm Suppression (daisy chain effect)
Inter-working Scenarios: CFM to E-LMI
CE
MPLS Core MPLS

Ethernet
Access Access
CE 1 uPE B nPE B nPE A uPE A CE 2

Service Layer
OAM
CFM to E-LMI I/W
E-LMI Network Layer
OAM
Provider Level CFM

CFM @ Provider Level acts as MEN OAM: provides EVC Status and Remote UNI Status/Name to E-LMI
– Port State TLV of CC Messages carry remote UNI status
– MEP Name TLV of CC Messages carry remote UNI name
– Status of remote MEP in CCDB indicates EVC State
Inter-working Scenarios: 802.3ah to CFM
CE
MPLS Core MPLS

Ethernet
Access Access
CE 1 uPE B PE-Agg B nPE B nPE A uPE A CE 2

Service Layer
OAM
AIS AIS CC
Transport
Layer OAM
802.3ah 802.3ah to CFM I/W 802.3ah
Link Layer Defects detected by 802.3ah, relayed to CFM on same device.

CFM notifies remote devices of localized fault.
Two variants:
CC based (802.3ah on edge of domain)
AIS based (802.3ah within domain)
Inter-working Scenarios:
MPLS PW OAM to CFM
CE
MPLS Core MPLS

Ethernet
Access Access
PW OAM to
CE 1 uPE B PE-Agg B nPE B CFM I/W nPE A uPE A CE 2
Service Layer
OAM
AIS AIS
Transport
Layer OAM
VCCV-BFD
D-LDP
Directed-LDP & VCCV (BFD mode) running between nPEs.
D-LDP for defect notification, VCCV for defect detection
3-way I/W function at nPE
Requires CFM AIS/RDI
Inter-working Scenarios:
MPLS PW OAM to E-LMI
CE
MPLS Core
PW OAM to
CE 1 PE B PE A CE 2
E-LMI I/W
Service Layer
OAM
E-LMI E-LMI
Transport
Layer OAM
VCCV-BFD
D-LDP
Directed-LDP & VCCV (BFD mode) running between PEs.
D-LDP for defect notification, VCCV for defect detection
Defects detected/communicated by PW OAM are relayed to E-LMI via I/W function on PE.
Ethernet OAM
Summary
E-OAM Toolkit for Fault Management (1 of 5)
Fault
Fault
Detection
Detection Fault
Fault
Detection
Detection
Fault
Fault
Notification
Notification
Fault Fault
Fault Fault
Recovery Verification
Fault
Fault
Isolation
Isolation
Fault Detection
Mechanism Detectable Faults
• Unintended connectivity/service leaks

802.1ag Continuity • Unexpected sites
• Loss of connectivity to a site
Check (CC) • Link Connectivity failure
(CFM) • Device failure (soft & hard)
• Forwarding plane loops
• CFM Configuration Errors
Link • Unidirectional Link

802.3ah
Monitoring • Slowly deteriorating link quality (Frame/Symbol
Errors)
Fault Notification
Mechanism Triggers Fault
Fault
Fault Notification
Notification Fault
Alarm • Loss of CFM Continuity Check

Fault Fault
Detection Notification
Y.1731 Indication • Indication from Server Layer OAM

Signal (AIS) • Indication from lower ME Level Fault Fault
Fault Fault
CFM Domain Recovery
Recovery
Verification
Verification
Fault
Fault
Isolation
802.1ag • Unidirectional service Isolation
Remote Defect Connectivity (p2p)

(CFM) Indication (RDI) • Partial service connectivity (mp)
Remote • Link Fault (receive path)

Failure • Dying Gasp
802.3ah Indication • Critical Event
Event • Error thresholds exceeded

Notification (frames/symbols per interval)
Status • EVC Status Change

E-LMI Message • Remote UNI(s) Status Change
Fault Fault
Fault Fault
Fault
Fault
Fault
Fault Fault
Fault
Verification
Verification
Fault
Fault
Isolation
Isolation
Fault Verification
Mechanism Capability
802.1ag • Per EVC MAC Ping (source to single

Loopback destination)
(CFM) • Verify bidirectional connectivity between two CFM
Maintenance Points (for varied frame sizes)
• Per EVC MAC Ping (source to all

Multicast destinations)
Y.1731 • Verify bidirectional connectivity between one CFM
Loopback Maintenance End Point and all other End Points of
a service (for varied frame sizes)
Fault Fault
Fault Fault
Fault Fault
Fault Fault
Fault
Fault Fault
Fault
Isolation
Isolation
Isolation
Isolation
Fault Isolation
• Per EVC MAC Traceroute

802.1ag • Discover Maintenance Intermediate Points on path
(CFM) Link Trace from source End Point to destination End Point
• Report Ingress Action, Relay Action, Egress
Action hop by hop.
• Report encountered ACLs or STP-blocked ports
Fault Fault
Fault Fault
Fault
Fault Fault
Fault
Fault
Fault
Recovery
Recovery
Fault
Fault
Isolation
Isolation
Fault Recovery
• Use CFM mechanisms for monitoring redundant

G.8031 ETH-APS paths (order of msec)
Non-OAM xSTP
Mechanisms
E-OAM Toolkit for Performance Management
Frame loss measurement
Delay measurement
Delay variation (Jitter) measurement
Availability measurement
Performance Management
SLA CFM • Uni-directional and Round Trip Latency

Loopback • Unidirectional Jitter
• Unidirectional Frame Loss
Jitter Probe
LMM / 1DM / • Loss Measurement

Y.1731 DMM • One way delay
• Two way delay
E-OAM Toolkit for Configuration Management
SP Network
Access Core
Customer
Equipment
E-LMI
N-PE 1 N-PE 3 U-PE D
U-PE A MPLS CE
CE
SONET/SDH
Native
Ethernet
PW
CE N-PE 4 CE
U-PE B MPLS U-PE C
N-PE 2
Ethernet UNI Ethernet UNI

Configuration Management
CE-VLAN / • Reduce Customer configuration errors

• Ease deployment for Managed Services / Low-
EVC Map Touch Turn-key CE
E-LMI
• Improve service performance by egress CE
BW Profile Shaping
• Reduce Policing configurations on SP gear
Which Protocol to Use & Where?
CFM
SP Network
E-LMI Access Core
Customer
Equipment U-PE D
802.3ah
N-PE 1 N-PE 3
U-PE A MPLS CE
CE
SONET/SDH
802.3ah Native
Ethernet
PW
CE N-PE 4 CE
U-PE B MPLS U-PE C
N-PE 2
Ethernet UNI Ethernet UNI

CFM CFM
Choice of protocols, their locality & span of operation varies with network topology &
services – many variations possible
Restrictions (inherent to protocol definitions):
E-LMI on UNI (uPE to CE)
802.3ah on pt-to-pt (emulated ok) 802.3 PHY
CFM/Y.1731 require Ethernet MAC Layer awareness
Q and A

Ethernet Oam Tutorial Srinath Beldona

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Ethernet Oam Tutorial Srinath Beldona

Uploaded by

Copyright:

Available Formats

Ethernet OAM Tutorial

Srinath Beldona sbeldona@cisco.com

 We value your feedback,

 Please switch off your mobile phones!

User Facing Provider Edge (U-PE) Metro C

Ethernet Multipoint Service—EMS SP

Ethernet Relay Multipoint Service—ERMS SP

Primary Focus of Ethernet OAM protocols is on W ÅÆ E interactions

Service Layer Service OAM

Network Layer Network Network

 Model is iterative (layer stacking) & relative (service layer for

Accelerate Migration to Carrier Ethernet

 Ethernet OAM Provides Manageability to Customer Premise

System and Control Infrastructure

Customer MPLS SONET/SDH Customer

 Standardized as IEEE 802.3ah Ethernet in the First Mile (EFM)

Carrier Ethernet Capabilities Benefits

 MEF 10: Service Attributes

 Enable monitoring & verification of SLAs

 Provide End-to-End OAM

 Facilitate deployment of Ethernet services over

 Deliver SP’s customers packaged plug and play

 IEEE 802.1ag: Connectivity Fault Management (CFM)

Ethernet MPLS Core MPLS

CE 1 uPE B PE-Agg B nPE B nPE A uPE A CE 2

Ethernet MPLS Core MPLS

802.3ah 802.3ah 802.3ah

 Link Level OAM

Ethernet MPLS Core MPLS

Customer Service Provider Customer

Ethernet MPLS Ethernet

 Customer contracts with Provider for end-to-end service.

 Domains may nest or touch, but must never intersect.

Maintenance Points Loopback Points

 At (at least) Operator and Physical levels, there may be

“Physical” level of Connectivity Fault Management

 There are levels below CFM.

 Each Level’s Maintenance Points are the next-higher-level’s

 Each Level’s Loopback Points are invisible to all higher

Maintenance Points Loopback Points

Maintenance Points Loopback Points

 Each Level’s Maintenance Points are the next-higher-

Customer Service Provider Customer

Ethernet MPLS Ethernet

Customer 802.3ah Eth in First Mile:

ITU-T SG 13 and SG 15:

 Maintenance Domains & Levels (continued…)

As of this writing, higher Maintenance Levels are more “Phyward”, lower

MEP MIP MIP MEP

MEP MIP MIP MEP Customer

MIP MIP MIP MIP

well-known Maintenance Level (0 to 7)

Customer Operator A Operator B Customer

Customer src Operator A dest Operator B Customer

MEP MIP MIP MEP

Customer src Operator A Operator B dest Customer

MEP MIP MIP MEP

4 Level 4 MIP Gi3/48

1. Configure Global parameters/attributes

2. Configure Global parameters/attributes

4. Configure Operator Level MIPs

5. Configure per-Domain parameters

1. Configure Service IDs

2. Configure Customer Level MIPs (optional)

We value your feedback,

Please switch off your mobile phones!

Model is iterative (layer stacking) & relative (service layer for

Ethernet OAM Provides Manageability to Customer Premise

Standardized as IEEE 802.3ah Ethernet in the First Mile (EFM)

MEF 10: Service Attributes

Enable monitoring & verification of SLAs

Provide End-to-End OAM

Facilitate deployment of Ethernet services over

Deliver SP’s customers packaged plug and play

IEEE 802.1ag: Connectivity Fault Management (CFM)

Link Level OAM

Customer contracts with Provider for end-to-end service.

Domains may nest or touch, but must never intersect.

At (at least) Operator and Physical levels, there may be

There are levels below CFM.

Each Level’s Maintenance Points are the next-higher-level’s

Each Level’s Loopback Points are invisible to all higher

Each Level’s Maintenance Points are the next-higher-

Maintenance Domains & Levels (continued…)

Allows configuring static list of expected remote MEPs

Show ethernet cfm errors

Traceroute shows location and reason for failure.

Ethernet Services Model & MEF

Dynamic Data (Learned via MEN OAM)

Based on Polling procedure invoked by CE

Ethernet LMI Configuration Commands

Global Config a. EVC NAME

Multi Point EVC: b. Num of UNIs associated

Note: Default is - bundle-multiplex

Global: Enables Ethernet LMI per Box

Interface: Enables Ethernet LMI per interface/UNI

Start with root words “show ethernet service”

Start with root words “show ethernet lmi”

denver-c3750me#show ethernet service evc

Operates on a single point-to-point link between 2 devices

What is 802.3ah Ethernet OAM?

OAM Remote Failure Indication

OAM Link Monitoring

OAM Remote Loopback