You are on page 1of 58

INTEROPERABILITY – THE FOUNDATION OF ETHERNET SUCCESS

Moderator: David J. Rodgers, Teledyne LeCroy


Presenters:
Shawn Nicholl, Xilinx – Enabling Interoperability Today for Next-Generation Ethernet
Solutions
Jeff Twombly, Credo – Interoperability – 400G Beyond – The Case for Active Cables
Paul Brooks, Viavi – Driving Interoperability Through Test & Measurement
Ray Nering, Cisco – Interoperability – The Foundation of Ethernet Success – A Systems
Vendor Perspective
April 16, 2020

www.ethernetalliance.org
© 2017 Ethernet Alliance
Presentation Disclaimer

The views being presented in this educational material


on the respective IEEE 802.3 standards under
consideration are the views of the author(s), and do NOT
represent a formal position or interpretation of the
respective standard by The Ethernet Alliance.

2
© 2017 Ethernet Alliance
The Ethernet Alliance
A Global Community of End Users, System Vendors,
Component Suppliers and Academia
§ Our Mission
• To promote industry awareness, acceptance and advancement
of technology and products based on, or dependent upon, both
existing and emerging IEEE 802 Ethernet standards and their
management.
• To accelerate industry adoption and remove barriers to market
entry by providing a cohesive, market-responsive, industry
voice.
• Provide resources to establish and demonstrate multi-vendor
interoperability.

3
© 2017 Ethernet Alliance
Ethernet Alliance Strategy
Expanding the Ethernet Ecosystem and Supporting Ethernet Development

§ Global Outreach § Collaboration


o Cultivate diverse, worldwide membership
o Facilitate interoperability testing events

§ Technology Advancement o Industry Plug Fests supporting member and


technology initiatives
o Support and advocacy of standards
development o Partnerships with peer organizations
o PoE Certification Program o Involvement with SIGs and MSAs

§ Thought Leadership § Promotion and Marketing of Ethernet


o Annual Ethernet Roadmap
o Industry media and analyst engagement
o Technology Exploration Forums
o Tradeshows and events
o Blogs, white papers, industry articles

4
© 2017 Ethernet Alliance
The Ethernet Alliance - Investment in Multi-vendor
Interoperability
• Plugfests in the last 12 months
– High Speed Networking
• Past Trade Shows
– OFC
– SC
– ECOC
– Other
• PoE Certification Program
– Gen 1 (.3af / .3at)
– Gen 2 (.3bt)

5
© 2017 Ethernet Alliance
ETHERNET INTEROPERABILITY –
TEST AND MEASUREMENT CONSIDERATIONS

David J. Rodgers – Teledyne LeCroy PSG

April 16, 2020

www.ethernetalliance.org
© 2017 Ethernet Alliance
Test and Measurement considerations for
High-Speed Ethernet applications
§ Ethernet Standards Evolving at Breakneck Pace
• 25GbE to 100GbE, now PAM4 50GbE, 200&400GbE
• Soon, 100GbE to 800GbE
• Automotive, Industrial, Commercial
§ Ethernet Fabrics Fueling Storage Explosion
o Speed and Optimization meeting QOS Expectations
• NVM-oF, iSCSI, FCoE, NFS, IBXoE, FCIP, iSER, iWARP, RoCE, Routable
RoCE (v2), and so on, and so on…
§ Standards beget Interoperability?
o Interpretation and implementation differences abound

7
© 2017 Ethernet Alliance
Effective Observation
Fabric Management
Utility/Hypervisor
Traffic Tap and DPI
(Wireshark®)
Line Rate
Analysis

8
© 2017 Ethernet Alliance
“You Can’t Test What You Can’t Measure!”
• Hardware Test tools have to change to keep up with market and
technology demands
• New Speeds adding new complexities
– NRZ to PAM4 signaling
• There is a “protocol” to the Phy
– Auto-negotiation
– Link Training
– FEC

• No two vendors implementations are identical

9
© 2017 Ethernet Alliance
Test and Measurement – The Bell Weather
§ T&M vendors; We’re On the Leading Edge!
• Partners in Pain of “being first”
• Traditional – Signal Integrity Tools
• New to the Scene – Protocol Specific Tools
§ Purpose Built Protocol Tools!
o Compliment to, not replacement for Traditional Tools
• Optimized for the Fabric/Device under test
o Becoming essential in all HW/SW test environments
§ The Goal; Testing must be “standardized” and repeatable
o Interop PlugFests, 3rd party testing services

10
© 2017 Ethernet Alliance
Key Interoperability Challenges
• Identifying Participants
– Characterizing Functionality of All Ecosystem Players
• Determining Root Cause
– Eliminate “finger pointing”
• Crafting the Solution
• Remediation Validation
– Test the fix
• Timely Resolution!

11
© 2017 Ethernet Alliance
Test & Measurement – Navigating the #NextEthernetEra
• Ubiquitous deployment requires vendor interoperability
• Establishing the “link” has become more
complicated than ever
• It’s imperative to know what’s “on the wire”
– Testing no longer “ends” at the connector
• Integrating legacy and new Ethernet
technologies creating new challenges

12
4/16/20
© 2017 Ethernet Alliance
Test & Measurement – ReCap
• Ethernet is a Juggernaut
• Content Delivery and Storage Demands are High
• Consistent and Predictable Interoperation
is Mandatory
• Speed adds Exponential Influences on the
Eco-System
• Testing, Testing, Testing!
– Tool Sets and Methodologies Must Evolve

13
© 2017 Ethernet Alliance
ENABLING INTEROPERABILITY TODAY FOR
NEXT-GENERATION ETHERNET SOLUTIONS

Shawn Nicholl - Xilinx

April 16, 2020

www.ethernetalliance.org
© 2017 Ethernet Alliance
Programmability for Early Adopters
• Early developers of Ethernet solutions are
constantly challenged
• Specs are in flux
• Proof of Concept development
• Interoperability testing

• Programmable platforms (FPGA and ACAP) are the


solution
• Early implementations without "locking in" the final
solution
• Test & Measurement equipment

• Programmability enables compliance testing


through evolving standards
• Vendors may need to tweak implementations

15
© 2017 Ethernet Alliance
Programmability and Continuous Development

• Recent trend to in-fill Ethernet rates Source: QSFP-DD MSA


( http://www.qsfp-dd.com/ )
• 25GE, 50GE, 200GE
• NBASE-T (2.5GE, 5GE)

• Continuous evolution of products


• Evolution of supported Ethernet rates for a port
• Evolution of connectivity, form factors (QSFP-DD/OSFP) and break-out
• Evolution of applications requiring wire-speed support

• Programmability enables continuous integration


Source: OSFP MSA
• Agile development for products ( https://osfpmsa.org )

16
© 2017 Ethernet Alliance
Building Blocks for Interoperability
• All network traffic passes over electrical interfaces of
silicon devices
• Cutting edge transceivers operate at 56G PAM4 and 112G PAM4
serial signaling rates
• Ethernet applications are emerging to take advantage of those
rates, while also expanding to leverage lower rates

• Applications need flexible Serdes


• PAM4 and NRZ signaling
• Wide range of rates

• Forward Error Correction


• PAM4 Bit Error Ratio (BER) demands use of FEC

17
© 2017 Ethernet Alliance
Creating Interoperable Solutions

• Judicious hardening of functions


• 16nm devices have 100 Gbps FEC in the Serdes
• 7nm devices have 600 Gbps protocol blocks
• Area and power efficient

• Platform for interoperability


• Develop new Ethernet products
• Ensure legacy applications continue to work

• Adaptable engines
• Future-proof for emerging protocols
• Programmability for applications evolution

18
© 2017 Ethernet Alliance
5G Wireless Application – Radio over Ethernet (RoE)

IEEE 1914.3 RoE CPRI


Ethernet Backhaul
IEEE 802.3br CPRI
Frame Preemption
eCPRI
IEEE 802.1CM
TSN eCPRI

19
© 2017 Ethernet Alliance
Protocol Offload and NFV Acceleration

Distributed
Secure
Network
Ethernet
Functions

Unencrypted 100GE
Encrypted 400GE
Encrypted 100GE

Unencrypted 50GE
PCIe to Host
Encrypted 25GE

20
© 2017 Ethernet Alliance
Ethernet Aggregation Application – FEC Variety

100G CAUI-4
RS(528,514) FEC 400GAUI-8
RS(544,514) FEC
200GAUI-4
RS(544,514) FEC
25GAUI
RS(528,514) FEC
SFI
CL74 KR FEC

21
© 2017 Ethernet Alliance
Ethernet-Centric World

• World is becoming Ethernet-dominated


• Countless new signaling and port rates
• Longevity of legacy rates

• Desire a scalable paradigm


• Supportable by everyone - from developers
to testers
• Supportable over a range of markets
• Ethernet link interoperability and reliability
should be a given for developers

• Testing Automation
• Need automation at all levels

22
© 2017 Ethernet Alliance
Test Automation with Programmable Devices

• Pre-hardware simulations
• Powerful regressions
• Constrained random test
• Shared vectors ensure interoperability

• Hardware-based Proof of Concept


• Enables early interop and in-system testing
• In-house or canned platform

• Ethernet test equipment automation hooks


• Script-based controls
• Library re-use over multiple generations

23
© 2017 Ethernet Alliance
Standards Development and Interoperability

• Industry must unite to develop interoperable networking standards


• Standard and industry bodies like IEEE, OIF, EA, and ETC foster a culture of
openness
• Create an improved solution without re-inventing the wheel
• Leverage existing solutions for newer market requirements like 800GE

• Strong, well-written standards lead to interoperability


• Whole industry benefits from hard work done at the ground floor
• Faster bring-ups and less cost wasted in field debug
• Companies should become contributors to standards

• Commercial benefits
• Common goal and common development focus
• Volume improves cost (economies of scale)

24
© 2017 Ethernet Alliance
Tradeshows and Interop events

• Multi-vendor interoperability testing is critical


• Customers want the plug-and-play nature of Ethernet

• Ethernet Alliance has taken a leadership role


• Live demonstrations at tradeshows
• Hot-plug events

• Interoperability events bring many vendors together


• Ensure that their equipment talks with each other

25
© 2017 Ethernet Alliance
Summary

• Developers of interoperable Ethernet technologies


benefit from programmable platforms
• Programmability enables continuous evolution of products
• FPGA and ACAP devices provide the building blocks

• In an Ethernet-dominated world, interoperability is


supported through:
• Efficient test automation
• Strong contribution to standards and industry bodies
• Live tradeshow demonstrations and hot plug events

• Get involved!

26
© 2017 Ethernet Alliance
INTEROPERABILITY –
400G BEYOND – THE CASE FOR ACTIVE CABLES

Jeff Twombly – CREDO

April 16, 2020

www.ethernetalliance.org
© 2017 Ethernet Alliance
Direct Attached Cables – Old Trusty Work Horse
• DAC has been the industry’s favorite
– Low cost
– Draws no power
– Availability

• But, as our industry moves beyond 25Gb NRZ, DACs have


become problematic
– Performance, interoperability, gauge density, routing challenges,…

28
© 2017 Ethernet Alliance
400G and Beyond - Problems with DACs
• DAC places burden of equalization on the switches
– What works on one port may not on the other
• There are established training protocols; however,
– “No two vendors implementations are identical”
• Routing challenges for emerging DDC routers and switches
– 100’s of fabric connections required
• As transfer rates increase to 800G
– Testing and interoperability issues become more problematic

29
© 2017 Ethernet Alliance
400Gb and 800Gb Ethernet Link Budgets & Cu Gauge
Scale Bend
400Gb Ethernet LR Link Budget OD Cross Radius
(mm) Section (mm)

400G DAC 1m – 30 AWG 2 x 6.7 33.5

400G DAC 2m – 28 AWG 2 x 7.9 39.5

400G DAC 2.5m – 26 AWG 11.4 55.0

800Gb Ethernet LR Link Budget

Scale Bend
OD Cross Radius
(mm) Section (mm)

800G DAC 1m – 28 AWG 2 x 6.7 33.5

800G DAC 2m – 26 AWG 11.4 55.0

www.ethernetalliance.org 30
© 2017 Ethernet Alliance
Adoption of 400Gb In the Data Center
2020 DDC 400G Applications
• DR4 Optics for Spine Interconnects
400G PAM4
• 400G CLOS AEC for Fabric Interconnects
time
19” Rack
32 Ports

128xDD 32 Ports

256 – 312 Cable CLOS Fabric


Fiber 32 Ports
32 Ports

64xDD Fabric

Fiber
64xDD Fabric
64xDD Fabric
64xDD Fabric
32 Ports
100G NRZ 32 Ports
128xDD
Fiber 32 Ports
32 Ports
31

31
© 2017 Ethernet Alliance
100Tb Ethernet DDC – Cable Management
Vertical Scale Bend
Cable OD Cross Radius
Manager (mm) Section (mm)
19” Rack #1 19” Rack #2
400G DAC 1m – 30 AWG 2 x 6.7 33.5
32 Ports 32 Ports

128xDD 32 Ports 128xDD 32 Ports 400G DAC 2m – 28 AWG 2 x 7.9 39.5


Fiber 32 Ports Fiber 32 Ports
32 Ports 32 Ports 400G DAC 2.5m – 26 AWG 11.4 55.0

Cable Fabric
Ingress/Egress Fiber

64xDD Fabric 64xDD Fabric


HiWire AEC – 30 AWG 2 x 6.5 32.5
64xDD Fabric 64xDD Fabric
64xDD Fabric 64xDD Fabric
HiWire CLOS <1.5m – 34 AWG 6 24
64xDD Fabric 64xDD Fabric
32 Ports 32 Ports
128
HiWire CLOS 1.5m–3m – 32 AWG 8 32
128xDD 32 Ports 128xDD 32 Ports
Fiber 32 Ports Fiber 32 Ports
32 Ports 32 Ports
• HiWire CLOS AECs consume up to 70% less volume than DACs
• HiWire CLOS can be routed with 15cm Cable Managers
Fiber/Cu divider
• DACs require >50cm cable managers
• Can break cages due to weight and stress
• Mixed DACs and Optics have shown signal integrity issues
Switch Racks stacked next to each other at the T2
Cross section of 128 cable fabric going up/down DACs are too thick and bend radius is too large
(256 Total Fabric Cables Required) for 400G the Distributed Disaggregated Chassis Use
Even NIC-TOR – 400G DACs don’t fit well and won’t
see mainstream adoption for 3-4 more years

32
© 2017 Ethernet Alliance
At 400Gb and 800Gb we need Active Cables
• At 400Gb, an active cable (such as a HiWire CLOS AEC):
– Consumes 70% less volume than a passive DAC (1x6mm vs. 1x11.4mm)
– Consumes 70% less power than an AOC
– Enables routing densities to >500 cables per rack

• At 800Gb, the choice is more stark


– Retimed front panel (Transceivers, AOC, AEC)
– Switches that support the DAC LR reach will consume 15-20% more power
• For what purpose? Opportunity for ASICs to move to optimized C2M reach & power…
– Improbable to route 800Gb DACs at even TOR-NIC densities in a data center

• The time has come to move away from the complexities of managing DAC
– Move to fully deterministic and persistent connections
– Better TTM and reduced OpEX

33
© 2017 Ethernet Alliance
Active Cable – Testing and Interoperability

• Leverage Ethernet Alliance’s plugfest to ensure interoperability


among active cable, switch, server, and test equipment
suppliers.
• Leverage industry consortium to ensure simplified application
use cases and guarantee plug & play performance.

34
© 2017 Ethernet Alliance
DRIVING INTEROPERABILITY THROUGH TEST &
MEASUREMENT

Paul Brooks – VIAVI Solutions

April 16, 2020

www.ethernetalliance.org
© 2017 Ethernet Alliance
Ethernet – Success through interop and flexibility
• Ethernet is the dominant solution for high speed interconnect
of communication and computing devices.
• With a choice of PMDs to go a few cms over a backplane or
10’s km over fiber
• Pluggable optical modules have allowed Ethernet to address
dynamic needs in a flexible ‘pay and adopt’ as you grow
approach
• Ensuring the validation, functionality of pluggable optical
modules across an open ecosystem has been a key element.

36
© 2017 Ethernet Alliance
Client Pluggable Optics Today
400GE
QSFP56-DD
Standard mechanical QSFPx– ‘multi-lane’ 400GE & 800GE
OSFP
form factors and module
40GE 100GE 200GE
management and host QSFP+ QSFP28 QSFP56

interfaces allow
interchange of modules 100GE
CFP4
and ‘pay as you grow’ 100GE
100GE
equipment. CFP
CFP2
• Client optical interfaces are deployed in volumes of
millions as pluggable, multi vendor, commoditized
products based on robust and clear standards to rates of
100GE 400G today.
50GE SFP56-DD
SFP56 • The vast majority of the photonic interfaces use direct
25GE
CPRI10 detect optical links using simple NRZ modulation. From
10GE SFP28 400G there has been a move to PAM-4 based signalling
CPRI up to 9
SFP – ‘single lane’
for both the electrical and optical links.
SFP+

© 2017 Ethernet Alliance


Inside a 400G QSFP-DD

Host AUI-8 inter-op & margin.


Equalizer set-up
Thermal – module & system

ROSA, TOSA BW, linearity,


power, performance
Double stacked connector DSP – host at PAM-4 25Gbd, line
SI issues at 25Gbd PAM-4 side 100G/lambda or ZR coherent
Mechanical robustness

Photonic inter-op & margin


Complex module management MIS – real time
software to manage events like LOS. Multiple
complex parameters like OSNR and PMD to be Mechanical integration
read-off and checked against alarm levels. Pre-FEC
BER to be tracked.

© 2017 Ethernet Alliance


Critical Areas for Inter-op
Mechanical & thermal environment

Data interface
FEC, eCPRI, Ethernet
P
h
o
DSP,
Optical i/f t
controller, Electrical interface
o
PMD, optics F/W & Phy, skew
n
S/W etc
i
c
s Management interface
MIS, I^2C, peek/poke

Optical - Line side Pluggable module


Electrical - Host side

© 2017 Ethernet Alliance


Succesful inter-op means an Integrated Environment
Power
Cooling
PHY layer The elements required for
Ethernet & PCS successful inter-op must be
MIS validated together. The close
Triggers interaction of host
FEC (power/thermal/mechanical,
Photonic data & phy/electrical and the
module management are key to
the open ecosystem. Also
critical is the inter-operability of
the optical side. With the
emergence of pluggable
coherent modules, especially
the expected wide scale take up
of ZR modules at 400G will
cause addition challenges.

© 2017 Ethernet Alliance


Legacy or State of the art?
• A modern host has to consistently
manage and support a wide range
of potential clients.
• Take a QSFP-DD slot in a network
element
– QSFP+ 40G
• What type of – QSFP28 – 100G
module? – QSFP56 – 200G
• SFF or CMIS – QSFP-DD – 400G
– Optical or DAC
• Supported modes
– SFF or CMIS
• AUI set-up – I^2C speed & termination
• LOS – Breakout or ?

• Traffic
• A modern host must be able to
• power manage this in a dynamic and
consistent manner
• Consistent behaviour under fault
conditions like LOS?

41
© 2017 Ethernet Alliance
Summary - Ethernet Interop
• Pluggable optics give Ethernet the powerful ability to scale and
grow with the right reach (PMD) for each application
– From copper DAC to ZR optics for 80km+
• Pluggable optics have evolved significantly from the simple E/O
and O/E converters of past generations to highly integrated
and complex systems with demanding management and
control as well as high speed electrical & photonic interfaces
• All of these need to be orchestrated together to continue the
open ‘plug, play & grow’ ecosystem that has made Ethernet
the success it is.

42
© 2017 Ethernet Alliance
INTEROPERABILITY – THE FOUNDATION OF
ETHERNET SUCCESS, A SYSTEMS VENDOR
PERSPECTIVE

Ray Nering – Cisco Systems


April 16, 2020

www.ethernetalliance.org
© 2017 Ethernet Alliance
Industry Relies on Ethernet Interoperability

• Cisco has relied on Ethernet requirements since it first


product in 1986
• Today, Cisco’s product line is broad, diverse and still
relies on Ethernet requirements
• Any product must be able to interoperate with other
vendors to be a relevant player in the market
• Ethernet standards, industry standards organizations (IEEE, etc), MSAs and industry
collaboration are the key to maintaining an ecosystem that promotes interoperability
• At 100GE and below Interop wasn’t trivial
• PAM4 technology, everything is new: Optics, SERDES, CMIS, form factors, standards

© 2017 Ethernet Alliance


Pluggable Optics are GREAT

Standards (IEEE, MSA, ) try to guarantee interop and uniformity

Optical Signaling (module <-> module)


Electrical Signaling (module<-> line-card High Speed)
Control Plane (module<-> linecard Low Speed)

Successful implementation on 1-100 GbE client optics, i.e. Switch to Switch, Switch to Router interfaces
Generally within same buiding, campus or data center (<10 km)

© 2017 Ethernet Alliance


100GbE Pluggable Optic Interoperation
• Industry has been releasing platforms w/ 100GbE ports since 2010 timeframe
• OEMs have been producing platforms w/ 100GbE ports since 2010
• They have a range of 100G port form factors and all must interoperate optically for all
time
• CFP, CFP2, CPAK, CFP4, QSFP28 and QSFP-DD…

• A CFP 100BASE-LR4 shipped in 2011 must be able to interoperate with all other
100GBASE-LR4 modules no matter the vendor or the form factor
• Customer may buy a 100G-LR4 and it should work in any platform w/ a compatible port

© 2017 Ethernet Alliance


100GbE was the beginning of more complex
host/module interactions

1, 10, 40
100 GbE
25Gbit/s I/O 3.5 W power
GbE

Equalization on electrical channel

Fixed or adaptive Equalization

Low power management Strong Equal. @


CDR
CDR Rx
Laser turn on

Diagnostic monitoring

Optical and environmental monitoring Complex


Provisioning based on state machine interactions
with Host

© 2017 Ethernet Alliance


What happens if……
Vendor A does not test full compliance and has little margin on equalization settings

On this unit, no errors

Error free area guardband

On this unit, CRC errors for no


apparent reason

© 2017 Ethernet Alliance


What happens if……
Vendor does test full compiance and has plenty of margin

On this unit, no errors

Error free area guardband

On this unit, no errors

© 2017 Ethernet Alliance


What happens if……
Vendor A has limited acceptance Vendor B has wide acceptance
for fast electrical signals from linecard for fast electrical signals from linecard

Error free only in this condition: Error free wide range: perfomance consistent
performance depended on port used across all ports

© 2017 Ethernet Alliance


Impact on the Transition to PAM4
• Industry turned to PAM4 to increase data rate beyond 25Gb/s
• PAM4 is much more sensitive to:
• Noise
• Reflections
• Non-linearities
• Baseline wander
• Receiver design is much more complicated
• IEEE has done a great job to embrace PAM4 with standards
• 802.3bs/cd/ck/cm/cn/ct/cw/cp/cu….
• There will be improvements as we learn more

© 2017 Ethernet Alliance


400 GbE Increases Complexity Further
100 GbE (LR4) 400 GbE (FR4)
• 400G PAM4 requires DSP and strong equalizations
to enable client optics. IDEAL
• DSPs are not as powerful (and power-hungry) as
the ones used on line-side optics.
• Yet interop between different DSPs is a critical MEASURED
item

POST DSP EQUALIZER

© 2017 Ethernet Alliance


400G DSP interop
• Interop testing over three 400G vendors
• Vendor A and C use same DSP
• Vendor B uses different DSP
• Tests identified significant issues with
interoperating with anything but itself

Tx A Tx B Tx C Tx A Tx B Tx C Tx A Tx B Tx C

Rx A Rx B Rx C

© 2017 Ethernet Alliance


Best practice: module level testing
• TX
Optical • RX

Interoperability
• TX
Electrical • RX
• Alarms
Control • Monitoring (DOM)

Across full range of

Temperature Voltage DC noise

© 2017 Ethernet Alliance


Best practice: system level testing
Software Interactions
Module & Fiber Plug /Remove
Power Cycle
Shut/No Shut

Across full range of

Temperature Voltage Platforms

© 2017 Ethernet Alliance


Faster speeds =
Key takeaways More complexity
Coverage
Barebone
More sensistive
Best Tier 1 optics Tier 2 optics
Practice supplier supplier
optics
supplier
linecards/module interactions
Basic optical compliance

Full optical compliance

Test over worst case fibers


Critical features for
Full electrical compliance
seamless interop
Basic Monitoring and control compl.
between modules
Full Monitoring and control compliance
on different
Full testing/regressionover MCN
platform
Interoperability with other optics

System level testing

Test across all T/V corners

© 2017 Ethernet Alliance


Conclusion
• Ethernet interoperability is literally the foundation of
networking today
• Lower speeds (>100GbE, NRZ) interop was straight forward
• Higher speeds (50G+) w/ PAM4 signaling interop has become
much more complicated
• Interactions between optics, platforms and SW are critical
• Corner testing not only at the optical level but also across
platforms & SW are critical for robust interoperability

57
© 2017 Ethernet Alliance
If you have any questions or comments, please email
admin@ethernetalliance.org

Ethernet Alliance: visit www.ethernetalliance.org


Join the Ethernet Alliance LinkedIn group
Follow @EthernetAllianc on Twitter

Visit the Ethernet Alliance on Facebook

www.ethernetalliance.org 58
© 2017 Ethernet Alliance

You might also like