Maximizing network Capacity, Reach and Value

Over land, under sea, worldwide www.xtera.com 1

Developing OTN Interface
Standards for Beyond 100G







Abstract

Work is underway to define Ethernet and OTN interfaces for client signals greater than
100 Gbit/s. For the OTN interface, three possible directions are being investigated thus far.
Many questions must be answered and contributions are expected. Experts from all over the
world got together in late February 2013 in Dallas, Texas, USA at an ITU-T Q11/15 meeting
hosted by Xtera Communications to continue their work in this area. This paper includes
some of the latest developments from this meeting.






Maximizing network Capacity, Reach and Value
Over land, under sea, worldwide www.xtera.com 2
Introduction

Industry drivers for increased transport and switching capacity continue to include growth of
the Internet, greater prevalence of video content, and more dependence on network based
storage. The new applications continue to be IP/Ethernet-based packet applications that rely
on routers and packet switches. In many cases, these devices operate more efficiently with
fewer high-capacity ports than they do with many low-capacity ports, even if the total
bandwidth supported in the two cases is the same. Partly for that reason, IEEE 802.3
specified 100 Gigabit Ethernet (GbE) in the past few years. The Telecommunication
Standardization Sector of International Telecommunication Union (ITU-T) extended the OTN
(Optical Transport Network) standards to support 100GbE and other combinations of lower
rate clients totaling around 100 Gbit/s.



Figure 1: OTN Frame structure

As networks continue to grow in capacity requirements, the efforts are already underway to
define the next higher rate client interface. The beyond 100G standards activities started
with work done by the IEEE as part of the Bandwidth Assessment project. The completion of
this project at the end of 2011 kicked off a series of activities relating to high bit rate systems
beyond 100G.
The first step is to decide on the bit rate for the next high speed interface. In general, large
bandwidth users and carriers favored the terabit per second system. Most component and
system vendors supported 400-Gbit/s approaches. The IEEE held several meetings trying to
weigh the pluses and minuses of each option in order to choose between 400 Gbit/s and
1 Tbit/s. During the March 2013 meeting in Orlando, Florida, USA, the IEEE approved the
formation of a 400GbE working group. The first working group meeting will be held at the
next IEEE meeting in May 2013. The decision is based on the ability to re-use much of the
technologies developed for 100GbE. This approach could reduce the time to market and


Maximizing network Capacity, Reach and Value
Over land, under sea, worldwide www.xtera.com 3
development costs, therefore encourages earlier adaptation of the next generation higher bit
rate systems.
While the IEEE 802.3 400GbE working group is starting its work on 400GbE, corresponding
activities at the ITU-T have also started. Because most optical network operators prefer to
manage their optical links using OTN performance measurements rather than Ethernet ones,
the ITU-T is beginning work on defining the next OTN frame structure, beyond 100G. It is
noted that in this paper, the bit rates referenced are nominal and not actual bit rates.
Interested readers should see the relevant standards for the actual bit rates of any signal.
For example, the so-called 100-Gbit/s OTU4 (Optical Transport Unit 4) actually has a bit rate
of 111,809,973.568 kbit/s plus or minus 20 parts per million [1].

OTN Line Signal
(ITU-T G.709)
OTUk Line Rate
(kbit/s)
0PUk Payload Rate
(kbit/s)
OTU Frequency
Accuracy
(Parts per Million)
OTU1 2,666,057 2,488,320 +/- 20
OTU 2 10,709,225 10,037,629 +/- 20
OTU 2e 11,095,727 10.356,012 +/- 100
OTU3 43,018,413 40,150,519 +/- 20
OTU3e2 44,583,355 41,611,131 +/- 100
OTU4 111,809,973 100,376298 +/- 20



Background on ITU-T Structure

The ITU-T is a technical subcommittee of the United Nations, tasked with developing global
standards related to the telecommunications industry. It is subdivided into Study Groups,
where Study Group 15 is the lead study group for optical networking. Study groups are
subdivided further into Working Parties and Questions, where the actual technical work is
conducted. Question 11 of Study Group 15 is the group responsible for developing the next
OTN frame structure to be included in ITU-T Recommendation G.709 [1].
All the work is contribution-driven, meaning it is conducted based on discussion around
proposals submitted by member companies. The best solutions result from collaboration by
industry experts from all over the world, focused on a solution that will be the best for the
global industry and not favoring any particular company or region of the world. A good global
standard largely eliminates the need for additional regional standards and makes it possible
for companies to build and sell products globally, by complying with the relevant ITU-T
standards (called Recommendations).
Having dependence on a single set of standards also reduces development costs and offers
the opportunity for volume based pricing. Maintaining the lowest cost brings the potential for
higher supplier, manufacturer, and service provider profits. Most importantly, adherence to
global standards should also result in lower cost of service for consumers.
For these reasons, companies collaborate to develop OTN standards.


Maximizing network Capacity, Reach and Value
Over land, under sea, worldwide www.xtera.com 4
Components of an OTN Interface Definition Beyond 100G

The work to define and standardize an OTN interface beyond 100G is not limited to the
definition of a new frame structure (to be added to ITU-T G.709 Recommendation).
Standardization in complementary areas is also necessary and will require enhancements to
other ITU-T Recommendations (or the development of new ones). The other areas include a
physical interface component ([2], developed in Question 6 of Study Group 15 Q6/15), an
equipment management component ([3, 4], developed in Q9/15 in the past, in Q11/15 going
forward), and potentially even components impacting overall network architecture ([5],
developed in Q12/15) and network management ([6, 7], developed in Q14/15).
Q11/15 is leading the way by defining the framing structure, collaborating with the other
Questions along the way. Once a framing structure is defined, then the other aspects can be
completed as well.



Framing Structure for the Next OTN Interface

In the early stages, all proposals for how to frame a client signal at a bit rate greater than
100 Gbit/s were being accepted and added to a G.709 Beyond 100G Living List. This list
was used to track all the questions that need to be answered, and the contributions and
proposals related to each one. It should eventually facilitate an objective comparison of the
options and selection of a final version for inclusion in a future version of ITU-T G.709
Recommendation.

Initially, all of the proposals for the OTN frame beyond 100G fit into one of three categories.
1. The first was to define a fixed OTN frame (ODU5 Optical Data Unit 5 and OTU5
Optical Transport Unit 5) at a single bit rate, much like most lower rate OTN signals
(ODUk/OTUk, k= 1, 2, 2e, 3, 4).
2. The second was to define a flexible single-OTU with a higher bit rate that would
always be managed as a single signal like the existing OTUk. It was flexible in that
the frame structure might be applied to any number of higher bit rates (e.g.
400 Gbit/s, 500 Gbit/s, and 1 Tbit/s).
3. The final option was to define a flexible multi-OTU structure where the new beyond
100G signals may be inverse multiplexed into multiple separate ODU/OTU, possibly
even reusing the existing ODUk/OTUk. The multiple signals might be managed
separately or together, given they would each carry full ODUk/OTUk overhead. At
the receiving end, the constituent ODUk/OTUk components would be deconstructed
and the original beyond 100G signals reassembled.



Maximizing network Capacity, Reach and Value
Over land, under sea, worldwide www.xtera.com 5


Figure 2: Single ODUx/OTUx/OChx



Figure 3: Multiple ODUxi/OTUxi/OChxi
slice
Fiber
Central
Frequency
single signal
frequency slot
...
Multiple Optical
Carriers
Flexi Grid
Central
Frequency
single signal
frequency slot
Multiple Optical
Carriers
Frequency Slot
ODUx(H)
OTUxi OTUxi
n*ODU4
Central
Frequency
single signal
frequency slot
Multiple Optical
Carriers
OTUxi
OChxi OChxi OChxi
ODUxi ODUxi ODUxi
Frequency Slot Frequency Slot
...
ODUx(L)
Client
OTLk.n
electrical
lanes not
shown
slice
Fiber
...
single signal
frequency slot
Single Optical
Carrier
single signal
frequency slot
single signal
frequency slot
50GHz
(Fixed Grid)
50GHz
(Fixed Grid)
50GHz
(Fixed Grid)
Single Optical
Carrier
Single Optical
Carrier
Frequency Slot
Central
Frequency
Central
Frequency
Central
Frequency
...
OChx
ODUx(H)
Packet
n*ODU4
Frequency Slot Frequency Slot
ODUx(L)
O
T
L
1
0
0
G
.4
#
1
OTLk.n OTLk.n OTLk.n
1T = 10* 100Gbit/s
25G electrical lane signal
OTUx(H)
O
T
L
1
0
0
G
.4
#
2
O
T
L
1
0
0
G
.4
#
3
O
T
L
1
0
0
G
.4
#
O
T
L
1
0
0
G
.4
#
1
O
T
L
1
0
0
G
.4
#
2
O
T
L
1
0
0
G
.4
#
3
O
T
L
1
0
0
G
.4
#
O
T
L
1
0
0
G
.4
#
1
O
T
L
1
0
0
G
.4
#
2
O
T
L
1
0
0
G
.4
#
3
O
T
L
1
0
0
G
.4
#
1Tbit/s = 10*100G
OS OS OS
OCh: Optical Channel
OS: Optical Signal


Maximizing network Capacity, Reach and Value
Over land, under sea, worldwide www.xtera.com 6
Both proposed flexible options were based on smaller payload increments, called tributary
slots. The frame structure would be defined based on the number of tributary slots it
contained and might only be resized (if resizing is standardized) in increments of those
tributary slots.
Today, ODUk are made up of 1.25-Gbit/s tributary slots (each one capable of carrying an
ODU0-framed Gigabit Ethernet signal). However, there is concern that having such fine
granularity at rates beyond 100G would be unnecessary and costly. So, proposals for the
tributary slot size for a new interface above 100G currently range from using the same
1.25-Gbit/s tributary slot all the way up to using 400-Gbit/s tributary slots. Most early
proposals were focused on using 100-Gbit/s tributary slots, but in the February meeting the
consensus of Q11/15 was to use 10-Gbit/s tributary slots as the working assumption going
forward. Signals at lower rates than the tributary slot will require an additional stage of
multiplexing to be carried in the new beyond 100G interface. For example, a 1 GbE (or any
other client signal below 10 Gbit/s) could not be efficiently mapped directly into an interface
with 10 Gbit/s tributary slots, and would need to first be mapped into an ODU at 10G or
higher, and then that higher rate ODU mapped into the new beyond 100G frame.

The tributary slot size to be used is only one of many questions that need to be answered,
regardless of which framing choice is selected. These must be explored to understand the
implications of the different options. The additional questions include those in the following
list:
What is the actual bit rate of the new OTN frame(s)?
What bit rate(s) will be used for the multi-lanes on the line side?
How will client signals be mapped into the payload area?
Can multiple frequency slots or wavelengths be used to support the new interface?
Must those frequency slots be of equal size?
What FEC (forward error correction) will be used for the interface?
How much skew compensation must be supported?

The different proposals may have very different answers to the questions listed above and as
a result, identifying the pros and cons of the different proposals will be a complex problem of
many variables. It will be important for the interested parties to fully understand each
proposal to be able to identify the important differences and implications of those differences.

For example, consider the fact that all digital signals beyond 100 Gbit/s are expected to be
transmitted in multiple digital lanes, like 100 GbE and the OTN digital frame defined to carry
it, OTU4. These parallel digital signals are then to be carried on multiple optical wavelengths
or optical sub-carriers within a single optical wavelength. If multiple optical wavelengths are
used, they may be closely packed in a single optical block of spectrum, or they might be
scattered across the channels within a single fiber, across multiple fibers in the same cable,
or even across multiple routes through a network.
If one were to select the flexible multi-OTU option, then there is overhead to monitor and
manage the transport of signals across multiple wavelengths or even multiple fibers.
However, managing the constituent OTU signals as a single entity may be far more
complicated with that option, depending on how widely the constituent signals are spread.


Maximizing network Capacity, Reach and Value
Over land, under sea, worldwide www.xtera.com 7
Recombining the different digital lanes becomes more complex the further the signals are
scattered in the optical transport, given the potentially higher and higher skew (time offset
resulting from different propagation times) between the lanes. However, one may minimize
waste of optical spectrum if the entire beyond 100G signal does not have to fit within a single
wavelength or contiguous group of wavelengths.



Three Applications/Interfaces and FEC

Another thing to keep in mind in following or contributing to the work on the next OTN frame
is the fact that there are multiple interfaces that will utilize it.
Existing OTUk (k=1, 2, 2e, 3, 4) frame structures are mapped into optical channels (OCh)
and used in three different applications. The interfaces in the three applications have
different levels of standardization, which will be true of the next rate as well.
The most highly standardized is the inter-domain interface (IrDI), used for interconnection
between different types of equipment regardless of the manufacturer. It is fully standardized
to ensure interoperability on the client side of transport equipment.
The second interface type has a frame definition and some functional specifications to
support them, but is not fully standardized for interoperability between equipment of different
vendors. This is the multi-vendor intra-domain interface (IaDI). It is used when one is
traversing a standardized transport link, meeting the specifications of a black link [8, 9], for
example. Interoperability between different vendor equipment may be possible, though it
may require negotiations and adjustments between the two equipment manufacturers
attempted to interoperate.
The third and final interface type is the single vendor IaDI. In this final case, no
interoperability is expected with other vendors, so even the frame structure may be modified
to accommodate a vendors unique development choices.
The two IaDI interface applications are typically used on the line side of optical transport
equipment, though they may also be used on the client side where interoperability with other
vendor equipment is not required.
With the different interface applications in mind, the standardization efforts will focus on the
IrDI and multi-vendor IaDI specifications.

One should keep in mind that the different applications are likely to require different choices
for FEC. FEC is used to correct errors digitally, improving performance of an optical
connection or extending the reach over which it can be operated cleanly.
In the past, Q11/15 considered standardizing the use of a stronger FEC code for the OTU4
IrDI than the Reed-Solomon (255,239) (RS(255,239)), specified in G.709 for other OTUk.
However, it was noted at the time that even the OTU4 IrDI is a multi-lane interface and not a
serial stream of bits at over 100 Gbit/s. Analysis by Q6/15 showed that given the IrDI is a
client side interface, and not a long haul interface, the performance of RS(255,239) was
sufficient.


Maximizing network Capacity, Reach and Value
Over land, under sea, worldwide www.xtera.com 8
Most optical transport manufacturers offer stronger FEC on their long haul IaDI interfaces.
Given those are not expected to be interoperable between different vendors equipment, the
FEC does not require ITU-T standardization. As part of the new frame definition, Q11/15 will
need to determine (with input from Q6/15) whether RS(255,239) is sufficient for the new IrDI
or if a stronger FEC will need to be selected for standardization in G.709.



Latest Development Based on the February 2013 Interim
Meeting in Dallas, Texas, USA

An ITU-T SG15/Q11 Experts Meeting was held in Dallas, Texas, USA attended by experts all
over the world. The approach and objectives for this meeting were:
Eventually develop a concise (simple) specification of the digital signals (electrical
lanes) that are passed between the electrical (Q11/15) domain and the optical
(Q6/15) domain;
Minimize the overall complexity of converting a high rate (Beyond 100G or B100G) bit
stream (OTUCn) from electrical into one or more optical signals that can be
transmitted over network media channels and converting the optical back into the
(original) electrical bit stream;
Explore the method to compensate for the skew between the electrical bit streams
carried by the OCh signals.

Several assumptions were agreed to at this meeting. These assumptions are important to
assure progress will be made on a uniform and consistent manner. The assumptions are:
Processing of the interface signal will be based on electrical lanes:
o Candidate rates include 28G, 40G, 56G
o Selection will be influenced by IEEE decisions on 400GbE
Objective to use a common electrical lane format and framer chip for:
o 400GbE;
o Multi-vendor Inter-Domain Interface (IrDI); and
o Single-vendor Intra-Domain Interface (IaDI).
Frame format based on n x 100G (frame increments) or m x lane rate increments
(e.g. m x 28G, 40G, 56G), standardizing only certain values of n or m
o Initial standardized IrDI rate would be ~400G and capable of carrying 400GbE

The group agreed at the meeting that it was premature to make any major decisions on
defining B100G OTN interfaces at that time. A more detailed list of questions that still need
to be answered was created to help guide future contributions on the subject. Though many
of the drivers for specifying new higher rate interfaces are known, there were differing
opinions on the urgency for the new standards. Future meetings will continue to discuss
proposals for how to specify the new OTN interfaces. As the options are better understood,
the best solutions will be selected.


Maximizing network Capacity, Reach and Value
Over land, under sea, worldwide www.xtera.com 9
As that is done, the advantages, consequences, and risks related to specifying the interface
too early or too late will be weighed. Once the interfaces are sufficiently specified and the
advantages are greater than the negative consequences and risks, it is expected that the
ITU-T will finalize the next OTN interface standards. Exactly when that will be is still
unknown, though the earliest possible conclusion is in 2014.



Summary

There are a lot of activities in progress for the development of standards for OTN interfaces
beyond 100G. The IEEE 802.3 group is also working hard to define the 400G Ethernet
interfaces. With the ITU-T and the IEEE working closely together and in parallel, it is
expected that the standards required for future beyond 100G systems will be available in the
next one to three years (2014-2016).





References

[1] ITU-T Recommendation G.709/Y.1331 (02/2012) Interfaces for the optical transport
network, pre-published.
[2] ITU-T Recommendation G.959.1 (02/2012) Optical transport network physical layer
interfaces.
[3] ITU-T Recommendation G.798 (04/2012) Characteristics of optical transport network
hierarchy equipment functional blocks.
[4] ITU-T Recommendation G.798.1 (04/2011) Types and characteristics of optical transport
network equipment.
[5] ITU-T Recommendation G.872 (07/2010) Architecture of optical transport networks.
[6] ITU-T Recommendation G.874 (04/2012) Management aspects of optical transport
network elements.
[7] ITU-T Recommendation G.874.1 (01/2002) Optical transport network (OTN): Protocol-
neutral management information model for the network element view.
[8] ITU-T Recommendation G.698.1 (11/2009) Multichannel DWDM applications with single-
channel optical interfaces.
[9] ITU-T Recommendation G.698.2 (11/2009) Amplified multichannel dense wavelength
division multiplexing applications with single channel optical interfaces.



Maximizing network Capacity, Reach and Value
Over land, under sea, worldwide www.xtera.com 10















Maximizing Network Capacity, Reach and Value
Over land, under sea, worldwide


Edition Date: May 2013
Version: 1.0