31.

1 Designing QoS for the MPLS VPN

Introduction 

Your colleague Dennis is an expert on MPLS. You decide to take some time to accompany
him on his assignments. One of his customers has asked to implement QoS on an MPLS
VPN. You are unaware of how to do this, so, during lunch, Dennis gives you an overview on
QoS in MPLS VPNs.

Dennis bets you that if you are able to answer his questions about the subjects he’s just taught
you, he will pay for lunch. Do you want to earn a free lunch?

Choose an option:

 Review one or more topics:

o Need for QoS in MPLS VPN
o Layer 2 Private WAN QoS Administration
o Fully Meshed MPLS VPN QoS Administration
o MPLS DiffServ Tunneling Modes
o Example: MPLS VPN QoS Roles
 There is no Challenge for this section. Once you are done with this section, proceed to
the next section.
31.2 Designing QoS for the MPLS VPN

Need for QoS in MPLS VPN 

MPLS VPNs are rapidly gaining popularity as private WAN alternatives. MPLS VPNs
provide fully meshed Layer 3 virtual WAN services to all interconnected CE routers.

Facts about QoS for MPLS VPNs:

 MPLS VPNs provide fully meshed Layer 3 virtual WAN services to all
interconnected CE routers.

 To achieve end-to-end service levels, both enterprise and service provider QoS
designs must be consistent and complementary.

Enterprise customer subscribers must closely cooperate with their service providers to ensure
end-to-end service levels; they can no longer achieve these service levels independent of the
service provider policies.

The MPLS VPN QoS design can be viewed from two distinct perspectives:

 The enterprise customer subscribing to the MPLS VPN service

 The service provider provisioning edge and core QoS within the MPLS VPN service

To achieve end-to-end service levels, both enterprise and service provider QoS designs must
be consistent and complementary. Many enterprise customers turn to service providers that
offer MPLS VPN services as private WAN alternatives. One of the main reasons for using an
MPLS VPN is the any-to-any connectivity capabilities of MPLS VPNs. However, this full-
mesh nature in itself poses significant QoS implications to enterprise customers and service
providers alike—namely, that they both need to comanage QoS in a cooperative and
complementary fashion to achieve end-to-end service levels.
The service provider IP core is used to provide high-speed packet transport. Therefore, all the
markings, policing, and shaping should be performed only at the PE router on the PE-to-CE
link, not in the core. Only the edge requires a complex QoS policy. In the core, only queuing
and dropping are required. The operation of queuing and dropping will be based on the
markings that are done at the PE.

The reason for these procedures is the any-to-any and full-mesh nature of MPLS VPNs,
where enterprise subscribers depend on their service providers to provision PE-to-CE QoS
policies that are consistent with their CE-to-PE policies.

In addition to these PE-to-CE policies, service providers will likely implement ingress
policers on their PEs to identify whether the traffic flows from the customer are in or out of
contract. Optionally, service providers may also provision QoS policies within their core
networks by using DiffServ or MPLS TE.

The role of QoS over the MPLS VPN:

 Managing packet loss and jitter.

 Shaping traffic to contracted service rates.

 Performing hierarchical queuing and dropping within these shaped rates.

 Policing traffic classes according to contracted rates.

 Restoring packet markings.

 Mapping enterprise-to-service provider class of service markings.

In addition to the dual role of QoS in the private WAN and branch networks (namely that of
managing packet loss and jitter by queuing policies and of enhancing classification
granularity by leveraging deep packet inspection engines), the role of QoS over the MPLS
VPN may be expanded to include the following:

 Shaping traffic to contracted service rates.

 Performing hierarchical queuing and dropping within these shaped rates.

 Mapping enterprise-to-service provider class of service markings.

 Policing traffic classes according to contracted rates.

 Restoring packet markings.

31.3 Designing QoS for the MPLS VPN

Layer 2 Private WAN QoS Administration 

MPLS VPNs offer a full mesh of connectivity between the campus and branch networks. It is
this fully meshed characteristic of MPLS VPNs that presents a significant QoS design
implication, as compared to the traditional Layer 2 private WAN QoS design: private WANs
(which are usually deployed in either a point-to-point or a hub-and-spoke topology).

QoS administration paradigm under Layer 2 private WAN design:

 Most Layer 2 WAN designs revolve around a hub-and-spoke model.

 QoS primarily is administered at the hub router.

 Enterprise hub controls QoS for campus-to-branch and branch-to-branch traffic.

It is this fully meshed characteristic of MPLS VPNs that presents a significant QoS design
implication, as compared to the traditional Layer 2 private WAN QoS design: private WANs
(which are usually deployed in either a point-to-point or a hub-and-spoke topology).
Because of cost, scalability, and manageability constraints, traditional private WAN designs
rarely use full-mesh models. Instead, most Layer 2 WAN designs revolve around a hub-and-
spoke model, implementing either a centralized hub design or the more efficient regional hub
design. In such hub-and-spoke designs, QoS is primarily administered at the hub router
(WAN aggregator) by the enterprise. As long as the service provider meets the contracted
service levels, the packets that are received at remote branches will reflect the scheduling
policies of the hub router. The WAN aggregator controls not only campus-to-branch traffic,
but also branch-to-branch traffic (which is homed through the hub). Under traditional hub-
and-spoke models, QoS is principally administered by the enterprise customer, as shown in
the figure.

31.4 Designing QoS for the MPLS VPN

Fully Meshed MPLS VPN QoS

Administration 

With MPLS VPN service offerings that inherently offer full-mesh connectivity, the QoS
administration paradigm shifts.

QoS administration paradigm shifts under an MPLS VPN full-mesh design:

 The hub router still administers QoS for all campus-to-branch traffic, but it no longer
fully controls the QoS for branch-to-branch traffic.

 To guarantee service levels, the service provider needs to provision QoS that is
compatible with enterprise policies on all PE links to remote branches.
In a full-mesh design, the hub router still administers QoS for all campus-to-branch traffic,
but it no longer fully controls the QoS for branch-to-branch traffic. Although it might appear
that the only required workaround for this new scenario is to ensure that QoS is provisioned
on all branch routers, this workaround is insufficient because it addresses only a part of the
issue.

For example, consider the case of provisioning any-to-any multimedia conferencing. As with
a traditional Layer 2 WAN design, a scheduling policy to prioritize multimedia conferencing
on the WAN aggregator is required. The enterprise must then properly provision similar
priority scheduling for multimedia conferencing on the branch routers also. In this manner,
any multimedia-conferencing calls from the campus to the branch (and also from branch to
branch) are protected against traffic of lesser importance flowing between the same sites. The
complexity of the fully meshed model arises when you consider that contending traffic might
not always come from the same sites but could come from any site. Furthermore, the
enterprise no longer fully controls QoS for branch-to-branch traffic because this traffic is no
longer homed through a hub. If a multimedia-conferencing call is set up between two
branches and a user from one of the branches also initiates a large FTP download from the
central site, the potential for oversubscription of the PE-to-CE link from the fully meshed
MPLS VPN cloud into one of the branches becomes very real, likely causing drops from the
multimedia-conferencing call.

The only way to guarantee service levels in such a scenario is for the service provider to
provision QoS scheduling that is compatible with enterprise policies on all PE links to remote
branches. This setup is what creates the paradigm shift in QoS administration for fully
meshed topologies—namely, enterprises and service providers must cooperate to jointly
administer QoS over MPLS VPNs, as shown in the figure.

Therefore, queuing policies are mandatory on CE and PE router egress edges because of the
full-mesh implications of MPLS VPNs. In addition, PE routers will have ingress policing
policies to enforce SLAs.

QoS policies on provider core routers are optional. Such policies are optional because some
service providers overprovision their MPLS core networks, and, as such, they do not require
any additional QoS policies within their backbones. However, other providers might
implement simplified DiffServ policies within their cores or might even deploy MPLS TE to
process congestion scenarios within their backbones.
31.5 Designing QoS for the MPLS VPN

MPLS DiffServ Tunneling Modes 

DiffServ tunneling modes introduce a new PHB, which allows differentiated QoS in a service
provider network. The tunneling mode is defined at the edge of the network, normally in the
PE LSRs (both ingress and egress).

Facts about CE-SP-CE communication in an MPLS VPN:

 Customer marks IP packets with DSCP marker.

 Service provider uses MPLS EXP bit to enforce QoS policy.

 Service provider can maintain QoS transparency for the customer or not.

There are three distinct modes of MPLS DiffServ tunneling:

 Uniform mode

 Short pipe mode

 Pipe mode

MPLS can be used to tunnel the QoS markings of a packet and create QoS transparency for
the customer. It is possible to mark the MPLS EXP field (in a service provider network)
independently of the PHB marked by the customer in the IP precedence or DSCP fields. A
service provider may choose from an existing array of classification criteria, including or
excluding the IP PHB marking, to classify those packets into a different PHB. The PHB
behavior is then marked only in the MPLS EXP field during label imposition. This marking is
useful to a service provider that requires SLA enforcement of the customer packets by
promoting or demoting the PHB of a packet, without regard to the QoS marking scheme and
without overwriting the IP PHB markings of the customer.

Some service providers re-mark packets at Layer 3 to indicate whether traffic is in contract or
out of contract. This re-marking is not always desirable from the standpoint of the enterprise
customer. Because MPLS labels include three bits that are commonly used for QoS marking,
it is possible to tunnel DiffServ, that is, to preserve Layer 3 DiffServ markings through a
service provider MPLS VPN cloud, while still performing re-marking (via MPLS EXP bits)
within the cloud to indicate in- or out-of-contract traffic.

It is up to the service provider to define the CoS models they will offer to their subscribers.
There is no one-size-fits-all model, because these CoS models are often a key component of
the competitive differentiation strategy of a service provider. Most service providers offer
four- and six-class QoS models, with a few offering eight (or more) classes of service.

Whenever multiple service provider models are presented as options, it is recommended that
the subscriber choose the model that most closely aligns with its strategic end-to-end QoS
model.

RFC 3270 defines three distinct modes of MPLS DiffServ tunneling:

 Uniform mode: The uniform mode has only one layer of QoS, which reaches end-to-
end. The ingress PE router copies the DSCP from the incoming IP packet into the
MPLS EXP bits of the imposed labels. As the EXP bits travel through the core, they
may or may not be modified by intermediate P routers. At the egress P router, the
EXP bits are copied to the EXP bits of the newly exposed label. Finally at the egress
PE router the EXP bits are copied to the DSCP bits of the newly exposed IP packet.

 Short pipe mode: The short pipe mode uses the same rules and techniques across the
core. The difference is at the egress PE router the newly exposed IP packets are
classified for outbound queuing based on the IP PHB from the DSCP value of this IP
packet.

 Pipe mode: The pipe mode uses two layers of QoS. First, there is underlying QoS for
the data, which remains unchanged when traversing the core. Second, there is a per-
core QoS, which is separate from that of the underlying IP packets. This per-core QoS
PHB remains transparent to end users. When a packet reaches the edge of the MPLS
core, the egress PE router classifies the newly exposed IP packets for outbound
queuing based on the MPLS PHB from the EXP bits of the recently removed label.

The differences between DiffServ modes are as follows:

 Whether customer and service provider are in the same QoS domain or not

 Whether service provider maintains QoS transparency for the customer or not
  Whether customer or service provider QoS policy is implied on the PE egress router

Default behavior of the DSCP and MPLS EXP bits as packet travels from CE to CE:

 The IP precedence of the incoming IP packet is copied to the MPLS EXP bits of all
pushed labels.

 The EXP is copied to the new labels that are swapped and pushed during forwarding
or imposition.

 At label disposition, the EXP bits are not copied to the IP precedence or DSCP field
of the newly exposed IP packet.

The default behavior of the DSCP MPLS EXP bits as a packet travels from one CE router to
another CE router across an MPLS core is as follows:

 Imposition of the label (IP to label):

1. The IP precedence of the incoming IP packet is copied to the MPLS EXP bits
of all pushed labels.

2. The first three bits of the DSCP bit are copied to the MPLS EXP bits of all
pushed labels.

3. This technique is also known as ToS reflection.

 MPLS forwarding (label to label):

1. The EXP is copied to the new labels that are swapped and pushed during
forwarding or imposition.

2. At label imposition, the underlying labels are not modified with the value of
the new label that is being added to the current label stack.

3. At label disposition, the EXP bits are not copied to the newly exposed label
EXP bits.
  Disposition of the label (label to IP):

1. At label disposition, the EXP bits are not copied to the IP precedence or DSCP
field of the newly exposed IP packet.

MPLS DiffServ uniform mode characteristics:

 The customer and service provider share the same DiffServ domain.

 The customer IP precedence or DSCP is copied into the MPLS EXP field on ingress
(happens by default).

 The MPLS EXP bits are propagated down into the IP precedence or DSCP field on
egress (configuration task performed by service provider).

Uniform mode is generally used when the customer and service provider share the same
DiffServ domain, as in the case of an enterprise deploying its own MPLS VPN core. The
outmost header is always used as the single meaningful information source about the QoS
PHB. On the MPLS label imposition, the IP precedence classification is copied into the
outermost experimental field of the label (default behavior). As the EXP bits travel through
the core, they may or may not be modified by intermediate P routers. On egress of the service
provider network, when the label is popped, the router propagates the EXP bits down into the
IP precedence or the DSCP field (needs to be configured by the service provider on the egress
PE router).
As shown in the figure, the enterprise customer DSCP markings have been re-marked in
transit by the service provider in the MPLS uniform DiffServ tunneling model.

MPLS DiffServ short pipe mode characteristics:

 The customer and service provider are in different DiffServ domains.

 The service provider enforces its own DiffServ policy.

 The service provider maintains DiffServ transparency.

 On the PE egress, the customer QoS policy is implied.

Short pipe mode is used when the customer and service provider are in different DiffServ
domains. Short pipe mode is useful when the service provider wants to enforce its own
DiffServ policy while maintaining DiffServ transparency. The outermost label is utilized as
the single meaningful information source as it relates to the QoS PHB of the service provider.
On the MPLS label imposition, the IP classification is not copied into the EXP of the
outermost label. Rather, based on the QoS policy of the service provider, an appropriate value
for the MPLS EXP is set on the ingress PE. The MPLS EXP value could be different from
the original IP precedence or the DSCP. The MPLS EXP will accomplish the CoS marking
on the topmost label but will preserve the underlying IP DSCP. If the service provider
reclassifies the traffic in the MPLS cloud for any reason, the EXP value of the topmost label
is changed. On the egress of the service provider network, when the label is popped, the PE
router will not affect the value of the underlying DSCP information. In this way, the MPLS
EXP is not propagated to the DSCP field. Therefore, the DSCP transparency is maintained.

As shown in the figure, MPLS EXP values can be marked in any way that the service
provider wants to convey local significance (and have no relation to the customer packet-
marking values). Therefore, the customer markings are preserved in transit and are available
to the customer as the packet exits the MPLS VPN.

Note that the egress PE in short pipe mode uses the original IP precedence or DSCP to
classify the packet that it sends to the enterprise network. The enterprise set the original IP
precedence per its own QoS policy. The service provider may apply enterprise QoS policy at
the egress PE for traffic that goes toward the CE. In this example, the PE implements per-
customer egress QoS policies for traffic toward the CE, granting customers maximum control
of the QoS treatment for the packet through the MPLS VPN.

In the case of any re-marking occurrence within the service provider MPLS VPN cloud,
changes are limited to the MPLS EXP re-marking only and are not propagated down to the
ToS byte of the underlying IP packet.

MPLS DiffServ pipe mode characteristics:

 The customer and service provider are in different DiffServ domains.

 The service provider enforces its own DiffServ policy.

  The service provider maintains DiffServ transparency.

 On the PE egress, the service provider QoS policy is implied.

The main difference between the short pipe mode and pipe mode MPLS DiffServ tunneling is
that the PE egress policies (toward the customer CEs) are provisioned according to the
service provider explicit markings and re-markings, not the enterprise customer IP DiffServ
markings (although these customer IP DiffServ markings are preserved). When a packet
reaches the edge of the MPLS core, the egress PE router classifies the newly exposed IP
packets for outbound queuing, based on the MPLS PHB from the EXP bits of the recently
removed label. As with the short pipe mode, any changes to label markings that occur within
the service provider cloud do not get propagated to the IP ToS byte when the packet leaves
the MPLS network. The figure illustrates the pipe mode MPLS DiffServ tunneling operation.

This implementation avoids the additional operational overhead of per-customer

configurations on each egress interface on the egress PE router. As shown in the figure, the
pipe mode operation is identical to the short pipe, with the sole exception being that the final
PE egress queuing policies are based on the service provider markings (not the customer
markings).

31.6 Designing QoS for the MPLS VPN

Example: MPLS VPN QoS Roles 

Sometimes the number of service provider CoS classes will match (or exceed) the number of
application classes that an enterprise has defined in its strategic end-to-end QoS policy.
However, this setup might not always be the case. If the number of enterprise application
classes exceeds the number of service provider CoS classes, the enterprise administrator will
need to map into the service provider model, tactically and efficiently collapsing and
combining application classes and performing any required re-marking in the process.
The QoS policy elements can be grouped into roles that various router nodes serve within the
MPLS VPN.

Here are some recommendations when it comes to enterprise-to-service provider mapping:

 Efficiently map enterprise application classes to service provider CoS classes.

 Balance service-level requirements for real-time voice and video applications with
service provider premiums for real-time bandwidth.

 Avoid mixing control plane traffic with data plane traffic in a single service provider
CoS.

 Separate TCP traffic from UDP traffic when mapping to service provider CoS classes.

Most service providers use the DSCP marking of packets that are offered to them to
determine which service provider CoS the packet should be assigned to. Therefore,
enterprises must mark (or re-mark) their traffic consistent with the service provider admission
criteria in order to gain the appropriate level of service.

A general DiffServ principle is to mark or trust traffic as close to the source as

administratively and technically possible. However, certain traffic types might need to be re-
marked before handoff to the service provider to gain admission to the correct class. If such a
re-marking is required, it is recommended that the re-marking be performed at the CE egress
edge, not within the campus. Service provider service offerings will likely evolve or expand
over time, and adjusting to such changes will be easier to manage if re-marking is performed
at the CE egress edge only.

The specific QoS policies for these roles are as follows:

1. Campus CE ingress/internal QoS: Ingress/internal QoS policies may be applied (if
required).

2. CE LAN edge:

o Ingress DSCP-trust should be enabled (enabled by default).

o Ingress NBAR2 classification and marking policies may be applied.

o Egress LLQ/CBWFQ/WRED policies may be applied (if required).

3. CE VPN edge:

o Ingress DSCP-trust should be enabled (enabled by default).

o Ingress NBAR2 classification and marking policies may be applied (to restore
DSCP markings lost in transit).

o Egress LLQ/CBWFQ/WRED policies may be applied (if required).

o Egress LLQ/CBWFQ/WRED policies should be applied.

o Egress hierarchical shaping with nested LLQ/CBWFQ/WRED policies may be

applied.

o Egress DSCP re-marking policies may be applied (to map application classes
into specific service provider classes of service).

4. PE ingress/internal: Ingress/internal QoS policies may be applied (if required).

5. PE customer-facing edge:

o Ingress DSCP trust should be enabled (enabled by default).

o Ingress policing policies to meter customer traffic should be applied.

o Ingress MPLS tunneling mode policies may be applied.

o Egress MPLS tunneling mode policies may be applied.

o Egress LLQ/CBWFQ/WRED policies should be applied.

6. PE core-facing edge:

o Ingress DSCP-trust should be enabled (enabled by default).

o Ingress policing policies to meter customer traffic should be applied.

o Egress MPLS EXP-based LLQ/CBWFQ policies should be applied.

 o Egress MPLS EXP-based WRED policies may be applied.

7. P (core router) ingress/internal QoS: Ingress/internal QoS policies may be applied (if
required).

8. P edges:

o Ingress DSCP-trust should be enabled (enabled by default).

o Egress MPLS EXP-based LLQ/CBWFQ policies may be applied (unless core

is overprovisioned or has MPLS TE enabled).

o Egress MPLS EXP-based WRED policies may be applied.