Exchange Server 2013

Architecture
Ross Smith IV
Principal Program Manager, Exchange Server

Disclaimer
2012 Microsoft Corporation. All rights reserved.
Microsoft, Office 365, and other product and service
names are or may be registered trademarks and/or
trademarks in the U.S. and/or other countries.
The information herein is for informational purposes
only and represents the current view of Microsoft
Corporation as of the date of this
presentation. Because Microsoft must respond to
changing market conditions, it should not be
interpreted to be a commitment on the part of
Microsoft, and Microsoft cannot guarantee the
accuracy of any information provided after the date of
this presentation. MICROSOFT MAKES NO
WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS
TO THE INFORMATION IN THIS PRESENTATION.

Agenda
Agenda

Discuss the new server

architecture in Exchange 2013

Discuss the Client Access

server role

Discuss the Mailbox

server role

E2007/E2010 Server Role Architecture

Enterprise Network
5 major roles
Tightly
coupled
Functionality

Geo affinity

Forefront
Online
Protection for
Exchange

External
SMTP
servers

Edge Transport
Routing and AV/AS

Hub Transport
Routing & policy

Mailbox
Storage of
mailbox items

Versioning
User partitioning

Mobile
phone

Layer 7 LB

Web
browser
Outlook
(remote user)

Phone system
(PBX or VOIP)

Outlook (local user)

Unified Messaging
Voice mail and
voice access

Client Access
Client connectivity
Web services

Line of
business
application
AD

Whats wrong with the existing model?

Exchange deployments are overly complicated

Doing Exchange load balancing right is hard and often requires
expensive solutions
Requiring session affinity at the load balancer significantly impacts scalability
Many hardware load balancing solutions are expensive and thus, are a luxury
many of our customers cant afford or dont have the expertise to deploy

Customers deploy based on dedicated server roles

This means that in many cases, hardware is deployed that is unutilized (e.g.,
DIMM slots and disk slots, etc.) or under-utilized

Too many namespaces are required (especially in site resilient designs)

Exchange: The Evolution

LB
L7 LB
CAS
Ex

MBX
Ex

HT
Ex

SAN

MBX
Ex

Exchange: The Evolution

2000/2003

2007

2010

2013

Role differentiation through

manual configuration

Separate roles for ease of

deployment and mgmt.
segmentation

Separate HA solutions for

each role

Simplify for scale, balanced

utilization, isolation

Introduced the DAG

Integrate HA for all roles

Rich management
experience using RBAC

Simplify network
architecture

Hardware solutions for

reliability ($$$$)

Support cheaper storage

Leaves resources on the

ground in each role

Ex

Ex
SAN

Ex

Ex

CAS

HT

MBX

MBX

L7 LB

LB

The new server role

architecture

Exchange 2013 Architecture Theme

Use Building Blocks to facilitate deployments at all scales from self-hosted
small organizations to Office 365
Server role evolution
Network layer improvements
Versioning and inter-op principles

Benefits

Hardware efficiency
Deployment simplicity
Low friction cross-version inter-op
Failure isolation

Exchange 2013 Server Role Architecture

2 Building
Blocks

Database
Availability Group
Evolution of
E2010 DAG
Includes core
server protocols

Loosely
coupled

Functionality
Versioning
User partitioning
Geo affinity

Forefront
Online
Protection for
Exchange

External
SMTP
servers

Edge Transport
Routing and AV/AS

CAS Array

DAG

CAS

MBX

CAS

MBX

CAS

MBX

CAS

MBX

CAS

MBX

Layer 4LB

Client Access Array

Evolution of
E2010 CAS Array
SMTP Front-End

Enterprise Network

Mobile
phone
Web
browser
Outlook
(remote user)

Outlook (local user)

AD

Line of
business
Phone system
application (PBX or VOIP)

Every Server is an Island

EWS protocol
MRS proxy protocol
SMTP

Protocols,
Server Agents

Business Logic

Storage

MRS MRSProxy

EWS

Transport

Custom WS

MRS MRSProxy

Transport

RPC CA

Assistants

Assistants

RPC CA

XSO

Mail Item

XSO

Mail Item

CTS

Other API

CTS

Other API

Store

Content index

Store

Content index

ESE

File system

ESE

File system

Server1 (Vn)

Banned
E2010

Server2 (Vn+1)

EWS

Functional Layering
E2010 Architecture

L7LB

Load Balancer
AuthN, Proxy, Re-direct

CAS, HT, UM

AuthN, Proxy, Re-direct

Protocols, API, Biz-logic

Assistants, Store, CI
Store, CI

MBX2013

Protocols, API, Biz-logic

MBX

CAS2013

Hardware LB

E2013 Architecture

The key to enlightenment

User
For a given mailboxs connectivity, the
protocol being used is always served by
the protocol instance that is local to the
active database copy

CAS

This means that the rendering for clients like

OWA occurs on the Mailbox server

This means that Transport transcoding is

occurring on the Mailbox server etc

DAG1

MBX-A

MBX-B

Client Access Protocol

Proxying

What is the CAS2013 role?

CAS2013 is comprised of three components:

Thin, stateless (protocol session) servers organized in a load balanced

configuration

Client protocols (HTTP, IMAP, POP)

SMTP
UM Call Router

Session affinity NOT required at the load balancer

Provides a unified namespace and authentication for clients

Where the logic lives to route a specific protocol request to the correct
destination end point
Capable of supporting legacy servers with redirect or proxy logic

Is a domain-joined machine in the corporate forest

CAS2013 Client Protocol Architecture

OWA

Outlook

EAS

EAC

PowerShell

IMAP

SMTP

Telephony

Load
Balancer

Redirect

IIS

CAS2013

POP
IMAP

HTTP Proxy
POP
IMAP

HTTP

UM

SMTP
POP
IMAP

IIS

MBX2013

SMTP

Transport

UM

RpcProxy
RPS

OWA, EAS, EWS, ECP, OAB

RPC CA

MDB

MailQ

SIP
+
RTP

Outlook Connectivity

RPC/HTTP and the death of RPC/TCP

RPCProxy.dll

What are the benefits?

Does not require a RPC CAS array namespace for the DAG
No longer have to worry about The Exchange administrator has made a
change that requires you to quit and restart Outlook during mailbox
moves or *over events
Extremely reliable and stable connectivity model the RPC session is
always on the MBX2013 server hosting the active database copy

What changes?
RPC end point for Outlook client is now a GUID (and SMTP suffix)
Support for internal and external Outlook Anywhere namespaces

Third-Party MAPI Products

Third-party MAPI products will need to use RPC/HTTP to connect to
CAS2013
Exchange 2013 will be the last release that supports a MAPI/CDO
download
Third-parties must move to EWS in the future

The MAPI/CDO download will be updated to include support for

RPC/HTTP connectivity
Will require third-party application configuration; either by
programmatically editing a dynamic MAPI profile or setting registry keys
Legacy environments can continue to use RPC/TCP

CAS2013 Client Protocol Connectivity Flow

HTTP

HTTP

Load Balancer

HTTP Proxy
HTTP

MBX

CAS
IIS
HTTP Proxy
HTTP

MBX

Site Boundary

IIS

Site Boundary

CAS

Load Balancer
HTTP

MBX

Protocol Head

Protocol Head

Protocol Head

DB

DB

DB

Local Proxy Request

OWA Cross-Site Redirect Request

Cross-Site Proxy Request

Trade-Offs

Whos it for?

Exchange Load Balancing Options

Generalist IT admin

Those with increased

network flexibility

Those who want to

maximize server
availability

Functionality
Simplicity
+ Simple, fast, no affinity LB
+ Single, unified namespace
+ Minimal networking skillset
- Per Server Availability

+ Simple, fast, no affinity LB

+ Per protocol availability

+ Per protocol availability

+ Single, unified namespace

- One namespace per protocol

- SSL termination @ LB
- Requires increase networking
skillset

A Single Common Namespace Example

Geographical DNS Solution
Sue

(somewhere in NA)

mail.contoso.com

DNS Resolution
Round-Robin between # of VIPs

VIP #1

DAG

Sue

DNS Resolution via Geo-DNS

Round-Robin between # of VIPs

VIP #2

VIP #3

DAG

(traveling
in APAC)

VIP #4

CAS2013 Client Protocol Benefits

Simplifies the network layer

Deployment flexibility

Simplifies upgrade and inter-op

No longer requires session affinity at the load balancer

Just get the traffic to CAS2013 and let it handle the affinity
CAS2013 can be farther away from MBX2013 and still offer good client
performance (because it is a 1:1 proxy)
Removes the need for RPC Client Access arrays
CAS2013 provides more deployment flexibility; for example, consolidate to
fewer sites
Can deploy a single world-wide namespace
Designed to proxy to multiple Mailbox server versions, up and down level
DAGs can be replaced with E2013 at any desired pace

The Mailbox Server Role

The Mailbox Server Role

A server that hosts all the components that process,

render and store the data
Clients do not connect directly to MBX2013 servers;
connectivity is through CAS2013
Evolution of E2010 DAG
Collection of servers that form a HA unit
Databases are replicated between servers in a given
DAG
Servers can be in different locations, for site resiliency
Maximum of 16 Mailbox servers
50 database copies / server

MBX1

MBX2

MBX16

The New Store Process

Store is effectively made up of three processes
Replication service
Store service process/controller
Store worker process

Replication service initiates failovers and is responsible for issuing

mount/dismount operations
Store service process/controller manages the store worker processes
Each database has its own Store worker process

Exchange IOPS Trend

DB IOPS/Mailbox

+99%

0.8
0.6

reduction!

0.4
0.2

0
Exchange 2003

Exchange 2007

Exchange 2010

Exchange 2013

Large Mailboxes for the win!

Large Mailbox Size 100GB+
Aggregate Mailbox = Primary Mailbox +
Archive Mailbox + Recoverable Items
1-2 years of mail (minimum)

Increased knowledge worker

productivity
Eliminate or reduce PST reliance

Eliminate or reduce third-party

archive solutions
Outlook 2013 allows you to control
OST size!
Gives more options around mailbox
deployments

1 Day

150

11 MB

1 Month

3300

242 MB

1 Year

39000

2.8 GB

2 Years

78000

5.6 GB

4 Years

156000

11.2 GB

New Exchange Search Infrastructure

Leverages Search Foundation
Common, actively developed search platform used across Office server
products
Does consume more memory (1/6 available memory) to improve query
performance

Provides
Significantly improved query performance compared to E2010
Significantly improved indexing performance compared to E2010

Feature parity with E2010 search

Leverages the same cmdlets like Get-MailboxDatabaseCopyStatus

Exchange Indexing

Reduced Processing of Body and Attachments

MBX2013

MBX2013

Transport
Transport

Content Transformation
Service

Mailbox
Store

Log

DB

Mailbox

Local Delivery
ExSearch

Reliable
Event

CTS

Index Node

Read
Content

Passive

Log

Idx

DB

Idx

Public Folders
Dawn of a New Age

Architectural bet

Public folders are based on the mailbox architecture

Details
Private
logon

Public Logon

Public
logon

CAS 2013

MBX2013

Hierarchy Mailbox

Content Mailbox

MBX2013

MBX2013

Hierarchy is stored in PF mailboxes (one writeable)

Content can be broken up and placed in multiple
mailboxes
The hierarchy folder points to the target content mailbox
Uses same HA mechanism as mailboxes
No separate replication mechanism
Single-master model
Similar administrative features to current PFs (setting
quota, expiry, etc.)
No end-user changes (looks just like todays PFs)

Not all public folder usage scenarios

are best served by public folders

Transport Architecture

Transport Components on Client Access

Front-End Transport Service

Handles all inbound and outbound external SMTP traffic for the
organization, as well as client endpoint for SMTP traffic
Does not replace the Edge Transport Server Role

Functions as a layer 7 proxy and has full access to protocol

conversation
Will not queue mail locally and will be completely stateless
All outbound traffic appears to come from the CAS2013
Listens on TCP25 and TCP587 (two receive connectors)

Processing Inbound Messages

Externa
l Server

CAS

MBX

1. New SMTP Connection

2. CAS performs envelope filtering
3. CAS determines route to best MBX
server
4. Message delivery begins
1. If successful, CAS returns 250 OK
acknowledgement to external
server
2. If unsuccessful, CAS returns 421
response

Benefits of SMTP Front-End Service

The SMTP Front-End Service provides:
Protocol level filtering performs connection, recipient, sender and
protocol filtering
Network protection centralized, load balanced egress/ingress point for
the organization
Mailbox locator avoids unnecessary hops by determining the best
MBX2013 to deliver the message
Load balanced solution for client/application SMTP submissions

Scales based on number of connections just add more servers

Transport Components on Mailbox

Transport in MBX2013 has been broken down into three components

Transport Service - Stateful and handles SMTP mail flow for the organization
and performs content inspection (Was previously referred to as Hub
Transport)
Mailbox Transport Delivery Service - Receives mail from the Transport service
and deliveries to the Mailbox Database
Mailbox Transport Submission Service - Takes mail from the Mailbox Databases
and submits to the Transport service

Mailbox Transport is stateless and does not have a persistent storage

mechanism
Mailbox Transport performs content conversion

Transport Components on Mailbox

Responsibilities

Receives all inbound mail to the organization (Proxied through CAS

or direct)
Submits all outbound mail from the organization (Proxied through
CAS or direct)
Handles all internal message processing such as Transport Rules,
Content Filtering, and Anti-Virus
Performs mail flow routing
Queue messages
Supports SMTP extensibility

Routing Optimizations

Next hop selection is broken down into distinct delivery groups:

Routable DAG
Mailbox Delivery Group
Connector Source Servers
AD Site (Hub Sites; Edge Subscriptions)
Server list (DG expansion servers)

Queuing is per delivery group, connector, or mailbox

Once message is received at final destination, Transport will deliver the
message via SMTP to Mailbox Transport on the server hosting the active
database copy
Send/Delivery-Agent Connectors can have source servers from multiple
DAGs or AD Sites, and can be proxied through CAS

Mail Delivery
CAS /
MBX

DAG
MBX-1
SMTP

MBX-2

Transport
SMTP

Mailbox
Transport

SMTP

Transport
Mailbox
Transport

MAPI

DB1

DB2

DB1

DB2

Service Availability
Improvements

Service Availability Improvements

Best copy selection now includes health of services when
selecting best copy
Failover time reductions
Lagged copy enhancements
Simpler site resilience strategy
Managed Availability

Managed Availability

Monitoring and recovery

infrastructure is integrated with
Exchanges high availability
solution
Detects and recovers from
problems as they occur and are
discovered
Is user focused if you cant
measure it, you cannot monitor it

Managed Availability

Managed Availability + Retriesstuff breaks and the Experience does not

LB

CAS-1

DAG
MBX-1

OWA

DB1

DB2

DB1

DB2

DB1

DB2

MBX-2

CAS-2

OWA

MBX-3
OWA

OWA send
OWA failure
OWA failure detected
OWA restart service
OWA restart complete
OWA verified as healthy
OWA send
OWA failure
OWA failure detected
OWA restart service
OWA restart service failed
Failover servers databases
OWA service restarts
OWA verified as healthy
Server becomes good
failover target (again)

Transport High Availability Improvements

Every message is redundantly persisted before its receipt is
acknowledged to the sender
Delivered messages are kept redundant in transport similar to
active messages
Every DAG represents a transport HA boundary and owns its
HA implementation
If you have a stretched DAG, you also have transport site resilience

Resubmits due to transport DB loss or MDB *over are fully

automatic and do not require any manual involvement

Transport HA

CAS2013 or
MBX2013

SMTP
250
OK
R1, R2, R3

250 OK

Transport

Transport

1. Maintain a copy of the message in the queue

database but dont acknowledge the DATA verb
2. Generate a shadow copy on another MBX2013
server in the DAG (remote site preferred)
3. Wait for acknowledgement from the shadow server
4. Send acknowledgement to SMTP client
5. Delete message from queue after SafetyNet
threshold has expired

Transport

Transport

R1
R1,R3
R2, R3
R2

Mail.que

Mail.que

MBX Transport

Mail.que

MBX Transport

Mail.que

MBX Transport

MBX Transport

250 OK

Store
DB
2

Log

MBX1

Transport

R3

DB
1

Log

DB
2

Mail.que

MBX Transport

MBX5

DB
1

Log

DB
2

Log

DB
4

MBX6

MBX3

Store
DB
1

Log

DB
2

MBX Transport

Store
Log

DB
4

MBX4

Mail.que

MBX Transport

DB
3

Log

Transport
Mail.que

Store
DB
3

Log

Transport

MBX Transport

Store
DB
4

250 OK

Transport

Mail.que

DB
3

MBX2

Store
Site Boundary

DB
1

Store

MBX7

Store
DB
3

Log

DB
4

MBX8

Facilitates deployments at all scales from self-hosted

small organizations to Office 365
Provides more flexibility in namespace management

All core Exchange functionality for a given mailbox is

served by the MBX2013 server where that mailboxs
database is currently activated
Simplifies the network layer
Transport protection is built-in

All components in a given server upgraded together

No need to juggle with CAS <-> MBX versions
separately

Utilize CPU core increase, cheaper RAM

Utilize capacity effectively
Fewer disks/server => simpler server SKUs

Thank you!